diff --git a/README.md b/README.md
index 716c55e..c24862f 100644
--- a/README.md
+++ b/README.md
@@ -799,3 +799,69 @@ Pri interpretácii výsledkov treba zohľadniť:
 - nebol zatiaľ zrobený benchmark základného modelu `slovak-nlp/mistral-sk-7b` bez adaptéru.
 
 
+
+
+---
+
+# Optuna – výber learning rate na 30k Aya príkladoch
+
+Po sérii 100k porovnávacích experimentov som na Hruške vyskúšal menší skúšobný experiment s automatickým výberom learning rate pomocou framewrku Optuna. Cieľom bolo vybrať sľubnú oblasť LR pre následné overenie na väčšej dátovej množine. Táto časť sumarizuje **osem úspešne dokončených trialov**.
+
+Konfigurácie, súhrnné výsledky, grafy a podrobné záznamy úspešných trialov sú v priečinku [optuna_30k_experiments/](optuna_30k_experiments/).
+
+Tréningové logy všetkých ôsmich dokončených trialov sú v priečinku [logs/](optuna_30k_experiments/logs/).
+
+## Metodika a spoločné nastavenia
+
+Optimalizovaný bol iba parameter `learning_rate` v rozsahu **2e-5 až 2e-4**. Prvé štyri referenčné hodnoty boli vložené ručne: `2e-5`, `5e-5`, `1e-4` a `2e-4`. Ďalšie hodnoty navrhovala Optuna.
+
+
+| Parameter | Hodnota |
+|---|---|
+| Základný model | `slovak-nlp/mistral-sk-7b` |
+| Dataset | fixný `aya_slovak_hpo_30k`, bez streamingu |
+| Rozdelenie dát | 29 000 tréningových + 1 000 validačných príkladov |
+| Hardvér | 1× NVIDIA RTX A4500 na Hruške, GPU 0 |
+| Compute type | `bf16` |
+| Framework | LLaMA-Factory + Unsloth |
+| Fine-tuning | LoRA, bez nastavenej 4-bit kvantizácie |
+| Template / max. dĺžka | `alpaca` / 1024 tokenov |
+| Packing / train on prompt | `false` / `false` |
+| LoRA rank / alpha / dropout | 16 / 32 / 0.0 |
+| LoRA target moduly | `q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj` |
+| Batch size / gradient accumulation | 1 / 16, effective batch 16 |
+| Rozsah tréningu | 1 epocha, 1 813 optimizer krokov |
+| Optimizer / scheduler | AdamW (`adamw_torch`) / cosine |
+| Warmup | 150 krokov |
+| Max grad norm | 0.5 |
+| Eval interval / eval batch size | každých 300 krokov + finálna evaluácia / 1 |
+| Seed | 42 |
+
+
+## Výsledky ôsmich dokončených trialov
+
+| Poradie | Trial | Learning rate | Finálny eval loss | Maximum zaznamenaného grad_norm |
+|---:|---:|---:|---:|---:|
+| 1 | 3 | 0.0002000000 | 0.740409 | 14.7127 |
+| 2 | 8 | 0.0001864520 | 0.740567 | 3.4996 |
+| 3 | 4 | 0.0001869045 | 0.741250 | 29.0258 |
+| 4 | 2 | 0.0001000000 | 0.752890 | 7.8035 |
+| 5 | 6 | 0.0000625750 | 0.766744 | 8.2603 |
+| 6 | 1 | 0.0000500000 | 0.774108 | 9.5442 |
+| 7 | 7 | 0.0000202781 | 0.800064 | 16.1318 |
+| 8 | 0 | 0.0000200000 | 0.800524 | 17.8756 |
+
+Presné hodnoty LR, finálne aj minimálne eval loss, kroky, training loss a runtime sú v [CSV súhrne](optuna_30k_experiments/results/optuna_lr_results.csv).
+
+![Finálny eval loss v závislosti od learning rate](optuna_30k_experiments/results/lr_vs_eval_loss.png)
+
+![Maximum zaznamenaného grad_norm v závislosti od learning rate](optuna_30k_experiments/results/lr_vs_max_grad_norm.png)
+
+
+## Súhrn
+
+- Najnižší finálny eval loss dosiahol **trial 3 s LR 0.0002: 0.740409**.
+- Trial 8 s LR približne **0.00018645** dosiahol veľmi blízky výsledok **0.740567**; rozdiel je približne 0.000157.
+- Na tomto 30k tréningu mali nižšie skúšané LR vyšší finálny eval loss. 
+- Najlepší skúšaný LR leží na hornej hranici rozsahu.
+
diff --git a/optuna_30k_experiments/configs/base_hpo_config.yaml b/optuna_30k_experiments/configs/base_hpo_config.yaml
new file mode 100644
index 0000000..5f92d5f
--- /dev/null
+++ b/optuna_30k_experiments/configs/base_hpo_config.yaml
@@ -0,0 +1,50 @@
+stage: sft
+do_train: true
+do_eval: true
+model_name_or_path: slovak-nlp/mistral-sk-7b
+trust_remote_code: true
+dataset_dir: data
+dataset: aya_slovak_hpo_30k
+template: alpaca
+cutoff_len: 1024
+packing: false
+train_on_prompt: false
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.0
+lora_target: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
+per_device_train_batch_size: 1
+per_device_eval_batch_size: 1
+gradient_accumulation_steps: 16
+learning_rate: 0.0001
+lr_scheduler_type: cosine
+warmup_steps: 150
+max_grad_norm: 0.5
+bf16: true
+fp16: false
+optim: adamw_torch
+flash_attn: auto
+plot_loss: true
+ddp_timeout: 180000000
+ddp_find_unused_parameters: false
+include_num_input_tokens_seen: true
+enable_thinking: false
+use_unsloth: true
+use_unsloth_gc: true
+streaming: false
+eval_strategy: steps
+val_size: 1000
+eval_steps: 300
+overwrite_cache: false
+overwrite_output_dir: true
+output_dir: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/template
+logging_steps: 10
+save_steps: 1800
+save_total_limit: 1
+report_to: none
+seed: 42
+accelerator_config:
+  dispatch_batches: false
+  split_batches: false
+num_train_epochs: 1.0
diff --git a/optuna_30k_experiments/configs/trial_000_lr_2.000e-05.yaml b/optuna_30k_experiments/configs/trial_000_lr_2.000e-05.yaml
new file mode 100644
index 0000000..25672e9
--- /dev/null
+++ b/optuna_30k_experiments/configs/trial_000_lr_2.000e-05.yaml
@@ -0,0 +1,50 @@
+stage: sft
+do_train: true
+do_eval: true
+model_name_or_path: slovak-nlp/mistral-sk-7b
+trust_remote_code: true
+dataset_dir: data
+dataset: aya_slovak_hpo_30k
+template: alpaca
+cutoff_len: 1024
+packing: false
+train_on_prompt: false
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.0
+lora_target: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
+per_device_train_batch_size: 1
+per_device_eval_batch_size: 1
+gradient_accumulation_steps: 16
+learning_rate: 2.0e-05
+lr_scheduler_type: cosine
+warmup_steps: 150
+max_grad_norm: 0.5
+bf16: true
+fp16: false
+optim: adamw_torch
+flash_attn: auto
+plot_loss: true
+ddp_timeout: 180000000
+ddp_find_unused_parameters: false
+include_num_input_tokens_seen: true
+enable_thinking: false
+use_unsloth: true
+use_unsloth_gc: true
+streaming: false
+eval_strategy: steps
+val_size: 1000
+eval_steps: 300
+overwrite_cache: false
+overwrite_output_dir: true
+output_dir: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_000_lr_2.000e-05
+logging_steps: 10
+save_steps: 1800
+save_total_limit: 1
+report_to: none
+seed: 42
+accelerator_config:
+  dispatch_batches: false
+  split_batches: false
+num_train_epochs: 1.0
diff --git a/optuna_30k_experiments/configs/trial_001_lr_5.000e-05.yaml b/optuna_30k_experiments/configs/trial_001_lr_5.000e-05.yaml
new file mode 100644
index 0000000..c4987b1
--- /dev/null
+++ b/optuna_30k_experiments/configs/trial_001_lr_5.000e-05.yaml
@@ -0,0 +1,50 @@
+stage: sft
+do_train: true
+do_eval: true
+model_name_or_path: slovak-nlp/mistral-sk-7b
+trust_remote_code: true
+dataset_dir: data
+dataset: aya_slovak_hpo_30k
+template: alpaca
+cutoff_len: 1024
+packing: false
+train_on_prompt: false
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.0
+lora_target: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
+per_device_train_batch_size: 1
+per_device_eval_batch_size: 1
+gradient_accumulation_steps: 16
+learning_rate: 5.0e-05
+lr_scheduler_type: cosine
+warmup_steps: 150
+max_grad_norm: 0.5
+bf16: true
+fp16: false
+optim: adamw_torch
+flash_attn: auto
+plot_loss: true
+ddp_timeout: 180000000
+ddp_find_unused_parameters: false
+include_num_input_tokens_seen: true
+enable_thinking: false
+use_unsloth: true
+use_unsloth_gc: true
+streaming: false
+eval_strategy: steps
+val_size: 1000
+eval_steps: 300
+overwrite_cache: false
+overwrite_output_dir: true
+output_dir: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_001_lr_5.000e-05
+logging_steps: 10
+save_steps: 1800
+save_total_limit: 1
+report_to: none
+seed: 42
+accelerator_config:
+  dispatch_batches: false
+  split_batches: false
+num_train_epochs: 1.0
diff --git a/optuna_30k_experiments/configs/trial_002_lr_1.000e-04.yaml b/optuna_30k_experiments/configs/trial_002_lr_1.000e-04.yaml
new file mode 100644
index 0000000..f7965c8
--- /dev/null
+++ b/optuna_30k_experiments/configs/trial_002_lr_1.000e-04.yaml
@@ -0,0 +1,50 @@
+stage: sft
+do_train: true
+do_eval: true
+model_name_or_path: slovak-nlp/mistral-sk-7b
+trust_remote_code: true
+dataset_dir: data
+dataset: aya_slovak_hpo_30k
+template: alpaca
+cutoff_len: 1024
+packing: false
+train_on_prompt: false
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.0
+lora_target: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
+per_device_train_batch_size: 1
+per_device_eval_batch_size: 1
+gradient_accumulation_steps: 16
+learning_rate: 0.0001
+lr_scheduler_type: cosine
+warmup_steps: 150
+max_grad_norm: 0.5
+bf16: true
+fp16: false
+optim: adamw_torch
+flash_attn: auto
+plot_loss: true
+ddp_timeout: 180000000
+ddp_find_unused_parameters: false
+include_num_input_tokens_seen: true
+enable_thinking: false
+use_unsloth: true
+use_unsloth_gc: true
+streaming: false
+eval_strategy: steps
+val_size: 1000
+eval_steps: 300
+overwrite_cache: false
+overwrite_output_dir: true
+output_dir: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_002_lr_1.000e-04
+logging_steps: 10
+save_steps: 1800
+save_total_limit: 1
+report_to: none
+seed: 42
+accelerator_config:
+  dispatch_batches: false
+  split_batches: false
+num_train_epochs: 1.0
diff --git a/optuna_30k_experiments/configs/trial_003_lr_2.000e-04.yaml b/optuna_30k_experiments/configs/trial_003_lr_2.000e-04.yaml
new file mode 100644
index 0000000..6b20820
--- /dev/null
+++ b/optuna_30k_experiments/configs/trial_003_lr_2.000e-04.yaml
@@ -0,0 +1,50 @@
+stage: sft
+do_train: true
+do_eval: true
+model_name_or_path: slovak-nlp/mistral-sk-7b
+trust_remote_code: true
+dataset_dir: data
+dataset: aya_slovak_hpo_30k
+template: alpaca
+cutoff_len: 1024
+packing: false
+train_on_prompt: false
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.0
+lora_target: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
+per_device_train_batch_size: 1
+per_device_eval_batch_size: 1
+gradient_accumulation_steps: 16
+learning_rate: 0.0002
+lr_scheduler_type: cosine
+warmup_steps: 150
+max_grad_norm: 0.5
+bf16: true
+fp16: false
+optim: adamw_torch
+flash_attn: auto
+plot_loss: true
+ddp_timeout: 180000000
+ddp_find_unused_parameters: false
+include_num_input_tokens_seen: true
+enable_thinking: false
+use_unsloth: true
+use_unsloth_gc: true
+streaming: false
+eval_strategy: steps
+val_size: 1000
+eval_steps: 300
+overwrite_cache: false
+overwrite_output_dir: true
+output_dir: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_003_lr_2.000e-04
+logging_steps: 10
+save_steps: 1800
+save_total_limit: 1
+report_to: none
+seed: 42
+accelerator_config:
+  dispatch_batches: false
+  split_batches: false
+num_train_epochs: 1.0
diff --git a/optuna_30k_experiments/configs/trial_004_lr_1.869e-04.yaml b/optuna_30k_experiments/configs/trial_004_lr_1.869e-04.yaml
new file mode 100644
index 0000000..8a503cb
--- /dev/null
+++ b/optuna_30k_experiments/configs/trial_004_lr_1.869e-04.yaml
@@ -0,0 +1,50 @@
+stage: sft
+do_train: true
+do_eval: true
+model_name_or_path: slovak-nlp/mistral-sk-7b
+trust_remote_code: true
+dataset_dir: data
+dataset: aya_slovak_hpo_30k
+template: alpaca
+cutoff_len: 1024
+packing: false
+train_on_prompt: false
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.0
+lora_target: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
+per_device_train_batch_size: 1
+per_device_eval_batch_size: 1
+gradient_accumulation_steps: 16
+learning_rate: 0.00018690446893209942
+lr_scheduler_type: cosine
+warmup_steps: 150
+max_grad_norm: 0.5
+bf16: true
+fp16: false
+optim: adamw_torch
+flash_attn: auto
+plot_loss: true
+ddp_timeout: 180000000
+ddp_find_unused_parameters: false
+include_num_input_tokens_seen: true
+enable_thinking: false
+use_unsloth: true
+use_unsloth_gc: true
+streaming: false
+eval_strategy: steps
+val_size: 1000
+eval_steps: 300
+overwrite_cache: false
+overwrite_output_dir: true
+output_dir: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_004_lr_1.869e-04
+logging_steps: 10
+save_steps: 1800
+save_total_limit: 1
+report_to: none
+seed: 42
+accelerator_config:
+  dispatch_batches: false
+  split_batches: false
+num_train_epochs: 1.0
diff --git a/optuna_30k_experiments/configs/trial_006_lr_6.257e-05.yaml b/optuna_30k_experiments/configs/trial_006_lr_6.257e-05.yaml
new file mode 100644
index 0000000..5238637
--- /dev/null
+++ b/optuna_30k_experiments/configs/trial_006_lr_6.257e-05.yaml
@@ -0,0 +1,50 @@
+stage: sft
+do_train: true
+do_eval: true
+model_name_or_path: slovak-nlp/mistral-sk-7b
+trust_remote_code: true
+dataset_dir: data
+dataset: aya_slovak_hpo_30k
+template: alpaca
+cutoff_len: 1024
+packing: false
+train_on_prompt: false
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.0
+lora_target: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
+per_device_train_batch_size: 1
+per_device_eval_batch_size: 1
+gradient_accumulation_steps: 16
+learning_rate: 6.257495330311199e-05
+lr_scheduler_type: cosine
+warmup_steps: 150
+max_grad_norm: 0.5
+bf16: true
+fp16: false
+optim: adamw_torch
+flash_attn: auto
+plot_loss: true
+ddp_timeout: 180000000
+ddp_find_unused_parameters: false
+include_num_input_tokens_seen: true
+enable_thinking: false
+use_unsloth: true
+use_unsloth_gc: true
+streaming: false
+eval_strategy: steps
+val_size: 1000
+eval_steps: 300
+overwrite_cache: false
+overwrite_output_dir: true
+output_dir: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_006_lr_6.257e-05
+logging_steps: 10
+save_steps: 1800
+save_total_limit: 1
+report_to: none
+seed: 42
+accelerator_config:
+  dispatch_batches: false
+  split_batches: false
+num_train_epochs: 1.0
diff --git a/optuna_30k_experiments/configs/trial_007_lr_2.028e-05.yaml b/optuna_30k_experiments/configs/trial_007_lr_2.028e-05.yaml
new file mode 100644
index 0000000..2a3029c
--- /dev/null
+++ b/optuna_30k_experiments/configs/trial_007_lr_2.028e-05.yaml
@@ -0,0 +1,50 @@
+stage: sft
+do_train: true
+do_eval: true
+model_name_or_path: slovak-nlp/mistral-sk-7b
+trust_remote_code: true
+dataset_dir: data
+dataset: aya_slovak_hpo_30k
+template: alpaca
+cutoff_len: 1024
+packing: false
+train_on_prompt: false
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.0
+lora_target: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
+per_device_train_batch_size: 1
+per_device_eval_batch_size: 1
+gradient_accumulation_steps: 16
+learning_rate: 2.0278096180761755e-05
+lr_scheduler_type: cosine
+warmup_steps: 150
+max_grad_norm: 0.5
+bf16: true
+fp16: false
+optim: adamw_torch
+flash_attn: auto
+plot_loss: true
+ddp_timeout: 180000000
+ddp_find_unused_parameters: false
+include_num_input_tokens_seen: true
+enable_thinking: false
+use_unsloth: true
+use_unsloth_gc: true
+streaming: false
+eval_strategy: steps
+val_size: 1000
+eval_steps: 300
+overwrite_cache: false
+overwrite_output_dir: true
+output_dir: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_007_lr_2.028e-05
+logging_steps: 10
+save_steps: 1800
+save_total_limit: 1
+report_to: none
+seed: 42
+accelerator_config:
+  dispatch_batches: false
+  split_batches: false
+num_train_epochs: 1.0
diff --git a/optuna_30k_experiments/configs/trial_008_lr_1.865e-04.yaml b/optuna_30k_experiments/configs/trial_008_lr_1.865e-04.yaml
new file mode 100644
index 0000000..744552e
--- /dev/null
+++ b/optuna_30k_experiments/configs/trial_008_lr_1.865e-04.yaml
@@ -0,0 +1,50 @@
+stage: sft
+do_train: true
+do_eval: true
+model_name_or_path: slovak-nlp/mistral-sk-7b
+trust_remote_code: true
+dataset_dir: data
+dataset: aya_slovak_hpo_30k
+template: alpaca
+cutoff_len: 1024
+packing: false
+train_on_prompt: false
+finetuning_type: lora
+lora_rank: 16
+lora_alpha: 32
+lora_dropout: 0.0
+lora_target: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
+per_device_train_batch_size: 1
+per_device_eval_batch_size: 1
+gradient_accumulation_steps: 16
+learning_rate: 0.00018645202585249604
+lr_scheduler_type: cosine
+warmup_steps: 150
+max_grad_norm: 0.5
+bf16: true
+fp16: false
+optim: adamw_torch
+flash_attn: auto
+plot_loss: true
+ddp_timeout: 180000000
+ddp_find_unused_parameters: false
+include_num_input_tokens_seen: true
+enable_thinking: false
+use_unsloth: true
+use_unsloth_gc: true
+streaming: false
+eval_strategy: steps
+val_size: 1000
+eval_steps: 300
+overwrite_cache: false
+overwrite_output_dir: true
+output_dir: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_008_lr_1.865e-04
+logging_steps: 10
+save_steps: 1800
+save_total_limit: 1
+report_to: none
+seed: 42
+accelerator_config:
+  dispatch_batches: false
+  split_batches: false
+num_train_epochs: 1.0
diff --git a/optuna_30k_experiments/logs/trial_000_lr_2.000e-05.log b/optuna_30k_experiments/logs/trial_000_lr_2.000e-05.log
new file mode 100644
index 0000000..5483024
--- /dev/null
+++ b/optuna_30k_experiments/logs/trial_000_lr_2.000e-05.log
@@ -0,0 +1,12418 @@
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/torch/cuda/__init__.py:734: UserWarning: Can't initialize NVML
+  warnings.warn("Can't initialize NVML")
+[INFO|2026-10-12 20:00:26] llamafactory.hparams.parser:651 >> Process rank: 0, world size: 1, device: cuda:0, distributed training: False, compute dtype: torch.bfloat16
+[INFO|configuration_utils.py:765] 2026-10-12 20:00:26,781 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-12 20:00:26,790 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
+[INFO|configuration_utils.py:765] 2026-10-12 20:00:28,414 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-12 20:00:28,418 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|configuration_utils.py:765] 2026-10-12 20:00:28,561 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-12 20:00:28,565 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-12 20:00:29] llamafactory.data.loader:144 >> Loading dataset aya_slovak_hpo_30k.jsonl...
+training example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28828, 2176, 28711, 28863, 29114, 10268, 28709, 6612, 28714, 2892, 493, 363, 6294, 10719, 363, 3811, 28723, 7379, 626, 28889, 307, 14637, 311, 28768, 2218, 22885, 3748, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 284, 8341, 6729, 345, 28738, 28739, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28723, 524, 4634, 28711, 570, 1879, 261, 28863, 532, 613, 14483, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 345, 15962, 28739, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 1879, 15173, 537, 284, 8341, 705, 28723, 367, 280, 28879, 570, 4415, 28716, 28797, 9335, 3175, 29826, 626, 323, 412, 8514, 16986, 29826, 28708, 345, 15962, 28739, 1879, 15173, 537, 1221, 4566, 28718, 325, 28707, 28723, 461, 28723, 724, 494, 8891, 24160, 28725, 8582, 28862, 24160, 28725, 20532, 28764, 271, 28725, 438, 29365, 15745, 1319, 12035, 28862, 8269, 284, 28827, 2762, 28709, 264, 4798, 8359, 1450, 8325, 12121, 1798, 420, 762, 1089, 290, 28906, 18765, 724, 2331, 29114, 1221, 4304, 1780, 363, 13881, 29826, 316, 2085, 28708, 28723, 22627, 28873, 263, 335, 20208, 284, 28827, 2762, 28709, 264, 4798, 5424, 10056, 1286, 290, 28906, 18765, 724, 2331, 29114, 11323, 10719, 12490, 363, 13881, 29826, 316, 284, 8341, 2390, 28723, 22127, 280, 28879, 21022, 1150, 424, 724, 28827, 1144, 28873, 340, 17066, 270, 21022, 485, 284, 8341, 705, 28725, 446, 6909, 28797, 637, 28705, 29114, 28708, 28879, 4798, 6612, 28827, 1100, 28768, 28863, 28723, 320, 412, 28879, 637, 15612, 28716, 28889, 28726, 1150, 424, 19523, 412, 28873, 276, 28797, 2390, 284, 8341, 2390, 363, 23407, 11277, 9538, 2085, 28708, 28723, 9532, 2144, 686, 7865, 28718, 341, 14475, 1687, 28706, 18037, 452, 270, 8269, 284, 8341, 705, 28725, 290, 4827, 486, 2710, 2144, 332, 1800, 16344, 29114, 6588, 269, 28862, 28711, 28863, 686, 1653, 5388, 28718, 710, 261, 1296, 28709, 284, 8341, 705, 28723, 351, 28711, 1371, 28797, 1776, 29826, 8269, 284, 8341, 705, 268, 28863, 1776, 29826, 8269, 1879, 3497, 598, 8269, 8631, 28879, 279, 412, 28725, 12121, 7988, 268, 28863, 24270, 12115, 28797, 1879, 5626, 263, 28862, 8269, 8631, 28879, 279, 412, 28723, 318, 15612, 1473, 276, 15690, 2245, 626, 15690, 307, 14637, 311, 28768, 300, 15469, 28711, 570, 446, 450, 4798, 29826, 333, 28729, 1879, 319, 3976, 294, 405, 28768, 4882, 405, 28725, 534, 28724, 2710, 724, 313, 4827, 2245, 626, 28889, 20532, 28764, 271, 28723, 10650, 300, 12235, 28827, 28873, 424, 268, 27172, 2245, 28723, 1054, 15504, 28879, 424, 724, 313, 8768, 4545, 504, 270, 8269, 2124, 2245, 626, 12689, 4584, 10906, 28718, 710, 18782, 28879, 28715, 28797, 1135, 10356, 28723, 2645, 28764, 12275, 28820, 28873, 29114, 13067, 549, 2892, 5872, 3811, 8631, 25991, 3026, 29114, 408, 28906, 28764, 485, 2082, 1403, 12121, 1798, 1773, 29826, 28729, 22160, 710, 408, 28906, 28764, 485, 1135, 10874, 1776, 4536, 28873, 300, 23407, 11277, 9538, 2085, 28708, 28723, 22127, 3361, 13470, 1150, 317, 28873, 424, 2082, 7330, 511, 2085, 28708, 28723, 401, 492, 13591, 28820, 261, 2220, 28862, 10766, 381, 28827, 486, 29114, 363, 28879, 4689, 3418, 28879, 21775, 29114, 280, 28723, 418, 28820, 11277, 28716, 6294, 26177, 903, 12689, 6612, 20122, 28733, 12868, 291, 2124, 2085, 28708, 1879, 6670, 28862, 9348, 493, 686, 375, 5620, 386, 28862, 28827, 28725, 23374, 4536, 25306, 16388, 957, 450, 15612, 28764, 2126, 29114, 511, 2152, 363, 25769, 14110, 17413, 26177, 903, 28873, 15808, 19396, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 7379, 1879, 277, 7939, 1879, 724, 313, 8768, 2245, 28718, 363, 446, 551, 9538, 12121, 1798, 24473, 21694, 300, 261, 5869, 28723, 384, 28728, 491, 28712, 4566, 15469, 28711, 570, 1879, 2245, 28725, 534, 28724, 2710, 7463, 15612, 13335, 28728, 4371, 1144, 28723, 10650, 300, 15469, 28711, 570, 1879, 1160, 28709, 28879, 3907, 13288, 491, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 28725, 534, 28724, 637, 686, 598, 5237, 5945, 17984, 28718, 2117, 18904, 3146, 515, 13288, 25208, 28723, 367, 280, 28879, 570, 12472, 1006, 5872, 28863, 358, 446, 311, 4845, 1879, 6149, 783, 494, 28718, 2245, 28718, 363, 17984, 28718, 405, 13288, 491, 28723, 13343, 29826, 28729, 385, 29114, 284, 8341, 705, 28747, 399, 3711, 3175, 29826, 626, 12990, 17984, 28718, 2117, 268, 6125, 15690, 345, 28738, 28739, 16986, 29826, 28708, 1773, 29826, 10641, 2124, 345, 28738, 28739, 582, 20524, 12214, 1773, 29826, 28729, 385, 29114, 284, 8341, 705, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 15173, 537, 28718, 325, 849, 3748, 28725, 446, 6909, 28820, 3061, 28720, 300, 28827, 1520, 6612, 749, 1780, 20532, 28764, 271, 586, 28873, 28710, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28731, 1879, 686, 6729, 1773, 29826, 28729, 22160, 284, 8341, 705, 28723, 550, 11245, 412, 28747, 550, 11245, 412, 1683, 28827, 16032, 271, 1342, 4295, 3191, 316, 28729, 6449, 2245, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 345, 28741, 28739, 1879, 22954, 338, 28718, 297, 12689, 345, 28741, 28739, 686, 2129, 28710, 29114, 16986, 28863, 358, 28723, 17563, 971, 28747, 17563, 971, 582, 20524, 12214, 1342, 28764, 1193, 1342, 4295, 5765, 6175, 2959, 15416, 28710, 13288, 491, 3589, 28723, 9532, 484, 28717, 3811, 18904, 17101, 29114, 22399, 971, 28725, 15612, 13335, 28728, 4371, 28711, 570, 13288, 20996, 28725, 446, 6909, 28797, 484, 28717, 3811, 18904, 17101, 29114, 28725, 264, 2513, 300, 8631, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 268, 345, 28741, 28739, 264, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Zhrnúť tento článok v jednej vete. Textový nástroj je ikona, ktorá sa podobá písmenu "T" v paneli nástrojov vľavo. Kliknite na túto ikonu. Použite rozbaľovaciu ponuku vedľa "Character" v hornej časti obrazovky na výber písma. Použite druhé rozbaľovacie menu vedľa "Character" na výber formátu (t. j. pravidelný, tučný, kurzor, atď.). Tradičné písmo ako Times New Roman alebo Garamond môže pridať formálny vzhľad loga. Bezšerifové písmo ako Helvetica môže pridať modernejší vzhľad písmu. Nepoužívajte príliš dekoratívne písma, ktoré sa ťažko čítajú. Tiež sa vyhýbajte zmiešanému písmu v návrhu loga. Ak si z internetu stiahnete bezplatné písma, mali by ste si uvedomiť licenčnú zmluvu pre tieto písma. Mnohé voľné písma sú voľné na osobné použitie, ale nie sú povolené na komerčné použitie. S vybraným textovým nástrojom kliknite kdekoľvek na grafickej doske, aby ste pridali textový kurzor. Potom napíšte svoj text. Zvážte pridanie samostatného textového objektu pre každé slovo. Obzvlášť ak plánujete používať rôzne farby alebo veľkosti pre rôzne slová vo vašom návrhu loga. Nepridávaj ešte farbu do loga. Farebná tlač nemusí byť vždy možnosťou. Návrh jednoduchého čierno-bieleho loga na začiatok zabezpečí, že vaše logo bude vyzerať dobre v jeho najjednoduchšej forme. Môžete tiež použiť nástroj Text na ceste na pridanie textu v kruhu alebo zakrivenom tvare. Dvakrát kliknite na text, aby ste ho vyzdvihli. Potom kliknite na položku znak v hornej časti obrazovky, aby sa zobrazila ponuka nastavenia znakov. Použite nasledujúce kroky na úpravu textu v ponuke znak. Veľkosť písma: Rozbaľovacia ponuka s malým "T" vedľa veľkého "T" upravuje veľkosť písma. Môžete tiež použiť nástroj výberu (ikona, ktorá pripomína čierny kurzor myši v paneli nástrojov vľavo) na zmenu veľkosti písma. Vedenie: Vedenie mení priestor medzi riadkami textu. Použite rozbaľovaciu ponuku vedľa ikony, ktorá sa podobá "A" na vrchu iného "A" zmeniť vedúce. Kerning: Kerning upravuje medzery medzi konkrétnymi znakmi. Ak chcete nastaviť kerning, vyzdvihnite znaky, ktoré chcete nastaviť, a potom použite rozbaľovaciu ponuku vedľa ikony s "A" aVyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+labels:
+Vyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+eval example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28965, 28709, 637, 8631, 25991, 3026, 28725, 13067, 10531, 28729, 350, 2591, 686, 3023, 28862, 28827, 284, 28873, 269, 13096, 28711, 28863, 6149, 3399, 28804, 550, 13881, 29826, 316, 300, 1879, 7871, 338, 28820, 6936, 1150, 28863, 7340, 6957, 8905, 3907, 12235, 28827, 28873, 424, 5765, 772, 28725, 446, 6909, 28889, 14848, 912, 12214, 4321, 28720, 751, 29365, 28723, 351, 28906, 18765, 298, 486, 29114, 28705, 28740, 387, 28705, 28750, 28734, 22116, 28827, 28723, 8294, 772, 28747, 13, 13, 27332, 12107, 28747, 13, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Čo sa používa, ak škodca zničí pšeničnú úrodu? Vzhľadom na predchádzajúcu otázku napíšte kontext, ktorý obsahuje odpoveď. Môže to byť 1 - 20 vetí. Kontext:
+
+### Response:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+labels:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+[INFO|configuration_utils.py:765] 2026-10-12 20:00:30,063 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-12 20:00:30,066 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-12 20:00:30] llamafactory.model.model_utils.kv_cache:144 >> KV cache is disabled during training.
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/unsloth/__init__.py:165: UserWarning: WARNING: Unsloth should be imported before [trl, transformers, peft] to ensure all optimizations are applied. Your code may run slower or encounter memory issues without these optimizations.
+
+Please restructure your imports with 'import unsloth' at the top of your file.
+  from ._gpu_init import *
+🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/torch/cuda/__init__.py:734: UserWarning: Can't initialize NVML
+  warnings.warn("Can't initialize NVML")
+🦥 Unsloth Zoo will now patch everything to make training faster!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: WARNING `trust_remote_code` is True.
+Are you certain you want to do remote code execution?
+==((====))==  Unsloth 2026.6.1: Fast Mistral patching. Transformers: 5.5.0.
+   \\   /|    NVIDIA RTX A4500. Num GPUs = 1. Max memory: 19.698 GB. Platform: Linux.
+O^O/ \_/ \    Torch: 2.6.0+cu124. CUDA: 8.6. CUDA Toolkit: 12.4. Triton: 3.2.0
+\        /    Bfloat16 = TRUE. FA [Xformers = 0.0.29.post3. FA2 = False]
+ "-____-"     Free license: http://github.com/unslothai/unsloth
+Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading weights file model.safetensors from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/model.safetensors.index.json
+Will use dtype=torch.bfloat16 as defined in model's config object
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 2,
+  "use_cache": true
+}
+
+
+Loading weights:   0%|          | 0/291 [00:00<?, ?it/s]
+Loading weights:   2%|▏         | 5/291 [00:00<00:06, 42.38it/s]
+Loading weights:   5%|▌         | 15/291 [00:00<00:03, 69.47it/s]
+Loading weights:   9%|▉         | 27/291 [00:00<00:02, 90.82it/s]
+Loading weights:  14%|█▎        | 40/291 [00:00<00:02, 100.14it/s]
+Loading weights:  18%|█▊        | 51/291 [00:00<00:02, 89.27it/s] 
+Loading weights:  22%|██▏       | 63/291 [00:00<00:02, 97.77it/s]
+Loading weights:  25%|██▌       | 74/291 [00:00<00:02, 98.51it/s]
+Loading weights:  29%|██▉       | 85/291 [00:00<00:02, 94.88it/s]
+Loading weights:  33%|███▎      | 95/291 [00:01<00:02, 92.07it/s]
+Loading weights:  36%|███▌      | 105/291 [00:01<00:02, 90.12it/s]
+Loading weights:  40%|████      | 117/291 [00:01<00:01, 94.74it/s]
+Loading weights:  45%|████▍     | 130/291 [00:01<00:01, 98.46it/s]
+Loading weights:  48%|████▊     | 141/291 [00:01<00:01, 96.02it/s]
+Loading weights:  53%|█████▎    | 153/291 [00:01<00:01, 100.07it/s]
+Loading weights:  56%|█████▋    | 164/291 [00:01<00:01, 99.73it/s] 
+Loading weights:  60%|██████    | 175/291 [00:01<00:01, 97.75it/s]
+Loading weights:  64%|██████▎   | 185/291 [00:01<00:01, 96.02it/s]
+Loading weights:  67%|██████▋   | 195/291 [00:02<00:01, 88.99it/s]
+Loading weights:  70%|███████   | 204/291 [00:02<00:00, 88.66it/s]
+Loading weights:  75%|███████▍  | 217/291 [00:02<00:00, 98.48it/s]
+Loading weights:  79%|███████▊  | 229/291 [00:02<00:00, 101.46it/s]
+Loading weights:  82%|████████▏ | 240/291 [00:02<00:00, 88.17it/s] 
+Loading weights:  87%|████████▋ | 252/291 [00:02<00:00, 95.84it/s]
+Loading weights:  91%|█████████ | 265/291 [00:02<00:00, 101.66it/s]
+Loading weights:  95%|█████████▍| 276/291 [00:02<00:00, 93.81it/s] 
+Loading weights: 100%|█████████▉| 290/291 [00:03<00:00, 105.40it/s]
+Loading weights: 100%|██████████| 291/291 [00:03<00:00, 95.84it/s] 
+loading configuration file generation_config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/generation_config.json
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2
+}
+
+Could not locate the custom_generate/generate.py inside slovak-nlp/mistral-sk-7b.
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: slovak-nlp/mistral-sk-7b had a bad pad_token (</s>). Using pad_token = <unk>.
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:68: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_fwd
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:85: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_bwd
+[INFO|2026-10-12 20:00:47] llamafactory.model.model_utils.checkpointing:144 >> Gradient checkpointing enabled.
+[INFO|2026-10-12 20:00:47] llamafactory.model.adapter:144 >> Upcasting trainable params to float32.
+[INFO|2026-10-12 20:00:47] llamafactory.model.adapter:144 >> Fine-tuning method: LoRA
+Unsloth 2026.6.1 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers.
+[INFO|2026-10-12 20:00:50] llamafactory.model.loader:144 >> trainable params: 41,943,040 || all params: 7,283,675,136 || trainable%: 0.5758
+The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'pad_token_id': 2}.
+==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 1
+   \\   /|    Num examples = 29,000 | Num Epochs = 1 | Total steps = 1,813
+O^O/ \_/ \    Batch size per device = 1 | Gradient accumulation steps = 16
+\        /    Data Parallel GPUs = 1 | Total batch size (1 x 16 x 1) = 16
+ "-____-"     Trainable parameters = 41,943,040 of 7,283,675,136 (0.58% trained)
+
+  0%|          | 0/1813 [00:00<?, ?it/s]`use_return_dict` is deprecated! Use `return_dict` instead!
+Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
+Unsloth: Will smartly offload gradients to save VRAM!
+
+  0%|          | 1/1813 [00:07<3:32:15,  7.03s/it]
+  0%|          | 2/1813 [00:13<3:24:36,  6.78s/it]
+  0%|          | 3/1813 [00:18<3:02:58,  6.07s/it]
+  0%|          | 4/1813 [00:23<2:42:42,  5.40s/it]
+  0%|          | 5/1813 [00:29<2:55:53,  5.84s/it]
+  0%|          | 6/1813 [00:37<3:11:11,  6.35s/it]
+  0%|          | 7/1813 [00:43<3:11:17,  6.36s/it]
+  0%|          | 8/1813 [00:48<2:58:29,  5.93s/it]
+  0%|          | 9/1813 [00:55<3:11:50,  6.38s/it]
+  1%|          | 10/1813 [01:01<3:03:22,  6.10s/it]
+                                                   
+{'loss': '1.804', 'grad_norm': '1.93', 'learning_rate': '1.2e-06', 'epoch': '0.005517', 'num_input_tokens_seen': 65928, 'train_runtime': '61.43', 'train_tokens_per_second': '1073'}
+
+  1%|          | 10/1813 [01:01<3:03:22,  6.10s/it]
+  1%|          | 11/1813 [01:07<3:03:13,  6.10s/it]
+  1%|          | 12/1813 [01:15<3:17:31,  6.58s/it]
+  1%|          | 13/1813 [01:19<2:54:50,  5.83s/it]
+  1%|          | 14/1813 [01:25<2:56:22,  5.88s/it]
+  1%|          | 15/1813 [01:30<2:48:03,  5.61s/it]
+  1%|          | 16/1813 [01:36<2:51:17,  5.72s/it]
+  1%|          | 17/1813 [01:42<2:56:35,  5.90s/it]
+  1%|          | 18/1813 [01:48<2:57:04,  5.92s/it]
+  1%|          | 19/1813 [01:55<3:03:25,  6.13s/it]
+  1%|          | 20/1813 [02:01<3:01:48,  6.08s/it]
+                                                   
+{'loss': '1.684', 'grad_norm': '2.422', 'learning_rate': '2.533e-06', 'epoch': '0.01103', 'num_input_tokens_seen': 131352, 'train_runtime': '121.1', 'train_tokens_per_second': '1084'}
+
+  1%|          | 20/1813 [02:01<3:01:48,  6.08s/it]
+  1%|          | 21/1813 [02:06<2:58:49,  5.99s/it]
+  1%|          | 22/1813 [02:11<2:47:58,  5.63s/it]
+  1%|▏         | 23/1813 [02:17<2:45:30,  5.55s/it]
+  1%|▏         | 24/1813 [02:22<2:48:24,  5.65s/it]
+  1%|▏         | 25/1813 [02:30<3:02:24,  6.12s/it]
+  1%|▏         | 26/1813 [02:37<3:12:57,  6.48s/it]
+  1%|▏         | 27/1813 [02:43<3:07:51,  6.31s/it]
+  2%|▏         | 28/1813 [02:50<3:13:20,  6.50s/it]
+  2%|▏         | 29/1813 [02:58<3:25:14,  6.90s/it]
+  2%|▏         | 30/1813 [03:02<3:04:50,  6.22s/it]
+                                                   
+{'loss': '1.691', 'grad_norm': '3.562', 'learning_rate': '3.867e-06', 'epoch': '0.01655', 'num_input_tokens_seen': 197376, 'train_runtime': '182.8', 'train_tokens_per_second': '1080'}
+
+  2%|▏         | 30/1813 [03:02<3:04:50,  6.22s/it]
+  2%|▏         | 31/1813 [03:08<3:04:24,  6.21s/it]
+  2%|▏         | 32/1813 [03:14<2:55:29,  5.91s/it]
+  2%|▏         | 33/1813 [03:20<2:55:33,  5.92s/it]
+  2%|▏         | 34/1813 [03:27<3:09:35,  6.39s/it]
+  2%|▏         | 35/1813 [03:34<3:11:30,  6.46s/it]
+  2%|▏         | 36/1813 [03:40<3:13:14,  6.52s/it]
+  2%|▏         | 37/1813 [03:46<3:01:20,  6.13s/it]
+  2%|▏         | 38/1813 [03:51<2:51:14,  5.79s/it]
+  2%|▏         | 39/1813 [03:56<2:49:34,  5.74s/it]
+  2%|▏         | 40/1813 [04:02<2:50:00,  5.75s/it]
+                                                   
+{'loss': '1.516', 'grad_norm': '1.376', 'learning_rate': '5.2e-06', 'epoch': '0.02207', 'num_input_tokens_seen': 261168, 'train_runtime': '242.5', 'train_tokens_per_second': '1077'}
+
+  2%|▏         | 40/1813 [04:02<2:50:00,  5.75s/it]
+  2%|▏         | 41/1813 [04:07<2:45:30,  5.60s/it]
+  2%|▏         | 42/1813 [04:13<2:49:46,  5.75s/it]
+  2%|▏         | 43/1813 [04:21<3:07:12,  6.35s/it]
+  2%|▏         | 44/1813 [04:28<3:09:27,  6.43s/it]
+  2%|▏         | 45/1813 [04:35<3:19:33,  6.77s/it]
+  3%|▎         | 46/1813 [04:41<3:13:24,  6.57s/it]
+  3%|▎         | 47/1813 [04:47<3:05:53,  6.32s/it]
+  3%|▎         | 48/1813 [04:54<3:08:55,  6.42s/it]
+  3%|▎         | 49/1813 [05:00<3:07:38,  6.38s/it]
+  3%|▎         | 50/1813 [05:06<3:07:29,  6.38s/it]
+                                                   
+{'loss': '1.534', 'grad_norm': '1.469', 'learning_rate': '6.533e-06', 'epoch': '0.02759', 'num_input_tokens_seen': 330104, 'train_runtime': '307', 'train_tokens_per_second': '1075'}
+
+  3%|▎         | 50/1813 [05:06<3:07:29,  6.38s/it]
+  3%|▎         | 51/1813 [05:14<3:18:46,  6.77s/it]
+  3%|▎         | 52/1813 [05:24<3:41:36,  7.55s/it]
+  3%|▎         | 53/1813 [05:30<3:29:13,  7.13s/it]
+  3%|▎         | 54/1813 [05:37<3:33:24,  7.28s/it]
+  3%|▎         | 55/1813 [05:44<3:26:11,  7.04s/it]
+  3%|▎         | 56/1813 [05:50<3:22:39,  6.92s/it]
+  3%|▎         | 57/1813 [05:55<3:05:32,  6.34s/it]
+  3%|▎         | 58/1813 [06:04<3:23:18,  6.95s/it]
+  3%|▎         | 59/1813 [06:11<3:23:22,  6.96s/it]
+  3%|▎         | 60/1813 [06:16<3:08:03,  6.44s/it]
+                                                   
+{'loss': '1.412', 'grad_norm': '1.321', 'learning_rate': '7.867e-06', 'epoch': '0.0331', 'num_input_tokens_seen': 407792, 'train_runtime': '376.5', 'train_tokens_per_second': '1083'}
+
+  3%|▎         | 60/1813 [06:16<3:08:03,  6.44s/it]
+  3%|▎         | 61/1813 [06:21<2:57:03,  6.06s/it]
+  3%|▎         | 62/1813 [06:27<2:54:18,  5.97s/it]
+  3%|▎         | 63/1813 [06:33<2:51:46,  5.89s/it]
+  4%|▎         | 64/1813 [06:38<2:50:11,  5.84s/it]
+  4%|▎         | 65/1813 [06:46<3:02:44,  6.27s/it]
+  4%|▎         | 66/1813 [06:52<3:03:39,  6.31s/it]
+  4%|▎         | 67/1813 [06:58<2:57:35,  6.10s/it]
+  4%|▍         | 68/1813 [07:04<2:59:50,  6.18s/it]
+  4%|▍         | 69/1813 [07:11<3:08:22,  6.48s/it]
+  4%|▍         | 70/1813 [07:19<3:16:38,  6.77s/it]
+                                                   
+{'loss': '1.263', 'grad_norm': '1.095', 'learning_rate': '9.2e-06', 'epoch': '0.03862', 'num_input_tokens_seen': 474368, 'train_runtime': '439.1', 'train_tokens_per_second': '1080'}
+
+  4%|▍         | 70/1813 [07:19<3:16:38,  6.77s/it]
+  4%|▍         | 71/1813 [07:24<3:06:53,  6.44s/it]
+  4%|▍         | 72/1813 [07:33<3:25:35,  7.09s/it]
+  4%|▍         | 73/1813 [07:38<3:11:16,  6.60s/it]
+  4%|▍         | 74/1813 [07:45<3:07:34,  6.47s/it]
+  4%|▍         | 75/1813 [07:51<3:08:13,  6.50s/it]
+  4%|▍         | 76/1813 [07:57<3:03:00,  6.32s/it]
+  4%|▍         | 77/1813 [08:04<3:06:34,  6.45s/it]
+  4%|▍         | 78/1813 [08:09<2:57:39,  6.14s/it]
+  4%|▍         | 79/1813 [08:15<2:54:05,  6.02s/it]
+  4%|▍         | 80/1813 [08:21<2:54:27,  6.04s/it]
+                                                   
+{'loss': '1.277', 'grad_norm': '1.363', 'learning_rate': '1.053e-05', 'epoch': '0.04414', 'num_input_tokens_seen': 541616, 'train_runtime': '501.5', 'train_tokens_per_second': '1080'}
+
+  4%|▍         | 80/1813 [08:21<2:54:27,  6.04s/it]
+  4%|▍         | 81/1813 [08:27<2:52:49,  5.99s/it]
+  5%|▍         | 82/1813 [08:34<3:01:26,  6.29s/it]
+  5%|▍         | 83/1813 [08:41<3:08:39,  6.54s/it]
+  5%|▍         | 84/1813 [08:48<3:13:44,  6.72s/it]
+  5%|▍         | 85/1813 [08:54<3:08:48,  6.56s/it]
+  5%|▍         | 86/1813 [09:01<3:08:35,  6.55s/it]
+  5%|▍         | 87/1813 [09:06<2:56:44,  6.14s/it]
+  5%|▍         | 88/1813 [09:12<2:57:15,  6.17s/it]
+  5%|▍         | 89/1813 [09:18<2:56:17,  6.14s/it]
+  5%|▍         | 90/1813 [09:25<2:58:51,  6.23s/it]
+                                                   
+{'loss': '1.184', 'grad_norm': '1.008', 'learning_rate': '1.187e-05', 'epoch': '0.04966', 'num_input_tokens_seen': 608792, 'train_runtime': '565.3', 'train_tokens_per_second': '1077'}
+
+  5%|▍         | 90/1813 [09:25<2:58:51,  6.23s/it]
+  5%|▌         | 91/1813 [09:31<2:55:27,  6.11s/it]
+  5%|▌         | 92/1813 [09:36<2:47:31,  5.84s/it]
+  5%|▌         | 93/1813 [09:42<2:48:20,  5.87s/it]
+  5%|▌         | 94/1813 [09:48<2:51:15,  5.98s/it]
+  5%|▌         | 95/1813 [09:54<2:54:25,  6.09s/it]
+  5%|▌         | 96/1813 [10:00<2:52:36,  6.03s/it]
+  5%|▌         | 97/1813 [10:07<2:56:29,  6.17s/it]
+  5%|▌         | 98/1813 [10:12<2:47:05,  5.85s/it]
+  5%|▌         | 99/1813 [10:17<2:44:07,  5.75s/it]
+  6%|▌         | 100/1813 [10:24<2:48:38,  5.91s/it]
+                                                    
+{'loss': '1.247', 'grad_norm': '1.117', 'learning_rate': '1.32e-05', 'epoch': '0.05517', 'num_input_tokens_seen': 670936, 'train_runtime': '624.1', 'train_tokens_per_second': '1075'}
+
+  6%|▌         | 100/1813 [10:24<2:48:38,  5.91s/it]
+  6%|▌         | 101/1813 [10:30<2:54:40,  6.12s/it]
+  6%|▌         | 102/1813 [10:37<2:59:43,  6.30s/it]
+  6%|▌         | 103/1813 [10:43<2:57:40,  6.23s/it]
+  6%|▌         | 104/1813 [10:49<2:58:26,  6.26s/it]
+  6%|▌         | 105/1813 [10:55<2:52:55,  6.07s/it]
+  6%|▌         | 106/1813 [11:01<2:50:18,  5.99s/it]
+  6%|▌         | 107/1813 [11:07<2:51:54,  6.05s/it]
+  6%|▌         | 108/1813 [11:13<2:54:25,  6.14s/it]
+  6%|▌         | 109/1813 [11:19<2:53:50,  6.12s/it]
+  6%|▌         | 110/1813 [11:26<2:58:48,  6.30s/it]
+                                                    
+{'loss': '1.203', 'grad_norm': '1.138', 'learning_rate': '1.453e-05', 'epoch': '0.06069', 'num_input_tokens_seen': 738632, 'train_runtime': '686.6', 'train_tokens_per_second': '1076'}
+
+  6%|▌         | 110/1813 [11:26<2:58:48,  6.30s/it]
+  6%|▌         | 111/1813 [11:33<3:03:27,  6.47s/it]
+  6%|▌         | 112/1813 [11:40<3:06:32,  6.58s/it]
+  6%|▌         | 113/1813 [11:46<3:05:09,  6.54s/it]
+  6%|▋         | 114/1813 [11:54<3:13:24,  6.83s/it]
+  6%|▋         | 115/1813 [12:00<3:06:38,  6.60s/it]
+  6%|▋         | 116/1813 [12:07<3:09:10,  6.69s/it]
+  6%|▋         | 117/1813 [12:12<3:00:49,  6.40s/it]
+  7%|▋         | 118/1813 [12:18<2:56:45,  6.26s/it]
+  7%|▋         | 119/1813 [12:25<2:57:56,  6.30s/it]
+  7%|▋         | 120/1813 [12:31<2:57:16,  6.28s/it]
+                                                    
+{'loss': '1.217', 'grad_norm': '1.645', 'learning_rate': '1.587e-05', 'epoch': '0.06621', 'num_input_tokens_seen': 808824, 'train_runtime': '751.5', 'train_tokens_per_second': '1076'}
+
+  7%|▋         | 120/1813 [12:31<2:57:16,  6.28s/it]
+  7%|▋         | 121/1813 [12:35<2:38:47,  5.63s/it]
+  7%|▋         | 122/1813 [12:40<2:32:51,  5.42s/it]
+  7%|▋         | 123/1813 [12:46<2:40:56,  5.71s/it]
+  7%|▋         | 124/1813 [12:52<2:36:35,  5.56s/it]
+  7%|▋         | 125/1813 [12:58<2:44:53,  5.86s/it]
+  7%|▋         | 126/1813 [13:04<2:47:11,  5.95s/it]
+  7%|▋         | 127/1813 [13:09<2:35:12,  5.52s/it]
+  7%|▋         | 128/1813 [13:13<2:27:10,  5.24s/it]
+  7%|▋         | 129/1813 [13:21<2:50:16,  6.07s/it]
+  7%|▋         | 130/1813 [13:28<2:54:37,  6.23s/it]
+                                                    
+{'loss': '1.151', 'grad_norm': '1.008', 'learning_rate': '1.72e-05', 'epoch': '0.07172', 'num_input_tokens_seen': 869720, 'train_runtime': '808.6', 'train_tokens_per_second': '1076'}
+
+  7%|▋         | 130/1813 [13:28<2:54:37,  6.23s/it]
+  7%|▋         | 131/1813 [13:34<2:51:07,  6.10s/it]
+  7%|▋         | 132/1813 [13:39<2:43:06,  5.82s/it]
+  7%|▋         | 133/1813 [13:45<2:41:54,  5.78s/it]
+  7%|▋         | 134/1813 [13:50<2:37:17,  5.62s/it]
+  7%|▋         | 135/1813 [13:55<2:34:29,  5.52s/it]
+  8%|▊         | 136/1813 [14:01<2:36:34,  5.60s/it]
+  8%|▊         | 137/1813 [14:07<2:40:25,  5.74s/it]
+  8%|▊         | 138/1813 [14:14<2:51:00,  6.13s/it]
+  8%|▊         | 139/1813 [14:21<2:55:02,  6.27s/it]
+  8%|▊         | 140/1813 [14:28<3:01:12,  6.50s/it]
+                                                    
+{'loss': '1.147', 'grad_norm': '1.75', 'learning_rate': '1.853e-05', 'epoch': '0.07724', 'num_input_tokens_seen': 932000, 'train_runtime': '868.3', 'train_tokens_per_second': '1073'}
+
+  8%|▊         | 140/1813 [14:28<3:01:12,  6.50s/it]
+  8%|▊         | 141/1813 [14:34<2:55:02,  6.28s/it]
+  8%|▊         | 142/1813 [14:42<3:08:42,  6.78s/it]
+  8%|▊         | 143/1813 [14:47<2:56:31,  6.34s/it]
+  8%|▊         | 144/1813 [14:52<2:49:06,  6.08s/it]
+  8%|▊         | 145/1813 [14:59<2:58:14,  6.41s/it]
+  8%|▊         | 146/1813 [15:06<2:57:10,  6.38s/it]
+  8%|▊         | 147/1813 [15:11<2:50:30,  6.14s/it]
+  8%|▊         | 148/1813 [15:17<2:50:16,  6.14s/it]
+  8%|▊         | 149/1813 [15:24<2:50:01,  6.13s/it]
+  8%|▊         | 150/1813 [15:30<2:52:11,  6.21s/it]
+                                                    
+{'loss': '1.179', 'grad_norm': '1.548', 'learning_rate': '1.987e-05', 'epoch': '0.08276', 'num_input_tokens_seen': 997296, 'train_runtime': '930.5', 'train_tokens_per_second': '1072'}
+
+  8%|▊         | 150/1813 [15:30<2:52:11,  6.21s/it]
+  8%|▊         | 151/1813 [15:36<2:47:28,  6.05s/it]
+  8%|▊         | 152/1813 [15:41<2:43:39,  5.91s/it]
+  8%|▊         | 153/1813 [15:47<2:42:17,  5.87s/it]
+  8%|▊         | 154/1813 [15:53<2:46:08,  6.01s/it]
+  9%|▊         | 155/1813 [16:00<2:55:09,  6.34s/it]
+  9%|▊         | 156/1813 [16:07<2:52:32,  6.25s/it]
+  9%|▊         | 157/1813 [16:13<2:52:06,  6.24s/it]
+  9%|▊         | 158/1813 [16:18<2:43:45,  5.94s/it]
+  9%|▉         | 159/1813 [16:24<2:42:19,  5.89s/it]
+  9%|▉         | 160/1813 [16:30<2:41:22,  5.86s/it]
+                                                    
+{'loss': '1.111', 'grad_norm': '1.715', 'learning_rate': '2e-05', 'epoch': '0.08828', 'num_input_tokens_seen': 1060096, 'train_runtime': '990', 'train_tokens_per_second': '1071'}
+
+  9%|▉         | 160/1813 [16:30<2:41:22,  5.86s/it]
+  9%|▉         | 161/1813 [16:36<2:42:20,  5.90s/it]
+  9%|▉         | 162/1813 [16:40<2:30:50,  5.48s/it]
+  9%|▉         | 163/1813 [16:47<2:42:09,  5.90s/it]
+  9%|▉         | 164/1813 [16:54<2:54:39,  6.35s/it]
+  9%|▉         | 165/1813 [17:01<2:55:05,  6.37s/it]
+  9%|▉         | 166/1813 [17:08<3:06:14,  6.78s/it]
+  9%|▉         | 167/1813 [17:15<3:05:03,  6.75s/it]
+  9%|▉         | 168/1813 [17:22<3:08:45,  6.88s/it]
+  9%|▉         | 169/1813 [17:28<2:58:30,  6.51s/it]
+  9%|▉         | 170/1813 [17:35<2:59:40,  6.56s/it]
+                                                    
+{'loss': '1.116', 'grad_norm': '0.8596', 'learning_rate': '1.999e-05', 'epoch': '0.09379', 'num_input_tokens_seen': 1130368, 'train_runtime': '1055', 'train_tokens_per_second': '1071'}
+
+  9%|▉         | 170/1813 [17:35<2:59:40,  6.56s/it]
+  9%|▉         | 171/1813 [17:40<2:49:46,  6.20s/it]
+  9%|▉         | 172/1813 [17:46<2:45:24,  6.05s/it]
+ 10%|▉         | 173/1813 [17:52<2:48:29,  6.16s/it]
+ 10%|▉         | 174/1813 [17:58<2:45:02,  6.04s/it]
+ 10%|▉         | 175/1813 [18:04<2:47:02,  6.12s/it]
+ 10%|▉         | 176/1813 [18:09<2:39:32,  5.85s/it]
+ 10%|▉         | 177/1813 [18:17<2:52:25,  6.32s/it]
+ 10%|▉         | 178/1813 [18:22<2:40:20,  5.88s/it]
+ 10%|▉         | 179/1813 [18:28<2:40:09,  5.88s/it]
+ 10%|▉         | 180/1813 [18:34<2:41:26,  5.93s/it]
+                                                    
+{'loss': '1.104', 'grad_norm': '1.564', 'learning_rate': '1.998e-05', 'epoch': '0.09931', 'num_input_tokens_seen': 1193192, 'train_runtime': '1114', 'train_tokens_per_second': '1071'}
+
+ 10%|▉         | 180/1813 [18:34<2:41:26,  5.93s/it]
+ 10%|▉         | 181/1813 [18:39<2:40:17,  5.89s/it]
+ 10%|█         | 182/1813 [18:45<2:38:18,  5.82s/it]
+ 10%|█         | 183/1813 [18:51<2:37:06,  5.78s/it]
+ 10%|█         | 184/1813 [18:56<2:34:04,  5.68s/it]
+ 10%|█         | 185/1813 [19:04<2:53:04,  6.38s/it]
+ 10%|█         | 186/1813 [19:11<2:52:23,  6.36s/it]
+ 10%|█         | 187/1813 [19:17<2:49:53,  6.27s/it]
+ 10%|█         | 188/1813 [19:22<2:41:08,  5.95s/it]
+ 10%|█         | 189/1813 [19:28<2:46:14,  6.14s/it]
+ 10%|█         | 190/1813 [19:35<2:50:16,  6.30s/it]
+                                                    
+{'loss': '1.055', 'grad_norm': '1.565', 'learning_rate': '1.997e-05', 'epoch': '0.1048', 'num_input_tokens_seen': 1259336, 'train_runtime': '1176', 'train_tokens_per_second': '1071'}
+
+ 10%|█         | 190/1813 [19:35<2:50:16,  6.30s/it]
+ 11%|█         | 191/1813 [19:41<2:43:52,  6.06s/it]
+ 11%|█         | 192/1813 [19:47<2:48:38,  6.24s/it]
+ 11%|█         | 193/1813 [19:54<2:51:57,  6.37s/it]
+ 11%|█         | 194/1813 [20:00<2:49:12,  6.27s/it]
+ 11%|█         | 195/1813 [20:08<3:00:10,  6.68s/it]
+ 11%|█         | 196/1813 [20:13<2:48:53,  6.27s/it]
+ 11%|█         | 197/1813 [20:20<2:55:57,  6.53s/it]
+ 11%|█         | 198/1813 [20:27<3:02:42,  6.79s/it]
+ 11%|█         | 199/1813 [20:34<3:02:08,  6.77s/it]
+ 11%|█         | 200/1813 [20:41<3:00:30,  6.71s/it]
+                                                    
+{'loss': '1.151', 'grad_norm': '1.74', 'learning_rate': '1.996e-05', 'epoch': '0.1103', 'num_input_tokens_seen': 1331224, 'train_runtime': '1241', 'train_tokens_per_second': '1072'}
+
+ 11%|█         | 200/1813 [20:41<3:00:30,  6.71s/it]
+ 11%|█         | 201/1813 [20:46<2:52:13,  6.41s/it]
+ 11%|█         | 202/1813 [20:52<2:43:27,  6.09s/it]
+ 11%|█         | 203/1813 [20:57<2:35:28,  5.79s/it]
+ 11%|█▏        | 204/1813 [21:04<2:45:05,  6.16s/it]
+ 11%|█▏        | 205/1813 [21:09<2:39:41,  5.96s/it]
+ 11%|█▏        | 206/1813 [21:15<2:40:27,  5.99s/it]
+ 11%|█▏        | 207/1813 [21:21<2:38:43,  5.93s/it]
+ 11%|█▏        | 208/1813 [21:29<2:53:02,  6.47s/it]
+ 12%|█▏        | 209/1813 [21:34<2:39:28,  5.97s/it]
+ 12%|█▏        | 210/1813 [21:40<2:39:27,  5.97s/it]
+                                                    
+{'loss': '1.027', 'grad_norm': '1.89', 'learning_rate': '1.994e-05', 'epoch': '0.1159', 'num_input_tokens_seen': 1392472, 'train_runtime': '1300', 'train_tokens_per_second': '1071'}
+
+ 12%|█▏        | 210/1813 [21:40<2:39:27,  5.97s/it]
+ 12%|█▏        | 211/1813 [21:48<3:01:13,  6.79s/it]
+ 12%|█▏        | 212/1813 [21:54<2:52:02,  6.45s/it]
+ 12%|█▏        | 213/1813 [22:00<2:44:26,  6.17s/it]
+ 12%|█▏        | 214/1813 [22:05<2:36:18,  5.87s/it]
+ 12%|█▏        | 215/1813 [22:10<2:34:35,  5.80s/it]
+ 12%|█▏        | 216/1813 [22:15<2:26:55,  5.52s/it]
+ 12%|█▏        | 217/1813 [22:21<2:24:57,  5.45s/it]
+ 12%|█▏        | 218/1813 [22:27<2:31:24,  5.70s/it]
+ 12%|█▏        | 219/1813 [22:33<2:34:31,  5.82s/it]
+ 12%|█▏        | 220/1813 [22:39<2:37:45,  5.94s/it]
+                                                    
+{'loss': '1.116', 'grad_norm': '1.319', 'learning_rate': '1.992e-05', 'epoch': '0.1214', 'num_input_tokens_seen': 1454040, 'train_runtime': '1360', 'train_tokens_per_second': '1069'}
+
+ 12%|█▏        | 220/1813 [22:39<2:37:45,  5.94s/it]
+ 12%|█▏        | 221/1813 [22:46<2:42:15,  6.12s/it]
+ 12%|█▏        | 222/1813 [22:51<2:35:44,  5.87s/it]
+ 12%|█▏        | 223/1813 [22:59<2:49:12,  6.39s/it]
+ 12%|█▏        | 224/1813 [23:04<2:38:23,  5.98s/it]
+ 12%|█▏        | 225/1813 [23:11<2:46:39,  6.30s/it]
+ 12%|█▏        | 226/1813 [23:18<2:51:46,  6.49s/it]
+ 13%|█▎        | 227/1813 [23:23<2:45:26,  6.26s/it]
+ 13%|█▎        | 228/1813 [23:30<2:46:50,  6.32s/it]
+ 13%|█▎        | 229/1813 [23:37<2:53:09,  6.56s/it]
+ 13%|█▎        | 230/1813 [23:42<2:41:36,  6.13s/it]
+                                                    
+{'loss': '1.068', 'grad_norm': '2.185', 'learning_rate': '1.989e-05', 'epoch': '0.1269', 'num_input_tokens_seen': 1522056, 'train_runtime': '1422', 'train_tokens_per_second': '1070'}
+
+ 13%|█▎        | 230/1813 [23:42<2:41:36,  6.13s/it]
+ 13%|█▎        | 231/1813 [23:47<2:32:48,  5.80s/it]
+ 13%|█▎        | 232/1813 [23:54<2:45:48,  6.29s/it]
+ 13%|█▎        | 233/1813 [24:00<2:36:05,  5.93s/it]
+ 13%|█▎        | 234/1813 [24:05<2:29:48,  5.69s/it]
+ 13%|█▎        | 235/1813 [24:11<2:38:30,  6.03s/it]
+ 13%|█▎        | 236/1813 [24:19<2:51:01,  6.51s/it]
+ 13%|█▎        | 237/1813 [24:26<2:54:30,  6.64s/it]
+ 13%|█▎        | 238/1813 [24:32<2:47:56,  6.40s/it]
+ 13%|█▎        | 239/1813 [24:40<2:57:44,  6.78s/it]
+ 13%|█▎        | 240/1813 [24:46<2:51:51,  6.56s/it]
+                                                    
+{'loss': '1.05', 'grad_norm': '1.283', 'learning_rate': '1.986e-05', 'epoch': '0.1324', 'num_input_tokens_seen': 1590200, 'train_runtime': '1486', 'train_tokens_per_second': '1070'}
+
+ 13%|█▎        | 240/1813 [24:46<2:51:51,  6.56s/it]
+ 13%|█▎        | 241/1813 [24:52<2:50:01,  6.49s/it]
+ 13%|█▎        | 242/1813 [24:58<2:48:25,  6.43s/it]
+ 13%|█▎        | 243/1813 [25:04<2:43:19,  6.24s/it]
+ 13%|█▎        | 244/1813 [25:11<2:50:33,  6.52s/it]
+ 14%|█▎        | 245/1813 [25:18<2:56:08,  6.74s/it]
+ 14%|█▎        | 246/1813 [25:24<2:47:54,  6.43s/it]
+ 14%|█▎        | 247/1813 [25:31<2:51:07,  6.56s/it]
+ 14%|█▎        | 248/1813 [25:39<2:59:48,  6.89s/it]
+ 14%|█▎        | 249/1813 [25:44<2:46:42,  6.40s/it]
+ 14%|█▍        | 250/1813 [25:50<2:40:23,  6.16s/it]
+                                                    
+{'loss': '1.151', 'grad_norm': '1.517', 'learning_rate': '1.983e-05', 'epoch': '0.1379', 'num_input_tokens_seen': 1658648, 'train_runtime': '1550', 'train_tokens_per_second': '1070'}
+
+ 14%|█▍        | 250/1813 [25:50<2:40:23,  6.16s/it]
+ 14%|█▍        | 251/1813 [25:54<2:29:04,  5.73s/it]
+ 14%|█▍        | 252/1813 [26:00<2:25:51,  5.61s/it]
+ 14%|█▍        | 253/1813 [26:06<2:30:16,  5.78s/it]
+ 14%|█▍        | 254/1813 [26:13<2:41:54,  6.23s/it]
+ 14%|█▍        | 255/1813 [26:20<2:44:53,  6.35s/it]
+ 14%|█▍        | 256/1813 [26:25<2:39:19,  6.14s/it]
+ 14%|█▍        | 257/1813 [26:32<2:43:54,  6.32s/it]
+ 14%|█▍        | 258/1813 [26:39<2:46:33,  6.43s/it]
+ 14%|█▍        | 259/1813 [26:43<2:31:31,  5.85s/it]
+ 14%|█▍        | 260/1813 [26:49<2:34:04,  5.95s/it]
+                                                    
+{'loss': '0.9606', 'grad_norm': '2.42', 'learning_rate': '1.979e-05', 'epoch': '0.1434', 'num_input_tokens_seen': 1722792, 'train_runtime': '1610', 'train_tokens_per_second': '1070'}
+
+ 14%|█▍        | 260/1813 [26:49<2:34:04,  5.95s/it]
+ 14%|█▍        | 261/1813 [26:57<2:44:36,  6.36s/it]
+ 14%|█▍        | 262/1813 [27:04<2:47:24,  6.48s/it]
+ 15%|█▍        | 263/1813 [27:10<2:48:46,  6.53s/it]
+ 15%|█▍        | 264/1813 [27:16<2:46:24,  6.45s/it]
+ 15%|█▍        | 265/1813 [27:22<2:42:55,  6.32s/it]
+ 15%|█▍        | 266/1813 [27:27<2:29:51,  5.81s/it]
+ 15%|█▍        | 267/1813 [27:32<2:26:04,  5.67s/it]
+ 15%|█▍        | 268/1813 [27:37<2:21:24,  5.49s/it]
+ 15%|█▍        | 269/1813 [27:44<2:26:33,  5.70s/it]
+ 15%|█▍        | 270/1813 [27:50<2:29:22,  5.81s/it]
+                                                    
+{'loss': '0.9791', 'grad_norm': '1.702', 'learning_rate': '1.975e-05', 'epoch': '0.149', 'num_input_tokens_seen': 1786808, 'train_runtime': '1670', 'train_tokens_per_second': '1070'}
+
+ 15%|█▍        | 270/1813 [27:50<2:29:22,  5.81s/it]
+ 15%|█▍        | 271/1813 [27:58<2:46:20,  6.47s/it]
+ 15%|█▌        | 272/1813 [28:03<2:38:43,  6.18s/it]
+ 15%|█▌        | 273/1813 [28:12<2:55:32,  6.84s/it]
+ 15%|█▌        | 274/1813 [28:18<2:55:18,  6.83s/it]
+ 15%|█▌        | 275/1813 [28:24<2:43:28,  6.38s/it]
+ 15%|█▌        | 276/1813 [28:29<2:35:12,  6.06s/it]
+ 15%|█▌        | 277/1813 [28:35<2:37:38,  6.16s/it]
+ 15%|█▌        | 278/1813 [28:42<2:38:15,  6.19s/it]
+ 15%|█▌        | 279/1813 [28:49<2:49:57,  6.65s/it]
+ 15%|█▌        | 280/1813 [28:56<2:49:33,  6.64s/it]
+                                                    
+{'loss': '1.057', 'grad_norm': '1.626', 'learning_rate': '1.97e-05', 'epoch': '0.1545', 'num_input_tokens_seen': 1858112, 'train_runtime': '1737', 'train_tokens_per_second': '1070'}
+
+ 15%|█▌        | 280/1813 [28:56<2:49:33,  6.64s/it]
+ 15%|█▌        | 281/1813 [29:02<2:41:48,  6.34s/it]
+ 16%|█▌        | 282/1813 [29:07<2:33:51,  6.03s/it]
+ 16%|█▌        | 283/1813 [29:12<2:26:26,  5.74s/it]
+ 16%|█▌        | 284/1813 [29:18<2:24:41,  5.68s/it]
+ 16%|█▌        | 285/1813 [29:24<2:31:54,  5.96s/it]
+ 16%|█▌        | 286/1813 [29:30<2:31:27,  5.95s/it]
+ 16%|█▌        | 287/1813 [29:36<2:32:44,  6.01s/it]
+ 16%|█▌        | 288/1813 [29:42<2:27:34,  5.81s/it]
+ 16%|█▌        | 289/1813 [29:48<2:31:46,  5.98s/it]
+ 16%|█▌        | 290/1813 [29:55<2:37:21,  6.20s/it]
+                                                    
+{'loss': '1.031', 'grad_norm': '1.415', 'learning_rate': '1.966e-05', 'epoch': '0.16', 'num_input_tokens_seen': 1920600, 'train_runtime': '1795', 'train_tokens_per_second': '1070'}
+
+ 16%|█▌        | 290/1813 [29:55<2:37:21,  6.20s/it]
+ 16%|█▌        | 291/1813 [30:02<2:44:23,  6.48s/it]
+ 16%|█▌        | 292/1813 [30:09<2:50:39,  6.73s/it]
+ 16%|█▌        | 293/1813 [30:15<2:44:36,  6.50s/it]
+ 16%|█▌        | 294/1813 [30:22<2:50:55,  6.75s/it]
+ 16%|█▋        | 295/1813 [30:30<2:55:50,  6.95s/it]
+ 16%|█▋        | 296/1813 [30:36<2:46:57,  6.60s/it]
+ 16%|█▋        | 297/1813 [30:42<2:44:23,  6.51s/it]
+ 16%|█▋        | 298/1813 [30:47<2:33:16,  6.07s/it]
+ 16%|█▋        | 299/1813 [30:53<2:31:13,  5.99s/it]
+ 17%|█▋        | 300/1813 [30:58<2:28:43,  5.90s/it]
+                                                    
+{'loss': '1.088', 'grad_norm': '1.404', 'learning_rate': '1.961e-05', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312, 'train_runtime': '1859', 'train_tokens_per_second': '1070'}
+
+ 17%|█▋        | 300/1813 [30:58<2:28:43,  5.90s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.77it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.09it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.01it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.83it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.34it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.92it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.28it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:35,  9.76it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.20it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.96it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.76it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.83it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.32it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.45it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.21it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.04it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.52it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.81it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.23it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.09it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.81it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:26,  9.97it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:30,  9.54it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.84it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.92it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  8.99it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.86it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25, 10.05it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.41it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.04it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.41it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.98it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:38,  8.56it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.78it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.48it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.66it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.43it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.56it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.74it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.70it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.73it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.89it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.67it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.79it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.59it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.92it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.51it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.68it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.51it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.36it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.20it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.96it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.48it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.99it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.36it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.88it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.35it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.68it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.75it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.79it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.80it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.21it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.28it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.24it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.57it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.42it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.18it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.57it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.80it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.01it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.86it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.73it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.21it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.56it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.97it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.61it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.44it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.20it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.29it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.93it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:30,  6.91it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.76it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.21it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.19it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.65it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.38it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.80it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.24it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.37it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.07it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.26it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.44it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.94it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.56it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.03it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.80it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.50it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.31it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  6.98it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:29,  6.16it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.55it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.14it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.47it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.79it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.82it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.31it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.44it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.97it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.97it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.38it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.55it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.13it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:47,  9.74it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.77it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.85it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.05it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.36it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.84it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.76it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.74it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.82it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.64it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.10it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.44it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:40,  8.89it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.83it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.51it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.06it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.54it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.50it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.55it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.69it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.90it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.57it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.64it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.95it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.91it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.87it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.23it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.04it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.71it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.34it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.05it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.30it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.11it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.60it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.53it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.88it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.52it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.61it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.67it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.15it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.59it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:33,  7.30it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.70it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.46it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.13it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.38it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  8.99it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.85it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.16it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.96it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.20it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.23it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.30it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.01it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.09it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.00it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.77it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.72it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.97it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.72it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.00it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.03it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.52it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.10it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.44it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.56it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.05it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.78it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.65it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.72it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.31it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.26it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.28it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.80it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.95it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  8.87it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:11, 10.09it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.62it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.64it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:14,  7.55it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.79it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.52it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.69it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.84it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.04it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.32it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.76it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.03it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.32it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.58it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.69it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.53it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.88it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.12it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.61it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.06it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.64it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.32it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.85it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.42it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.44it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.23it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  7.94it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.13it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.75it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.77it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.40it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.05it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.43it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.91it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.35it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.27it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.39it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  6.91it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.69it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.59it/s]
+                                                    
+
+                                                   
+{'eval_loss': '0.907', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.38', 'eval_steps_per_second': '7.38', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312}
+
+ 17%|█▋        | 300/1813 [33:14<2:28:43,  5.90s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.59it/s]
+
+                                                   
+ 17%|█▋        | 301/1813 [33:19<19:27:06, 46.31s/it]
+ 17%|█▋        | 302/1813 [33:24<14:10:48, 33.78s/it]
+ 17%|█▋        | 303/1813 [33:30<10:46:32, 25.69s/it]
+ 17%|█▋        | 304/1813 [33:38<8:28:41, 20.23s/it] 
+ 17%|█▋        | 305/1813 [33:44<6:42:38, 16.02s/it]
+ 17%|█▋        | 306/1813 [33:50<5:25:24, 12.96s/it]
+ 17%|█▋        | 307/1813 [33:55<4:28:26, 10.69s/it]
+ 17%|█▋        | 308/1813 [34:01<3:48:53,  9.13s/it]
+ 17%|█▋        | 309/1813 [34:07<3:27:55,  8.30s/it]
+ 17%|█▋        | 310/1813 [34:12<3:02:02,  7.27s/it]
+                                                    
+{'loss': '1.005', 'grad_norm': '2.029', 'learning_rate': '1.955e-05', 'epoch': '0.171', 'num_input_tokens_seen': 2050672, 'train_runtime': '2053', 'train_tokens_per_second': '999.1'}
+
+ 17%|█▋        | 310/1813 [34:12<3:02:02,  7.27s/it]
+ 17%|█▋        | 311/1813 [34:19<2:59:34,  7.17s/it]
+ 17%|█▋        | 312/1813 [34:26<3:00:13,  7.20s/it]
+ 17%|█▋        | 313/1813 [34:33<2:56:20,  7.05s/it]
+ 17%|█▋        | 314/1813 [34:38<2:44:11,  6.57s/it]
+ 17%|█▋        | 315/1813 [34:45<2:46:05,  6.65s/it]
+ 17%|█▋        | 316/1813 [34:52<2:43:03,  6.54s/it]
+ 17%|█▋        | 317/1813 [34:57<2:36:48,  6.29s/it]
+ 18%|█▊        | 318/1813 [35:03<2:29:28,  6.00s/it]
+ 18%|█▊        | 319/1813 [35:07<2:19:46,  5.61s/it]
+ 18%|█▊        | 320/1813 [35:13<2:20:55,  5.66s/it]
+                                                    
+{'loss': '1.086', 'grad_norm': '1.108', 'learning_rate': '1.949e-05', 'epoch': '0.1766', 'num_input_tokens_seen': 2118000, 'train_runtime': '2114', 'train_tokens_per_second': '1002'}
+
+ 18%|█▊        | 320/1813 [35:13<2:20:55,  5.66s/it]
+ 18%|█▊        | 321/1813 [35:19<2:21:53,  5.71s/it]
+ 18%|█▊        | 322/1813 [35:25<2:22:05,  5.72s/it]
+ 18%|█▊        | 323/1813 [35:31<2:25:40,  5.87s/it]
+ 18%|█▊        | 324/1813 [35:38<2:33:33,  6.19s/it]
+ 18%|█▊        | 325/1813 [35:45<2:38:42,  6.40s/it]
+ 18%|█▊        | 326/1813 [35:51<2:35:47,  6.29s/it]
+ 18%|█▊        | 327/1813 [35:56<2:31:19,  6.11s/it]
+ 18%|█▊        | 328/1813 [36:03<2:33:27,  6.20s/it]
+ 18%|█▊        | 329/1813 [36:08<2:29:22,  6.04s/it]
+ 18%|█▊        | 330/1813 [36:14<2:26:53,  5.94s/it]
+                                                    
+{'loss': '1.003', 'grad_norm': '1.572', 'learning_rate': '1.943e-05', 'epoch': '0.1821', 'num_input_tokens_seen': 2184640, 'train_runtime': '2175', 'train_tokens_per_second': '1005'}
+
+ 18%|█▊        | 330/1813 [36:14<2:26:53,  5.94s/it]
+ 18%|█▊        | 331/1813 [36:20<2:28:37,  6.02s/it]
+ 18%|█▊        | 332/1813 [36:25<2:21:28,  5.73s/it]
+ 18%|█▊        | 333/1813 [36:31<2:23:12,  5.81s/it]
+ 18%|█▊        | 334/1813 [36:37<2:22:46,  5.79s/it]
+ 18%|█▊        | 335/1813 [36:45<2:37:24,  6.39s/it]
+ 19%|█▊        | 336/1813 [36:52<2:42:13,  6.59s/it]
+ 19%|█▊        | 337/1813 [36:58<2:36:39,  6.37s/it]
+ 19%|█▊        | 338/1813 [37:04<2:35:47,  6.34s/it]
+ 19%|█▊        | 339/1813 [37:10<2:31:03,  6.15s/it]
+ 19%|█▉        | 340/1813 [37:17<2:38:21,  6.45s/it]
+                                                    
+{'loss': '0.99', 'grad_norm': '1.818', 'learning_rate': '1.937e-05', 'epoch': '0.1876', 'num_input_tokens_seen': 2251392, 'train_runtime': '2238', 'train_tokens_per_second': '1006'}
+
+ 19%|█▉        | 340/1813 [37:17<2:38:21,  6.45s/it]
+ 19%|█▉        | 341/1813 [37:24<2:38:39,  6.47s/it]
+ 19%|█▉        | 342/1813 [37:31<2:46:17,  6.78s/it]
+ 19%|█▉        | 343/1813 [37:37<2:39:21,  6.50s/it]
+ 19%|█▉        | 344/1813 [37:43<2:34:26,  6.31s/it]
+ 19%|█▉        | 345/1813 [37:50<2:42:02,  6.62s/it]
+ 19%|█▉        | 346/1813 [37:57<2:40:47,  6.58s/it]
+ 19%|█▉        | 347/1813 [38:02<2:29:17,  6.11s/it]
+ 19%|█▉        | 348/1813 [38:08<2:31:54,  6.22s/it]
+ 19%|█▉        | 349/1813 [38:16<2:47:35,  6.87s/it]
+ 19%|█▉        | 350/1813 [38:24<2:53:21,  7.11s/it]
+                                                    
+{'loss': '1.095', 'grad_norm': '1.301', 'learning_rate': '1.93e-05', 'epoch': '0.1931', 'num_input_tokens_seen': 2324712, 'train_runtime': '2305', 'train_tokens_per_second': '1009'}
+
+ 19%|█▉        | 350/1813 [38:24<2:53:21,  7.11s/it]
+ 19%|█▉        | 351/1813 [38:31<2:51:10,  7.03s/it]
+ 19%|█▉        | 352/1813 [38:37<2:42:03,  6.66s/it]
+ 19%|█▉        | 353/1813 [38:42<2:33:03,  6.29s/it]
+ 20%|█▉        | 354/1813 [38:48<2:27:31,  6.07s/it]
+ 20%|█▉        | 355/1813 [38:54<2:26:03,  6.01s/it]
+ 20%|█▉        | 356/1813 [39:00<2:32:21,  6.27s/it]
+ 20%|█▉        | 357/1813 [39:06<2:30:02,  6.18s/it]
+ 20%|█▉        | 358/1813 [39:13<2:32:07,  6.27s/it]
+ 20%|█▉        | 359/1813 [39:19<2:27:31,  6.09s/it]
+ 20%|█▉        | 360/1813 [39:26<2:33:57,  6.36s/it]
+                                                    
+{'loss': '1.044', 'grad_norm': '1.511', 'learning_rate': '1.923e-05', 'epoch': '0.1986', 'num_input_tokens_seen': 2390776, 'train_runtime': '2366', 'train_tokens_per_second': '1010'}
+
+ 20%|█▉        | 360/1813 [39:26<2:33:57,  6.36s/it]
+ 20%|█▉        | 361/1813 [39:31<2:26:00,  6.03s/it]
+ 20%|█▉        | 362/1813 [39:37<2:23:48,  5.95s/it]
+ 20%|██        | 363/1813 [39:44<2:30:43,  6.24s/it]
+ 20%|██        | 364/1813 [39:50<2:33:08,  6.34s/it]
+ 20%|██        | 365/1813 [39:56<2:29:45,  6.21s/it]
+ 20%|██        | 366/1813 [40:03<2:35:19,  6.44s/it]
+ 20%|██        | 367/1813 [40:09<2:32:17,  6.32s/it]
+ 20%|██        | 368/1813 [40:15<2:27:24,  6.12s/it]
+ 20%|██        | 369/1813 [40:20<2:18:49,  5.77s/it]
+ 20%|██        | 370/1813 [40:26<2:20:40,  5.85s/it]
+                                                    
+{'loss': '1.029', 'grad_norm': '2.127', 'learning_rate': '1.916e-05', 'epoch': '0.2041', 'num_input_tokens_seen': 2454808, 'train_runtime': '2426', 'train_tokens_per_second': '1012'}
+
+ 20%|██        | 370/1813 [40:26<2:20:40,  5.85s/it]
+ 20%|██        | 371/1813 [40:31<2:16:43,  5.69s/it]
+ 21%|██        | 372/1813 [40:39<2:35:59,  6.50s/it]
+ 21%|██        | 373/1813 [40:46<2:34:05,  6.42s/it]
+ 21%|██        | 374/1813 [40:53<2:41:00,  6.71s/it]
+ 21%|██        | 375/1813 [41:00<2:42:01,  6.76s/it]
+ 21%|██        | 376/1813 [41:05<2:33:16,  6.40s/it]
+ 21%|██        | 377/1813 [41:12<2:31:33,  6.33s/it]
+ 21%|██        | 378/1813 [41:18<2:30:42,  6.30s/it]
+ 21%|██        | 379/1813 [41:23<2:23:22,  6.00s/it]
+ 21%|██        | 380/1813 [41:28<2:18:24,  5.79s/it]
+                                                    
+{'loss': '1.005', 'grad_norm': '2.369', 'learning_rate': '1.908e-05', 'epoch': '0.2097', 'num_input_tokens_seen': 2520472, 'train_runtime': '2489', 'train_tokens_per_second': '1013'}
+
+ 21%|██        | 380/1813 [41:28<2:18:24,  5.79s/it]
+ 21%|██        | 381/1813 [41:36<2:27:39,  6.19s/it]
+ 21%|██        | 382/1813 [41:42<2:33:05,  6.42s/it]
+ 21%|██        | 383/1813 [41:49<2:31:30,  6.36s/it]
+ 21%|██        | 384/1813 [41:55<2:30:12,  6.31s/it]
+ 21%|██        | 385/1813 [42:01<2:30:33,  6.33s/it]
+ 21%|██▏       | 386/1813 [42:06<2:18:03,  5.80s/it]
+ 21%|██▏       | 387/1813 [42:12<2:19:37,  5.87s/it]
+ 21%|██▏       | 388/1813 [42:19<2:28:43,  6.26s/it]
+ 21%|██▏       | 389/1813 [42:27<2:38:46,  6.69s/it]
+ 22%|██▏       | 390/1813 [42:33<2:38:22,  6.68s/it]
+                                                    
+{'loss': '1.04', 'grad_norm': '1.429', 'learning_rate': '1.9e-05', 'epoch': '0.2152', 'num_input_tokens_seen': 2591512, 'train_runtime': '2554', 'train_tokens_per_second': '1015'}
+
+ 22%|██▏       | 390/1813 [42:33<2:38:22,  6.68s/it]
+ 22%|██▏       | 391/1813 [42:40<2:35:26,  6.56s/it]
+ 22%|██▏       | 392/1813 [42:46<2:35:31,  6.57s/it]
+ 22%|██▏       | 393/1813 [42:53<2:37:12,  6.64s/it]
+ 22%|██▏       | 394/1813 [42:58<2:26:22,  6.19s/it]
+ 22%|██▏       | 395/1813 [43:04<2:22:27,  6.03s/it]
+ 22%|██▏       | 396/1813 [43:10<2:24:12,  6.11s/it]
+ 22%|██▏       | 397/1813 [43:15<2:16:31,  5.79s/it]
+ 22%|██▏       | 398/1813 [43:22<2:21:27,  6.00s/it]
+ 22%|██▏       | 399/1813 [43:29<2:33:35,  6.52s/it]
+ 22%|██▏       | 400/1813 [43:38<2:45:28,  7.03s/it]
+                                                    
+{'loss': '1.023', 'grad_norm': '1.554', 'learning_rate': '1.891e-05', 'epoch': '0.2207', 'num_input_tokens_seen': 2660336, 'train_runtime': '2618', 'train_tokens_per_second': '1016'}
+
+ 22%|██▏       | 400/1813 [43:38<2:45:28,  7.03s/it]
+ 22%|██▏       | 401/1813 [43:45<2:46:30,  7.08s/it]
+ 22%|██▏       | 402/1813 [43:50<2:30:06,  6.38s/it]
+ 22%|██▏       | 403/1813 [43:56<2:27:51,  6.29s/it]
+ 22%|██▏       | 404/1813 [44:02<2:31:19,  6.44s/it]
+ 22%|██▏       | 405/1813 [44:10<2:35:25,  6.62s/it]
+ 22%|██▏       | 406/1813 [44:14<2:22:29,  6.08s/it]
+ 22%|██▏       | 407/1813 [44:19<2:10:03,  5.55s/it]
+ 23%|██▎       | 408/1813 [44:24<2:11:24,  5.61s/it]
+ 23%|██▎       | 409/1813 [44:30<2:13:56,  5.72s/it]
+ 23%|██▎       | 410/1813 [44:36<2:13:12,  5.70s/it]
+                                                    
+{'loss': '1.043', 'grad_norm': '2.289', 'learning_rate': '1.883e-05', 'epoch': '0.2262', 'num_input_tokens_seen': 2722056, 'train_runtime': '2677', 'train_tokens_per_second': '1017'}
+
+ 23%|██▎       | 410/1813 [44:36<2:13:12,  5.70s/it]
+ 23%|██▎       | 411/1813 [44:42<2:17:32,  5.89s/it]
+ 23%|██▎       | 412/1813 [44:49<2:22:27,  6.10s/it]
+ 23%|██▎       | 413/1813 [44:56<2:30:44,  6.46s/it]
+ 23%|██▎       | 414/1813 [45:02<2:24:16,  6.19s/it]
+ 23%|██▎       | 415/1813 [45:08<2:25:35,  6.25s/it]
+ 23%|██▎       | 416/1813 [45:15<2:27:42,  6.34s/it]
+ 23%|██▎       | 417/1813 [45:22<2:36:48,  6.74s/it]
+ 23%|██▎       | 418/1813 [45:28<2:30:38,  6.48s/it]
+ 23%|██▎       | 419/1813 [45:35<2:31:43,  6.53s/it]
+ 23%|██▎       | 420/1813 [45:41<2:26:51,  6.33s/it]
+                                                    
+{'loss': '1.032', 'grad_norm': '1.158', 'learning_rate': '1.874e-05', 'epoch': '0.2317', 'num_input_tokens_seen': 2790960, 'train_runtime': '2741', 'train_tokens_per_second': '1018'}
+
+ 23%|██▎       | 420/1813 [45:41<2:26:51,  6.33s/it]
+ 23%|██▎       | 421/1813 [45:47<2:27:07,  6.34s/it]
+ 23%|██▎       | 422/1813 [45:55<2:34:11,  6.65s/it]
+ 23%|██▎       | 423/1813 [46:01<2:30:20,  6.49s/it]
+ 23%|██▎       | 424/1813 [46:07<2:28:25,  6.41s/it]
+ 23%|██▎       | 425/1813 [46:13<2:23:35,  6.21s/it]
+ 23%|██▎       | 426/1813 [46:20<2:31:18,  6.55s/it]
+ 24%|██▎       | 427/1813 [46:26<2:26:47,  6.35s/it]
+ 24%|██▎       | 428/1813 [46:32<2:24:42,  6.27s/it]
+ 24%|██▎       | 429/1813 [46:37<2:17:20,  5.95s/it]
+ 24%|██▎       | 430/1813 [46:45<2:27:45,  6.41s/it]
+                                                    
+{'loss': '1.018', 'grad_norm': '1.421', 'learning_rate': '1.864e-05', 'epoch': '0.2372', 'num_input_tokens_seen': 2860096, 'train_runtime': '2805', 'train_tokens_per_second': '1020'}
+
+ 24%|██▎       | 430/1813 [46:45<2:27:45,  6.41s/it]
+ 24%|██▍       | 431/1813 [46:50<2:23:45,  6.24s/it]
+ 24%|██▍       | 432/1813 [46:56<2:19:52,  6.08s/it]
+ 24%|██▍       | 433/1813 [47:01<2:14:42,  5.86s/it]
+ 24%|██▍       | 434/1813 [47:08<2:18:54,  6.04s/it]
+ 24%|██▍       | 435/1813 [47:14<2:18:28,  6.03s/it]
+ 24%|██▍       | 436/1813 [47:21<2:22:54,  6.23s/it]
+ 24%|██▍       | 437/1813 [47:27<2:20:30,  6.13s/it]
+ 24%|██▍       | 438/1813 [47:33<2:23:02,  6.24s/it]
+ 24%|██▍       | 439/1813 [47:39<2:22:19,  6.22s/it]
+ 24%|██▍       | 440/1813 [47:46<2:23:56,  6.29s/it]
+                                                    
+{'loss': '0.9504', 'grad_norm': '2.44', 'learning_rate': '1.855e-05', 'epoch': '0.2428', 'num_input_tokens_seen': 2924856, 'train_runtime': '2866', 'train_tokens_per_second': '1020'}
+
+ 24%|██▍       | 440/1813 [47:46<2:23:56,  6.29s/it]
+ 24%|██▍       | 441/1813 [47:52<2:22:21,  6.23s/it]
+ 24%|██▍       | 442/1813 [47:58<2:21:45,  6.20s/it]
+ 24%|██▍       | 443/1813 [48:05<2:26:09,  6.40s/it]
+ 24%|██▍       | 444/1813 [48:10<2:21:01,  6.18s/it]
+ 25%|██▍       | 445/1813 [48:18<2:28:54,  6.53s/it]
+ 25%|██▍       | 446/1813 [48:24<2:27:42,  6.48s/it]
+ 25%|██▍       | 447/1813 [48:30<2:25:28,  6.39s/it]
+ 25%|██▍       | 448/1813 [48:37<2:25:21,  6.39s/it]
+ 25%|██▍       | 449/1813 [48:42<2:19:14,  6.13s/it]
+ 25%|██▍       | 450/1813 [48:48<2:15:59,  5.99s/it]
+                                                    
+{'loss': '1.071', 'grad_norm': '1.767', 'learning_rate': '1.845e-05', 'epoch': '0.2483', 'num_input_tokens_seen': 2990040, 'train_runtime': '2928', 'train_tokens_per_second': '1021'}
+
+ 25%|██▍       | 450/1813 [48:48<2:15:59,  5.99s/it]
+ 25%|██▍       | 451/1813 [48:55<2:26:24,  6.45s/it]
+ 25%|██▍       | 452/1813 [49:01<2:21:28,  6.24s/it]
+ 25%|██▍       | 453/1813 [49:06<2:13:06,  5.87s/it]
+ 25%|██▌       | 454/1813 [49:13<2:20:18,  6.19s/it]
+ 25%|██▌       | 455/1813 [49:20<2:22:32,  6.30s/it]
+ 25%|██▌       | 456/1813 [49:24<2:11:27,  5.81s/it]
+ 25%|██▌       | 457/1813 [49:31<2:15:13,  5.98s/it]
+ 25%|██▌       | 458/1813 [49:39<2:29:17,  6.61s/it]
+ 25%|██▌       | 459/1813 [49:46<2:31:29,  6.71s/it]
+ 25%|██▌       | 460/1813 [49:52<2:29:04,  6.61s/it]
+                                                    
+{'loss': '1.123', 'grad_norm': '1.4', 'learning_rate': '1.834e-05', 'epoch': '0.2538', 'num_input_tokens_seen': 3058848, 'train_runtime': '2993', 'train_tokens_per_second': '1022'}
+
+ 25%|██▌       | 460/1813 [49:52<2:29:04,  6.61s/it]
+ 25%|██▌       | 461/1813 [49:58<2:24:30,  6.41s/it]
+ 25%|██▌       | 462/1813 [50:06<2:31:53,  6.75s/it]
+ 26%|██▌       | 463/1813 [50:11<2:25:57,  6.49s/it]
+ 26%|██▌       | 464/1813 [50:17<2:16:41,  6.08s/it]
+ 26%|██▌       | 465/1813 [50:22<2:13:13,  5.93s/it]
+ 26%|██▌       | 466/1813 [50:29<2:17:33,  6.13s/it]
+ 26%|██▌       | 467/1813 [50:35<2:14:48,  6.01s/it]
+ 26%|██▌       | 468/1813 [50:41<2:17:58,  6.15s/it]
+ 26%|██▌       | 469/1813 [50:47<2:19:49,  6.24s/it]
+ 26%|██▌       | 470/1813 [50:54<2:21:44,  6.33s/it]
+                                                    
+{'loss': '1.005', 'grad_norm': '1.333', 'learning_rate': '1.824e-05', 'epoch': '0.2593', 'num_input_tokens_seen': 3126680, 'train_runtime': '3054', 'train_tokens_per_second': '1024'}
+
+ 26%|██▌       | 470/1813 [50:54<2:21:44,  6.33s/it]
+ 26%|██▌       | 471/1813 [51:00<2:16:17,  6.09s/it]
+ 26%|██▌       | 472/1813 [51:06<2:19:15,  6.23s/it]
+ 26%|██▌       | 473/1813 [51:13<2:22:02,  6.36s/it]
+ 26%|██▌       | 474/1813 [51:19<2:19:18,  6.24s/it]
+ 26%|██▌       | 475/1813 [51:24<2:11:56,  5.92s/it]
+ 26%|██▋       | 476/1813 [51:30<2:12:15,  5.94s/it]
+ 26%|██▋       | 477/1813 [51:36<2:16:10,  6.12s/it]
+ 26%|██▋       | 478/1813 [51:43<2:21:32,  6.36s/it]
+ 26%|██▋       | 479/1813 [51:49<2:16:15,  6.13s/it]
+ 26%|██▋       | 480/1813 [51:55<2:18:09,  6.22s/it]
+                                                    
+{'loss': '1.101', 'grad_norm': '1.691', 'learning_rate': '1.813e-05', 'epoch': '0.2648', 'num_input_tokens_seen': 3193704, 'train_runtime': '3116', 'train_tokens_per_second': '1025'}
+
+ 26%|██▋       | 480/1813 [51:55<2:18:09,  6.22s/it]
+ 27%|██▋       | 481/1813 [52:02<2:19:21,  6.28s/it]
+ 27%|██▋       | 482/1813 [52:09<2:23:35,  6.47s/it]
+ 27%|██▋       | 483/1813 [52:17<2:32:30,  6.88s/it]
+ 27%|██▋       | 484/1813 [52:22<2:25:39,  6.58s/it]
+ 27%|██▋       | 485/1813 [52:30<2:33:54,  6.95s/it]
+ 27%|██▋       | 486/1813 [52:37<2:30:53,  6.82s/it]
+ 27%|██▋       | 487/1813 [52:44<2:31:50,  6.87s/it]
+ 27%|██▋       | 488/1813 [52:50<2:26:11,  6.62s/it]
+ 27%|██▋       | 489/1813 [52:57<2:30:39,  6.83s/it]
+ 27%|██▋       | 490/1813 [53:03<2:27:00,  6.67s/it]
+                                                    
+{'loss': '1.045', 'grad_norm': '1.845', 'learning_rate': '1.802e-05', 'epoch': '0.2703', 'num_input_tokens_seen': 3268064, 'train_runtime': '3184', 'train_tokens_per_second': '1026'}
+
+ 27%|██▋       | 490/1813 [53:03<2:27:00,  6.67s/it]
+ 27%|██▋       | 491/1813 [53:10<2:27:11,  6.68s/it]
+ 27%|██▋       | 492/1813 [53:18<2:32:44,  6.94s/it]
+ 27%|██▋       | 493/1813 [53:23<2:22:09,  6.46s/it]
+ 27%|██▋       | 494/1813 [53:29<2:16:11,  6.20s/it]
+ 27%|██▋       | 495/1813 [53:36<2:21:38,  6.45s/it]
+ 27%|██▋       | 496/1813 [53:42<2:20:09,  6.39s/it]
+ 27%|██▋       | 497/1813 [53:47<2:13:08,  6.07s/it]
+ 27%|██▋       | 498/1813 [53:54<2:19:50,  6.38s/it]
+ 28%|██▊       | 499/1813 [54:00<2:17:06,  6.26s/it]
+ 28%|██▊       | 500/1813 [54:09<2:35:26,  7.10s/it]
+                                                    
+{'loss': '1.139', 'grad_norm': '1.167', 'learning_rate': '1.79e-05', 'epoch': '0.2759', 'num_input_tokens_seen': 3341536, 'train_runtime': '3250', 'train_tokens_per_second': '1028'}
+
+ 28%|██▊       | 500/1813 [54:09<2:35:26,  7.10s/it]
+ 28%|██▊       | 501/1813 [54:16<2:29:35,  6.84s/it]
+ 28%|██▊       | 502/1813 [54:21<2:22:37,  6.53s/it]
+ 28%|██▊       | 503/1813 [54:26<2:12:00,  6.05s/it]
+ 28%|██▊       | 504/1813 [54:32<2:11:42,  6.04s/it]
+ 28%|██▊       | 505/1813 [54:39<2:18:09,  6.34s/it]
+ 28%|██▊       | 506/1813 [54:45<2:15:38,  6.23s/it]
+ 28%|██▊       | 507/1813 [54:53<2:22:31,  6.55s/it]
+ 28%|██▊       | 508/1813 [54:57<2:11:41,  6.05s/it]
+ 28%|██▊       | 509/1813 [55:02<2:04:19,  5.72s/it]
+ 28%|██▊       | 510/1813 [55:08<2:00:55,  5.57s/it]
+                                                    
+{'loss': '0.9646', 'grad_norm': '2.307', 'learning_rate': '1.779e-05', 'epoch': '0.2814', 'num_input_tokens_seen': 3402712, 'train_runtime': '3308', 'train_tokens_per_second': '1029'}
+
+ 28%|██▊       | 510/1813 [55:08<2:00:55,  5.57s/it]
+ 28%|██▊       | 511/1813 [55:13<1:59:36,  5.51s/it]
+ 28%|██▊       | 512/1813 [55:20<2:08:04,  5.91s/it]
+ 28%|██▊       | 513/1813 [55:27<2:15:46,  6.27s/it]
+ 28%|██▊       | 514/1813 [55:34<2:19:06,  6.43s/it]
+ 28%|██▊       | 515/1813 [55:39<2:11:34,  6.08s/it]
+ 28%|██▊       | 516/1813 [55:45<2:09:53,  6.01s/it]
+ 29%|██▊       | 517/1813 [55:52<2:14:03,  6.21s/it]
+ 29%|██▊       | 518/1813 [55:58<2:14:03,  6.21s/it]
+ 29%|██▊       | 519/1813 [56:04<2:14:54,  6.26s/it]
+ 29%|██▊       | 520/1813 [56:11<2:17:44,  6.39s/it]
+                                                    
+{'loss': '1.007', 'grad_norm': '1.624', 'learning_rate': '1.767e-05', 'epoch': '0.2869', 'num_input_tokens_seen': 3469624, 'train_runtime': '3371', 'train_tokens_per_second': '1029'}
+
+ 29%|██▊       | 520/1813 [56:11<2:17:44,  6.39s/it]
+ 29%|██▊       | 521/1813 [56:15<2:03:40,  5.74s/it]
+ 29%|██▉       | 522/1813 [56:21<2:04:34,  5.79s/it]
+ 29%|██▉       | 523/1813 [56:27<2:06:25,  5.88s/it]
+ 29%|██▉       | 524/1813 [56:33<2:08:58,  6.00s/it]
+ 29%|██▉       | 525/1813 [56:38<2:02:24,  5.70s/it]
+ 29%|██▉       | 526/1813 [56:45<2:09:02,  6.02s/it]
+ 29%|██▉       | 527/1813 [56:52<2:12:53,  6.20s/it]
+ 29%|██▉       | 528/1813 [56:59<2:17:45,  6.43s/it]
+ 29%|██▉       | 529/1813 [57:06<2:21:18,  6.60s/it]
+ 29%|██▉       | 530/1813 [57:11<2:15:58,  6.36s/it]
+                                                    
+{'loss': '0.9814', 'grad_norm': '2.366', 'learning_rate': '1.754e-05', 'epoch': '0.2924', 'num_input_tokens_seen': 3534904, 'train_runtime': '3432', 'train_tokens_per_second': '1030'}
+
+ 29%|██▉       | 530/1813 [57:11<2:15:58,  6.36s/it]
+ 29%|██▉       | 531/1813 [57:20<2:27:35,  6.91s/it]
+ 29%|██▉       | 532/1813 [57:26<2:23:16,  6.71s/it]
+ 29%|██▉       | 533/1813 [57:33<2:25:36,  6.83s/it]
+ 29%|██▉       | 534/1813 [57:40<2:28:01,  6.94s/it]
+ 30%|██▉       | 535/1813 [57:45<2:11:27,  6.17s/it]
+ 30%|██▉       | 536/1813 [57:50<2:04:33,  5.85s/it]
+ 30%|██▉       | 537/1813 [57:56<2:10:03,  6.12s/it]
+ 30%|██▉       | 538/1813 [58:02<2:06:40,  5.96s/it]
+ 30%|██▉       | 539/1813 [58:08<2:08:41,  6.06s/it]
+ 30%|██▉       | 540/1813 [58:14<2:08:24,  6.05s/it]
+                                                    
+{'loss': '0.9969', 'grad_norm': '1.442', 'learning_rate': '1.742e-05', 'epoch': '0.2979', 'num_input_tokens_seen': 3603488, 'train_runtime': '3495', 'train_tokens_per_second': '1031'}
+
+ 30%|██▉       | 540/1813 [58:14<2:08:24,  6.05s/it]
+ 30%|██▉       | 541/1813 [58:19<2:02:08,  5.76s/it]
+ 30%|██▉       | 542/1813 [58:25<2:00:10,  5.67s/it]
+ 30%|██▉       | 543/1813 [58:32<2:09:23,  6.11s/it]
+ 30%|███       | 544/1813 [58:40<2:23:20,  6.78s/it]
+ 30%|███       | 545/1813 [58:47<2:22:08,  6.73s/it]
+ 30%|███       | 546/1813 [58:53<2:19:53,  6.62s/it]
+ 30%|███       | 547/1813 [59:01<2:25:59,  6.92s/it]
+ 30%|███       | 548/1813 [59:08<2:29:07,  7.07s/it]
+ 30%|███       | 549/1813 [59:14<2:19:11,  6.61s/it]
+ 30%|███       | 550/1813 [59:20<2:13:24,  6.34s/it]
+                                                    
+{'loss': '0.9527', 'grad_norm': '2.147', 'learning_rate': '1.729e-05', 'epoch': '0.3034', 'num_input_tokens_seen': 3676576, 'train_runtime': '3560', 'train_tokens_per_second': '1033'}
+
+ 30%|███       | 550/1813 [59:20<2:13:24,  6.34s/it]
+ 30%|███       | 551/1813 [59:26<2:10:32,  6.21s/it]
+ 30%|███       | 552/1813 [59:31<2:05:09,  5.96s/it]
+ 31%|███       | 553/1813 [59:36<1:58:01,  5.62s/it]
+ 31%|███       | 554/1813 [59:42<2:04:51,  5.95s/it]
+ 31%|███       | 555/1813 [59:47<1:58:12,  5.64s/it]
+ 31%|███       | 556/1813 [59:53<2:00:06,  5.73s/it]
+ 31%|███       | 557/1813 [1:00:00<2:06:32,  6.05s/it]
+ 31%|███       | 558/1813 [1:00:06<2:03:18,  5.89s/it]
+ 31%|███       | 559/1813 [1:00:11<1:57:54,  5.64s/it]
+ 31%|███       | 560/1813 [1:00:17<2:02:52,  5.88s/it]
+                                                      
+{'loss': '1.011', 'grad_norm': '1.074', 'learning_rate': '1.716e-05', 'epoch': '0.309', 'num_input_tokens_seen': 3738056, 'train_runtime': '3618', 'train_tokens_per_second': '1033'}
+
+ 31%|███       | 560/1813 [1:00:17<2:02:52,  5.88s/it]
+ 31%|███       | 561/1813 [1:00:23<2:01:14,  5.81s/it]
+ 31%|███       | 562/1813 [1:00:29<2:05:34,  6.02s/it]
+ 31%|███       | 563/1813 [1:00:36<2:10:35,  6.27s/it]
+ 31%|███       | 564/1813 [1:00:42<2:08:17,  6.16s/it]
+ 31%|███       | 565/1813 [1:00:47<2:03:36,  5.94s/it]
+ 31%|███       | 566/1813 [1:00:54<2:06:25,  6.08s/it]
+ 31%|███▏      | 567/1813 [1:00:59<2:02:53,  5.92s/it]
+ 31%|███▏      | 568/1813 [1:01:06<2:09:16,  6.23s/it]
+ 31%|███▏      | 569/1813 [1:01:14<2:17:42,  6.64s/it]
+ 31%|███▏      | 570/1813 [1:01:21<2:22:51,  6.90s/it]
+                                                      
+{'loss': '1.012', 'grad_norm': '1.21', 'learning_rate': '1.703e-05', 'epoch': '0.3145', 'num_input_tokens_seen': 3808208, 'train_runtime': '3682', 'train_tokens_per_second': '1034'}
+
+ 31%|███▏      | 570/1813 [1:01:21<2:22:51,  6.90s/it]
+ 31%|███▏      | 571/1813 [1:01:28<2:22:45,  6.90s/it]
+ 32%|███▏      | 572/1813 [1:01:34<2:16:51,  6.62s/it]
+ 32%|███▏      | 573/1813 [1:01:39<2:07:42,  6.18s/it]
+ 32%|███▏      | 574/1813 [1:01:45<2:04:19,  6.02s/it]
+ 32%|███▏      | 575/1813 [1:01:52<2:07:31,  6.18s/it]
+ 32%|███▏      | 576/1813 [1:01:56<1:56:53,  5.67s/it]
+ 32%|███▏      | 577/1813 [1:02:03<2:02:27,  5.94s/it]
+ 32%|███▏      | 578/1813 [1:02:09<2:02:43,  5.96s/it]
+ 32%|███▏      | 579/1813 [1:02:15<2:03:19,  6.00s/it]
+ 32%|███▏      | 580/1813 [1:02:21<2:04:13,  6.05s/it]
+                                                      
+{'loss': '1.013', 'grad_norm': '1.458', 'learning_rate': '1.689e-05', 'epoch': '0.32', 'num_input_tokens_seen': 3872328, 'train_runtime': '3741', 'train_tokens_per_second': '1035'}
+
+ 32%|███▏      | 580/1813 [1:02:21<2:04:13,  6.05s/it]
+ 32%|███▏      | 581/1813 [1:02:27<2:01:41,  5.93s/it]
+ 32%|███▏      | 582/1813 [1:02:32<1:57:52,  5.75s/it]
+ 32%|███▏      | 583/1813 [1:02:38<1:57:08,  5.71s/it]
+ 32%|███▏      | 584/1813 [1:02:44<2:00:54,  5.90s/it]
+ 32%|███▏      | 585/1813 [1:02:50<2:01:38,  5.94s/it]
+ 32%|███▏      | 586/1813 [1:02:56<2:04:31,  6.09s/it]
+ 32%|███▏      | 587/1813 [1:03:02<2:04:18,  6.08s/it]
+ 32%|███▏      | 588/1813 [1:03:09<2:05:42,  6.16s/it]
+ 32%|███▏      | 589/1813 [1:03:15<2:07:17,  6.24s/it]
+ 33%|███▎      | 590/1813 [1:03:22<2:07:23,  6.25s/it]
+                                                      
+{'loss': '0.9103', 'grad_norm': '1.84', 'learning_rate': '1.675e-05', 'epoch': '0.3255', 'num_input_tokens_seen': 3936024, 'train_runtime': '3802', 'train_tokens_per_second': '1035'}
+
+ 33%|███▎      | 590/1813 [1:03:22<2:07:23,  6.25s/it]
+ 33%|███▎      | 591/1813 [1:03:27<2:04:18,  6.10s/it]
+ 33%|███▎      | 592/1813 [1:03:33<2:03:39,  6.08s/it]
+ 33%|███▎      | 593/1813 [1:03:40<2:05:59,  6.20s/it]
+ 33%|███▎      | 594/1813 [1:03:46<2:04:20,  6.12s/it]
+ 33%|███▎      | 595/1813 [1:03:51<2:01:45,  6.00s/it]
+ 33%|███▎      | 596/1813 [1:03:57<2:00:38,  5.95s/it]
+ 33%|███▎      | 597/1813 [1:04:03<1:56:52,  5.77s/it]
+ 33%|███▎      | 598/1813 [1:04:08<1:55:29,  5.70s/it]
+ 33%|███▎      | 599/1813 [1:04:14<1:58:09,  5.84s/it]
+ 33%|███▎      | 600/1813 [1:04:21<2:04:53,  6.18s/it]
+                                                      
+{'loss': '0.9823', 'grad_norm': '1.286', 'learning_rate': '1.661e-05', 'epoch': '0.331', 'num_input_tokens_seen': 3999176, 'train_runtime': '3862', 'train_tokens_per_second': '1036'}
+
+ 33%|███▎      | 600/1813 [1:04:21<2:04:53,  6.18s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.68it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.23it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.32it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.08it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  7.00it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.83it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:43,  6.00it/s]
+
+  2%|▏         | 19/1000 [00:02<03:11,  5.12it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.58it/s]
+
+  2%|▏         | 22/1000 [00:03<02:37,  6.22it/s]
+
+  2%|▏         | 23/1000 [00:03<02:55,  5.58it/s]
+
+  2%|▎         | 25/1000 [00:03<02:14,  7.27it/s]
+
+  3%|▎         | 26/1000 [00:04<02:36,  6.20it/s]
+
+  3%|▎         | 28/1000 [00:04<02:43,  5.96it/s]
+
+  3%|▎         | 29/1000 [00:04<02:33,  6.33it/s]
+
+  3%|▎         | 30/1000 [00:04<02:43,  5.94it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.98it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.10it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:37,  9.71it/s]
+
+  5%|▌         | 53/1000 [00:07<01:27, 10.77it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.65it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.25it/s]
+
+  6%|▌         | 59/1000 [00:08<01:55,  8.15it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.31it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.67it/s]
+
+  6%|▋         | 65/1000 [00:08<01:52,  8.33it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.54it/s]
+
+  7%|▋         | 69/1000 [00:09<01:53,  8.22it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.97it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.70it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.10it/s]
+
+  8%|▊         | 75/1000 [00:10<02:15,  6.85it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.66it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.81it/s]
+
+  8%|▊         | 82/1000 [00:10<01:40,  9.14it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  7.99it/s]
+
+  8%|▊         | 85/1000 [00:11<02:15,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:34,  5.93it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.37it/s]
+
+  9%|▉         | 89/1000 [00:12<02:12,  6.85it/s]
+
+  9%|▉         | 90/1000 [00:12<02:07,  7.13it/s]
+
+  9%|▉         | 91/1000 [00:12<02:31,  6.01it/s]
+
+  9%|▉         | 92/1000 [00:12<02:28,  6.12it/s]
+
+  9%|▉         | 93/1000 [00:12<02:37,  5.74it/s]
+
+  9%|▉         | 94/1000 [00:13<02:19,  6.50it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:52,  5.24it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:12,  6.80it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:44,  8.60it/s]
+
+ 10%|█         | 101/1000 [00:13<01:30,  9.93it/s]
+
+ 10%|█         | 103/1000 [00:14<01:54,  7.80it/s]
+
+ 10%|█         | 104/1000 [00:14<01:53,  7.87it/s]
+
+ 11%|█         | 106/1000 [00:14<01:38,  9.06it/s]
+
+ 11%|█         | 108/1000 [00:14<01:27, 10.25it/s]
+
+ 11%|█         | 110/1000 [00:14<01:20, 11.04it/s]
+
+ 11%|█         | 112/1000 [00:14<01:14, 11.91it/s]
+
+ 11%|█▏        | 114/1000 [00:15<01:11, 12.42it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.74it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.18it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.89it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:48,  8.07it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.71it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.69it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.13it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.63it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.95it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  8.94it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:37,  8.83it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25, 10.02it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.38it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:26,  9.86it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:17, 10.98it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.35it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.63it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.95it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.59it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.66it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:22<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.14it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.14it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.46it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.56it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.74it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.89it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.09it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.47it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.74it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.17it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.64it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.76it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.63it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.93it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.50it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.47it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.66it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.75it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.16it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:29<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.54it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.43it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.36it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.97it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.47it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.36it/s]
+
+ 24%|██▍       | 242/1000 [00:32<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.46it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:34<01:36,  7.70it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.52it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:44,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.05it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.74it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.78it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.01it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.84it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.07it/s]
+
+ 30%|███       | 305/1000 [00:41<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.94it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.78it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.17it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.23it/s]
+
+ 32%|███▏      | 315/1000 [00:42<01:06, 10.23it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.57it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:43<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:44<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.37it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.78it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:06,  9.99it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.85it/s]
+
+ 34%|███▍      | 344/1000 [00:46<01:26,  7.58it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:25,  7.70it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.20it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.40it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:47<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.63it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.54it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.94it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.59it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.43it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:58,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.19it/s]
+
+ 38%|███▊      | 379/1000 [00:51<01:15,  8.18it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.21it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.56it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.07it/s]
+
+ 39%|███▊      | 387/1000 [00:52<01:21,  7.56it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.29it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.83it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.47it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  8.99it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.52it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:09,  8.71it/s]
+
+ 40%|███▉      | 398/1000 [00:53<01:23,  7.18it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:41,  5.92it/s]
+
+ 40%|████      | 400/1000 [00:54<01:51,  5.36it/s]
+
+ 40%|████      | 402/1000 [00:54<01:44,  5.74it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.23it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.59it/s]
+
+ 41%|████      | 407/1000 [00:54<01:23,  7.07it/s]
+
+ 41%|████      | 408/1000 [00:55<01:36,  6.15it/s]
+
+ 41%|████      | 410/1000 [00:55<01:33,  6.29it/s]
+
+ 41%|████      | 411/1000 [00:55<01:44,  5.64it/s]
+
+ 41%|████      | 412/1000 [00:55<01:53,  5.17it/s]
+
+ 41%|████▏     | 414/1000 [00:56<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:26,  6.78it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.29it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.47it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.67it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.90it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.35it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:12,  7.83it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.53it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.01it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.77it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.51it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.24it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.71it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.47it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.80it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.24it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.81it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.61it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.73it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.00it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.30it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:04<01:20,  6.50it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.05it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.31it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.19it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.98it/s]
+
+ 50%|████▉     | 496/1000 [01:06<01:00,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.42it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.53it/s]
+
+ 50%|█████     | 503/1000 [01:08<01:13,  6.79it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.42it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:09<00:48,  9.97it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.52it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.10it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.07it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.38it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:12<00:48,  9.72it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:15<00:59,  7.50it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.94it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.76it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.01it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.84it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.04it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.43it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.82it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.72it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.96it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.58it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.90it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.79it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.68it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.09it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:50,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.72it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.79it/s]
+
+ 62%|██████▏   | 616/1000 [01:24<00:50,  7.61it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.74it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.08it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.20it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.45it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.62it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.43it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.94it/s]
+
+ 64%|██████▍   | 640/1000 [01:27<00:40,  8.99it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.86it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.51it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.05it/s]
+
+ 65%|██████▍   | 647/1000 [01:28<00:46,  7.55it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.49it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.55it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.30it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:45,  7.67it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.89it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.08it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:57,  5.98it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.54it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.65it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.97it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.92it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.88it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.83it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:48,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+
+ 70%|███████   | 702/1000 [01:36<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.21it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:42,  6.85it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.50it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:45,  6.42it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.61it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:37,  7.74it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.24it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.00it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.78it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.91it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.45it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.14it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:39<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.14it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.61it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.31it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.50it/s]
+
+ 74%|███████▎  | 735/1000 [01:40<00:26,  9.88it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.41it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.69it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.27it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.34it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.59it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.41it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.63it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.67it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.70it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  7.88it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:36,  6.29it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.71it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.83it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.56it/s]
+
+ 78%|███████▊  | 780/1000 [01:46<00:31,  6.92it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.34it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.14it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.98it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.21it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.96it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.20it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.96it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.18it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.21it/s]
+
+ 81%|████████▏ | 813/1000 [01:50<00:20,  9.04it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.29it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.12it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.19it/s]
+
+ 82%|████████▏ | 821/1000 [01:51<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.99it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:53<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.08it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.39it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.75it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.23it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.52it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.47it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.86it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.46it/s]
+
+ 86%|████████▌ | 855/1000 [01:56<00:14, 10.05it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.18it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.51it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.74it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.62it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.72it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.30it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.25it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.27it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.62it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:59<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.16it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.30it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:14,  7.47it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.70it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.47it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.64it/s]
+
+ 90%|█████████ | 901/1000 [02:01<00:12,  7.64it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.88it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.39it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  6.94it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.36it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.57it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:09,  8.82it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.07it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.58it/s]
+
+ 92%|█████████▏| 919/1000 [02:04<00:14,  5.70it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.93it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:05<00:09,  8.09it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.66it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:06<00:10,  6.59it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.13it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.27it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.80it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.02it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.12it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.33it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.70it/s]
+
+ 95%|█████████▌| 950/1000 [02:08<00:05,  8.82it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:09<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.41it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.33it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.00it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.41it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.77it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.79it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.43it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+
+ 99%|█████████▊| 986/1000 [02:13<00:01,  8.37it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:14<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.8479', 'eval_runtime': '135.6', 'eval_samples_per_second': '7.376', 'eval_steps_per_second': '7.376', 'epoch': '0.331', 'num_input_tokens_seen': 3999176}
+
+ 33%|███▎      | 600/1813 [1:06:37<2:04:53,  6.18s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+
+                                                   
+ 33%|███▎      | 601/1813 [1:06:41<15:36:24, 46.36s/it]
+ 33%|███▎      | 602/1813 [1:06:46<11:24:42, 33.92s/it]
+ 33%|███▎      | 603/1813 [1:06:52<8:32:49, 25.43s/it] 
+ 33%|███▎      | 604/1813 [1:06:58<6:36:38, 19.68s/it]
+ 33%|███▎      | 605/1813 [1:07:06<5:25:38, 16.17s/it]
+ 33%|███▎      | 606/1813 [1:07:14<4:32:11, 13.53s/it]
+ 33%|███▎      | 607/1813 [1:07:20<3:47:22, 11.31s/it]
+ 34%|███▎      | 608/1813 [1:07:26<3:18:12,  9.87s/it]
+ 34%|███▎      | 609/1813 [1:07:33<2:56:57,  8.82s/it]
+ 34%|███▎      | 610/1813 [1:07:39<2:41:36,  8.06s/it]
+                                                      
+{'loss': '1.024', 'grad_norm': '1.188', 'learning_rate': '1.647e-05', 'epoch': '0.3366', 'num_input_tokens_seen': 4066376, 'train_runtime': '4059', 'train_tokens_per_second': '1002'}
+
+ 34%|███▎      | 610/1813 [1:07:39<2:41:36,  8.06s/it]
+ 34%|███▎      | 611/1813 [1:07:44<2:26:06,  7.29s/it]
+ 34%|███▍      | 612/1813 [1:07:49<2:09:23,  6.46s/it]
+ 34%|███▍      | 613/1813 [1:07:55<2:06:17,  6.31s/it]
+ 34%|███▍      | 614/1813 [1:08:01<2:05:03,  6.26s/it]
+ 34%|███▍      | 615/1813 [1:08:06<2:00:28,  6.03s/it]
+ 34%|███▍      | 616/1813 [1:08:12<1:56:14,  5.83s/it]
+ 34%|███▍      | 617/1813 [1:08:17<1:54:13,  5.73s/it]
+ 34%|███▍      | 618/1813 [1:08:25<2:03:17,  6.19s/it]
+ 34%|███▍      | 619/1813 [1:08:30<1:56:06,  5.83s/it]
+ 34%|███▍      | 620/1813 [1:08:36<2:01:12,  6.10s/it]
+                                                      
+{'loss': '0.906', 'grad_norm': '6.695', 'learning_rate': '1.633e-05', 'epoch': '0.3421', 'num_input_tokens_seen': 4123976, 'train_runtime': '4117', 'train_tokens_per_second': '1002'}
+
+ 34%|███▍      | 620/1813 [1:08:36<2:01:12,  6.10s/it]
+ 34%|███▍      | 621/1813 [1:08:43<2:06:02,  6.34s/it]
+ 34%|███▍      | 622/1813 [1:08:51<2:13:19,  6.72s/it]
+ 34%|███▍      | 623/1813 [1:08:56<2:02:30,  6.18s/it]
+ 34%|███▍      | 624/1813 [1:09:02<2:03:40,  6.24s/it]
+ 34%|███▍      | 625/1813 [1:09:07<1:56:48,  5.90s/it]
+ 35%|███▍      | 626/1813 [1:09:11<1:46:33,  5.39s/it]
+ 35%|███▍      | 627/1813 [1:09:18<1:51:46,  5.65s/it]
+ 35%|███▍      | 628/1813 [1:09:24<1:54:54,  5.82s/it]
+ 35%|███▍      | 629/1813 [1:09:32<2:09:47,  6.58s/it]
+ 35%|███▍      | 630/1813 [1:09:39<2:11:35,  6.67s/it]
+                                                      
+{'loss': '0.9852', 'grad_norm': '1.124', 'learning_rate': '1.618e-05', 'epoch': '0.3476', 'num_input_tokens_seen': 4191992, 'train_runtime': '4180', 'train_tokens_per_second': '1003'}
+
+ 35%|███▍      | 630/1813 [1:09:39<2:11:35,  6.67s/it]
+ 35%|███▍      | 631/1813 [1:09:46<2:11:08,  6.66s/it]
+ 35%|███▍      | 632/1813 [1:09:52<2:07:16,  6.47s/it]
+ 35%|███▍      | 633/1813 [1:09:58<2:02:54,  6.25s/it]
+ 35%|███▍      | 634/1813 [1:10:05<2:09:07,  6.57s/it]
+ 35%|███▌      | 635/1813 [1:10:11<2:09:24,  6.59s/it]
+ 35%|███▌      | 636/1813 [1:10:17<2:00:27,  6.14s/it]
+ 35%|███▌      | 637/1813 [1:10:24<2:05:57,  6.43s/it]
+ 35%|███▌      | 638/1813 [1:10:29<2:00:37,  6.16s/it]
+ 35%|███▌      | 639/1813 [1:10:35<1:55:34,  5.91s/it]
+ 35%|███▌      | 640/1813 [1:10:41<1:57:53,  6.03s/it]
+                                                      
+{'loss': '1.004', 'grad_norm': '1.587', 'learning_rate': '1.603e-05', 'epoch': '0.3531', 'num_input_tokens_seen': 4257760, 'train_runtime': '4241', 'train_tokens_per_second': '1004'}
+
+ 35%|███▌      | 640/1813 [1:10:41<1:57:53,  6.03s/it]
+ 35%|███▌      | 641/1813 [1:10:46<1:54:55,  5.88s/it]
+ 35%|███▌      | 642/1813 [1:10:54<2:02:35,  6.28s/it]
+ 35%|███▌      | 643/1813 [1:11:00<2:04:54,  6.41s/it]
+ 36%|███▌      | 644/1813 [1:11:06<1:59:53,  6.15s/it]
+ 36%|███▌      | 645/1813 [1:11:12<2:00:22,  6.18s/it]
+ 36%|███▌      | 646/1813 [1:11:18<1:59:44,  6.16s/it]
+ 36%|███▌      | 647/1813 [1:11:25<2:01:52,  6.27s/it]
+ 36%|███▌      | 648/1813 [1:11:32<2:06:56,  6.54s/it]
+ 36%|███▌      | 649/1813 [1:11:38<2:02:07,  6.30s/it]
+ 36%|███▌      | 650/1813 [1:11:43<1:56:24,  6.01s/it]
+                                                      
+{'loss': '1.01', 'grad_norm': '1.884', 'learning_rate': '1.588e-05', 'epoch': '0.3586', 'num_input_tokens_seen': 4322136, 'train_runtime': '4303', 'train_tokens_per_second': '1004'}
+
+ 36%|███▌      | 650/1813 [1:11:43<1:56:24,  6.01s/it]
+ 36%|███▌      | 651/1813 [1:11:50<2:00:14,  6.21s/it]
+ 36%|███▌      | 652/1813 [1:11:56<2:03:27,  6.38s/it]
+ 36%|███▌      | 653/1813 [1:12:02<1:57:23,  6.07s/it]
+ 36%|███▌      | 654/1813 [1:12:07<1:54:28,  5.93s/it]
+ 36%|███▌      | 655/1813 [1:12:14<1:57:24,  6.08s/it]
+ 36%|███▌      | 656/1813 [1:12:19<1:54:43,  5.95s/it]
+ 36%|███▌      | 657/1813 [1:12:25<1:51:26,  5.78s/it]
+ 36%|███▋      | 658/1813 [1:12:31<1:55:40,  6.01s/it]
+ 36%|███▋      | 659/1813 [1:12:38<1:59:07,  6.19s/it]
+ 36%|███▋      | 660/1813 [1:12:44<1:56:43,  6.07s/it]
+                                                      
+{'loss': '0.917', 'grad_norm': '1.545', 'learning_rate': '1.572e-05', 'epoch': '0.3641', 'num_input_tokens_seen': 4385688, 'train_runtime': '4364', 'train_tokens_per_second': '1005'}
+
+ 36%|███▋      | 660/1813 [1:12:44<1:56:43,  6.07s/it]
+ 36%|███▋      | 661/1813 [1:12:50<1:56:41,  6.08s/it]
+ 37%|███▋      | 662/1813 [1:12:56<1:56:42,  6.08s/it]
+ 37%|███▋      | 663/1813 [1:13:03<2:00:28,  6.29s/it]
+ 37%|███▋      | 664/1813 [1:13:09<2:00:11,  6.28s/it]
+ 37%|███▋      | 665/1813 [1:13:14<1:52:09,  5.86s/it]
+ 37%|███▋      | 666/1813 [1:13:19<1:48:41,  5.69s/it]
+ 37%|███▋      | 667/1813 [1:13:26<1:55:17,  6.04s/it]
+ 37%|███▋      | 668/1813 [1:13:32<1:57:29,  6.16s/it]
+ 37%|███▋      | 669/1813 [1:13:38<1:55:39,  6.07s/it]
+ 37%|███▋      | 670/1813 [1:13:44<1:52:04,  5.88s/it]
+                                                      
+{'loss': '1.024', 'grad_norm': '1.735', 'learning_rate': '1.557e-05', 'epoch': '0.3697', 'num_input_tokens_seen': 4449456, 'train_runtime': '4424', 'train_tokens_per_second': '1006'}
+
+ 37%|███▋      | 670/1813 [1:13:44<1:52:04,  5.88s/it]
+ 37%|███▋      | 671/1813 [1:13:50<1:53:29,  5.96s/it]
+ 37%|███▋      | 672/1813 [1:13:57<1:58:22,  6.22s/it]
+ 37%|███▋      | 673/1813 [1:14:03<1:58:09,  6.22s/it]
+ 37%|███▋      | 674/1813 [1:14:10<2:01:33,  6.40s/it]
+ 37%|███▋      | 675/1813 [1:14:16<1:59:33,  6.30s/it]
+ 37%|███▋      | 676/1813 [1:14:22<1:56:32,  6.15s/it]
+ 37%|███▋      | 677/1813 [1:14:30<2:07:41,  6.74s/it]
+ 37%|███▋      | 678/1813 [1:14:35<1:59:23,  6.31s/it]
+ 37%|███▋      | 679/1813 [1:14:40<1:53:35,  6.01s/it]
+ 38%|███▊      | 680/1813 [1:14:46<1:52:34,  5.96s/it]
+                                                      
+{'loss': '0.9882', 'grad_norm': '1.562', 'learning_rate': '1.541e-05', 'epoch': '0.3752', 'num_input_tokens_seen': 4514984, 'train_runtime': '4487', 'train_tokens_per_second': '1006'}
+
+ 38%|███▊      | 680/1813 [1:14:46<1:52:34,  5.96s/it]
+ 38%|███▊      | 681/1813 [1:14:53<1:56:05,  6.15s/it]
+ 38%|███▊      | 682/1813 [1:14:58<1:49:15,  5.80s/it]
+ 38%|███▊      | 683/1813 [1:15:04<1:50:34,  5.87s/it]
+ 38%|███▊      | 684/1813 [1:15:10<1:54:45,  6.10s/it]
+ 38%|███▊      | 685/1813 [1:15:18<2:03:57,  6.59s/it]
+ 38%|███▊      | 686/1813 [1:15:25<2:02:16,  6.51s/it]
+ 38%|███▊      | 687/1813 [1:15:32<2:09:24,  6.90s/it]
+ 38%|███▊      | 688/1813 [1:15:37<1:58:06,  6.30s/it]
+ 38%|███▊      | 689/1813 [1:15:46<2:11:20,  7.01s/it]
+ 38%|███▊      | 690/1813 [1:15:53<2:11:48,  7.04s/it]
+                                                      
+{'loss': '1.015', 'grad_norm': '1.654', 'learning_rate': '1.525e-05', 'epoch': '0.3807', 'num_input_tokens_seen': 4587840, 'train_runtime': '4554', 'train_tokens_per_second': '1008'}
+
+ 38%|███▊      | 690/1813 [1:15:53<2:11:48,  7.04s/it]
+ 38%|███▊      | 691/1813 [1:15:58<2:02:46,  6.57s/it]
+ 38%|███▊      | 692/1813 [1:16:04<1:58:50,  6.36s/it]
+ 38%|███▊      | 693/1813 [1:16:10<1:53:03,  6.06s/it]
+ 38%|███▊      | 694/1813 [1:16:16<1:53:08,  6.07s/it]
+ 38%|███▊      | 695/1813 [1:16:22<1:52:20,  6.03s/it]
+ 38%|███▊      | 696/1813 [1:16:30<2:02:14,  6.57s/it]
+ 38%|███▊      | 697/1813 [1:16:34<1:51:53,  6.02s/it]
+ 38%|███▊      | 698/1813 [1:16:40<1:47:36,  5.79s/it]
+ 39%|███▊      | 699/1813 [1:16:46<1:53:09,  6.09s/it]
+ 39%|███▊      | 700/1813 [1:16:53<1:55:58,  6.25s/it]
+                                                      
+{'loss': '1.028', 'grad_norm': '1.539', 'learning_rate': '1.509e-05', 'epoch': '0.3862', 'num_input_tokens_seen': 4653312, 'train_runtime': '4613', 'train_tokens_per_second': '1009'}
+
+ 39%|███▊      | 700/1813 [1:16:53<1:55:58,  6.25s/it]
+ 39%|███▊      | 701/1813 [1:17:00<2:02:14,  6.60s/it]
+ 39%|███▊      | 702/1813 [1:17:07<2:00:03,  6.48s/it]
+ 39%|███▉      | 703/1813 [1:17:15<2:08:21,  6.94s/it]
+ 39%|███▉      | 704/1813 [1:17:21<2:02:56,  6.65s/it]
+ 39%|███▉      | 705/1813 [1:17:28<2:05:27,  6.79s/it]
+ 39%|███▉      | 706/1813 [1:17:34<2:03:32,  6.70s/it]
+ 39%|███▉      | 707/1813 [1:17:44<2:19:50,  7.59s/it]
+ 39%|███▉      | 708/1813 [1:17:51<2:18:46,  7.53s/it]
+ 39%|███▉      | 709/1813 [1:17:58<2:14:11,  7.29s/it]
+ 39%|███▉      | 710/1813 [1:18:04<2:06:24,  6.88s/it]
+                                                      
+{'loss': '0.9832', 'grad_norm': '1.263', 'learning_rate': '1.492e-05', 'epoch': '0.3917', 'num_input_tokens_seen': 4732080, 'train_runtime': '4684', 'train_tokens_per_second': '1010'}
+
+ 39%|███▉      | 710/1813 [1:18:04<2:06:24,  6.88s/it]
+ 39%|███▉      | 711/1813 [1:18:11<2:07:01,  6.92s/it]
+ 39%|███▉      | 712/1813 [1:18:18<2:05:50,  6.86s/it]
+ 39%|███▉      | 713/1813 [1:18:23<1:59:57,  6.54s/it]
+ 39%|███▉      | 714/1813 [1:18:30<1:58:09,  6.45s/it]
+ 39%|███▉      | 715/1813 [1:18:36<1:59:45,  6.54s/it]
+ 39%|███▉      | 716/1813 [1:18:43<2:01:58,  6.67s/it]
+ 40%|███▉      | 717/1813 [1:18:50<1:59:24,  6.54s/it]
+ 40%|███▉      | 718/1813 [1:18:56<1:58:24,  6.49s/it]
+ 40%|███▉      | 719/1813 [1:19:01<1:52:11,  6.15s/it]
+ 40%|███▉      | 720/1813 [1:19:08<1:52:55,  6.20s/it]
+                                                      
+{'loss': '1.04', 'grad_norm': '1.533', 'learning_rate': '1.476e-05', 'epoch': '0.3972', 'num_input_tokens_seen': 4799552, 'train_runtime': '4748', 'train_tokens_per_second': '1011'}
+
+ 40%|███▉      | 720/1813 [1:19:08<1:52:55,  6.20s/it]
+ 40%|███▉      | 721/1813 [1:19:14<1:55:15,  6.33s/it]
+ 40%|███▉      | 722/1813 [1:19:19<1:48:38,  5.97s/it]
+ 40%|███▉      | 723/1813 [1:19:25<1:44:17,  5.74s/it]
+ 40%|███▉      | 724/1813 [1:19:33<1:55:45,  6.38s/it]
+ 40%|███▉      | 725/1813 [1:19:41<2:06:39,  6.99s/it]
+ 40%|████      | 726/1813 [1:19:48<2:04:31,  6.87s/it]
+ 40%|████      | 727/1813 [1:19:53<1:59:14,  6.59s/it]
+ 40%|████      | 728/1813 [1:19:58<1:50:38,  6.12s/it]
+ 40%|████      | 729/1813 [1:20:04<1:48:15,  5.99s/it]
+ 40%|████      | 730/1813 [1:20:10<1:46:05,  5.88s/it]
+                                                      
+{'loss': '1.009', 'grad_norm': '1.975', 'learning_rate': '1.459e-05', 'epoch': '0.4028', 'num_input_tokens_seen': 4866864, 'train_runtime': '4810', 'train_tokens_per_second': '1012'}
+
+ 40%|████      | 730/1813 [1:20:10<1:46:05,  5.88s/it]
+ 40%|████      | 731/1813 [1:20:17<1:52:06,  6.22s/it]
+ 40%|████      | 732/1813 [1:20:23<1:51:46,  6.20s/it]
+ 40%|████      | 733/1813 [1:20:30<1:57:48,  6.54s/it]
+ 40%|████      | 734/1813 [1:20:37<1:59:42,  6.66s/it]
+ 41%|████      | 735/1813 [1:20:44<1:59:45,  6.67s/it]
+ 41%|████      | 736/1813 [1:20:50<1:55:35,  6.44s/it]
+ 41%|████      | 737/1813 [1:20:56<1:51:48,  6.23s/it]
+ 41%|████      | 738/1813 [1:21:01<1:45:23,  5.88s/it]
+ 41%|████      | 739/1813 [1:21:06<1:44:56,  5.86s/it]
+ 41%|████      | 740/1813 [1:21:13<1:48:31,  6.07s/it]
+                                                      
+{'loss': '0.9416', 'grad_norm': '1.222', 'learning_rate': '1.442e-05', 'epoch': '0.4083', 'num_input_tokens_seen': 4936352, 'train_runtime': '4874', 'train_tokens_per_second': '1013'}
+
+ 41%|████      | 740/1813 [1:21:13<1:48:31,  6.07s/it]
+ 41%|████      | 741/1813 [1:21:19<1:50:14,  6.17s/it]
+ 41%|████      | 742/1813 [1:21:27<1:56:54,  6.55s/it]
+ 41%|████      | 743/1813 [1:21:34<2:01:11,  6.80s/it]
+ 41%|████      | 744/1813 [1:21:41<2:01:27,  6.82s/it]
+ 41%|████      | 745/1813 [1:21:48<2:02:23,  6.88s/it]
+ 41%|████      | 746/1813 [1:21:54<1:57:35,  6.61s/it]
+ 41%|████      | 747/1813 [1:22:02<2:04:20,  7.00s/it]
+ 41%|████▏     | 748/1813 [1:22:07<1:56:02,  6.54s/it]
+ 41%|████▏     | 749/1813 [1:22:14<1:54:48,  6.47s/it]
+ 41%|████▏     | 750/1813 [1:22:20<1:50:56,  6.26s/it]
+                                                      
+{'loss': '1.02', 'grad_norm': '1.263', 'learning_rate': '1.425e-05', 'epoch': '0.4138', 'num_input_tokens_seen': 5011064, 'train_runtime': '4940', 'train_tokens_per_second': '1014'}
+
+ 41%|████▏     | 750/1813 [1:22:20<1:50:56,  6.26s/it]
+ 41%|████▏     | 751/1813 [1:22:25<1:48:56,  6.15s/it]
+ 41%|████▏     | 752/1813 [1:22:31<1:47:49,  6.10s/it]
+ 42%|████▏     | 753/1813 [1:22:38<1:49:57,  6.22s/it]
+ 42%|████▏     | 754/1813 [1:22:46<2:01:43,  6.90s/it]
+ 42%|████▏     | 755/1813 [1:22:51<1:50:46,  6.28s/it]
+ 42%|████▏     | 756/1813 [1:22:57<1:46:17,  6.03s/it]
+ 42%|████▏     | 757/1813 [1:23:02<1:44:40,  5.95s/it]
+ 42%|████▏     | 758/1813 [1:23:09<1:47:47,  6.13s/it]
+ 42%|████▏     | 759/1813 [1:23:16<1:52:05,  6.38s/it]
+ 42%|████▏     | 760/1813 [1:23:22<1:49:33,  6.24s/it]
+                                                      
+{'loss': '1.022', 'grad_norm': '1.735', 'learning_rate': '1.408e-05', 'epoch': '0.4193', 'num_input_tokens_seen': 5078960, 'train_runtime': '5002', 'train_tokens_per_second': '1015'}
+
+ 42%|████▏     | 760/1813 [1:23:22<1:49:33,  6.24s/it]
+ 42%|████▏     | 761/1813 [1:23:29<1:51:28,  6.36s/it]
+ 42%|████▏     | 762/1813 [1:23:35<1:51:49,  6.38s/it]
+ 42%|████▏     | 763/1813 [1:23:42<1:54:56,  6.57s/it]
+ 42%|████▏     | 764/1813 [1:23:49<1:57:05,  6.70s/it]
+ 42%|████▏     | 765/1813 [1:23:56<2:01:15,  6.94s/it]
+ 42%|████▏     | 766/1813 [1:24:03<1:58:04,  6.77s/it]
+ 42%|████▏     | 767/1813 [1:24:08<1:51:39,  6.40s/it]
+ 42%|████▏     | 768/1813 [1:24:13<1:39:38,  5.72s/it]
+ 42%|████▏     | 769/1813 [1:24:19<1:45:55,  6.09s/it]
+ 42%|████▏     | 770/1813 [1:24:26<1:47:21,  6.18s/it]
+                                                      
+{'loss': '0.9686', 'grad_norm': '1.488', 'learning_rate': '1.391e-05', 'epoch': '0.4248', 'num_input_tokens_seen': 5148504, 'train_runtime': '5066', 'train_tokens_per_second': '1016'}
+
+ 42%|████▏     | 770/1813 [1:24:26<1:47:21,  6.18s/it]
+ 43%|████▎     | 771/1813 [1:24:32<1:49:28,  6.30s/it]
+ 43%|████▎     | 772/1813 [1:24:40<1:53:47,  6.56s/it]
+ 43%|████▎     | 773/1813 [1:24:45<1:50:13,  6.36s/it]
+ 43%|████▎     | 774/1813 [1:24:51<1:44:27,  6.03s/it]
+ 43%|████▎     | 775/1813 [1:24:57<1:44:47,  6.06s/it]
+ 43%|████▎     | 776/1813 [1:25:04<1:48:16,  6.26s/it]
+ 43%|████▎     | 777/1813 [1:25:11<1:55:10,  6.67s/it]
+ 43%|████▎     | 778/1813 [1:25:16<1:47:37,  6.24s/it]
+ 43%|████▎     | 779/1813 [1:25:22<1:42:46,  5.96s/it]
+ 43%|████▎     | 780/1813 [1:25:28<1:46:18,  6.18s/it]
+                                                      
+{'loss': '0.914', 'grad_norm': '1.593', 'learning_rate': '1.373e-05', 'epoch': '0.4303', 'num_input_tokens_seen': 5215784, 'train_runtime': '5129', 'train_tokens_per_second': '1017'}
+
+ 43%|████▎     | 780/1813 [1:25:28<1:46:18,  6.18s/it]
+ 43%|████▎     | 781/1813 [1:25:35<1:49:44,  6.38s/it]
+ 43%|████▎     | 782/1813 [1:25:43<1:58:51,  6.92s/it]
+ 43%|████▎     | 783/1813 [1:25:51<1:59:21,  6.95s/it]
+ 43%|████▎     | 784/1813 [1:25:56<1:52:55,  6.58s/it]
+ 43%|████▎     | 785/1813 [1:26:03<1:54:26,  6.68s/it]
+ 43%|████▎     | 786/1813 [1:26:10<1:56:42,  6.82s/it]
+ 43%|████▎     | 787/1813 [1:26:16<1:48:58,  6.37s/it]
+ 43%|████▎     | 788/1813 [1:26:21<1:44:51,  6.14s/it]
+ 44%|████▎     | 789/1813 [1:26:27<1:40:42,  5.90s/it]
+ 44%|████▎     | 790/1813 [1:26:32<1:40:41,  5.91s/it]
+                                                      
+{'loss': '1.038', 'grad_norm': '2.029', 'learning_rate': '1.356e-05', 'epoch': '0.4359', 'num_input_tokens_seen': 5285160, 'train_runtime': '5193', 'train_tokens_per_second': '1018'}
+
+ 44%|████▎     | 790/1813 [1:26:32<1:40:41,  5.91s/it]
+ 44%|████▎     | 791/1813 [1:26:39<1:42:40,  6.03s/it]
+ 44%|████▎     | 792/1813 [1:26:46<1:49:24,  6.43s/it]
+ 44%|████▎     | 793/1813 [1:26:54<1:54:00,  6.71s/it]
+ 44%|████▍     | 794/1813 [1:27:00<1:53:30,  6.68s/it]
+ 44%|████▍     | 795/1813 [1:27:06<1:50:07,  6.49s/it]
+ 44%|████▍     | 796/1813 [1:27:13<1:52:27,  6.63s/it]
+ 44%|████▍     | 797/1813 [1:27:21<1:57:07,  6.92s/it]
+ 44%|████▍     | 798/1813 [1:27:27<1:52:49,  6.67s/it]
+ 44%|████▍     | 799/1813 [1:27:33<1:48:54,  6.44s/it]
+ 44%|████▍     | 800/1813 [1:27:39<1:45:38,  6.26s/it]
+                                                      
+{'loss': '0.9746', 'grad_norm': '1.493', 'learning_rate': '1.338e-05', 'epoch': '0.4414', 'num_input_tokens_seen': 5357720, 'train_runtime': '5259', 'train_tokens_per_second': '1019'}
+
+ 44%|████▍     | 800/1813 [1:27:39<1:45:38,  6.26s/it]
+ 44%|████▍     | 801/1813 [1:27:44<1:41:30,  6.02s/it]
+ 44%|████▍     | 802/1813 [1:27:50<1:41:53,  6.05s/it]
+ 44%|████▍     | 803/1813 [1:27:58<1:51:10,  6.60s/it]
+ 44%|████▍     | 804/1813 [1:28:04<1:49:13,  6.50s/it]
+ 44%|████▍     | 805/1813 [1:28:10<1:45:18,  6.27s/it]
+ 44%|████▍     | 806/1813 [1:28:16<1:43:27,  6.16s/it]
+ 45%|████▍     | 807/1813 [1:28:21<1:40:15,  5.98s/it]
+ 45%|████▍     | 808/1813 [1:28:29<1:49:16,  6.52s/it]
+ 45%|████▍     | 809/1813 [1:28:36<1:49:42,  6.56s/it]
+ 45%|████▍     | 810/1813 [1:28:44<1:55:59,  6.94s/it]
+                                                      
+{'loss': '0.986', 'grad_norm': '9.549', 'learning_rate': '1.32e-05', 'epoch': '0.4469', 'num_input_tokens_seen': 5430288, 'train_runtime': '5324', 'train_tokens_per_second': '1020'}
+
+ 45%|████▍     | 810/1813 [1:28:44<1:55:59,  6.94s/it]
+ 45%|████▍     | 811/1813 [1:28:50<1:53:55,  6.82s/it]
+ 45%|████▍     | 812/1813 [1:28:56<1:47:33,  6.45s/it]
+ 45%|████▍     | 813/1813 [1:29:01<1:43:12,  6.19s/it]
+ 45%|████▍     | 814/1813 [1:29:08<1:43:06,  6.19s/it]
+ 45%|████▍     | 815/1813 [1:29:12<1:35:35,  5.75s/it]
+ 45%|████▌     | 816/1813 [1:29:19<1:39:55,  6.01s/it]
+ 45%|████▌     | 817/1813 [1:29:25<1:37:16,  5.86s/it]
+ 45%|████▌     | 818/1813 [1:29:32<1:43:09,  6.22s/it]
+ 45%|████▌     | 819/1813 [1:29:38<1:44:31,  6.31s/it]
+ 45%|████▌     | 820/1813 [1:29:46<1:50:03,  6.65s/it]
+                                                      
+{'loss': '0.9449', 'grad_norm': '1.717', 'learning_rate': '1.302e-05', 'epoch': '0.4524', 'num_input_tokens_seen': 5497552, 'train_runtime': '5386', 'train_tokens_per_second': '1021'}
+
+ 45%|████▌     | 820/1813 [1:29:46<1:50:03,  6.65s/it]
+ 45%|████▌     | 821/1813 [1:29:52<1:49:35,  6.63s/it]
+ 45%|████▌     | 822/1813 [1:29:57<1:41:29,  6.14s/it]
+ 45%|████▌     | 823/1813 [1:30:02<1:35:20,  5.78s/it]
+ 45%|████▌     | 824/1813 [1:30:09<1:38:44,  5.99s/it]
+ 46%|████▌     | 825/1813 [1:30:15<1:41:20,  6.15s/it]
+ 46%|████▌     | 826/1813 [1:30:22<1:45:30,  6.41s/it]
+ 46%|████▌     | 827/1813 [1:30:28<1:43:10,  6.28s/it]
+ 46%|████▌     | 828/1813 [1:30:34<1:42:39,  6.25s/it]
+ 46%|████▌     | 829/1813 [1:30:40<1:40:31,  6.13s/it]
+ 46%|████▌     | 830/1813 [1:30:47<1:46:41,  6.51s/it]
+                                                      
+{'loss': '0.8374', 'grad_norm': '1.441', 'learning_rate': '1.284e-05', 'epoch': '0.4579', 'num_input_tokens_seen': 5563144, 'train_runtime': '5448', 'train_tokens_per_second': '1021'}
+
+ 46%|████▌     | 830/1813 [1:30:47<1:46:41,  6.51s/it]
+ 46%|████▌     | 831/1813 [1:30:54<1:44:43,  6.40s/it]
+ 46%|████▌     | 832/1813 [1:31:01<1:49:51,  6.72s/it]
+ 46%|████▌     | 833/1813 [1:31:07<1:47:42,  6.59s/it]
+ 46%|████▌     | 834/1813 [1:31:14<1:48:28,  6.65s/it]
+ 46%|████▌     | 835/1813 [1:31:20<1:45:47,  6.49s/it]
+ 46%|████▌     | 836/1813 [1:31:27<1:47:25,  6.60s/it]
+ 46%|████▌     | 837/1813 [1:31:35<1:53:06,  6.95s/it]
+ 46%|████▌     | 838/1813 [1:31:41<1:46:42,  6.57s/it]
+ 46%|████▋     | 839/1813 [1:31:47<1:44:52,  6.46s/it]
+ 46%|████▋     | 840/1813 [1:31:52<1:40:26,  6.19s/it]
+                                                      
+{'loss': '0.9971', 'grad_norm': '1.74', 'learning_rate': '1.266e-05', 'epoch': '0.4634', 'num_input_tokens_seen': 5634104, 'train_runtime': '5513', 'train_tokens_per_second': '1022'}
+
+ 46%|████▋     | 840/1813 [1:31:52<1:40:26,  6.19s/it]
+ 46%|████▋     | 841/1813 [1:31:58<1:38:56,  6.11s/it]
+ 46%|████▋     | 842/1813 [1:32:05<1:42:41,  6.35s/it]
+ 46%|████▋     | 843/1813 [1:32:11<1:41:15,  6.26s/it]
+ 47%|████▋     | 844/1813 [1:32:17<1:37:50,  6.06s/it]
+ 47%|████▋     | 845/1813 [1:32:23<1:39:20,  6.16s/it]
+ 47%|████▋     | 846/1813 [1:32:30<1:41:57,  6.33s/it]
+ 47%|████▋     | 847/1813 [1:32:37<1:45:28,  6.55s/it]
+ 47%|████▋     | 848/1813 [1:32:44<1:48:42,  6.76s/it]
+ 47%|████▋     | 849/1813 [1:32:49<1:40:20,  6.25s/it]
+ 47%|████▋     | 850/1813 [1:32:57<1:49:33,  6.83s/it]
+                                                      
+{'loss': '0.9587', 'grad_norm': '1.381', 'learning_rate': '1.248e-05', 'epoch': '0.469', 'num_input_tokens_seen': 5704744, 'train_runtime': '5578', 'train_tokens_per_second': '1023'}
+
+ 47%|████▋     | 850/1813 [1:32:57<1:49:33,  6.83s/it]
+ 47%|████▋     | 851/1813 [1:33:04<1:48:54,  6.79s/it]
+ 47%|████▋     | 852/1813 [1:33:09<1:38:34,  6.15s/it]
+ 47%|████▋     | 853/1813 [1:33:15<1:38:30,  6.16s/it]
+ 47%|████▋     | 854/1813 [1:33:22<1:41:11,  6.33s/it]
+ 47%|████▋     | 855/1813 [1:33:28<1:39:23,  6.23s/it]
+ 47%|████▋     | 856/1813 [1:33:34<1:38:20,  6.17s/it]
+ 47%|████▋     | 857/1813 [1:33:39<1:33:39,  5.88s/it]
+ 47%|████▋     | 858/1813 [1:33:43<1:24:01,  5.28s/it]
+ 47%|████▋     | 859/1813 [1:33:49<1:28:00,  5.53s/it]
+ 47%|████▋     | 860/1813 [1:33:55<1:32:24,  5.82s/it]
+                                                      
+{'loss': '0.9774', 'grad_norm': '1.452', 'learning_rate': '1.229e-05', 'epoch': '0.4745', 'num_input_tokens_seen': 5766424, 'train_runtime': '5636', 'train_tokens_per_second': '1023'}
+
+ 47%|████▋     | 860/1813 [1:33:55<1:32:24,  5.82s/it]
+ 47%|████▋     | 861/1813 [1:34:00<1:28:03,  5.55s/it]
+ 48%|████▊     | 862/1813 [1:34:07<1:33:35,  5.91s/it]
+ 48%|████▊     | 863/1813 [1:34:13<1:34:17,  5.96s/it]
+ 48%|████▊     | 864/1813 [1:34:17<1:25:41,  5.42s/it]
+ 48%|████▊     | 865/1813 [1:34:24<1:32:26,  5.85s/it]
+ 48%|████▊     | 866/1813 [1:34:30<1:32:17,  5.85s/it]
+ 48%|████▊     | 867/1813 [1:34:36<1:31:25,  5.80s/it]
+ 48%|████▊     | 868/1813 [1:34:41<1:30:22,  5.74s/it]
+ 48%|████▊     | 869/1813 [1:34:49<1:38:12,  6.24s/it]
+ 48%|████▊     | 870/1813 [1:34:57<1:45:41,  6.72s/it]
+                                                      
+{'loss': '0.9746', 'grad_norm': '1.21', 'learning_rate': '1.211e-05', 'epoch': '0.48', 'num_input_tokens_seen': 5831352, 'train_runtime': '5697', 'train_tokens_per_second': '1024'}
+
+ 48%|████▊     | 870/1813 [1:34:57<1:45:41,  6.72s/it]
+ 48%|████▊     | 871/1813 [1:35:02<1:39:55,  6.36s/it]
+ 48%|████▊     | 872/1813 [1:35:10<1:44:35,  6.67s/it]
+ 48%|████▊     | 873/1813 [1:35:16<1:45:15,  6.72s/it]
+ 48%|████▊     | 874/1813 [1:35:24<1:48:57,  6.96s/it]
+ 48%|████▊     | 875/1813 [1:35:32<1:52:48,  7.22s/it]
+ 48%|████▊     | 876/1813 [1:35:39<1:51:26,  7.14s/it]
+ 48%|████▊     | 877/1813 [1:35:47<1:55:40,  7.42s/it]
+ 48%|████▊     | 878/1813 [1:35:53<1:51:19,  7.14s/it]
+ 48%|████▊     | 879/1813 [1:36:01<1:53:36,  7.30s/it]
+ 49%|████▊     | 880/1813 [1:36:05<1:40:35,  6.47s/it]
+                                                      
+{'loss': '1', 'grad_norm': '2.895', 'learning_rate': '1.192e-05', 'epoch': '0.4855', 'num_input_tokens_seen': 5907208, 'train_runtime': '5766', 'train_tokens_per_second': '1025'}
+
+ 49%|████▊     | 880/1813 [1:36:05<1:40:35,  6.47s/it]
+ 49%|████▊     | 881/1813 [1:36:11<1:35:06,  6.12s/it]
+ 49%|████▊     | 882/1813 [1:36:17<1:35:00,  6.12s/it]
+ 49%|████▊     | 883/1813 [1:36:24<1:37:54,  6.32s/it]
+ 49%|████▉     | 884/1813 [1:36:30<1:36:19,  6.22s/it]
+ 49%|████▉     | 885/1813 [1:36:36<1:35:09,  6.15s/it]
+ 49%|████▉     | 886/1813 [1:36:42<1:35:54,  6.21s/it]
+ 49%|████▉     | 887/1813 [1:36:50<1:44:11,  6.75s/it]
+ 49%|████▉     | 888/1813 [1:36:57<1:44:21,  6.77s/it]
+ 49%|████▉     | 889/1813 [1:37:04<1:46:56,  6.94s/it]
+ 49%|████▉     | 890/1813 [1:37:10<1:43:56,  6.76s/it]
+                                                      
+{'loss': '0.9675', 'grad_norm': '1.243', 'learning_rate': '1.174e-05', 'epoch': '0.491', 'num_input_tokens_seen': 5978048, 'train_runtime': '5831', 'train_tokens_per_second': '1025'}
+
+ 49%|████▉     | 890/1813 [1:37:10<1:43:56,  6.76s/it]
+ 49%|████▉     | 891/1813 [1:37:16<1:40:22,  6.53s/it]
+ 49%|████▉     | 892/1813 [1:37:24<1:44:15,  6.79s/it]
+ 49%|████▉     | 893/1813 [1:37:30<1:40:24,  6.55s/it]
+ 49%|████▉     | 894/1813 [1:37:35<1:35:59,  6.27s/it]
+ 49%|████▉     | 895/1813 [1:37:43<1:40:59,  6.60s/it]
+ 49%|████▉     | 896/1813 [1:37:48<1:35:29,  6.25s/it]
+ 49%|████▉     | 897/1813 [1:37:54<1:33:30,  6.13s/it]
+ 50%|████▉     | 898/1813 [1:38:01<1:38:11,  6.44s/it]
+ 50%|████▉     | 899/1813 [1:38:09<1:42:24,  6.72s/it]
+ 50%|████▉     | 900/1813 [1:38:16<1:47:00,  7.03s/it]
+                                                      
+{'loss': '1.015', 'grad_norm': '10.01', 'learning_rate': '1.155e-05', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856, 'train_runtime': '5897', 'train_tokens_per_second': '1026'}
+
+ 50%|████▉     | 900/1813 [1:38:16<1:47:00,  7.03s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.72it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.46it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.07it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.01it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.83it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.06it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.37it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.92it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.75it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.31it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.21it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:35,  9.76it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:26,  6.31it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.85it/s]
+
+  8%|▊         | 82/1000 [00:10<01:40,  9.11it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  7.97it/s]
+
+  8%|▊         | 85/1000 [00:11<02:15,  6.76it/s]
+
+  9%|▊         | 86/1000 [00:11<02:34,  5.93it/s]
+
+  9%|▊         | 87/1000 [00:12<02:50,  5.36it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  6.98it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:37,  5.77it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:12,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.38it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.65it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.29it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.42it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.19it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.03it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.52it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.80it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.24it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.65it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.89it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.05it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.91it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.08it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.92it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.04it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.39it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.83it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.11it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:15,  6.13it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.21it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.72it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.42it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.79it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.84it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.37it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.10it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.62it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.47it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.20it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.64it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.75it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.62it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.91it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.47it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.67it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.49it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.79it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.36it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.98it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.48it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.37it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.46it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.64it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.74it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.04it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.75it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.78it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.20it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.13it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.77it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.09it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.16it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:41,  6.80it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.80it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.26it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.56it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.88it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.21it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.40it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.94it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.57it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.41it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.75it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.88it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.18it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.19it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.64it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.36it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.77it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.74it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.23it/s]
+
+ 41%|████      | 406/1000 [00:54<01:10,  8.48it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.44it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.85it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.56it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.01it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:48, 11.76it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.49it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.31it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.24it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.51it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.83it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.20it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.37it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:25,  6.17it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.17it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:39,  5.28it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.07it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:20,  6.49it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.04it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.29it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.61it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.98it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.96it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.27it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.52it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.10it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:12,  6.64it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.76it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:26,  5.53it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.56it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.49it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:14,  6.36it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.10it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.72it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.68it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.35it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.49it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.94it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.86it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.84it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.58it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.52it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.38it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.74it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.80it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.75it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.11it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.04it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.45it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.63it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.44it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.73it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.18it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.71it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.57it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.59it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.22it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.62it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.97it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.92it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.87it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.59it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.01it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.61it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.72it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.36it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.63it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.53it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.54it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.80it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.32it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.44it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.67it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.24it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.72it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.67it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.62it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.63it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  8.98it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.16it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.84it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.00it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.63it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.15it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.95it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:19,  9.98it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:17, 11.04it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  8.99it/s]
+
+ 81%|████████▏ | 814/1000 [01:49<00:21,  8.84it/s]
+
+ 82%|████████▏ | 816/1000 [01:50<00:18,  9.90it/s]
+
+ 82%|████████▏ | 818/1000 [01:50<00:21,  8.35it/s]
+
+ 82%|████████▏ | 820/1000 [01:50<00:23,  7.54it/s]
+
+ 82%|████████▏ | 822/1000 [01:50<00:20,  8.70it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.72it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.54it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.84it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.62it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:33,  5.13it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:30,  5.65it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  6.97it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.71it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.21it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.00it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.50it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.86it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.05it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.75it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.61it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.65it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.28it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.23it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.26it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.86it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.86it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.78it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.77it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.84it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.78it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  6.95it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.45it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  7.95it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.59it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.42it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.79it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.60it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.29it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.80it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.15it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.07it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.38it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.84it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.41it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.63it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.69it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  5.97it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.40it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.39it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.31it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.00it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  7.99it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.17it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.79it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.81it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.44it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.05it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.37it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.52it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.8215', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.384', 'eval_steps_per_second': '7.384', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856}
+
+ 50%|████▉     | 900/1813 [1:40:32<1:47:00,  7.03s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+
+                                                   
+ 50%|████▉     | 901/1813 [1:40:38<12:00:38, 47.41s/it]
+ 50%|████▉     | 902/1813 [1:40:44<8:50:32, 34.94s/it] 
+ 50%|████▉     | 903/1813 [1:40:50<6:38:19, 26.26s/it]
+ 50%|████▉     | 904/1813 [1:40:56<5:05:38, 20.17s/it]
+ 50%|████▉     | 905/1813 [1:41:04<4:09:36, 16.49s/it]
+ 50%|████▉     | 906/1813 [1:41:09<3:18:21, 13.12s/it]
+ 50%|█████     | 907/1813 [1:41:13<2:38:29, 10.50s/it]
+ 50%|█████     | 908/1813 [1:41:19<2:15:15,  8.97s/it]
+ 50%|█████     | 909/1813 [1:41:25<2:03:59,  8.23s/it]
+ 50%|█████     | 910/1813 [1:41:31<1:54:03,  7.58s/it]
+                                                      
+{'loss': '0.951', 'grad_norm': '1.885', 'learning_rate': '1.137e-05', 'epoch': '0.5021', 'num_input_tokens_seen': 6115160, 'train_runtime': '6092', 'train_tokens_per_second': '1004'}
+
+ 50%|█████     | 910/1813 [1:41:31<1:54:03,  7.58s/it]
+ 50%|█████     | 911/1813 [1:41:37<1:43:46,  6.90s/it]
+ 50%|█████     | 912/1813 [1:41:42<1:37:00,  6.46s/it]
+ 50%|█████     | 913/1813 [1:41:47<1:31:05,  6.07s/it]
+ 50%|█████     | 914/1813 [1:41:52<1:24:38,  5.65s/it]
+ 50%|█████     | 915/1813 [1:42:00<1:33:38,  6.26s/it]
+ 51%|█████     | 916/1813 [1:42:07<1:37:28,  6.52s/it]
+ 51%|█████     | 917/1813 [1:42:14<1:39:03,  6.63s/it]
+ 51%|█████     | 918/1813 [1:42:19<1:34:56,  6.36s/it]
+ 51%|█████     | 919/1813 [1:42:26<1:35:05,  6.38s/it]
+ 51%|█████     | 920/1813 [1:42:31<1:31:29,  6.15s/it]
+                                                      
+{'loss': '0.996', 'grad_norm': '1.245', 'learning_rate': '1.118e-05', 'epoch': '0.5076', 'num_input_tokens_seen': 6178904, 'train_runtime': '6152', 'train_tokens_per_second': '1004'}
+
+ 51%|█████     | 920/1813 [1:42:31<1:31:29,  6.15s/it]
+ 51%|█████     | 921/1813 [1:42:37<1:29:49,  6.04s/it]
+ 51%|█████     | 922/1813 [1:42:43<1:28:00,  5.93s/it]
+ 51%|█████     | 923/1813 [1:42:49<1:28:19,  5.95s/it]
+ 51%|█████     | 924/1813 [1:42:56<1:33:02,  6.28s/it]
+ 51%|█████     | 925/1813 [1:43:02<1:31:48,  6.20s/it]
+ 51%|█████     | 926/1813 [1:43:09<1:34:06,  6.37s/it]
+ 51%|█████     | 927/1813 [1:43:15<1:32:57,  6.29s/it]
+ 51%|█████     | 928/1813 [1:43:21<1:30:39,  6.15s/it]
+ 51%|█████     | 929/1813 [1:43:27<1:31:33,  6.21s/it]
+ 51%|█████▏    | 930/1813 [1:43:34<1:34:42,  6.43s/it]
+                                                      
+{'loss': '0.952', 'grad_norm': '1.57', 'learning_rate': '1.099e-05', 'epoch': '0.5131', 'num_input_tokens_seen': 6244360, 'train_runtime': '6214', 'train_tokens_per_second': '1005'}
+
+ 51%|█████▏    | 930/1813 [1:43:34<1:34:42,  6.43s/it]
+ 51%|█████▏    | 931/1813 [1:43:41<1:36:30,  6.56s/it]
+ 51%|█████▏    | 932/1813 [1:43:46<1:32:02,  6.27s/it]
+ 51%|█████▏    | 933/1813 [1:43:52<1:31:16,  6.22s/it]
+ 52%|█████▏    | 934/1813 [1:44:00<1:34:55,  6.48s/it]
+ 52%|█████▏    | 935/1813 [1:44:08<1:45:32,  7.21s/it]
+ 52%|█████▏    | 936/1813 [1:44:15<1:42:35,  7.02s/it]
+ 52%|█████▏    | 937/1813 [1:44:21<1:39:12,  6.80s/it]
+ 52%|█████▏    | 938/1813 [1:44:28<1:40:31,  6.89s/it]
+ 52%|█████▏    | 939/1813 [1:44:34<1:35:43,  6.57s/it]
+ 52%|█████▏    | 940/1813 [1:44:41<1:37:37,  6.71s/it]
+                                                      
+{'loss': '0.9382', 'grad_norm': '1.969', 'learning_rate': '1.08e-05', 'epoch': '0.5186', 'num_input_tokens_seen': 6318968, 'train_runtime': '6282', 'train_tokens_per_second': '1006'}
+
+ 52%|█████▏    | 940/1813 [1:44:41<1:37:37,  6.71s/it]
+ 52%|█████▏    | 941/1813 [1:44:47<1:30:54,  6.25s/it]
+ 52%|█████▏    | 942/1813 [1:44:52<1:28:09,  6.07s/it]
+ 52%|█████▏    | 943/1813 [1:44:58<1:25:41,  5.91s/it]
+ 52%|█████▏    | 944/1813 [1:45:04<1:25:35,  5.91s/it]
+ 52%|█████▏    | 945/1813 [1:45:09<1:22:24,  5.70s/it]
+ 52%|█████▏    | 946/1813 [1:45:15<1:25:15,  5.90s/it]
+ 52%|█████▏    | 947/1813 [1:45:21<1:26:56,  6.02s/it]
+ 52%|█████▏    | 948/1813 [1:45:28<1:28:29,  6.14s/it]
+ 52%|█████▏    | 949/1813 [1:45:33<1:25:11,  5.92s/it]
+ 52%|█████▏    | 950/1813 [1:45:40<1:26:58,  6.05s/it]
+                                                      
+{'loss': '0.9812', 'grad_norm': '1.674', 'learning_rate': '1.061e-05', 'epoch': '0.5241', 'num_input_tokens_seen': 6379704, 'train_runtime': '6340', 'train_tokens_per_second': '1006'}
+
+ 52%|█████▏    | 950/1813 [1:45:40<1:26:58,  6.05s/it]
+ 52%|█████▏    | 951/1813 [1:45:45<1:23:58,  5.85s/it]
+ 53%|█████▎    | 952/1813 [1:45:51<1:22:28,  5.75s/it]
+ 53%|█████▎    | 953/1813 [1:45:57<1:24:06,  5.87s/it]
+ 53%|█████▎    | 954/1813 [1:46:02<1:23:46,  5.85s/it]
+ 53%|█████▎    | 955/1813 [1:46:08<1:24:00,  5.87s/it]
+ 53%|█████▎    | 956/1813 [1:46:14<1:23:24,  5.84s/it]
+ 53%|█████▎    | 957/1813 [1:46:20<1:23:13,  5.83s/it]
+ 53%|█████▎    | 958/1813 [1:46:24<1:17:21,  5.43s/it]
+ 53%|█████▎    | 959/1813 [1:46:31<1:19:56,  5.62s/it]
+ 53%|█████▎    | 960/1813 [1:46:39<1:31:13,  6.42s/it]
+                                                      
+{'loss': '0.9977', 'grad_norm': '1.467', 'learning_rate': '1.042e-05', 'epoch': '0.5297', 'num_input_tokens_seen': 6441656, 'train_runtime': '6399', 'train_tokens_per_second': '1007'}
+
+ 53%|█████▎    | 960/1813 [1:46:39<1:31:13,  6.42s/it]
+ 53%|█████▎    | 961/1813 [1:46:46<1:32:50,  6.54s/it]
+ 53%|█████▎    | 962/1813 [1:46:53<1:38:15,  6.93s/it]
+ 53%|█████▎    | 963/1813 [1:47:00<1:37:50,  6.91s/it]
+ 53%|█████▎    | 964/1813 [1:47:06<1:33:58,  6.64s/it]
+ 53%|█████▎    | 965/1813 [1:47:12<1:31:30,  6.48s/it]
+ 53%|█████▎    | 966/1813 [1:47:20<1:36:15,  6.82s/it]
+ 53%|█████▎    | 967/1813 [1:47:27<1:38:32,  6.99s/it]
+ 53%|█████▎    | 968/1813 [1:47:33<1:32:08,  6.54s/it]
+ 53%|█████▎    | 969/1813 [1:47:39<1:29:29,  6.36s/it]
+ 54%|█████▎    | 970/1813 [1:47:44<1:22:13,  5.85s/it]
+                                                      
+{'loss': '1.048', 'grad_norm': '2.189', 'learning_rate': '1.024e-05', 'epoch': '0.5352', 'num_input_tokens_seen': 6512320, 'train_runtime': '6464', 'train_tokens_per_second': '1007'}
+
+ 54%|█████▎    | 970/1813 [1:47:44<1:22:13,  5.85s/it]
+ 54%|█████▎    | 971/1813 [1:47:50<1:25:37,  6.10s/it]
+ 54%|█████▎    | 972/1813 [1:47:56<1:26:09,  6.15s/it]
+ 54%|█████▎    | 973/1813 [1:48:02<1:24:57,  6.07s/it]
+ 54%|█████▎    | 974/1813 [1:48:11<1:34:33,  6.76s/it]
+ 54%|█████▍    | 975/1813 [1:48:17<1:33:25,  6.69s/it]
+ 54%|█████▍    | 976/1813 [1:48:23<1:29:25,  6.41s/it]
+ 54%|█████▍    | 977/1813 [1:48:31<1:35:17,  6.84s/it]
+ 54%|█████▍    | 978/1813 [1:48:37<1:33:42,  6.73s/it]
+ 54%|█████▍    | 979/1813 [1:48:45<1:35:19,  6.86s/it]
+ 54%|█████▍    | 980/1813 [1:48:51<1:34:19,  6.79s/it]
+                                                      
+{'loss': '0.9625', 'grad_norm': '2.924', 'learning_rate': '1.005e-05', 'epoch': '0.5407', 'num_input_tokens_seen': 6586672, 'train_runtime': '6532', 'train_tokens_per_second': '1008'}
+
+ 54%|█████▍    | 980/1813 [1:48:51<1:34:19,  6.79s/it]
+ 54%|█████▍    | 981/1813 [1:48:58<1:33:47,  6.76s/it]
+ 54%|█████▍    | 982/1813 [1:49:03<1:26:46,  6.27s/it]
+ 54%|█████▍    | 983/1813 [1:49:08<1:22:23,  5.96s/it]
+ 54%|█████▍    | 984/1813 [1:49:14<1:19:50,  5.78s/it]
+ 54%|█████▍    | 985/1813 [1:49:20<1:23:07,  6.02s/it]
+ 54%|█████▍    | 986/1813 [1:49:28<1:29:19,  6.48s/it]
+ 54%|█████▍    | 987/1813 [1:49:34<1:27:25,  6.35s/it]
+ 54%|█████▍    | 988/1813 [1:49:39<1:24:22,  6.14s/it]
+ 55%|█████▍    | 989/1813 [1:49:45<1:20:47,  5.88s/it]
+ 55%|█████▍    | 990/1813 [1:49:53<1:30:02,  6.56s/it]
+                                                      
+{'loss': '0.9528', 'grad_norm': '1.624', 'learning_rate': '9.858e-06', 'epoch': '0.5462', 'num_input_tokens_seen': 6653096, 'train_runtime': '6593', 'train_tokens_per_second': '1009'}
+
+ 55%|█████▍    | 990/1813 [1:49:53<1:30:02,  6.56s/it]
+ 55%|█████▍    | 991/1813 [1:49:58<1:23:58,  6.13s/it]
+ 55%|█████▍    | 992/1813 [1:50:05<1:29:06,  6.51s/it]
+ 55%|█████▍    | 993/1813 [1:50:11<1:26:11,  6.31s/it]
+ 55%|█████▍    | 994/1813 [1:50:20<1:34:51,  6.95s/it]
+ 55%|█████▍    | 995/1813 [1:50:25<1:29:46,  6.59s/it]
+ 55%|█████▍    | 996/1813 [1:50:31<1:27:43,  6.44s/it]
+ 55%|█████▍    | 997/1813 [1:50:37<1:24:02,  6.18s/it]
+ 55%|█████▌    | 998/1813 [1:50:42<1:19:23,  5.85s/it]
+ 55%|█████▌    | 999/1813 [1:50:48<1:18:56,  5.82s/it]
+ 55%|█████▌    | 1000/1813 [1:50:53<1:16:29,  5.64s/it]
+                                                       
+{'loss': '1.037', 'grad_norm': '1.357', 'learning_rate': '9.669e-06', 'epoch': '0.5517', 'num_input_tokens_seen': 6716784, 'train_runtime': '6654', 'train_tokens_per_second': '1009'}
+
+ 55%|█████▌    | 1000/1813 [1:50:53<1:16:29,  5.64s/it]
+ 55%|█████▌    | 1001/1813 [1:51:00<1:20:30,  5.95s/it]
+ 55%|█████▌    | 1002/1813 [1:51:06<1:22:14,  6.08s/it]
+ 55%|█████▌    | 1003/1813 [1:51:13<1:26:00,  6.37s/it]
+ 55%|█████▌    | 1004/1813 [1:51:19<1:22:37,  6.13s/it]
+ 55%|█████▌    | 1005/1813 [1:51:26<1:26:46,  6.44s/it]
+ 55%|█████▌    | 1006/1813 [1:51:33<1:27:46,  6.53s/it]
+ 56%|█████▌    | 1007/1813 [1:51:39<1:25:18,  6.35s/it]
+ 56%|█████▌    | 1008/1813 [1:51:45<1:23:55,  6.25s/it]
+ 56%|█████▌    | 1009/1813 [1:51:51<1:23:41,  6.25s/it]
+ 56%|█████▌    | 1010/1813 [1:51:57<1:22:05,  6.13s/it]
+                                                       
+{'loss': '0.9886', 'grad_norm': '2.195', 'learning_rate': '9.481e-06', 'epoch': '0.5572', 'num_input_tokens_seen': 6783768, 'train_runtime': '6717', 'train_tokens_per_second': '1010'}
+
+ 56%|█████▌    | 1010/1813 [1:51:57<1:22:05,  6.13s/it]
+ 56%|█████▌    | 1011/1813 [1:52:04<1:25:24,  6.39s/it]
+ 56%|█████▌    | 1012/1813 [1:52:09<1:19:51,  5.98s/it]
+ 56%|█████▌    | 1013/1813 [1:52:13<1:14:30,  5.59s/it]
+ 56%|█████▌    | 1014/1813 [1:52:22<1:25:52,  6.45s/it]
+ 56%|█████▌    | 1015/1813 [1:52:27<1:21:41,  6.14s/it]
+ 56%|█████▌    | 1016/1813 [1:52:33<1:18:33,  5.91s/it]
+ 56%|█████▌    | 1017/1813 [1:52:40<1:22:48,  6.24s/it]
+ 56%|█████▌    | 1018/1813 [1:52:46<1:23:03,  6.27s/it]
+ 56%|█████▌    | 1019/1813 [1:52:51<1:18:37,  5.94s/it]
+ 56%|█████▋    | 1020/1813 [1:52:56<1:15:50,  5.74s/it]
+                                                       
+{'loss': '0.9086', 'grad_norm': '1.735', 'learning_rate': '9.292e-06', 'epoch': '0.5628', 'num_input_tokens_seen': 6844992, 'train_runtime': '6777', 'train_tokens_per_second': '1010'}
+
+ 56%|█████▋    | 1020/1813 [1:52:56<1:15:50,  5.74s/it]
+ 56%|█████▋    | 1021/1813 [1:53:04<1:21:29,  6.17s/it]
+ 56%|█████▋    | 1022/1813 [1:53:10<1:22:00,  6.22s/it]
+ 56%|█████▋    | 1023/1813 [1:53:18<1:27:24,  6.64s/it]
+ 56%|█████▋    | 1024/1813 [1:53:24<1:26:36,  6.59s/it]
+ 57%|█████▋    | 1025/1813 [1:53:33<1:36:45,  7.37s/it]
+ 57%|█████▋    | 1026/1813 [1:53:39<1:29:27,  6.82s/it]
+ 57%|█████▋    | 1027/1813 [1:53:45<1:27:19,  6.67s/it]
+ 57%|█████▋    | 1028/1813 [1:53:53<1:30:26,  6.91s/it]
+ 57%|█████▋    | 1029/1813 [1:53:59<1:28:49,  6.80s/it]
+ 57%|█████▋    | 1030/1813 [1:54:06<1:28:07,  6.75s/it]
+                                                       
+{'loss': '1.039', 'grad_norm': '1.241', 'learning_rate': '9.104e-06', 'epoch': '0.5683', 'num_input_tokens_seen': 6920856, 'train_runtime': '6846', 'train_tokens_per_second': '1011'}
+
+ 57%|█████▋    | 1030/1813 [1:54:06<1:28:07,  6.75s/it]
+ 57%|█████▋    | 1031/1813 [1:54:11<1:22:57,  6.37s/it]
+ 57%|█████▋    | 1032/1813 [1:54:18<1:23:36,  6.42s/it]
+ 57%|█████▋    | 1033/1813 [1:54:24<1:22:02,  6.31s/it]
+ 57%|█████▋    | 1034/1813 [1:54:29<1:19:10,  6.10s/it]
+ 57%|█████▋    | 1035/1813 [1:54:35<1:18:52,  6.08s/it]
+ 57%|█████▋    | 1036/1813 [1:54:42<1:21:20,  6.28s/it]
+ 57%|█████▋    | 1037/1813 [1:54:48<1:19:42,  6.16s/it]
+ 57%|█████▋    | 1038/1813 [1:54:55<1:20:38,  6.24s/it]
+ 57%|█████▋    | 1039/1813 [1:55:02<1:23:24,  6.47s/it]
+ 57%|█████▋    | 1040/1813 [1:55:09<1:25:21,  6.63s/it]
+                                                       
+{'loss': '0.9788', 'grad_norm': '1.29', 'learning_rate': '8.916e-06', 'epoch': '0.5738', 'num_input_tokens_seen': 6988568, 'train_runtime': '6909', 'train_tokens_per_second': '1012'}
+
+ 57%|█████▋    | 1040/1813 [1:55:09<1:25:21,  6.63s/it]
+ 57%|█████▋    | 1041/1813 [1:55:15<1:23:09,  6.46s/it]
+ 57%|█████▋    | 1042/1813 [1:55:21<1:23:14,  6.48s/it]
+ 58%|█████▊    | 1043/1813 [1:55:27<1:20:11,  6.25s/it]
+ 58%|█████▊    | 1044/1813 [1:55:32<1:16:49,  5.99s/it]
+ 58%|█████▊    | 1045/1813 [1:55:38<1:15:49,  5.92s/it]
+ 58%|█████▊    | 1046/1813 [1:55:44<1:15:53,  5.94s/it]
+ 58%|█████▊    | 1047/1813 [1:55:50<1:17:15,  6.05s/it]
+ 58%|█████▊    | 1048/1813 [1:55:57<1:20:06,  6.28s/it]
+ 58%|█████▊    | 1049/1813 [1:56:03<1:17:49,  6.11s/it]
+ 58%|█████▊    | 1050/1813 [1:56:09<1:17:17,  6.08s/it]
+                                                       
+{'loss': '0.9295', 'grad_norm': '3.211', 'learning_rate': '8.728e-06', 'epoch': '0.5793', 'num_input_tokens_seen': 7051808, 'train_runtime': '6969', 'train_tokens_per_second': '1012'}
+
+ 58%|█████▊    | 1050/1813 [1:56:09<1:17:17,  6.08s/it]
+ 58%|█████▊    | 1051/1813 [1:56:14<1:12:55,  5.74s/it]
+ 58%|█████▊    | 1052/1813 [1:56:19<1:12:32,  5.72s/it]
+ 58%|█████▊    | 1053/1813 [1:56:25<1:12:48,  5.75s/it]
+ 58%|█████▊    | 1054/1813 [1:56:31<1:13:12,  5.79s/it]
+ 58%|█████▊    | 1055/1813 [1:56:37<1:12:05,  5.71s/it]
+ 58%|█████▊    | 1056/1813 [1:56:43<1:13:16,  5.81s/it]
+ 58%|█████▊    | 1057/1813 [1:56:49<1:16:15,  6.05s/it]
+ 58%|█████▊    | 1058/1813 [1:56:54<1:12:01,  5.72s/it]
+ 58%|█████▊    | 1059/1813 [1:56:59<1:07:43,  5.39s/it]
+ 58%|█████▊    | 1060/1813 [1:57:06<1:13:14,  5.84s/it]
+                                                       
+{'loss': '0.9621', 'grad_norm': '1.232', 'learning_rate': '8.541e-06', 'epoch': '0.5848', 'num_input_tokens_seen': 7110624, 'train_runtime': '7026', 'train_tokens_per_second': '1012'}
+
+ 58%|█████▊    | 1060/1813 [1:57:06<1:13:14,  5.84s/it]
+ 59%|█████▊    | 1061/1813 [1:57:11<1:09:34,  5.55s/it]
+ 59%|█████▊    | 1062/1813 [1:57:16<1:08:17,  5.46s/it]
+ 59%|█████▊    | 1063/1813 [1:57:23<1:12:54,  5.83s/it]
+ 59%|█████▊    | 1064/1813 [1:57:30<1:18:57,  6.33s/it]
+ 59%|█████▊    | 1065/1813 [1:57:37<1:19:35,  6.38s/it]
+ 59%|█████▉    | 1066/1813 [1:57:44<1:23:42,  6.72s/it]
+ 59%|█████▉    | 1067/1813 [1:57:51<1:23:00,  6.68s/it]
+ 59%|█████▉    | 1068/1813 [1:57:56<1:16:59,  6.20s/it]
+ 59%|█████▉    | 1069/1813 [1:58:04<1:23:20,  6.72s/it]
+ 59%|█████▉    | 1070/1813 [1:58:09<1:18:39,  6.35s/it]
+                                                       
+{'loss': '0.9566', 'grad_norm': '1.71', 'learning_rate': '8.355e-06', 'epoch': '0.5903', 'num_input_tokens_seen': 7179728, 'train_runtime': '7090', 'train_tokens_per_second': '1013'}
+
+ 59%|█████▉    | 1070/1813 [1:58:09<1:18:39,  6.35s/it]
+ 59%|█████▉    | 1071/1813 [1:58:14<1:14:22,  6.01s/it]
+ 59%|█████▉    | 1072/1813 [1:58:21<1:15:06,  6.08s/it]
+ 59%|█████▉    | 1073/1813 [1:58:27<1:14:25,  6.03s/it]
+ 59%|█████▉    | 1074/1813 [1:58:32<1:12:56,  5.92s/it]
+ 59%|█████▉    | 1075/1813 [1:58:39<1:14:32,  6.06s/it]
+ 59%|█████▉    | 1076/1813 [1:58:45<1:16:06,  6.20s/it]
+ 59%|█████▉    | 1077/1813 [1:58:50<1:09:50,  5.69s/it]
+ 59%|█████▉    | 1078/1813 [1:58:55<1:07:20,  5.50s/it]
+ 60%|█████▉    | 1079/1813 [1:59:01<1:10:57,  5.80s/it]
+ 60%|█████▉    | 1080/1813 [1:59:08<1:12:58,  5.97s/it]
+                                                       
+{'loss': '0.9477', 'grad_norm': '1.467', 'learning_rate': '8.169e-06', 'epoch': '0.5959', 'num_input_tokens_seen': 7240984, 'train_runtime': '7148', 'train_tokens_per_second': '1013'}
+
+ 60%|█████▉    | 1080/1813 [1:59:08<1:12:58,  5.97s/it]
+ 60%|█████▉    | 1081/1813 [1:59:14<1:13:19,  6.01s/it]
+ 60%|█████▉    | 1082/1813 [1:59:19<1:11:43,  5.89s/it]
+ 60%|█████▉    | 1083/1813 [1:59:25<1:11:41,  5.89s/it]
+ 60%|█████▉    | 1084/1813 [1:59:31<1:11:39,  5.90s/it]
+ 60%|█████▉    | 1085/1813 [1:59:37<1:11:31,  5.89s/it]
+ 60%|█████▉    | 1086/1813 [1:59:41<1:04:13,  5.30s/it]
+ 60%|█████▉    | 1087/1813 [1:59:47<1:05:24,  5.41s/it]
+ 60%|██████    | 1088/1813 [1:59:52<1:06:14,  5.48s/it]
+ 60%|██████    | 1089/1813 [1:59:59<1:09:24,  5.75s/it]
+ 60%|██████    | 1090/1813 [2:00:05<1:09:59,  5.81s/it]
+                                                       
+{'loss': '0.9017', 'grad_norm': '1.965', 'learning_rate': '7.983e-06', 'epoch': '0.6014', 'num_input_tokens_seen': 7301824, 'train_runtime': '7205', 'train_tokens_per_second': '1013'}
+
+ 60%|██████    | 1090/1813 [2:00:05<1:09:59,  5.81s/it]
+ 60%|██████    | 1091/1813 [2:00:11<1:11:17,  5.92s/it]
+ 60%|██████    | 1092/1813 [2:00:17<1:11:01,  5.91s/it]
+ 60%|██████    | 1093/1813 [2:00:22<1:09:09,  5.76s/it]
+ 60%|██████    | 1094/1813 [2:00:28<1:10:45,  5.90s/it]
+ 60%|██████    | 1095/1813 [2:00:34<1:11:22,  5.96s/it]
+ 60%|██████    | 1096/1813 [2:00:40<1:09:04,  5.78s/it]
+ 61%|██████    | 1097/1813 [2:00:46<1:09:11,  5.80s/it]
+ 61%|██████    | 1098/1813 [2:00:53<1:13:28,  6.17s/it]
+ 61%|██████    | 1099/1813 [2:00:59<1:13:04,  6.14s/it]
+ 61%|██████    | 1100/1813 [2:01:07<1:19:12,  6.67s/it]
+                                                       
+{'loss': '0.9343', 'grad_norm': '1.767', 'learning_rate': '7.798e-06', 'epoch': '0.6069', 'num_input_tokens_seen': 7369528, 'train_runtime': '7267', 'train_tokens_per_second': '1014'}
+
+ 61%|██████    | 1100/1813 [2:01:07<1:19:12,  6.67s/it]
+ 61%|██████    | 1101/1813 [2:01:13<1:18:02,  6.58s/it]
+ 61%|██████    | 1102/1813 [2:01:20<1:19:21,  6.70s/it]
+ 61%|██████    | 1103/1813 [2:01:25<1:14:29,  6.29s/it]
+ 61%|██████    | 1104/1813 [2:01:33<1:19:03,  6.69s/it]
+ 61%|██████    | 1105/1813 [2:01:38<1:12:20,  6.13s/it]
+ 61%|██████    | 1106/1813 [2:01:43<1:10:21,  5.97s/it]
+ 61%|██████    | 1107/1813 [2:01:52<1:19:03,  6.72s/it]
+ 61%|██████    | 1108/1813 [2:01:57<1:14:46,  6.36s/it]
+ 61%|██████    | 1109/1813 [2:02:04<1:14:47,  6.37s/it]
+ 61%|██████    | 1110/1813 [2:02:11<1:18:53,  6.73s/it]
+                                                       
+{'loss': '0.9729', 'grad_norm': '1.818', 'learning_rate': '7.615e-06', 'epoch': '0.6124', 'num_input_tokens_seen': 7441040, 'train_runtime': '7332', 'train_tokens_per_second': '1015'}
+
+ 61%|██████    | 1110/1813 [2:02:11<1:18:53,  6.73s/it]
+ 61%|██████▏   | 1111/1813 [2:02:16<1:11:50,  6.14s/it]
+ 61%|██████▏   | 1112/1813 [2:02:22<1:10:33,  6.04s/it]
+ 61%|██████▏   | 1113/1813 [2:02:28<1:11:07,  6.10s/it]
+ 61%|██████▏   | 1114/1813 [2:02:35<1:15:03,  6.44s/it]
+ 62%|██████▏   | 1115/1813 [2:02:42<1:16:17,  6.56s/it]
+ 62%|██████▏   | 1116/1813 [2:02:48<1:15:20,  6.49s/it]
+ 62%|██████▏   | 1117/1813 [2:02:54<1:12:31,  6.25s/it]
+ 62%|██████▏   | 1118/1813 [2:02:59<1:08:17,  5.90s/it]
+ 62%|██████▏   | 1119/1813 [2:03:06<1:09:51,  6.04s/it]
+ 62%|██████▏   | 1120/1813 [2:03:12<1:12:24,  6.27s/it]
+                                                       
+{'loss': '0.9464', 'grad_norm': '17.88', 'learning_rate': '7.432e-06', 'epoch': '0.6179', 'num_input_tokens_seen': 7504976, 'train_runtime': '7393', 'train_tokens_per_second': '1015'}
+
+ 62%|██████▏   | 1120/1813 [2:03:12<1:12:24,  6.27s/it]
+ 62%|██████▏   | 1121/1813 [2:03:20<1:18:14,  6.78s/it]
+ 62%|██████▏   | 1122/1813 [2:03:25<1:11:48,  6.23s/it]
+ 62%|██████▏   | 1123/1813 [2:03:33<1:16:42,  6.67s/it]
+ 62%|██████▏   | 1124/1813 [2:03:39<1:12:58,  6.35s/it]
+ 62%|██████▏   | 1125/1813 [2:03:45<1:14:24,  6.49s/it]
+ 62%|██████▏   | 1126/1813 [2:03:52<1:13:15,  6.40s/it]
+ 62%|██████▏   | 1127/1813 [2:03:57<1:09:28,  6.08s/it]
+ 62%|██████▏   | 1128/1813 [2:04:02<1:07:16,  5.89s/it]
+ 62%|██████▏   | 1129/1813 [2:04:08<1:05:42,  5.76s/it]
+ 62%|██████▏   | 1130/1813 [2:04:13<1:04:06,  5.63s/it]
+                                                       
+{'loss': '0.942', 'grad_norm': '1.271', 'learning_rate': '7.249e-06', 'epoch': '0.6234', 'num_input_tokens_seen': 7568576, 'train_runtime': '7454', 'train_tokens_per_second': '1015'}
+
+ 62%|██████▏   | 1130/1813 [2:04:13<1:04:06,  5.63s/it]
+ 62%|██████▏   | 1131/1813 [2:04:19<1:04:07,  5.64s/it]
+ 62%|██████▏   | 1132/1813 [2:04:25<1:04:54,  5.72s/it]
+ 62%|██████▏   | 1133/1813 [2:04:31<1:07:11,  5.93s/it]
+ 63%|██████▎   | 1134/1813 [2:04:39<1:13:40,  6.51s/it]
+ 63%|██████▎   | 1135/1813 [2:04:46<1:14:54,  6.63s/it]
+ 63%|██████▎   | 1136/1813 [2:04:53<1:15:45,  6.71s/it]
+ 63%|██████▎   | 1137/1813 [2:04:59<1:15:00,  6.66s/it]
+ 63%|██████▎   | 1138/1813 [2:05:06<1:16:27,  6.80s/it]
+ 63%|██████▎   | 1139/1813 [2:05:12<1:12:30,  6.46s/it]
+ 63%|██████▎   | 1140/1813 [2:05:18<1:09:17,  6.18s/it]
+                                                       
+{'loss': '1.003', 'grad_norm': '1.604', 'learning_rate': '7.068e-06', 'epoch': '0.629', 'num_input_tokens_seen': 7639168, 'train_runtime': '7518', 'train_tokens_per_second': '1016'}
+
+ 63%|██████▎   | 1140/1813 [2:05:18<1:09:17,  6.18s/it]
+ 63%|██████▎   | 1141/1813 [2:05:24<1:09:03,  6.17s/it]
+ 63%|██████▎   | 1142/1813 [2:05:30<1:08:32,  6.13s/it]
+ 63%|██████▎   | 1143/1813 [2:05:36<1:07:34,  6.05s/it]
+ 63%|██████▎   | 1144/1813 [2:05:43<1:10:07,  6.29s/it]
+ 63%|██████▎   | 1145/1813 [2:05:49<1:11:05,  6.39s/it]
+ 63%|██████▎   | 1146/1813 [2:05:55<1:07:52,  6.11s/it]
+ 63%|██████▎   | 1147/1813 [2:06:02<1:12:40,  6.55s/it]
+ 63%|██████▎   | 1148/1813 [2:06:09<1:14:28,  6.72s/it]
+ 63%|██████▎   | 1149/1813 [2:06:15<1:10:48,  6.40s/it]
+ 63%|██████▎   | 1150/1813 [2:06:23<1:15:07,  6.80s/it]
+                                                       
+{'loss': '0.9546', 'grad_norm': '1.429', 'learning_rate': '6.888e-06', 'epoch': '0.6345', 'num_input_tokens_seen': 7707832, 'train_runtime': '7583', 'train_tokens_per_second': '1016'}
+
+ 63%|██████▎   | 1150/1813 [2:06:23<1:15:07,  6.80s/it]
+ 63%|██████▎   | 1151/1813 [2:06:27<1:07:08,  6.09s/it]
+ 64%|██████▎   | 1152/1813 [2:06:33<1:05:34,  5.95s/it]
+ 64%|██████▎   | 1153/1813 [2:06:40<1:09:12,  6.29s/it]
+ 64%|██████▎   | 1154/1813 [2:06:46<1:08:25,  6.23s/it]
+ 64%|██████▎   | 1155/1813 [2:06:52<1:09:03,  6.30s/it]
+ 64%|██████▍   | 1156/1813 [2:06:59<1:09:00,  6.30s/it]
+ 64%|██████▍   | 1157/1813 [2:07:06<1:13:34,  6.73s/it]
+ 64%|██████▍   | 1158/1813 [2:07:13<1:13:51,  6.77s/it]
+ 64%|██████▍   | 1159/1813 [2:07:21<1:15:22,  6.92s/it]
+ 64%|██████▍   | 1160/1813 [2:07:27<1:13:12,  6.73s/it]
+                                                       
+{'loss': '0.9628', 'grad_norm': '1.139', 'learning_rate': '6.709e-06', 'epoch': '0.64', 'num_input_tokens_seen': 7775872, 'train_runtime': '7647', 'train_tokens_per_second': '1017'}
+
+ 64%|██████▍   | 1160/1813 [2:07:27<1:13:12,  6.73s/it]
+ 64%|██████▍   | 1161/1813 [2:07:34<1:15:34,  6.95s/it]
+ 64%|██████▍   | 1162/1813 [2:07:41<1:14:59,  6.91s/it]
+ 64%|██████▍   | 1163/1813 [2:07:48<1:13:11,  6.76s/it]
+ 64%|██████▍   | 1164/1813 [2:07:54<1:12:05,  6.67s/it]
+ 64%|██████▍   | 1165/1813 [2:08:01<1:11:32,  6.62s/it]
+ 64%|██████▍   | 1166/1813 [2:08:07<1:10:07,  6.50s/it]
+ 64%|██████▍   | 1167/1813 [2:08:14<1:12:05,  6.70s/it]
+ 64%|██████▍   | 1168/1813 [2:08:20<1:11:09,  6.62s/it]
+ 64%|██████▍   | 1169/1813 [2:08:27<1:09:36,  6.49s/it]
+ 65%|██████▍   | 1170/1813 [2:08:32<1:07:19,  6.28s/it]
+                                                       
+{'loss': '0.9561', 'grad_norm': '1.446', 'learning_rate': '6.532e-06', 'epoch': '0.6455', 'num_input_tokens_seen': 7846688, 'train_runtime': '7713', 'train_tokens_per_second': '1017'}
+
+ 65%|██████▍   | 1170/1813 [2:08:32<1:07:19,  6.28s/it]
+ 65%|██████▍   | 1171/1813 [2:08:40<1:11:27,  6.68s/it]
+ 65%|██████▍   | 1172/1813 [2:08:47<1:13:33,  6.89s/it]
+ 65%|██████▍   | 1173/1813 [2:08:53<1:09:43,  6.54s/it]
+ 65%|██████▍   | 1174/1813 [2:08:58<1:05:47,  6.18s/it]
+ 65%|██████▍   | 1175/1813 [2:09:04<1:05:11,  6.13s/it]
+ 65%|██████▍   | 1176/1813 [2:09:11<1:06:10,  6.23s/it]
+ 65%|██████▍   | 1177/1813 [2:09:17<1:04:32,  6.09s/it]
+ 65%|██████▍   | 1178/1813 [2:09:23<1:04:01,  6.05s/it]
+ 65%|██████▌   | 1179/1813 [2:09:29<1:04:02,  6.06s/it]
+ 65%|██████▌   | 1180/1813 [2:09:35<1:05:53,  6.25s/it]
+                                                       
+{'loss': '0.944', 'grad_norm': '1.604', 'learning_rate': '6.355e-06', 'epoch': '0.651', 'num_input_tokens_seen': 7914072, 'train_runtime': '7776', 'train_tokens_per_second': '1018'}
+
+ 65%|██████▌   | 1180/1813 [2:09:35<1:05:53,  6.25s/it]
+ 65%|██████▌   | 1181/1813 [2:09:43<1:10:40,  6.71s/it]
+ 65%|██████▌   | 1182/1813 [2:09:51<1:13:43,  7.01s/it]
+ 65%|██████▌   | 1183/1813 [2:09:58<1:13:28,  7.00s/it]
+ 65%|██████▌   | 1184/1813 [2:10:04<1:09:50,  6.66s/it]
+ 65%|██████▌   | 1185/1813 [2:10:11<1:13:07,  6.99s/it]
+ 65%|██████▌   | 1186/1813 [2:10:19<1:14:01,  7.08s/it]
+ 65%|██████▌   | 1187/1813 [2:10:26<1:13:28,  7.04s/it]
+ 66%|██████▌   | 1188/1813 [2:10:31<1:07:58,  6.53s/it]
+ 66%|██████▌   | 1189/1813 [2:10:36<1:03:36,  6.12s/it]
+ 66%|██████▌   | 1190/1813 [2:10:43<1:04:21,  6.20s/it]
+                                                       
+{'loss': '0.9904', 'grad_norm': '1.677', 'learning_rate': '6.18e-06', 'epoch': '0.6566', 'num_input_tokens_seen': 7989064, 'train_runtime': '7843', 'train_tokens_per_second': '1019'}
+
+ 66%|██████▌   | 1190/1813 [2:10:43<1:04:21,  6.20s/it]
+ 66%|██████▌   | 1191/1813 [2:10:49<1:03:45,  6.15s/it]
+ 66%|██████▌   | 1192/1813 [2:10:55<1:04:54,  6.27s/it]
+ 66%|██████▌   | 1193/1813 [2:11:01<1:03:53,  6.18s/it]
+ 66%|██████▌   | 1194/1813 [2:11:08<1:04:47,  6.28s/it]
+ 66%|██████▌   | 1195/1813 [2:11:13<1:00:39,  5.89s/it]
+ 66%|██████▌   | 1196/1813 [2:11:19<1:02:15,  6.05s/it]
+ 66%|██████▌   | 1197/1813 [2:11:25<1:02:29,  6.09s/it]
+ 66%|██████▌   | 1198/1813 [2:11:31<1:01:23,  5.99s/it]
+ 66%|██████▌   | 1199/1813 [2:11:37<1:00:45,  5.94s/it]
+ 66%|██████▌   | 1200/1813 [2:11:44<1:05:04,  6.37s/it]
+                                                       
+{'loss': '0.8943', 'grad_norm': '1.651', 'learning_rate': '6.006e-06', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352, 'train_runtime': '7905', 'train_tokens_per_second': '1019'}
+
+ 66%|██████▌   | 1200/1813 [2:11:44<1:05:04,  6.37s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.74it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.09it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.01it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.82it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.65it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.09it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.10it/s]
+
+  4%|▍         | 44/1000 [00:06<01:34, 10.16it/s]
+
+  5%|▍         | 46/1000 [00:06<01:46,  8.95it/s]
+
+  5%|▍         | 47/1000 [00:06<02:10,  7.32it/s]
+
+  5%|▍         | 49/1000 [00:06<01:49,  8.71it/s]
+
+  5%|▌         | 51/1000 [00:07<01:36,  9.86it/s]
+
+  5%|▌         | 53/1000 [00:07<01:27, 10.86it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.69it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.28it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.32it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.74it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.56it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.86it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.20it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.62it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.28it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.42it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.19it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.03it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.52it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.80it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.19it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.71it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.13it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.69it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.12it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.62it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.95it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.02it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.88it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25, 10.04it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.42it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.05it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.40it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.68it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.83it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:58,  7.01it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:16,  6.08it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:14,  6.16it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:03,  6.68it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:38,  8.36it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.77it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:36,  8.53it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.48it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.74it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.70it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.73it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.89it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.62it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.74it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.66it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.77it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.63it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.92it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.51it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:22,  9.52it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.65it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:26,  9.08it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.47it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:29,  8.74it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:04,  6.29it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:04,  6.28it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.54it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.36it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.95it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.47it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.01it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.46it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.73it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.34it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.68it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.73it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.05it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.74it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.09it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.15it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.80it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.19it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.27it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.58it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.56it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:36,  6.94it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.36it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.02it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.81it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.02it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.87it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.21it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.40it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.89it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.94it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:25,  7.62it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.54it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.59it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.43it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.59it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.54it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.91it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.77it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.21it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.18it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.15it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.65it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.38it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.79it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.46it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.91it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.23it/s]
+
+ 41%|████      | 406/1000 [00:54<01:10,  8.48it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.38it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.43it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.82it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.80it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:10,  8.23it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:18,  7.37it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:27,  6.62it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.88it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.32it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.90it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:16,  7.51it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:13,  7.82it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.52it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.01it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.79it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.48it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.07it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.24it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.00it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.83it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.27it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.81it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:05,  8.12it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.33it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:25,  6.16it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.17it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:39,  5.28it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.08it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.51it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:05,  7.94it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:11,  7.25it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:22,  6.26it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:22,  6.26it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.97it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.60it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.20it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.26it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.97it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.42it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:08<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:09<00:49,  9.90it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.26it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.52it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.09it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.05it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.38it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.58it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:12<00:48,  9.71it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.49it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:15<00:59,  7.50it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.17it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.85it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.04it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.36it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.84it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.19it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:02,  6.46it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.86it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.80it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.10it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.47it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.45it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.73it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.94it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.16it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.57it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.33it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.71it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.57it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.25it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.65it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.99it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.88it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.23it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.71it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.37it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.17it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.60it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.53it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.52it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.78it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.44it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.69it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.24it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.70it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.64it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.46it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.78it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.60it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.33it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.96it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.21it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.96it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.84it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.63it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.82it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.16it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.22it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.22it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.06it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.31it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.21it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.83it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.12it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.44it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.15it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.56it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:13, 10.23it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.53it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.62it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.70it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.27it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.29it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.59it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.62it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.85it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.87it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.08it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.60it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.78it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.16it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.09it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.64it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.62it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.16it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.84it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.07it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.15it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.73it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.85it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.41it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.44it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.43it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.80it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.44it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.37it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.52it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.04it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                       
+
+                                                   
+{'eval_loss': '0.8076', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.385', 'eval_steps_per_second': '7.385', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352}
+
+ 66%|██████▌   | 1200/1813 [2:14:00<1:05:04,  6.37s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 66%|██████▌   | 1201/1813 [2:14:05<7:56:02, 46.67s/it]
+ 66%|██████▋   | 1202/1813 [2:14:11<5:52:41, 34.63s/it]
+ 66%|██████▋   | 1203/1813 [2:14:18<4:25:54, 26.16s/it]
+ 66%|██████▋   | 1204/1813 [2:14:24<3:25:44, 20.27s/it]
+ 66%|██████▋   | 1205/1813 [2:14:31<2:43:29, 16.13s/it]
+ 67%|██████▋   | 1206/1813 [2:14:37<2:12:28, 13.09s/it]
+ 67%|██████▋   | 1207/1813 [2:14:43<1:51:49, 11.07s/it]
+ 67%|██████▋   | 1208/1813 [2:14:51<1:41:17, 10.05s/it]
+ 67%|██████▋   | 1209/1813 [2:14:58<1:31:48,  9.12s/it]
+ 67%|██████▋   | 1210/1813 [2:15:04<1:22:20,  8.19s/it]
+                                                       
+{'loss': '0.9689', 'grad_norm': '1.872', 'learning_rate': '5.833e-06', 'epoch': '0.6676', 'num_input_tokens_seen': 8125224, 'train_runtime': '8104', 'train_tokens_per_second': '1003'}
+
+ 67%|██████▋   | 1210/1813 [2:15:04<1:22:20,  8.19s/it]
+ 67%|██████▋   | 1211/1813 [2:15:11<1:18:39,  7.84s/it]
+ 67%|██████▋   | 1212/1813 [2:15:15<1:09:00,  6.89s/it]
+ 67%|██████▋   | 1213/1813 [2:15:22<1:08:05,  6.81s/it]
+ 67%|██████▋   | 1214/1813 [2:15:31<1:12:47,  7.29s/it]
+ 67%|██████▋   | 1215/1813 [2:15:37<1:10:11,  7.04s/it]
+ 67%|██████▋   | 1216/1813 [2:15:44<1:10:45,  7.11s/it]
+ 67%|██████▋   | 1217/1813 [2:15:51<1:08:09,  6.86s/it]
+ 67%|██████▋   | 1218/1813 [2:15:57<1:06:48,  6.74s/it]
+ 67%|██████▋   | 1219/1813 [2:16:03<1:03:18,  6.39s/it]
+ 67%|██████▋   | 1220/1813 [2:16:11<1:08:49,  6.96s/it]
+                                                       
+{'loss': '1.006', 'grad_norm': '9.92', 'learning_rate': '5.662e-06', 'epoch': '0.6731', 'num_input_tokens_seen': 8197488, 'train_runtime': '8171', 'train_tokens_per_second': '1003'}
+
+ 67%|██████▋   | 1220/1813 [2:16:11<1:08:49,  6.96s/it]
+ 67%|██████▋   | 1221/1813 [2:16:16<1:04:34,  6.54s/it]
+ 67%|██████▋   | 1222/1813 [2:16:22<1:00:37,  6.15s/it]
+ 67%|██████▋   | 1223/1813 [2:16:28<59:32,  6.05s/it]  
+ 68%|██████▊   | 1224/1813 [2:16:32<56:12,  5.73s/it]
+ 68%|██████▊   | 1225/1813 [2:16:38<56:50,  5.80s/it]
+ 68%|██████▊   | 1226/1813 [2:16:44<57:30,  5.88s/it]
+ 68%|██████▊   | 1227/1813 [2:16:51<59:04,  6.05s/it]
+ 68%|██████▊   | 1228/1813 [2:16:56<56:08,  5.76s/it]
+ 68%|██████▊   | 1229/1813 [2:17:02<57:53,  5.95s/it]
+ 68%|██████▊   | 1230/1813 [2:17:08<57:29,  5.92s/it]
+                                                     
+{'loss': '0.8639', 'grad_norm': '1.422', 'learning_rate': '5.493e-06', 'epoch': '0.6786', 'num_input_tokens_seen': 8257288, 'train_runtime': '8229', 'train_tokens_per_second': '1003'}
+
+ 68%|██████▊   | 1230/1813 [2:17:08<57:29,  5.92s/it]
+ 68%|██████▊   | 1231/1813 [2:17:16<1:01:45,  6.37s/it]
+ 68%|██████▊   | 1232/1813 [2:17:23<1:04:12,  6.63s/it]
+ 68%|██████▊   | 1233/1813 [2:17:29<1:02:35,  6.47s/it]
+ 68%|██████▊   | 1234/1813 [2:17:35<1:00:44,  6.29s/it]
+ 68%|██████▊   | 1235/1813 [2:17:41<1:00:04,  6.24s/it]
+ 68%|██████▊   | 1236/1813 [2:17:49<1:06:24,  6.91s/it]
+ 68%|██████▊   | 1237/1813 [2:17:56<1:04:28,  6.72s/it]
+ 68%|██████▊   | 1238/1813 [2:18:01<1:00:29,  6.31s/it]
+ 68%|██████▊   | 1239/1813 [2:18:06<57:16,  5.99s/it]  
+ 68%|██████▊   | 1240/1813 [2:18:11<54:35,  5.72s/it]
+                                                     
+{'loss': '0.8751', 'grad_norm': '1.56', 'learning_rate': '5.325e-06', 'epoch': '0.6841', 'num_input_tokens_seen': 8327600, 'train_runtime': '8292', 'train_tokens_per_second': '1004'}
+
+ 68%|██████▊   | 1240/1813 [2:18:11<54:35,  5.72s/it]
+ 68%|██████▊   | 1241/1813 [2:18:17<53:48,  5.64s/it]
+ 69%|██████▊   | 1242/1813 [2:18:23<54:29,  5.73s/it]
+ 69%|██████▊   | 1243/1813 [2:18:29<55:35,  5.85s/it]
+ 69%|██████▊   | 1244/1813 [2:18:36<58:09,  6.13s/it]
+ 69%|██████▊   | 1245/1813 [2:18:43<1:01:05,  6.45s/it]
+ 69%|██████▊   | 1246/1813 [2:18:49<59:25,  6.29s/it]  
+ 69%|██████▉   | 1247/1813 [2:18:56<1:00:40,  6.43s/it]
+ 69%|██████▉   | 1248/1813 [2:19:02<1:01:28,  6.53s/it]
+ 69%|██████▉   | 1249/1813 [2:19:09<1:01:05,  6.50s/it]
+ 69%|██████▉   | 1250/1813 [2:19:14<56:26,  6.02s/it]  
+                                                     
+{'loss': '1.006', 'grad_norm': '2.001', 'learning_rate': '5.159e-06', 'epoch': '0.6897', 'num_input_tokens_seen': 8394824, 'train_runtime': '8354', 'train_tokens_per_second': '1005'}
+
+ 69%|██████▉   | 1250/1813 [2:19:14<56:26,  6.02s/it]
+ 69%|██████▉   | 1251/1813 [2:19:20<56:44,  6.06s/it]
+ 69%|██████▉   | 1252/1813 [2:19:26<55:56,  5.98s/it]
+ 69%|██████▉   | 1253/1813 [2:19:31<55:26,  5.94s/it]
+ 69%|██████▉   | 1254/1813 [2:19:38<57:07,  6.13s/it]
+ 69%|██████▉   | 1255/1813 [2:19:44<56:45,  6.10s/it]
+ 69%|██████▉   | 1256/1813 [2:19:50<55:59,  6.03s/it]
+ 69%|██████▉   | 1257/1813 [2:19:55<54:18,  5.86s/it]
+ 69%|██████▉   | 1258/1813 [2:20:02<56:18,  6.09s/it]
+ 69%|██████▉   | 1259/1813 [2:20:09<57:22,  6.21s/it]
+ 69%|██████▉   | 1260/1813 [2:20:15<59:14,  6.43s/it]
+                                                     
+{'loss': '0.948', 'grad_norm': '2.11', 'learning_rate': '4.995e-06', 'epoch': '0.6952', 'num_input_tokens_seen': 8461096, 'train_runtime': '8416', 'train_tokens_per_second': '1005'}
+
+ 69%|██████▉   | 1260/1813 [2:20:15<59:14,  6.43s/it]
+ 70%|██████▉   | 1261/1813 [2:20:24<1:03:39,  6.92s/it]
+ 70%|██████▉   | 1262/1813 [2:20:31<1:05:30,  7.13s/it]
+ 70%|██████▉   | 1263/1813 [2:20:39<1:06:05,  7.21s/it]
+ 70%|██████▉   | 1264/1813 [2:20:44<1:01:48,  6.75s/it]
+ 70%|██████▉   | 1265/1813 [2:20:49<57:12,  6.26s/it]  
+ 70%|██████▉   | 1266/1813 [2:20:57<1:01:00,  6.69s/it]
+ 70%|██████▉   | 1267/1813 [2:21:02<56:13,  6.18s/it]  
+ 70%|██████▉   | 1268/1813 [2:21:07<52:18,  5.76s/it]
+ 70%|██████▉   | 1269/1813 [2:21:12<50:35,  5.58s/it]
+ 70%|███████   | 1270/1813 [2:21:18<51:52,  5.73s/it]
+                                                     
+{'loss': '0.9557', 'grad_norm': '1.872', 'learning_rate': '4.832e-06', 'epoch': '0.7007', 'num_input_tokens_seen': 8529224, 'train_runtime': '8479', 'train_tokens_per_second': '1006'}
+
+ 70%|███████   | 1270/1813 [2:21:18<51:52,  5.73s/it]
+ 70%|███████   | 1271/1813 [2:21:23<50:07,  5.55s/it]
+ 70%|███████   | 1272/1813 [2:21:30<54:07,  6.00s/it]
+ 70%|███████   | 1273/1813 [2:21:37<56:26,  6.27s/it]
+ 70%|███████   | 1274/1813 [2:21:44<58:57,  6.56s/it]
+ 70%|███████   | 1275/1813 [2:21:51<59:12,  6.60s/it]
+ 70%|███████   | 1276/1813 [2:21:57<56:16,  6.29s/it]
+ 70%|███████   | 1277/1813 [2:22:04<58:07,  6.51s/it]
+ 70%|███████   | 1278/1813 [2:22:09<55:06,  6.18s/it]
+ 71%|███████   | 1279/1813 [2:22:14<50:57,  5.73s/it]
+ 71%|███████   | 1280/1813 [2:22:21<54:17,  6.11s/it]
+                                                     
+{'loss': '1.012', 'grad_norm': '1.767', 'learning_rate': '4.671e-06', 'epoch': '0.7062', 'num_input_tokens_seen': 8596456, 'train_runtime': '8541', 'train_tokens_per_second': '1006'}
+
+ 71%|███████   | 1280/1813 [2:22:21<54:17,  6.11s/it]
+ 71%|███████   | 1281/1813 [2:22:26<52:15,  5.89s/it]
+ 71%|███████   | 1282/1813 [2:22:33<53:29,  6.04s/it]
+ 71%|███████   | 1283/1813 [2:22:39<55:06,  6.24s/it]
+ 71%|███████   | 1284/1813 [2:22:45<54:02,  6.13s/it]
+ 71%|███████   | 1285/1813 [2:22:50<51:48,  5.89s/it]
+ 71%|███████   | 1286/1813 [2:22:58<55:32,  6.32s/it]
+ 71%|███████   | 1287/1813 [2:23:04<55:07,  6.29s/it]
+ 71%|███████   | 1288/1813 [2:23:09<52:50,  6.04s/it]
+ 71%|███████   | 1289/1813 [2:23:16<55:05,  6.31s/it]
+ 71%|███████   | 1290/1813 [2:23:24<57:23,  6.58s/it]
+                                                     
+{'loss': '1.026', 'grad_norm': '1.149', 'learning_rate': '4.512e-06', 'epoch': '0.7117', 'num_input_tokens_seen': 8662472, 'train_runtime': '8604', 'train_tokens_per_second': '1007'}
+
+ 71%|███████   | 1290/1813 [2:23:24<57:23,  6.58s/it]
+ 71%|███████   | 1291/1813 [2:23:30<56:25,  6.49s/it]
+ 71%|███████▏  | 1292/1813 [2:23:37<57:09,  6.58s/it]
+ 71%|███████▏  | 1293/1813 [2:23:44<58:34,  6.76s/it]
+ 71%|███████▏  | 1294/1813 [2:23:50<57:31,  6.65s/it]
+ 71%|███████▏  | 1295/1813 [2:23:56<55:45,  6.46s/it]
+ 71%|███████▏  | 1296/1813 [2:24:02<52:54,  6.14s/it]
+ 72%|███████▏  | 1297/1813 [2:24:07<50:35,  5.88s/it]
+ 72%|███████▏  | 1298/1813 [2:24:14<52:39,  6.13s/it]
+ 72%|███████▏  | 1299/1813 [2:24:21<54:55,  6.41s/it]
+ 72%|███████▏  | 1300/1813 [2:24:27<54:00,  6.32s/it]
+                                                     
+{'loss': '0.9216', 'grad_norm': '1.832', 'learning_rate': '4.355e-06', 'epoch': '0.7172', 'num_input_tokens_seen': 8730600, 'train_runtime': '8667', 'train_tokens_per_second': '1007'}
+
+ 72%|███████▏  | 1300/1813 [2:24:27<54:00,  6.32s/it]
+ 72%|███████▏  | 1301/1813 [2:24:33<53:11,  6.23s/it]
+ 72%|███████▏  | 1302/1813 [2:24:39<53:56,  6.33s/it]
+ 72%|███████▏  | 1303/1813 [2:24:47<57:09,  6.72s/it]
+ 72%|███████▏  | 1304/1813 [2:24:52<53:43,  6.33s/it]
+ 72%|███████▏  | 1305/1813 [2:25:00<56:31,  6.68s/it]
+ 72%|███████▏  | 1306/1813 [2:25:06<53:53,  6.38s/it]
+ 72%|███████▏  | 1307/1813 [2:25:12<53:21,  6.33s/it]
+ 72%|███████▏  | 1308/1813 [2:25:18<53:27,  6.35s/it]
+ 72%|███████▏  | 1309/1813 [2:25:24<53:02,  6.31s/it]
+ 72%|███████▏  | 1310/1813 [2:25:31<54:29,  6.50s/it]
+                                                     
+{'loss': '1.006', 'grad_norm': '1.317', 'learning_rate': '4.2e-06', 'epoch': '0.7228', 'num_input_tokens_seen': 8802000, 'train_runtime': '8732', 'train_tokens_per_second': '1008'}
+
+ 72%|███████▏  | 1310/1813 [2:25:31<54:29,  6.50s/it]
+ 72%|███████▏  | 1311/1813 [2:25:38<53:29,  6.39s/it]
+ 72%|███████▏  | 1312/1813 [2:25:43<51:29,  6.17s/it]
+ 72%|███████▏  | 1313/1813 [2:25:49<50:49,  6.10s/it]
+ 72%|███████▏  | 1314/1813 [2:25:55<51:07,  6.15s/it]
+ 73%|███████▎  | 1315/1813 [2:26:01<50:09,  6.04s/it]
+ 73%|███████▎  | 1316/1813 [2:26:07<49:48,  6.01s/it]
+ 73%|███████▎  | 1317/1813 [2:26:13<50:10,  6.07s/it]
+ 73%|███████▎  | 1318/1813 [2:26:19<49:12,  5.96s/it]
+ 73%|███████▎  | 1319/1813 [2:26:26<50:49,  6.17s/it]
+ 73%|███████▎  | 1320/1813 [2:26:32<52:12,  6.35s/it]
+                                                     
+{'loss': '1.006', 'grad_norm': '1.538', 'learning_rate': '4.048e-06', 'epoch': '0.7283', 'num_input_tokens_seen': 8866824, 'train_runtime': '8793', 'train_tokens_per_second': '1008'}
+
+ 73%|███████▎  | 1320/1813 [2:26:32<52:12,  6.35s/it]
+ 73%|███████▎  | 1321/1813 [2:26:39<53:13,  6.49s/it]
+ 73%|███████▎  | 1322/1813 [2:26:45<51:53,  6.34s/it]
+ 73%|███████▎  | 1323/1813 [2:26:52<52:32,  6.43s/it]
+ 73%|███████▎  | 1324/1813 [2:26:58<51:10,  6.28s/it]
+ 73%|███████▎  | 1325/1813 [2:27:04<49:41,  6.11s/it]
+ 73%|███████▎  | 1326/1813 [2:27:10<49:52,  6.14s/it]
+ 73%|███████▎  | 1327/1813 [2:27:17<53:30,  6.61s/it]
+ 73%|███████▎  | 1328/1813 [2:27:24<54:07,  6.70s/it]
+ 73%|███████▎  | 1329/1813 [2:27:31<52:40,  6.53s/it]
+ 73%|███████▎  | 1330/1813 [2:27:36<50:44,  6.30s/it]
+                                                     
+{'loss': '0.9392', 'grad_norm': '1.714', 'learning_rate': '3.897e-06', 'epoch': '0.7338', 'num_input_tokens_seen': 8935584, 'train_runtime': '8857', 'train_tokens_per_second': '1009'}
+
+ 73%|███████▎  | 1330/1813 [2:27:36<50:44,  6.30s/it]
+ 73%|███████▎  | 1331/1813 [2:27:42<49:03,  6.11s/it]
+ 73%|███████▎  | 1332/1813 [2:27:48<48:03,  5.99s/it]
+ 74%|███████▎  | 1333/1813 [2:27:53<47:24,  5.93s/it]
+ 74%|███████▎  | 1334/1813 [2:27:59<46:03,  5.77s/it]
+ 74%|███████▎  | 1335/1813 [2:28:04<45:32,  5.72s/it]
+ 74%|███████▎  | 1336/1813 [2:28:11<46:57,  5.91s/it]
+ 74%|███████▎  | 1337/1813 [2:28:18<50:49,  6.41s/it]
+ 74%|███████▍  | 1338/1813 [2:28:26<52:26,  6.63s/it]
+ 74%|███████▍  | 1339/1813 [2:28:31<48:46,  6.17s/it]
+ 74%|███████▍  | 1340/1813 [2:28:38<50:37,  6.42s/it]
+                                                     
+{'loss': '1.022', 'grad_norm': '1.407', 'learning_rate': '3.748e-06', 'epoch': '0.7393', 'num_input_tokens_seen': 9000416, 'train_runtime': '8918', 'train_tokens_per_second': '1009'}
+
+ 74%|███████▍  | 1340/1813 [2:28:38<50:37,  6.42s/it]
+ 74%|███████▍  | 1341/1813 [2:28:44<51:00,  6.48s/it]
+ 74%|███████▍  | 1342/1813 [2:28:50<47:58,  6.11s/it]
+ 74%|███████▍  | 1343/1813 [2:28:55<46:26,  5.93s/it]
+ 74%|███████▍  | 1344/1813 [2:29:01<46:33,  5.96s/it]
+ 74%|███████▍  | 1345/1813 [2:29:09<50:16,  6.45s/it]
+ 74%|███████▍  | 1346/1813 [2:29:15<50:02,  6.43s/it]
+ 74%|███████▍  | 1347/1813 [2:29:21<48:15,  6.21s/it]
+ 74%|███████▍  | 1348/1813 [2:29:27<48:11,  6.22s/it]
+ 74%|███████▍  | 1349/1813 [2:29:34<48:52,  6.32s/it]
+ 74%|███████▍  | 1350/1813 [2:29:39<47:13,  6.12s/it]
+                                                     
+{'loss': '0.9816', 'grad_norm': '2.242', 'learning_rate': '3.602e-06', 'epoch': '0.7448', 'num_input_tokens_seen': 9066864, 'train_runtime': '8980', 'train_tokens_per_second': '1010'}
+
+ 74%|███████▍  | 1350/1813 [2:29:39<47:13,  6.12s/it]
+ 75%|███████▍  | 1351/1813 [2:29:47<51:08,  6.64s/it]
+ 75%|███████▍  | 1352/1813 [2:29:53<50:10,  6.53s/it]
+ 75%|███████▍  | 1353/1813 [2:30:00<50:42,  6.61s/it]
+ 75%|███████▍  | 1354/1813 [2:30:07<51:00,  6.67s/it]
+ 75%|███████▍  | 1355/1813 [2:30:15<55:02,  7.21s/it]
+ 75%|███████▍  | 1356/1813 [2:30:21<51:30,  6.76s/it]
+ 75%|███████▍  | 1357/1813 [2:30:27<49:37,  6.53s/it]
+ 75%|███████▍  | 1358/1813 [2:30:33<49:15,  6.50s/it]
+ 75%|███████▍  | 1359/1813 [2:30:39<47:30,  6.28s/it]
+ 75%|███████▌  | 1360/1813 [2:30:45<46:47,  6.20s/it]
+                                                     
+{'loss': '0.9603', 'grad_norm': '2.247', 'learning_rate': '3.458e-06', 'epoch': '0.7503', 'num_input_tokens_seen': 9140520, 'train_runtime': '9046', 'train_tokens_per_second': '1010'}
+
+ 75%|███████▌  | 1360/1813 [2:30:45<46:47,  6.20s/it]
+ 75%|███████▌  | 1361/1813 [2:30:52<47:49,  6.35s/it]
+ 75%|███████▌  | 1362/1813 [2:30:59<48:12,  6.41s/it]
+ 75%|███████▌  | 1363/1813 [2:31:06<50:37,  6.75s/it]
+ 75%|███████▌  | 1364/1813 [2:31:12<49:15,  6.58s/it]
+ 75%|███████▌  | 1365/1813 [2:31:20<51:11,  6.85s/it]
+ 75%|███████▌  | 1366/1813 [2:31:26<49:27,  6.64s/it]
+ 75%|███████▌  | 1367/1813 [2:31:33<49:32,  6.66s/it]
+ 75%|███████▌  | 1368/1813 [2:31:38<46:15,  6.24s/it]
+ 76%|███████▌  | 1369/1813 [2:31:46<50:20,  6.80s/it]
+ 76%|███████▌  | 1370/1813 [2:31:52<48:50,  6.61s/it]
+                                                     
+{'loss': '0.9485', 'grad_norm': '1.725', 'learning_rate': '3.316e-06', 'epoch': '0.7559', 'num_input_tokens_seen': 9213048, 'train_runtime': '9113', 'train_tokens_per_second': '1011'}
+
+ 76%|███████▌  | 1370/1813 [2:31:52<48:50,  6.61s/it]
+ 76%|███████▌  | 1371/1813 [2:32:00<52:09,  7.08s/it]
+ 76%|███████▌  | 1372/1813 [2:32:06<47:53,  6.52s/it]
+ 76%|███████▌  | 1373/1813 [2:32:12<47:11,  6.44s/it]
+ 76%|███████▌  | 1374/1813 [2:32:18<46:14,  6.32s/it]
+ 76%|███████▌  | 1375/1813 [2:32:24<44:52,  6.15s/it]
+ 76%|███████▌  | 1376/1813 [2:32:29<43:20,  5.95s/it]
+ 76%|███████▌  | 1377/1813 [2:32:35<43:53,  6.04s/it]
+ 76%|███████▌  | 1378/1813 [2:32:41<42:42,  5.89s/it]
+ 76%|███████▌  | 1379/1813 [2:32:47<42:19,  5.85s/it]
+ 76%|███████▌  | 1380/1813 [2:32:54<44:31,  6.17s/it]
+                                                     
+{'loss': '0.9339', 'grad_norm': '1.861', 'learning_rate': '3.177e-06', 'epoch': '0.7614', 'num_input_tokens_seen': 9279088, 'train_runtime': '9174', 'train_tokens_per_second': '1011'}
+
+ 76%|███████▌  | 1380/1813 [2:32:54<44:31,  6.17s/it]
+ 76%|███████▌  | 1381/1813 [2:33:00<46:03,  6.40s/it]
+ 76%|███████▌  | 1382/1813 [2:33:06<43:43,  6.09s/it]
+ 76%|███████▋  | 1383/1813 [2:33:12<43:08,  6.02s/it]
+ 76%|███████▋  | 1384/1813 [2:33:17<42:14,  5.91s/it]
+ 76%|███████▋  | 1385/1813 [2:33:24<44:08,  6.19s/it]
+ 76%|███████▋  | 1386/1813 [2:33:31<44:35,  6.27s/it]
+ 77%|███████▋  | 1387/1813 [2:33:38<46:07,  6.50s/it]
+ 77%|███████▋  | 1388/1813 [2:33:43<43:33,  6.15s/it]
+ 77%|███████▋  | 1389/1813 [2:33:47<39:31,  5.59s/it]
+ 77%|███████▋  | 1390/1813 [2:33:54<41:43,  5.92s/it]
+                                                     
+{'loss': '0.9885', 'grad_norm': '1.879', 'learning_rate': '3.04e-06', 'epoch': '0.7669', 'num_input_tokens_seen': 9344880, 'train_runtime': '9234', 'train_tokens_per_second': '1012'}
+
+ 77%|███████▋  | 1390/1813 [2:33:54<41:43,  5.92s/it]
+ 77%|███████▋  | 1391/1813 [2:33:59<40:06,  5.70s/it]
+ 77%|███████▋  | 1392/1813 [2:34:06<41:50,  5.96s/it]
+ 77%|███████▋  | 1393/1813 [2:34:11<39:58,  5.71s/it]
+ 77%|███████▋  | 1394/1813 [2:34:17<41:42,  5.97s/it]
+ 77%|███████▋  | 1395/1813 [2:34:24<42:09,  6.05s/it]
+ 77%|███████▋  | 1396/1813 [2:34:30<41:55,  6.03s/it]
+ 77%|███████▋  | 1397/1813 [2:34:36<43:22,  6.26s/it]
+ 77%|███████▋  | 1398/1813 [2:34:41<39:50,  5.76s/it]
+ 77%|███████▋  | 1399/1813 [2:34:46<37:47,  5.48s/it]
+ 77%|███████▋  | 1400/1813 [2:34:53<40:10,  5.84s/it]
+                                                     
+{'loss': '0.9126', 'grad_norm': '1.179', 'learning_rate': '2.906e-06', 'epoch': '0.7724', 'num_input_tokens_seen': 9408280, 'train_runtime': '9293', 'train_tokens_per_second': '1012'}
+
+ 77%|███████▋  | 1400/1813 [2:34:53<40:10,  5.84s/it]
+ 77%|███████▋  | 1401/1813 [2:34:58<39:29,  5.75s/it]
+ 77%|███████▋  | 1402/1813 [2:35:04<40:10,  5.86s/it]
+ 77%|███████▋  | 1403/1813 [2:35:11<41:13,  6.03s/it]
+ 77%|███████▋  | 1404/1813 [2:35:17<41:23,  6.07s/it]
+ 77%|███████▋  | 1405/1813 [2:35:21<38:27,  5.66s/it]
+ 78%|███████▊  | 1406/1813 [2:35:27<38:10,  5.63s/it]
+ 78%|███████▊  | 1407/1813 [2:35:34<39:51,  5.89s/it]
+ 78%|███████▊  | 1408/1813 [2:35:39<38:10,  5.65s/it]
+ 78%|███████▊  | 1409/1813 [2:35:43<35:51,  5.33s/it]
+ 78%|███████▊  | 1410/1813 [2:35:49<36:25,  5.42s/it]
+                                                     
+{'loss': '0.9878', 'grad_norm': '2.114', 'learning_rate': '2.774e-06', 'epoch': '0.7779', 'num_input_tokens_seen': 9467408, 'train_runtime': '9349', 'train_tokens_per_second': '1013'}
+
+ 78%|███████▊  | 1410/1813 [2:35:49<36:25,  5.42s/it]
+ 78%|███████▊  | 1411/1813 [2:35:54<36:02,  5.38s/it]
+ 78%|███████▊  | 1412/1813 [2:36:00<37:51,  5.66s/it]
+ 78%|███████▊  | 1413/1813 [2:36:06<37:34,  5.64s/it]
+ 78%|███████▊  | 1414/1813 [2:36:12<38:31,  5.79s/it]
+ 78%|███████▊  | 1415/1813 [2:36:19<40:53,  6.17s/it]
+ 78%|███████▊  | 1416/1813 [2:36:25<40:43,  6.16s/it]
+ 78%|███████▊  | 1417/1813 [2:36:31<38:59,  5.91s/it]
+ 78%|███████▊  | 1418/1813 [2:36:36<38:37,  5.87s/it]
+ 78%|███████▊  | 1419/1813 [2:36:45<43:53,  6.68s/it]
+ 78%|███████▊  | 1420/1813 [2:36:49<39:20,  6.01s/it]
+                                                     
+{'loss': '0.9281', 'grad_norm': '1.699', 'learning_rate': '2.644e-06', 'epoch': '0.7834', 'num_input_tokens_seen': 9532720, 'train_runtime': '9410', 'train_tokens_per_second': '1013'}
+
+ 78%|███████▊  | 1420/1813 [2:36:49<39:20,  6.01s/it]
+ 78%|███████▊  | 1421/1813 [2:36:56<40:46,  6.24s/it]
+ 78%|███████▊  | 1422/1813 [2:37:02<39:03,  5.99s/it]
+ 78%|███████▊  | 1423/1813 [2:37:07<37:59,  5.84s/it]
+ 79%|███████▊  | 1424/1813 [2:37:14<40:38,  6.27s/it]
+ 79%|███████▊  | 1425/1813 [2:37:19<37:09,  5.75s/it]
+ 79%|███████▊  | 1426/1813 [2:37:27<42:27,  6.58s/it]
+ 79%|███████▊  | 1427/1813 [2:37:33<39:59,  6.22s/it]
+ 79%|███████▉  | 1428/1813 [2:37:39<40:29,  6.31s/it]
+ 79%|███████▉  | 1429/1813 [2:37:46<40:02,  6.26s/it]
+ 79%|███████▉  | 1430/1813 [2:37:52<41:00,  6.42s/it]
+                                                     
+{'loss': '0.9384', 'grad_norm': '1.819', 'learning_rate': '2.518e-06', 'epoch': '0.789', 'num_input_tokens_seen': 9602112, 'train_runtime': '9473', 'train_tokens_per_second': '1014'}
+
+ 79%|███████▉  | 1430/1813 [2:37:52<41:00,  6.42s/it]
+ 79%|███████▉  | 1431/1813 [2:37:59<40:34,  6.37s/it]
+ 79%|███████▉  | 1432/1813 [2:38:05<39:50,  6.28s/it]
+ 79%|███████▉  | 1433/1813 [2:38:11<39:53,  6.30s/it]
+ 79%|███████▉  | 1434/1813 [2:38:17<40:02,  6.34s/it]
+ 79%|███████▉  | 1435/1813 [2:38:23<38:44,  6.15s/it]
+ 79%|███████▉  | 1436/1813 [2:38:30<40:14,  6.40s/it]
+ 79%|███████▉  | 1437/1813 [2:38:36<39:01,  6.23s/it]
+ 79%|███████▉  | 1438/1813 [2:38:43<41:08,  6.58s/it]
+ 79%|███████▉  | 1439/1813 [2:38:49<39:20,  6.31s/it]
+ 79%|███████▉  | 1440/1813 [2:38:54<37:18,  6.00s/it]
+                                                     
+{'loss': '0.9421', 'grad_norm': '1.957', 'learning_rate': '2.394e-06', 'epoch': '0.7945', 'num_input_tokens_seen': 9667112, 'train_runtime': '9535', 'train_tokens_per_second': '1014'}
+
+ 79%|███████▉  | 1440/1813 [2:38:54<37:18,  6.00s/it]
+ 79%|███████▉  | 1441/1813 [2:39:01<38:09,  6.16s/it]
+ 80%|███████▉  | 1442/1813 [2:39:06<36:16,  5.87s/it]
+ 80%|███████▉  | 1443/1813 [2:39:11<34:31,  5.60s/it]
+ 80%|███████▉  | 1444/1813 [2:39:18<36:46,  5.98s/it]
+ 80%|███████▉  | 1445/1813 [2:39:24<36:05,  5.88s/it]
+ 80%|███████▉  | 1446/1813 [2:39:29<35:07,  5.74s/it]
+ 80%|███████▉  | 1447/1813 [2:39:37<39:37,  6.50s/it]
+ 80%|███████▉  | 1448/1813 [2:39:45<41:11,  6.77s/it]
+ 80%|███████▉  | 1449/1813 [2:39:51<40:41,  6.71s/it]
+ 80%|███████▉  | 1450/1813 [2:39:58<40:10,  6.64s/it]
+                                                     
+{'loss': '1.026', 'grad_norm': '1.678', 'learning_rate': '2.273e-06', 'epoch': '0.8', 'num_input_tokens_seen': 9735792, 'train_runtime': '9598', 'train_tokens_per_second': '1014'}
+
+ 80%|███████▉  | 1450/1813 [2:39:58<40:10,  6.64s/it]
+ 80%|████████  | 1451/1813 [2:40:05<40:39,  6.74s/it]
+ 80%|████████  | 1452/1813 [2:40:09<36:29,  6.06s/it]
+ 80%|████████  | 1453/1813 [2:40:17<40:25,  6.74s/it]
+ 80%|████████  | 1454/1813 [2:40:23<38:06,  6.37s/it]
+ 80%|████████  | 1455/1813 [2:40:29<37:56,  6.36s/it]
+ 80%|████████  | 1456/1813 [2:40:37<39:26,  6.63s/it]
+ 80%|████████  | 1457/1813 [2:40:41<34:49,  5.87s/it]
+ 80%|████████  | 1458/1813 [2:40:47<34:47,  5.88s/it]
+ 80%|████████  | 1459/1813 [2:40:54<36:50,  6.25s/it]
+ 81%|████████  | 1460/1813 [2:41:01<38:51,  6.60s/it]
+                                                     
+{'loss': '0.9611', 'grad_norm': '1.585', 'learning_rate': '2.154e-06', 'epoch': '0.8055', 'num_input_tokens_seen': 9802968, 'train_runtime': '9662', 'train_tokens_per_second': '1015'}
+
+ 81%|████████  | 1460/1813 [2:41:01<38:51,  6.60s/it]
+ 81%|████████  | 1461/1813 [2:41:07<37:49,  6.45s/it]
+ 81%|████████  | 1462/1813 [2:41:13<35:57,  6.15s/it]
+ 81%|████████  | 1463/1813 [2:41:20<38:12,  6.55s/it]
+ 81%|████████  | 1464/1813 [2:41:27<38:30,  6.62s/it]
+ 81%|████████  | 1465/1813 [2:41:35<41:13,  7.11s/it]
+ 81%|████████  | 1466/1813 [2:41:41<38:27,  6.65s/it]
+ 81%|████████  | 1467/1813 [2:41:48<39:37,  6.87s/it]
+ 81%|████████  | 1468/1813 [2:41:55<39:27,  6.86s/it]
+ 81%|████████  | 1469/1813 [2:42:04<42:43,  7.45s/it]
+ 81%|████████  | 1470/1813 [2:42:10<41:23,  7.24s/it]
+                                                     
+{'loss': '0.9531', 'grad_norm': '1.71', 'learning_rate': '2.038e-06', 'epoch': '0.811', 'num_input_tokens_seen': 9879232, 'train_runtime': '9731', 'train_tokens_per_second': '1015'}
+
+ 81%|████████  | 1470/1813 [2:42:10<41:23,  7.24s/it]
+ 81%|████████  | 1471/1813 [2:42:17<39:37,  6.95s/it]
+ 81%|████████  | 1472/1813 [2:42:24<39:16,  6.91s/it]
+ 81%|████████  | 1473/1813 [2:42:29<37:06,  6.55s/it]
+ 81%|████████▏ | 1474/1813 [2:42:36<37:05,  6.56s/it]
+ 81%|████████▏ | 1475/1813 [2:42:42<36:18,  6.44s/it]
+ 81%|████████▏ | 1476/1813 [2:42:48<35:50,  6.38s/it]
+ 81%|████████▏ | 1477/1813 [2:42:55<35:31,  6.34s/it]
+ 82%|████████▏ | 1478/1813 [2:43:00<34:27,  6.17s/it]
+ 82%|████████▏ | 1479/1813 [2:43:07<34:34,  6.21s/it]
+ 82%|████████▏ | 1480/1813 [2:43:14<36:03,  6.50s/it]
+                                                     
+{'loss': '0.9342', 'grad_norm': '1.206', 'learning_rate': '1.925e-06', 'epoch': '0.8166', 'num_input_tokens_seen': 9948032, 'train_runtime': '9794', 'train_tokens_per_second': '1016'}
+
+ 82%|████████▏ | 1480/1813 [2:43:14<36:03,  6.50s/it]
+ 82%|████████▏ | 1481/1813 [2:43:21<36:24,  6.58s/it]
+ 82%|████████▏ | 1482/1813 [2:43:26<34:46,  6.30s/it]
+ 82%|████████▏ | 1483/1813 [2:43:33<35:25,  6.44s/it]
+ 82%|████████▏ | 1484/1813 [2:43:39<35:16,  6.43s/it]
+ 82%|████████▏ | 1485/1813 [2:43:46<35:45,  6.54s/it]
+ 82%|████████▏ | 1486/1813 [2:43:52<33:50,  6.21s/it]
+ 82%|████████▏ | 1487/1813 [2:44:00<36:42,  6.76s/it]
+ 82%|████████▏ | 1488/1813 [2:44:06<36:20,  6.71s/it]
+ 82%|████████▏ | 1489/1813 [2:44:14<37:24,  6.93s/it]
+ 82%|████████▏ | 1490/1813 [2:44:21<37:10,  6.91s/it]
+                                                     
+{'loss': '1.015', 'grad_norm': '2.129', 'learning_rate': '1.815e-06', 'epoch': '0.8221', 'num_input_tokens_seen': 10021400, 'train_runtime': '9861', 'train_tokens_per_second': '1016'}
+
+ 82%|████████▏ | 1490/1813 [2:44:21<37:10,  6.91s/it]
+ 82%|████████▏ | 1491/1813 [2:44:26<35:12,  6.56s/it]
+ 82%|████████▏ | 1492/1813 [2:44:35<37:50,  7.07s/it]
+ 82%|████████▏ | 1493/1813 [2:44:41<36:06,  6.77s/it]
+ 82%|████████▏ | 1494/1813 [2:44:47<34:54,  6.57s/it]
+ 82%|████████▏ | 1495/1813 [2:44:55<37:06,  7.00s/it]
+ 83%|████████▎ | 1496/1813 [2:45:00<34:13,  6.48s/it]
+ 83%|████████▎ | 1497/1813 [2:45:05<32:19,  6.14s/it]
+ 83%|████████▎ | 1498/1813 [2:45:12<33:01,  6.29s/it]
+ 83%|████████▎ | 1499/1813 [2:45:17<30:31,  5.83s/it]
+ 83%|████████▎ | 1500/1813 [2:45:24<33:14,  6.37s/it]
+                                                     
+{'loss': '0.9054', 'grad_norm': '1.702', 'learning_rate': '1.708e-06', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904, 'train_runtime': '9925', 'train_tokens_per_second': '1017'}
+
+ 83%|████████▎ | 1500/1813 [2:45:24<33:14,  6.37s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.75it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.10it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.01it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.04it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.59it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.37it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.98it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.12it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.34it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.04it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.75it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.94it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.74it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.29it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.35it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.85it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.20it/s]
+
+  8%|▊         | 84/1000 [00:11<01:53,  8.04it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.94it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.62it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.30it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.43it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.20it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.03it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.52it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.80it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.21it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.65it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.90it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.02it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.88it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25, 10.06it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.48it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.90it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.03it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.43it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.66it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.62it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.82it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:23,  5.79it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.46it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.74it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.71it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.73it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.89it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.37it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.10it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.20it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.68it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.78it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.63it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.92it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.51it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.70it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.79it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.03it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.22it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.96it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.96it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.67it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.50it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.04it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.73it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.77it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.09it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.15it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.96it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.79it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.27it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.26it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.60it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.76it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.80it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.42it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.59it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.44it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:59,  5.31it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:06,  4.98it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:53,  5.57it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.52it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.69it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.05it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.20it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:30,  6.91it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.76it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.21it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.20it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.66it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.38it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.79it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.74it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.24it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.23it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.80it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.24it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.44it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.68it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.92it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.85it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.55it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.02it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.80it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.50it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.17it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.49it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.80it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.84it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.56it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.50it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.32it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.26it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.96it/s]
+
+ 50%|████▉     | 496/1000 [01:06<01:00,  8.39it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.04it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.60it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.10it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.72it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:19,  5.81it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:21,  5.67it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:28,  5.21it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:06,  6.88it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.01it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:10,  6.47it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.68it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.91it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.50it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.25it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.85it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.55it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.76it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.19it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.40it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.40it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.46it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.81it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.62it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.74it/s]
+
+ 62%|██████▏   | 618/1000 [01:24<00:48,  7.96it/s]
+
+ 62%|██████▏   | 620/1000 [01:24<00:39,  9.68it/s]
+
+ 62%|██████▏   | 622/1000 [01:24<00:46,  8.14it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:48,  7.74it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:52,  7.09it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:44,  8.44it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:53,  7.00it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<01:01,  6.06it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.78it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.42it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.68it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.04it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.94it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.18it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.95it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.58it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.09it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.51it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.57it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.71it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:45,  7.42it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.41it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.73it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:03,  5.30it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.60it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:04,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.69it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.92it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.89it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.85it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.85it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.15it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.82it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.25it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.41it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:48,  6.32it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.21it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.61it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.49it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.71it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.87it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.38it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.29it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.62it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.88it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.54it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.78it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.51it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.38it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.62it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.66it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.15it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.70it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.46it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.77it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.63it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  9.00it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.74it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.84it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.70it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.62it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.81it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.14it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.95it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.21it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.23it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.06it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.31it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.03it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.14it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.76it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.86it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.46it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.05it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.05it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.44it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.61it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:13, 10.50it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.59it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.25it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.22it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.24it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.53it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.94it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.16it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.75it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.85it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.32it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.59it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.78it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.93it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.15it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.63it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.08it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.84it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.03it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.13it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.26it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.02it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.33it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.65it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.80it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.40it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.15it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.39it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.39it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.48it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.43it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.79it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.83it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.8019', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.383', 'eval_steps_per_second': '7.383', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904}
+
+ 83%|████████▎ | 1500/1813 [2:47:40<33:14,  6.37s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+
+                                                   
+ 83%|████████▎ | 1501/1813 [2:47:46<4:04:41, 47.06s/it]
+ 83%|████████▎ | 1502/1813 [2:47:51<2:58:03, 34.35s/it]
+ 83%|████████▎ | 1503/1813 [2:47:57<2:14:01, 25.94s/it]
+ 83%|████████▎ | 1504/1813 [2:48:02<1:41:14, 19.66s/it]
+ 83%|████████▎ | 1505/1813 [2:48:08<1:19:03, 15.40s/it]
+ 83%|████████▎ | 1506/1813 [2:48:15<1:05:45, 12.85s/it]
+ 83%|████████▎ | 1507/1813 [2:48:20<53:51, 10.56s/it]  
+ 83%|████████▎ | 1508/1813 [2:48:27<48:22,  9.52s/it]
+ 83%|████████▎ | 1509/1813 [2:48:33<42:49,  8.45s/it]
+ 83%|████████▎ | 1510/1813 [2:48:39<39:02,  7.73s/it]
+                                                     
+{'loss': '0.9198', 'grad_norm': '1.94', 'learning_rate': '1.604e-06', 'epoch': '0.8331', 'num_input_tokens_seen': 10151928, 'train_runtime': '1.012e+04', 'train_tokens_per_second': '1003'}
+
+ 83%|████████▎ | 1510/1813 [2:48:39<39:02,  7.73s/it]
+ 83%|████████▎ | 1511/1813 [2:48:46<36:58,  7.35s/it]
+ 83%|████████▎ | 1512/1813 [2:48:51<34:07,  6.80s/it]
+ 83%|████████▎ | 1513/1813 [2:48:56<31:56,  6.39s/it]
+ 84%|████████▎ | 1514/1813 [2:49:03<32:03,  6.43s/it]
+ 84%|████████▎ | 1515/1813 [2:49:11<33:33,  6.76s/it]
+ 84%|████████▎ | 1516/1813 [2:49:18<33:47,  6.83s/it]
+ 84%|████████▎ | 1517/1813 [2:49:24<33:16,  6.75s/it]
+ 84%|████████▎ | 1518/1813 [2:49:32<34:31,  7.02s/it]
+ 84%|████████▍ | 1519/1813 [2:49:39<34:46,  7.10s/it]
+ 84%|████████▍ | 1520/1813 [2:49:45<33:40,  6.90s/it]
+                                                     
+{'loss': '0.9458', 'grad_norm': '1.957', 'learning_rate': '1.503e-06', 'epoch': '0.8386', 'num_input_tokens_seen': 10225024, 'train_runtime': '1.019e+04', 'train_tokens_per_second': '1004'}
+
+ 84%|████████▍ | 1520/1813 [2:49:45<33:40,  6.90s/it]
+ 84%|████████▍ | 1521/1813 [2:49:52<33:41,  6.92s/it]
+ 84%|████████▍ | 1522/1813 [2:49:59<32:52,  6.78s/it]
+ 84%|████████▍ | 1523/1813 [2:50:06<33:44,  6.98s/it]
+ 84%|████████▍ | 1524/1813 [2:50:13<32:49,  6.81s/it]
+ 84%|████████▍ | 1525/1813 [2:50:18<30:13,  6.30s/it]
+ 84%|████████▍ | 1526/1813 [2:50:24<30:33,  6.39s/it]
+ 84%|████████▍ | 1527/1813 [2:50:31<31:04,  6.52s/it]
+ 84%|████████▍ | 1528/1813 [2:50:38<31:00,  6.53s/it]
+ 84%|████████▍ | 1529/1813 [2:50:45<31:44,  6.71s/it]
+ 84%|████████▍ | 1530/1813 [2:50:51<30:19,  6.43s/it]
+                                                     
+{'loss': '0.9768', 'grad_norm': '1.286', 'learning_rate': '1.405e-06', 'epoch': '0.8441', 'num_input_tokens_seen': 10294368, 'train_runtime': '1.025e+04', 'train_tokens_per_second': '1004'}
+
+ 84%|████████▍ | 1530/1813 [2:50:51<30:19,  6.43s/it]
+ 84%|████████▍ | 1531/1813 [2:50:56<28:23,  6.04s/it]
+ 85%|████████▍ | 1532/1813 [2:51:03<29:31,  6.31s/it]
+ 85%|████████▍ | 1533/1813 [2:51:08<27:36,  5.92s/it]
+ 85%|████████▍ | 1534/1813 [2:51:16<30:30,  6.56s/it]
+ 85%|████████▍ | 1535/1813 [2:51:21<28:53,  6.23s/it]
+ 85%|████████▍ | 1536/1813 [2:51:29<30:46,  6.66s/it]
+ 85%|████████▍ | 1537/1813 [2:51:36<30:53,  6.71s/it]
+ 85%|████████▍ | 1538/1813 [2:51:41<28:01,  6.11s/it]
+ 85%|████████▍ | 1539/1813 [2:51:46<27:09,  5.95s/it]
+ 85%|████████▍ | 1540/1813 [2:51:52<26:23,  5.80s/it]
+                                                     
+{'loss': '0.9564', 'grad_norm': '2.047', 'learning_rate': '1.31e-06', 'epoch': '0.8497', 'num_input_tokens_seen': 10359416, 'train_runtime': '1.031e+04', 'train_tokens_per_second': '1005'}
+
+ 85%|████████▍ | 1540/1813 [2:51:52<26:23,  5.80s/it]
+ 85%|████████▍ | 1541/1813 [2:51:58<27:19,  6.03s/it]
+ 85%|████████▌ | 1542/1813 [2:52:04<27:32,  6.10s/it]
+ 85%|████████▌ | 1543/1813 [2:52:09<25:45,  5.73s/it]
+ 85%|████████▌ | 1544/1813 [2:52:14<24:31,  5.47s/it]
+ 85%|████████▌ | 1545/1813 [2:52:22<27:32,  6.17s/it]
+ 85%|████████▌ | 1546/1813 [2:52:28<27:16,  6.13s/it]
+ 85%|████████▌ | 1547/1813 [2:52:33<26:20,  5.94s/it]
+ 85%|████████▌ | 1548/1813 [2:52:42<29:25,  6.66s/it]
+ 85%|████████▌ | 1549/1813 [2:52:47<27:46,  6.31s/it]
+ 85%|████████▌ | 1550/1813 [2:52:55<29:14,  6.67s/it]
+                                                     
+{'loss': '1.061', 'grad_norm': '1.561', 'learning_rate': '1.218e-06', 'epoch': '0.8552', 'num_input_tokens_seen': 10427368, 'train_runtime': '1.038e+04', 'train_tokens_per_second': '1005'}
+
+ 85%|████████▌ | 1550/1813 [2:52:55<29:14,  6.67s/it]
+ 86%|████████▌ | 1551/1813 [2:53:03<30:33,  7.00s/it]
+ 86%|████████▌ | 1552/1813 [2:53:09<29:19,  6.74s/it]
+ 86%|████████▌ | 1553/1813 [2:53:15<28:04,  6.48s/it]
+ 86%|████████▌ | 1554/1813 [2:53:21<28:23,  6.58s/it]
+ 86%|████████▌ | 1555/1813 [2:53:28<28:26,  6.61s/it]
+ 86%|████████▌ | 1556/1813 [2:53:34<27:52,  6.51s/it]
+ 86%|████████▌ | 1557/1813 [2:53:42<29:13,  6.85s/it]
+ 86%|████████▌ | 1558/1813 [2:53:48<27:52,  6.56s/it]
+ 86%|████████▌ | 1559/1813 [2:53:54<27:26,  6.48s/it]
+ 86%|████████▌ | 1560/1813 [2:53:59<25:53,  6.14s/it]
+                                                     
+{'loss': '1.014', 'grad_norm': '1.497', 'learning_rate': '1.129e-06', 'epoch': '0.8607', 'num_input_tokens_seen': 10497472, 'train_runtime': '1.044e+04', 'train_tokens_per_second': '1006'}
+
+ 86%|████████▌ | 1560/1813 [2:53:59<25:53,  6.14s/it]
+ 86%|████████▌ | 1561/1813 [2:54:07<26:58,  6.42s/it]
+ 86%|████████▌ | 1562/1813 [2:54:13<26:54,  6.43s/it]
+ 86%|████████▌ | 1563/1813 [2:54:19<26:34,  6.38s/it]
+ 86%|████████▋ | 1564/1813 [2:54:25<26:16,  6.33s/it]
+ 86%|████████▋ | 1565/1813 [2:54:30<23:39,  5.72s/it]
+ 86%|████████▋ | 1566/1813 [2:54:36<23:46,  5.77s/it]
+ 86%|████████▋ | 1567/1813 [2:54:42<23:54,  5.83s/it]
+ 86%|████████▋ | 1568/1813 [2:54:46<22:26,  5.50s/it]
+ 87%|████████▋ | 1569/1813 [2:54:54<24:44,  6.08s/it]
+ 87%|████████▋ | 1570/1813 [2:55:01<26:05,  6.44s/it]
+                                                     
+{'loss': '0.899', 'grad_norm': '1.825', 'learning_rate': '1.044e-06', 'epoch': '0.8662', 'num_input_tokens_seen': 10564336, 'train_runtime': '1.05e+04', 'train_tokens_per_second': '1006'}
+
+ 87%|████████▋ | 1570/1813 [2:55:01<26:05,  6.44s/it]
+ 87%|████████▋ | 1571/1813 [2:55:06<24:38,  6.11s/it]
+ 87%|████████▋ | 1572/1813 [2:55:12<23:56,  5.96s/it]
+ 87%|████████▋ | 1573/1813 [2:55:18<23:38,  5.91s/it]
+ 87%|████████▋ | 1574/1813 [2:55:24<24:14,  6.09s/it]
+ 87%|████████▋ | 1575/1813 [2:55:30<23:47,  6.00s/it]
+ 87%|████████▋ | 1576/1813 [2:55:38<25:32,  6.46s/it]
+ 87%|████████▋ | 1577/1813 [2:55:45<26:01,  6.62s/it]
+ 87%|████████▋ | 1578/1813 [2:55:51<25:40,  6.55s/it]
+ 87%|████████▋ | 1579/1813 [2:55:57<25:12,  6.46s/it]
+ 87%|████████▋ | 1580/1813 [2:56:02<23:36,  6.08s/it]
+                                                     
+{'loss': '0.9687', 'grad_norm': '2.684', 'learning_rate': '9.612e-07', 'epoch': '0.8717', 'num_input_tokens_seen': 10628400, 'train_runtime': '1.056e+04', 'train_tokens_per_second': '1006'}
+
+ 87%|████████▋ | 1580/1813 [2:56:02<23:36,  6.08s/it]
+ 87%|████████▋ | 1581/1813 [2:56:08<23:24,  6.05s/it]
+ 87%|████████▋ | 1582/1813 [2:56:13<21:34,  5.60s/it]
+ 87%|████████▋ | 1583/1813 [2:56:19<22:12,  5.79s/it]
+ 87%|████████▋ | 1584/1813 [2:56:26<22:42,  5.95s/it]
+ 87%|████████▋ | 1585/1813 [2:56:33<24:11,  6.36s/it]
+ 87%|████████▋ | 1586/1813 [2:56:40<25:22,  6.71s/it]
+ 88%|████████▊ | 1587/1813 [2:56:48<26:15,  6.97s/it]
+ 88%|████████▊ | 1588/1813 [2:56:54<24:48,  6.62s/it]
+ 88%|████████▊ | 1589/1813 [2:56:59<23:08,  6.20s/it]
+ 88%|████████▊ | 1590/1813 [2:57:06<23:39,  6.36s/it]
+                                                     
+{'loss': '0.9394', 'grad_norm': '1.512', 'learning_rate': '8.82e-07', 'epoch': '0.8772', 'num_input_tokens_seen': 10695288, 'train_runtime': '1.063e+04', 'train_tokens_per_second': '1006'}
+
+ 88%|████████▊ | 1590/1813 [2:57:06<23:39,  6.36s/it]
+ 88%|████████▊ | 1591/1813 [2:57:12<23:26,  6.33s/it]
+ 88%|████████▊ | 1592/1813 [2:57:19<23:38,  6.42s/it]
+ 88%|████████▊ | 1593/1813 [2:57:25<23:56,  6.53s/it]
+ 88%|████████▊ | 1594/1813 [2:57:32<23:55,  6.55s/it]
+ 88%|████████▊ | 1595/1813 [2:57:39<24:36,  6.77s/it]
+ 88%|████████▊ | 1596/1813 [2:57:48<26:38,  7.36s/it]
+ 88%|████████▊ | 1597/1813 [2:57:55<25:44,  7.15s/it]
+ 88%|████████▊ | 1598/1813 [2:58:01<24:51,  6.94s/it]
+ 88%|████████▊ | 1599/1813 [2:58:07<23:10,  6.50s/it]
+ 88%|████████▊ | 1600/1813 [2:58:11<21:01,  5.92s/it]
+                                                     
+{'loss': '0.9872', 'grad_norm': '1.64', 'learning_rate': '8.061e-07', 'epoch': '0.8828', 'num_input_tokens_seen': 10767512, 'train_runtime': '1.069e+04', 'train_tokens_per_second': '1007'}
+
+ 88%|████████▊ | 1600/1813 [2:58:11<21:01,  5.92s/it]
+ 88%|████████▊ | 1601/1813 [2:58:19<23:01,  6.52s/it]
+ 88%|████████▊ | 1602/1813 [2:58:25<22:17,  6.34s/it]
+ 88%|████████▊ | 1603/1813 [2:58:32<22:37,  6.47s/it]
+ 88%|████████▊ | 1604/1813 [2:58:39<22:46,  6.54s/it]
+ 89%|████████▊ | 1605/1813 [2:58:47<24:33,  7.09s/it]
+ 89%|████████▊ | 1606/1813 [2:58:52<22:38,  6.56s/it]
+ 89%|████████▊ | 1607/1813 [2:58:59<22:49,  6.65s/it]
+ 89%|████████▊ | 1608/1813 [2:59:07<23:32,  6.89s/it]
+ 89%|████████▊ | 1609/1813 [2:59:12<21:51,  6.43s/it]
+ 89%|████████▉ | 1610/1813 [2:59:18<21:10,  6.26s/it]
+                                                     
+{'loss': '0.9559', 'grad_norm': '2.117', 'learning_rate': '7.334e-07', 'epoch': '0.8883', 'num_input_tokens_seen': 10840800, 'train_runtime': '1.076e+04', 'train_tokens_per_second': '1008'}
+
+ 89%|████████▉ | 1610/1813 [2:59:18<21:10,  6.26s/it]
+ 89%|████████▉ | 1611/1813 [2:59:23<19:42,  5.85s/it]
+ 89%|████████▉ | 1612/1813 [2:59:29<19:43,  5.89s/it]
+ 89%|████████▉ | 1613/1813 [2:59:33<18:34,  5.57s/it]
+ 89%|████████▉ | 1614/1813 [2:59:39<18:52,  5.69s/it]
+ 89%|████████▉ | 1615/1813 [2:59:46<19:44,  5.98s/it]
+ 89%|████████▉ | 1616/1813 [2:59:52<20:04,  6.11s/it]
+ 89%|████████▉ | 1617/1813 [2:59:59<20:25,  6.25s/it]
+ 89%|████████▉ | 1618/1813 [3:00:04<19:04,  5.87s/it]
+ 89%|████████▉ | 1619/1813 [3:00:10<18:42,  5.79s/it]
+ 89%|████████▉ | 1620/1813 [3:00:14<17:42,  5.50s/it]
+                                                     
+{'loss': '0.962', 'grad_norm': '1.801', 'learning_rate': '6.641e-07', 'epoch': '0.8938', 'num_input_tokens_seen': 10899640, 'train_runtime': '1.081e+04', 'train_tokens_per_second': '1008'}
+
+ 89%|████████▉ | 1620/1813 [3:00:14<17:42,  5.50s/it]
+ 89%|████████▉ | 1621/1813 [3:00:20<17:39,  5.52s/it]
+ 89%|████████▉ | 1622/1813 [3:00:28<19:38,  6.17s/it]
+ 90%|████████▉ | 1623/1813 [3:00:35<20:19,  6.42s/it]
+ 90%|████████▉ | 1624/1813 [3:00:41<19:48,  6.29s/it]
+ 90%|████████▉ | 1625/1813 [3:00:48<20:38,  6.59s/it]
+ 90%|████████▉ | 1626/1813 [3:00:54<19:58,  6.41s/it]
+ 90%|████████▉ | 1627/1813 [3:01:00<19:23,  6.26s/it]
+ 90%|████████▉ | 1628/1813 [3:01:06<18:56,  6.14s/it]
+ 90%|████████▉ | 1629/1813 [3:01:13<19:47,  6.45s/it]
+ 90%|████████▉ | 1630/1813 [3:01:18<18:34,  6.09s/it]
+                                                     
+{'loss': '0.9755', 'grad_norm': '2.885', 'learning_rate': '5.981e-07', 'epoch': '0.8993', 'num_input_tokens_seen': 10968528, 'train_runtime': '1.088e+04', 'train_tokens_per_second': '1008'}
+
+ 90%|████████▉ | 1630/1813 [3:01:18<18:34,  6.09s/it]
+ 90%|████████▉ | 1631/1813 [3:01:24<17:50,  5.88s/it]
+ 90%|█████████ | 1632/1813 [3:01:28<16:43,  5.55s/it]
+ 90%|█████████ | 1633/1813 [3:01:36<18:16,  6.09s/it]
+ 90%|█████████ | 1634/1813 [3:01:41<17:05,  5.73s/it]
+ 90%|█████████ | 1635/1813 [3:01:47<17:52,  6.03s/it]
+ 90%|█████████ | 1636/1813 [3:01:53<17:15,  5.85s/it]
+ 90%|█████████ | 1637/1813 [3:01:59<17:33,  5.98s/it]
+ 90%|█████████ | 1638/1813 [3:02:04<16:43,  5.73s/it]
+ 90%|█████████ | 1639/1813 [3:02:11<17:32,  6.05s/it]
+ 90%|█████████ | 1640/1813 [3:02:16<16:39,  5.78s/it]
+                                                     
+{'loss': '0.9798', 'grad_norm': '1.326', 'learning_rate': '5.354e-07', 'epoch': '0.9048', 'num_input_tokens_seen': 11030048, 'train_runtime': '1.094e+04', 'train_tokens_per_second': '1009'}
+
+ 90%|█████████ | 1640/1813 [3:02:16<16:39,  5.78s/it]
+ 91%|█████████ | 1641/1813 [3:02:22<16:32,  5.77s/it]
+ 91%|█████████ | 1642/1813 [3:02:28<16:57,  5.95s/it]
+ 91%|█████████ | 1643/1813 [3:02:34<16:55,  5.97s/it]
+ 91%|█████████ | 1644/1813 [3:02:41<17:27,  6.20s/it]
+ 91%|█████████ | 1645/1813 [3:02:50<20:02,  7.16s/it]
+ 91%|█████████ | 1646/1813 [3:02:57<19:23,  6.97s/it]
+ 91%|█████████ | 1647/1813 [3:03:03<18:18,  6.62s/it]
+ 91%|█████████ | 1648/1813 [3:03:09<18:04,  6.57s/it]
+ 91%|█████████ | 1649/1813 [3:03:14<16:41,  6.11s/it]
+ 91%|█████████ | 1650/1813 [3:03:19<15:39,  5.76s/it]
+                                                     
+{'loss': '0.9429', 'grad_norm': '1.733', 'learning_rate': '4.761e-07', 'epoch': '0.9103', 'num_input_tokens_seen': 11098768, 'train_runtime': '1.1e+04', 'train_tokens_per_second': '1009'}
+
+ 91%|█████████ | 1650/1813 [3:03:19<15:39,  5.76s/it]
+ 91%|█████████ | 1651/1813 [3:03:25<15:52,  5.88s/it]
+ 91%|█████████ | 1652/1813 [3:03:32<16:03,  5.98s/it]
+ 91%|█████████ | 1653/1813 [3:03:38<16:25,  6.16s/it]
+ 91%|█████████ | 1654/1813 [3:03:44<16:02,  6.05s/it]
+ 91%|█████████▏| 1655/1813 [3:03:50<16:09,  6.13s/it]
+ 91%|█████████▏| 1656/1813 [3:03:56<15:40,  5.99s/it]
+ 91%|█████████▏| 1657/1813 [3:04:04<16:51,  6.48s/it]
+ 91%|█████████▏| 1658/1813 [3:04:10<16:38,  6.44s/it]
+ 92%|█████████▏| 1659/1813 [3:04:17<16:52,  6.58s/it]
+ 92%|█████████▏| 1660/1813 [3:04:23<16:34,  6.50s/it]
+                                                     
+{'loss': '0.9938', 'grad_norm': '2.138', 'learning_rate': '4.202e-07', 'epoch': '0.9159', 'num_input_tokens_seen': 11168848, 'train_runtime': '1.106e+04', 'train_tokens_per_second': '1010'}
+
+ 92%|█████████▏| 1660/1813 [3:04:23<16:34,  6.50s/it]
+ 92%|█████████▏| 1661/1813 [3:04:30<16:53,  6.67s/it]
+ 92%|█████████▏| 1662/1813 [3:04:37<16:39,  6.62s/it]
+ 92%|█████████▏| 1663/1813 [3:04:44<16:53,  6.76s/it]
+ 92%|█████████▏| 1664/1813 [3:04:51<17:00,  6.85s/it]
+ 92%|█████████▏| 1665/1813 [3:04:58<17:31,  7.10s/it]
+ 92%|█████████▏| 1666/1813 [3:05:03<15:35,  6.36s/it]
+ 92%|█████████▏| 1667/1813 [3:05:10<15:45,  6.48s/it]
+ 92%|█████████▏| 1668/1813 [3:05:16<15:17,  6.33s/it]
+ 92%|█████████▏| 1669/1813 [3:05:26<17:48,  7.42s/it]
+ 92%|█████████▏| 1670/1813 [3:05:31<16:11,  6.79s/it]
+                                                     
+{'loss': '0.9557', 'grad_norm': '1.997', 'learning_rate': '3.677e-07', 'epoch': '0.9214', 'num_input_tokens_seen': 11244288, 'train_runtime': '1.113e+04', 'train_tokens_per_second': '1010'}
+
+ 92%|█████████▏| 1670/1813 [3:05:31<16:11,  6.79s/it]
+ 92%|█████████▏| 1671/1813 [3:05:36<14:58,  6.33s/it]
+ 92%|█████████▏| 1672/1813 [3:05:42<14:25,  6.14s/it]
+ 92%|█████████▏| 1673/1813 [3:05:48<14:05,  6.04s/it]
+ 92%|█████████▏| 1674/1813 [3:05:55<14:35,  6.30s/it]
+ 92%|█████████▏| 1675/1813 [3:06:02<15:05,  6.56s/it]
+ 92%|█████████▏| 1676/1813 [3:06:09<15:29,  6.78s/it]
+ 92%|█████████▏| 1677/1813 [3:06:16<15:04,  6.65s/it]
+ 93%|█████████▎| 1678/1813 [3:06:22<14:42,  6.54s/it]
+ 93%|█████████▎| 1679/1813 [3:06:28<14:05,  6.31s/it]
+ 93%|█████████▎| 1680/1813 [3:06:36<15:35,  7.04s/it]
+                                                     
+{'loss': '1.013', 'grad_norm': '1.755', 'learning_rate': '3.187e-07', 'epoch': '0.9269', 'num_input_tokens_seen': 11317056, 'train_runtime': '1.12e+04', 'train_tokens_per_second': '1011'}
+
+ 93%|█████████▎| 1680/1813 [3:06:36<15:35,  7.04s/it]
+ 93%|█████████▎| 1681/1813 [3:06:41<14:03,  6.39s/it]
+ 93%|█████████▎| 1682/1813 [3:06:47<13:20,  6.11s/it]
+ 93%|█████████▎| 1683/1813 [3:06:52<12:25,  5.74s/it]
+ 93%|█████████▎| 1684/1813 [3:06:58<13:01,  6.06s/it]
+ 93%|█████████▎| 1685/1813 [3:07:05<13:18,  6.24s/it]
+ 93%|█████████▎| 1686/1813 [3:07:12<13:35,  6.42s/it]
+ 93%|█████████▎| 1687/1813 [3:07:18<13:26,  6.40s/it]
+ 93%|█████████▎| 1688/1813 [3:07:25<13:35,  6.52s/it]
+ 93%|█████████▎| 1689/1813 [3:07:30<12:20,  5.97s/it]
+ 93%|█████████▎| 1690/1813 [3:07:36<12:33,  6.13s/it]
+                                                     
+{'loss': '0.9349', 'grad_norm': '1.355', 'learning_rate': '2.731e-07', 'epoch': '0.9324', 'num_input_tokens_seen': 11380560, 'train_runtime': '1.126e+04', 'train_tokens_per_second': '1011'}
+
+ 93%|█████████▎| 1690/1813 [3:07:36<12:33,  6.13s/it]
+ 93%|█████████▎| 1691/1813 [3:07:42<12:27,  6.13s/it]
+ 93%|█████████▎| 1692/1813 [3:07:50<13:14,  6.56s/it]
+ 93%|█████████▎| 1693/1813 [3:07:56<12:53,  6.45s/it]
+ 93%|█████████▎| 1694/1813 [3:08:05<13:58,  7.05s/it]
+ 93%|█████████▎| 1695/1813 [3:08:11<13:17,  6.76s/it]
+ 94%|█████████▎| 1696/1813 [3:08:16<12:32,  6.43s/it]
+ 94%|█████████▎| 1697/1813 [3:08:23<12:28,  6.45s/it]
+ 94%|█████████▎| 1698/1813 [3:08:28<11:41,  6.10s/it]
+ 94%|█████████▎| 1699/1813 [3:08:34<11:43,  6.17s/it]
+ 94%|█████████▍| 1700/1813 [3:08:41<11:39,  6.19s/it]
+                                                     
+{'loss': '0.9708', 'grad_norm': '2.135', 'learning_rate': '2.31e-07', 'epoch': '0.9379', 'num_input_tokens_seen': 11450712, 'train_runtime': '1.132e+04', 'train_tokens_per_second': '1011'}
+
+ 94%|█████████▍| 1700/1813 [3:08:41<11:39,  6.19s/it]
+ 94%|█████████▍| 1701/1813 [3:08:45<10:35,  5.68s/it]
+ 94%|█████████▍| 1702/1813 [3:08:50<10:11,  5.51s/it]
+ 94%|█████████▍| 1703/1813 [3:08:56<10:17,  5.61s/it]
+ 94%|█████████▍| 1704/1813 [3:09:04<11:19,  6.24s/it]
+ 94%|█████████▍| 1705/1813 [3:09:10<11:26,  6.35s/it]
+ 94%|█████████▍| 1706/1813 [3:09:17<11:20,  6.36s/it]
+ 94%|█████████▍| 1707/1813 [3:09:23<10:54,  6.17s/it]
+ 94%|█████████▍| 1708/1813 [3:09:28<10:10,  5.82s/it]
+ 94%|█████████▍| 1709/1813 [3:09:34<10:20,  5.96s/it]
+ 94%|█████████▍| 1710/1813 [3:09:40<10:11,  5.94s/it]
+                                                     
+{'loss': '0.9709', 'grad_norm': '2.15', 'learning_rate': '1.924e-07', 'epoch': '0.9434', 'num_input_tokens_seen': 11511984, 'train_runtime': '1.138e+04', 'train_tokens_per_second': '1012'}
+
+ 94%|█████████▍| 1710/1813 [3:09:40<10:11,  5.94s/it]
+ 94%|█████████▍| 1711/1813 [3:09:45<09:49,  5.78s/it]
+ 94%|█████████▍| 1712/1813 [3:09:51<09:47,  5.82s/it]
+ 94%|█████████▍| 1713/1813 [3:09:57<09:45,  5.86s/it]
+ 95%|█████████▍| 1714/1813 [3:10:03<09:40,  5.86s/it]
+ 95%|█████████▍| 1715/1813 [3:10:09<09:28,  5.80s/it]
+ 95%|█████████▍| 1716/1813 [3:10:14<09:06,  5.64s/it]
+ 95%|█████████▍| 1717/1813 [3:10:19<08:54,  5.56s/it]
+ 95%|█████████▍| 1718/1813 [3:10:26<09:11,  5.80s/it]
+ 95%|█████████▍| 1719/1813 [3:10:32<09:21,  5.98s/it]
+ 95%|█████████▍| 1720/1813 [3:10:38<09:19,  6.01s/it]
+                                                     
+{'loss': '0.9658', 'grad_norm': '1.579', 'learning_rate': '1.573e-07', 'epoch': '0.949', 'num_input_tokens_seen': 11571368, 'train_runtime': '1.144e+04', 'train_tokens_per_second': '1012'}
+
+ 95%|█████████▍| 1720/1813 [3:10:38<09:19,  6.01s/it]
+ 95%|█████████▍| 1721/1813 [3:10:45<09:52,  6.44s/it]
+ 95%|█████████▍| 1722/1813 [3:10:52<09:38,  6.36s/it]
+ 95%|█████████▌| 1723/1813 [3:10:58<09:26,  6.29s/it]
+ 95%|█████████▌| 1724/1813 [3:11:04<09:13,  6.22s/it]
+ 95%|█████████▌| 1725/1813 [3:11:11<09:36,  6.55s/it]
+ 95%|█████████▌| 1726/1813 [3:11:19<09:54,  6.83s/it]
+ 95%|█████████▌| 1727/1813 [3:11:25<09:28,  6.61s/it]
+ 95%|█████████▌| 1728/1813 [3:11:30<08:46,  6.19s/it]
+ 95%|█████████▌| 1729/1813 [3:11:35<08:20,  5.96s/it]
+ 95%|█████████▌| 1730/1813 [3:11:42<08:34,  6.20s/it]
+                                                     
+{'loss': '0.9777', 'grad_norm': '1.138', 'learning_rate': '1.256e-07', 'epoch': '0.9545', 'num_input_tokens_seen': 11640392, 'train_runtime': '1.15e+04', 'train_tokens_per_second': '1012'}
+
+ 95%|█████████▌| 1730/1813 [3:11:42<08:34,  6.20s/it]
+ 95%|█████████▌| 1731/1813 [3:11:48<08:30,  6.22s/it]
+ 96%|█████████▌| 1732/1813 [3:11:55<08:40,  6.42s/it]
+ 96%|█████████▌| 1733/1813 [3:12:00<08:03,  6.05s/it]
+ 96%|█████████▌| 1734/1813 [3:12:06<07:54,  6.01s/it]
+ 96%|█████████▌| 1735/1813 [3:12:13<07:57,  6.12s/it]
+ 96%|█████████▌| 1736/1813 [3:12:20<08:08,  6.35s/it]
+ 96%|█████████▌| 1737/1813 [3:12:26<07:57,  6.28s/it]
+ 96%|█████████▌| 1738/1813 [3:12:31<07:19,  5.86s/it]
+ 96%|█████████▌| 1739/1813 [3:12:39<07:59,  6.48s/it]
+ 96%|█████████▌| 1740/1813 [3:12:45<07:58,  6.55s/it]
+                                                     
+{'loss': '0.9616', 'grad_norm': '1.926', 'learning_rate': '9.755e-08', 'epoch': '0.96', 'num_input_tokens_seen': 11709120, 'train_runtime': '1.157e+04', 'train_tokens_per_second': '1012'}
+
+ 96%|█████████▌| 1740/1813 [3:12:45<07:58,  6.55s/it]
+ 96%|█████████▌| 1741/1813 [3:12:51<07:39,  6.38s/it]
+ 96%|█████████▌| 1742/1813 [3:12:58<07:43,  6.53s/it]
+ 96%|█████████▌| 1743/1813 [3:13:05<07:50,  6.72s/it]
+ 96%|█████████▌| 1744/1813 [3:13:13<07:55,  6.89s/it]
+ 96%|█████████▌| 1745/1813 [3:13:19<07:30,  6.63s/it]
+ 96%|█████████▋| 1746/1813 [3:13:26<07:37,  6.83s/it]
+ 96%|█████████▋| 1747/1813 [3:13:31<06:50,  6.21s/it]
+ 96%|█████████▋| 1748/1813 [3:13:38<07:09,  6.61s/it]
+ 96%|█████████▋| 1749/1813 [3:13:44<06:43,  6.30s/it]
+ 97%|█████████▋| 1750/1813 [3:13:49<06:17,  6.00s/it]
+                                                     
+{'loss': '0.9759', 'grad_norm': '1.339', 'learning_rate': '7.3e-08', 'epoch': '0.9655', 'num_input_tokens_seen': 11777768, 'train_runtime': '1.163e+04', 'train_tokens_per_second': '1013'}
+
+ 97%|█████████▋| 1750/1813 [3:13:49<06:17,  6.00s/it]
+ 97%|█████████▋| 1751/1813 [3:13:55<06:10,  5.98s/it]
+ 97%|█████████▋| 1752/1813 [3:14:01<06:01,  5.93s/it]
+ 97%|█████████▋| 1753/1813 [3:14:08<06:25,  6.42s/it]
+ 97%|█████████▋| 1754/1813 [3:14:16<06:32,  6.66s/it]
+ 97%|█████████▋| 1755/1813 [3:14:22<06:21,  6.58s/it]
+ 97%|█████████▋| 1756/1813 [3:14:27<05:53,  6.20s/it]
+ 97%|█████████▋| 1757/1813 [3:14:33<05:33,  5.96s/it]
+ 97%|█████████▋| 1758/1813 [3:14:38<05:21,  5.84s/it]
+ 97%|█████████▋| 1759/1813 [3:14:44<05:16,  5.86s/it]
+ 97%|█████████▋| 1760/1813 [3:14:49<04:59,  5.66s/it]
+                                                     
+{'loss': '0.9586', 'grad_norm': '1.957', 'learning_rate': '5.199e-08', 'epoch': '0.971', 'num_input_tokens_seen': 11843192, 'train_runtime': '1.169e+04', 'train_tokens_per_second': '1013'}
+
+ 97%|█████████▋| 1760/1813 [3:14:49<04:59,  5.66s/it]
+ 97%|█████████▋| 1761/1813 [3:14:55<04:48,  5.55s/it]
+ 97%|█████████▋| 1762/1813 [3:14:59<04:28,  5.26s/it]
+ 97%|█████████▋| 1763/1813 [3:15:07<04:58,  5.97s/it]
+ 97%|█████████▋| 1764/1813 [3:15:12<04:39,  5.70s/it]
+ 97%|█████████▋| 1765/1813 [3:15:17<04:26,  5.56s/it]
+ 97%|█████████▋| 1766/1813 [3:15:23<04:19,  5.52s/it]
+ 97%|█████████▋| 1767/1813 [3:15:28<04:13,  5.51s/it]
+ 98%|█████████▊| 1768/1813 [3:15:34<04:11,  5.59s/it]
+ 98%|█████████▊| 1769/1813 [3:15:40<04:07,  5.63s/it]
+ 98%|█████████▊| 1770/1813 [3:15:46<04:15,  5.93s/it]
+                                                     
+{'loss': '0.9701', 'grad_norm': '1.618', 'learning_rate': '3.453e-08', 'epoch': '0.9766', 'num_input_tokens_seen': 11903728, 'train_runtime': '1.175e+04', 'train_tokens_per_second': '1013'}
+
+ 98%|█████████▊| 1770/1813 [3:15:46<04:15,  5.93s/it]
+ 98%|█████████▊| 1771/1813 [3:15:53<04:13,  6.04s/it]
+ 98%|█████████▊| 1772/1813 [3:15:59<04:10,  6.12s/it]
+ 98%|█████████▊| 1773/1813 [3:16:06<04:15,  6.38s/it]
+ 98%|█████████▊| 1774/1813 [3:16:10<03:47,  5.84s/it]
+ 98%|█████████▊| 1775/1813 [3:16:18<03:59,  6.31s/it]
+ 98%|█████████▊| 1776/1813 [3:16:25<04:04,  6.61s/it]
+ 98%|█████████▊| 1777/1813 [3:16:33<04:07,  6.86s/it]
+ 98%|█████████▊| 1778/1813 [3:16:38<03:47,  6.49s/it]
+ 98%|█████████▊| 1779/1813 [3:16:44<03:30,  6.19s/it]
+ 98%|█████████▊| 1780/1813 [3:16:49<03:20,  6.07s/it]
+                                                     
+{'loss': '0.9802', 'grad_norm': '1.235', 'learning_rate': '2.062e-08', 'epoch': '0.9821', 'num_input_tokens_seen': 11972864, 'train_runtime': '1.181e+04', 'train_tokens_per_second': '1014'}
+
+ 98%|█████████▊| 1780/1813 [3:16:49<03:20,  6.07s/it]
+ 98%|█████████▊| 1781/1813 [3:16:56<03:14,  6.09s/it]
+ 98%|█████████▊| 1782/1813 [3:17:01<03:06,  6.02s/it]
+ 98%|█████████▊| 1783/1813 [3:17:08<03:01,  6.04s/it]
+ 98%|█████████▊| 1784/1813 [3:17:14<03:02,  6.28s/it]
+ 98%|█████████▊| 1785/1813 [3:17:21<02:54,  6.23s/it]
+ 99%|█████████▊| 1786/1813 [3:17:27<02:49,  6.26s/it]
+ 99%|█████████▊| 1787/1813 [3:17:32<02:37,  6.04s/it]
+ 99%|█████████▊| 1788/1813 [3:17:39<02:35,  6.22s/it]
+ 99%|█████████▊| 1789/1813 [3:17:48<02:45,  6.92s/it]
+ 99%|█████████▊| 1790/1813 [3:17:55<02:40,  6.97s/it]
+                                                     
+{'loss': '0.9638', 'grad_norm': '1.635', 'learning_rate': '1.028e-08', 'epoch': '0.9876', 'num_input_tokens_seen': 12043496, 'train_runtime': '1.188e+04', 'train_tokens_per_second': '1014'}
+
+ 99%|█████████▊| 1790/1813 [3:17:55<02:40,  6.97s/it]
+ 99%|█████████▉| 1791/1813 [3:18:00<02:21,  6.45s/it]
+ 99%|█████████▉| 1792/1813 [3:18:06<02:13,  6.38s/it]
+ 99%|█████████▉| 1793/1813 [3:18:13<02:09,  6.49s/it]
+ 99%|█████████▉| 1794/1813 [3:18:20<02:06,  6.64s/it]
+ 99%|█████████▉| 1795/1813 [3:18:27<01:59,  6.66s/it]
+ 99%|█████████▉| 1796/1813 [3:18:33<01:53,  6.68s/it]
+ 99%|█████████▉| 1797/1813 [3:18:39<01:44,  6.55s/it]
+ 99%|█████████▉| 1798/1813 [3:18:49<01:50,  7.37s/it]
+ 99%|█████████▉| 1799/1813 [3:18:55<01:40,  7.14s/it]
+ 99%|█████████▉| 1800/1813 [3:19:00<01:22,  6.36s/it]
+                                                     
+{'loss': '0.9417', 'grad_norm': '1.431', 'learning_rate': '3.497e-09', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448, 'train_runtime': '1.194e+04', 'train_tokens_per_second': '1015'}
+
+ 99%|█████████▉| 1800/1813 [3:19:00<01:22,  6.36s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.78it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.23it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.32it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.44it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.05it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.98it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.82it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.04it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.59it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:55,  5.58it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.35it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.11it/s]
+
+  4%|▍         | 39/1000 [00:05<02:07,  7.56it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.08it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.32it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.04it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.35it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.73it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.90it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.94it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.75it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.29it/s]
+
+  6%|▌         | 59/1000 [00:08<01:55,  8.18it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.86it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.18it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.94it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.51it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.77it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:48,  8.34it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.61it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.32it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.44it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.21it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.04it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.51it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.80it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.22it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.07it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.14it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  8.98it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.84it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25,  9.97it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.38it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.45it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:26,  9.88it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.01it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:26,  9.80it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:10,  6.48it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:47,  7.80it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:39,  8.47it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:48,  7.73it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:52,  7.43it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:11,  6.36it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:28,  5.64it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.16it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.76it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.82it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.15it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.43it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.79it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.56it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.49it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.74it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.89it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.37it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.10it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.74it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.68it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.78it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.64it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.95it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 210/1000 [00:27<01:32,  8.50it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.69it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.73it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.13it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.49it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.54it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.22it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.93it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.44it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.96it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.34it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.01it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.46it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:08,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:25,  8.72it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.66it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.32it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.72it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.99it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.06it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.87it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.16it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.13it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.77it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:40,  6.94it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.77it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.79it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.21it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.27it/s]
+
+ 32%|███▏      | 315/1000 [00:42<01:06, 10.24it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.57it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.03it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.80it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.01it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.86it/s]
+
+ 34%|███▍      | 344/1000 [00:46<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.56it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.37it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.58it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.43it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:24,  7.42it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:22,  7.60it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:18,  8.00it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:41,  6.18it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:54,  5.47it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:41,  6.16it/s]
+
+ 38%|███▊      | 379/1000 [00:51<01:16,  8.14it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.17it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.56it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.07it/s]
+
+ 39%|███▊      | 387/1000 [00:52<01:21,  7.56it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.30it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.84it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.50it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  9.01it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.52it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.84it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.67it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.51it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.43it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.82it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+
+ 41%|████▏     | 414/1000 [00:56<01:27,  6.73it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:27,  6.64it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:11,  8.10it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:09,  8.34it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.42it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.64it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.91it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.34it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.93it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:16,  7.52it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:13,  7.81it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.52it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.01it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.80it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.52it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.09it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.82it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.21it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.38it/s]
+
+ 47%|████▋     | 474/1000 [01:04<01:25,  6.18it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.19it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.29it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.10it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.51it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.04it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.29it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.61it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.20it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.44it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:08<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:09<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.58it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:14,  6.33it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:15,  6.26it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:58,  7.99it/s]
+
+ 53%|█████▎    | 533/1000 [01:12<00:48,  9.62it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:10,  6.63it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.32it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:20,  5.74it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:15<00:59,  7.52it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.17it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.94it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.76it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.87it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.07it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.36it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.76it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:07,  6.39it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:08,  6.29it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.81it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:14,  5.71it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.57it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.40it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:03,  6.28it/s]
+
+ 60%|██████    | 602/1000 [01:22<01:00,  6.61it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:10,  5.64it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.04it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:50,  7.81it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.44it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.71it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.78it/s]
+
+ 62%|██████▏   | 616/1000 [01:24<00:50,  7.61it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.74it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.10it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.44it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.63it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.43it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.96it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.18it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.09it/s]
+
+ 65%|██████▍   | 647/1000 [01:28<00:46,  7.57it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.51it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:56,  6.14it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:45,  7.53it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:39,  8.75it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.04it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:57,  5.96it/s]
+
+ 66%|██████▌   | 661/1000 [01:30<00:45,  7.52it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.46it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:03,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.58it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.63it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.91it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.89it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.86it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.14it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.83it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:48,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:36<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.69it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.35it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.79it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.92it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.14it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:39<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.14it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.61it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.31it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+
+ 74%|███████▎  | 735/1000 [01:40<00:26,  9.88it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.48it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.74it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.28it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.33it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.58it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.43it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.63it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.66it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.59it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:33,  7.29it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.67it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.45it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.76it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.13it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.37it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.77it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.61it/s]
+
+ 78%|███████▊  | 780/1000 [01:46<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.36it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.99it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|███████▉  | 799/1000 [01:48<00:30,  6.56it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:30,  6.53it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.70it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:24,  7.90it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:20,  9.36it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  9.10it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.35it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.38it/s]
+
+ 81%|████████▏ | 813/1000 [01:50<00:20,  9.10it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.35it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.21it/s]
+
+ 82%|████████▏ | 821/1000 [01:51<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.81it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.74it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.99it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.78it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:53<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.61it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.14it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.86it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:56<00:14, 10.05it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.51it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.76it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.64it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.74it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.27it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.24it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.41it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:14,  8.67it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.80it/s]
+
+ 88%|████████▊ | 881/1000 [01:59<00:15,  7.54it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.49it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.90it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.12it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.32it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.76it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.83it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.07it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.06it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:04<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.16it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.64it/s]
+
+ 93%|█████████▎| 927/1000 [02:05<00:09,  8.08it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.64it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:06<00:10,  6.60it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.84it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.12it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.33it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.69it/s]
+
+ 95%|█████████▌| 950/1000 [02:08<00:05,  8.81it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.40it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.15it/s]
+
+ 96%|█████████▌| 956/1000 [02:09<00:05,  7.39it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.39it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.80it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.43it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.40it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.79it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.81it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.42it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+
+ 99%|█████████▊| 986/1000 [02:13<00:01,  8.36it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:14<00:01,  6.14it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.04it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.70it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.62it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.8004', 'eval_runtime': '135.6', 'eval_samples_per_second': '7.375', 'eval_steps_per_second': '7.375', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448}
+
+ 99%|█████████▉| 1800/1813 [3:21:16<01:22,  6.36s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.62it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_000_lr_2.000e-05/checkpoint-1800
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+ 99%|█████████▉| 1801/1813 [3:21:24<09:33, 47.76s/it]
+ 99%|█████████▉| 1802/1813 [3:21:30<06:25, 35.00s/it]
+ 99%|█████████▉| 1803/1813 [3:21:35<04:21, 26.16s/it]
+100%|█████████▉| 1804/1813 [3:21:43<03:05, 20.58s/it]
+100%|█████████▉| 1805/1813 [3:21:49<02:09, 16.25s/it]
+100%|█████████▉| 1806/1813 [3:21:55<01:33, 13.35s/it]
+100%|█████████▉| 1807/1813 [3:22:03<01:09, 11.52s/it]
+100%|█████████▉| 1808/1813 [3:22:10<00:51, 10.30s/it]
+100%|█████████▉| 1809/1813 [3:22:16<00:36,  9.09s/it]
+100%|█████████▉| 1810/1813 [3:22:22<00:23,  7.99s/it]
+                                                     
+{'loss': '0.9077', 'grad_norm': '1.875', 'learning_rate': '2.855e-10', 'epoch': '0.9986', 'num_input_tokens_seen': 12187432, 'train_runtime': '1.214e+04', 'train_tokens_per_second': '1004'}
+
+100%|█████████▉| 1810/1813 [3:22:22<00:23,  7.99s/it]
+100%|█████████▉| 1811/1813 [3:22:27<00:14,  7.21s/it]
+100%|█████████▉| 1812/1813 [3:22:32<00:06,  6.61s/it]
+100%|██████████| 1813/1813 [3:22:35<00:00,  5.51s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.77it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.23it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.07it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.99it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.04it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:55,  5.58it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.35it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:43,  5.95it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.75it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.74it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.29it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.11it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.85it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.19it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.01it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.83it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.33it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.46it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.22it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.06it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.54it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.81it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.19it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.69it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.14it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.90it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.08it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.04it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.42it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.23it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:22,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.12it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.68it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.77it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.65it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.95it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.53it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.67it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.16it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.79it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.43it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.03it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.22it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.93it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.44it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.96it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:31,  8.34it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.01it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.73it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.35it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.52it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.05it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.01it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.84it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.75it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.78it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.84it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.11it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.75it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.07it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.94it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.78it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.78it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.19it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.24it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.23it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.57it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.88it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.57it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.86it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.63it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.94it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.58it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.42it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.23it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.77it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.90it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.20it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.19it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.64it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.36it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.78it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.82it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.44it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.93it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:16,  7.53it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.83it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.55it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.04it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.80it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.51it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.31it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.51it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.82it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.27it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.84it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.00it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.30it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.84it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.56it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.07it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.32it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50, 10.00it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.45it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.42it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.38it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.57it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.71it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.68it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.35it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.76it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.85it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:06,  6.54it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.55it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.58it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.90it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:02,  6.46it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.40it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.67it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.72it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.78it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.61it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.74it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.08it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  7.97it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.16it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.43it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.61it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.17it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.20it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.84it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.41it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.97it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.11it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  5.99it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.55it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.64it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.99it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.89it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:48,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.93it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.58it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.21it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.00it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.61it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.49it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.70it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.34it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.15it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.62it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.88it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.51it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.29it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.34it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.60it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.44it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.67it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.62it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.55it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:36,  6.55it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.42it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:30,  7.72it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.61it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.54it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.75it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.86it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.56it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.93it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.35it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.14it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.99it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.98it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.85it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.16it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.21it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.24it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.30it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.13it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.18it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.03it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.81it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.99it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.00it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.22it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.45it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.04it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.51it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.75it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.62it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.73it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.26it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.28it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.79it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.88it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.78it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.77it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.85it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.69it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.07it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.59it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.38it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.18it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.67it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.82it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.13it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.70it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.03it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.21it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.81it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.81it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.14it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.8005', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.387', 'eval_steps_per_second': '7.387', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:24:51<00:00,  5.51s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_000_lr_2.000e-05/checkpoint-1813
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+
+Training completed. Do not forget to share your model on huggingface.co/models =)
+
+
+
+                                                     
+{'train_runtime': '1.229e+04', 'train_samples_per_second': '2.359', 'train_steps_per_second': '0.147', 'train_loss': '1.019', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:24:52<00:00,  5.51s/it]
+100%|██████████| 1813/1813 [3:24:52<00:00,  6.78s/it]
+Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_000_lr_2.000e-05
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+***** train metrics *****
+  epoch                    =         1.0
+  num_input_tokens_seen    =    12201816
+  total_flos               = 487685654GF
+  train_loss               =      1.0191
+  train_runtime            =  3:24:52.13
+  train_samples_per_second =       2.359
+  train_steps_per_second   =       0.147
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_000_lr_2.000e-05/training_loss.png
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_000_lr_2.000e-05/training_eval_loss.png
+[WARNING|2026-10-12 23:25:44] llamafactory.extras.ploting:149 >> No metric eval_accuracy to plot.
+
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+  0%|          | 3/1000 [00:00<01:02, 15.88it/s]
+  0%|          | 5/1000 [00:00<02:17,  7.25it/s]
+  1%|          | 7/1000 [00:00<01:58,  8.35it/s]
+  1%|          | 9/1000 [00:01<02:32,  6.48it/s]
+  1%|          | 11/1000 [00:01<02:02,  8.10it/s]
+  1%|▏         | 13/1000 [00:01<02:20,  7.02it/s]
+  1%|▏         | 14/1000 [00:01<02:23,  6.85it/s]
+  2%|▏         | 15/1000 [00:02<02:44,  5.98it/s]
+  2%|▏         | 16/1000 [00:02<02:51,  5.74it/s]
+  2%|▏         | 18/1000 [00:02<02:41,  6.07it/s]
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+  2%|▏         | 20/1000 [00:03<03:32,  4.60it/s]
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+  2%|▎         | 25/1000 [00:03<02:12,  7.37it/s]
+  3%|▎         | 26/1000 [00:03<02:35,  6.26it/s]
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+  3%|▎         | 32/1000 [00:05<02:41,  5.98it/s]
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+  4%|▎         | 35/1000 [00:05<02:05,  7.66it/s]
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+  4%|▍         | 40/1000 [00:05<02:03,  7.74it/s]
+  4%|▍         | 42/1000 [00:06<01:45,  9.09it/s]
+  4%|▍         | 44/1000 [00:06<01:32, 10.31it/s]
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+  5%|▍         | 47/1000 [00:06<02:09,  7.37it/s]
+  5%|▍         | 49/1000 [00:06<01:48,  8.75it/s]
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+  5%|▌         | 53/1000 [00:07<01:26, 10.94it/s]
+  6%|▌         | 55/1000 [00:07<01:27, 10.75it/s]
+  6%|▌         | 57/1000 [00:07<01:41,  9.31it/s]
+  6%|▌         | 59/1000 [00:08<01:54,  8.21it/s]
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+  7%|▋         | 72/1000 [00:09<02:17,  6.73it/s]
+  7%|▋         | 74/1000 [00:09<01:53,  8.13it/s]
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+  8%|▊         | 82/1000 [00:10<01:39,  9.21it/s]
+  8%|▊         | 84/1000 [00:11<01:54,  8.03it/s]
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+  9%|▊         | 87/1000 [00:11<02:49,  5.39it/s]
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+  9%|▉         | 90/1000 [00:12<02:05,  7.26it/s]
+  9%|▉         | 91/1000 [00:12<02:29,  6.09it/s]
+  9%|▉         | 92/1000 [00:12<02:26,  6.18it/s]
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+ 10%|▉         | 99/1000 [00:13<01:47,  8.38it/s]
+ 10%|█         | 101/1000 [00:13<01:33,  9.65it/s]
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+ 11%|█         | 106/1000 [00:14<01:35,  9.34it/s]
+ 11%|█         | 108/1000 [00:14<01:25, 10.47it/s]
+ 11%|█         | 110/1000 [00:14<01:19, 11.24it/s]
+ 11%|█         | 112/1000 [00:14<01:13, 12.07it/s]
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.55it/s]
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.82it/s]
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.22it/s]
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.07it/s]
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.66it/s]
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.90it/s]
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.03it/s]
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.06it/s]
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.91it/s]
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.03it/s]
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.43it/s]
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.66it/s]
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.98it/s]
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.62it/s]
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.66it/s]
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.84it/s]
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.15it/s]
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.51it/s]
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+ 18%|█▊        | 182/1000 [00:24<02:22,  5.72it/s]
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.11it/s]
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.64it/s]
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.76it/s]
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.20it/s]
+ 20%|██        | 200/1000 [00:26<01:15, 10.63it/s]
+ 20%|██        | 202/1000 [00:26<01:31,  8.76it/s]
+ 20%|██        | 204/1000 [00:26<01:32,  8.61it/s]
+ 21%|██        | 206/1000 [00:26<01:20,  9.92it/s]
+ 21%|██        | 208/1000 [00:27<01:33,  8.52it/s]
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+ 21%|██        | 212/1000 [00:27<01:21,  9.68it/s]
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.16it/s]
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.83it/s]
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.38it/s]
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.96it/s]
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.47it/s]
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.98it/s]
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.34it/s]
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.24it/s]
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.96it/s]
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.03it/s]
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.05it/s]
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.74it/s]
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.99it/s]
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.06it/s]
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+ 30%|███       | 300/1000 [00:40<01:42,  6.84it/s]
+ 30%|███       | 301/1000 [00:40<01:38,  7.11it/s]
+ 30%|███       | 303/1000 [00:40<01:19,  8.75it/s]
+ 30%|███       | 304/1000 [00:40<01:26,  8.07it/s]
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+ 31%|███       | 309/1000 [00:41<01:18,  8.79it/s]
+ 31%|███       | 311/1000 [00:41<01:07, 10.19it/s]
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.26it/s]
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.24it/s]
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.56it/s]
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.88it/s]
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.05it/s]
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.21it/s]
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.93it/s]
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.54it/s]
+ 36%|███▌      | 361/1000 [00:47<01:07,  9.40it/s]
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.36it/s]
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.72it/s]
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.19it/s]
+ 38%|███▊      | 379/1000 [00:50<01:15,  8.17it/s]
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.19it/s]
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.55it/s]
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.08it/s]
+ 39%|███▊      | 387/1000 [00:51<01:21,  7.56it/s]
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.30it/s]
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.84it/s]
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.49it/s]
+ 39%|███▉      | 393/1000 [00:52<01:07,  9.00it/s]
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.52it/s]
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.83it/s]
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+ 40%|████      | 400/1000 [00:53<01:45,  5.67it/s]
+ 40%|████      | 402/1000 [00:54<01:41,  5.91it/s]
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+ 41%|████      | 406/1000 [00:54<01:09,  8.50it/s]
+ 41%|████      | 408/1000 [00:54<01:32,  6.39it/s]
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+ 41%|████      | 411/1000 [00:55<01:41,  5.83it/s]
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.05it/s]
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.46it/s]
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.96it/s]
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.84it/s]
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.52it/s]
+ 43%|████▎     | 433/1000 [00:58<00:51, 10.99it/s]
+ 44%|████▎     | 435/1000 [00:58<00:48, 11.75it/s]
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.49it/s]
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.07it/s]
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.29it/s]
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.19it/s]
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.50it/s]
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.27it/s]
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.84it/s]
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+ 47%|████▋     | 471/1000 [01:03<01:10,  7.54it/s]
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.38it/s]
+ 47%|████▋     | 474/1000 [01:03<01:22,  6.40it/s]
+ 48%|████▊     | 476/1000 [01:04<01:23,  6.30it/s]
+ 48%|████▊     | 477/1000 [01:04<01:36,  5.44it/s]
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.10it/s]
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.55it/s]
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.02it/s]
+ 48%|████▊     | 484/1000 [01:05<01:11,  7.23it/s]
+ 48%|████▊     | 485/1000 [01:05<01:22,  6.27it/s]
+ 49%|████▊     | 486/1000 [01:05<01:22,  6.27it/s]
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.96it/s]
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.60it/s]
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.18it/s]
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.25it/s]
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.96it/s]
+ 50%|████▉     | 496/1000 [01:06<01:01,  8.17it/s]
+ 50%|████▉     | 498/1000 [01:07<01:00,  8.26it/s]
+ 50%|████▉     | 499/1000 [01:07<01:16,  6.57it/s]
+ 50%|█████     | 500/1000 [01:07<01:25,  5.86it/s]
+ 50%|█████     | 502/1000 [01:07<01:06,  7.48it/s]
+ 50%|█████     | 503/1000 [01:07<01:13,  6.77it/s]
+ 50%|█████     | 505/1000 [01:08<01:15,  6.54it/s]
+ 51%|█████     | 506/1000 [01:08<01:25,  5.78it/s]
+ 51%|█████     | 508/1000 [01:08<01:06,  7.39it/s]
+ 51%|█████     | 510/1000 [01:08<00:54,  8.92it/s]
+ 51%|█████     | 512/1000 [01:08<00:49,  9.94it/s]
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.52it/s]
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.09it/s]
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.06it/s]
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.55it/s]
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.58it/s]
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.13it/s]
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.73it/s]
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.05it/s]
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.49it/s]
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.17it/s]
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.94it/s]
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.86it/s]
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.05it/s]
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.36it/s]
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.74it/s]
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.82it/s]
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.72it/s]
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.55it/s]
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.58it/s]
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.40it/s]
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+ 60%|██████    | 604/1000 [01:22<01:09,  5.66it/s]
+ 61%|██████    | 606/1000 [01:22<00:55,  7.08it/s]
+ 61%|██████    | 608/1000 [01:23<00:50,  7.71it/s]
+ 61%|██████    | 610/1000 [01:23<00:46,  8.37it/s]
+ 61%|██████    | 612/1000 [01:23<00:40,  9.65it/s]
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.75it/s]
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.58it/s]
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.73it/s]
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.08it/s]
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.02it/s]
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.19it/s]
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.44it/s]
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.61it/s]
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.18it/s]
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.44it/s]
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.18it/s]
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.70it/s]
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.91it/s]
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.47it/s]
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.64it/s]
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.70it/s]
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.98it/s]
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.93it/s]
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.88it/s]
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.83it/s]
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+ 70%|██████▉   | 696/1000 [01:34<00:48,  6.33it/s]
+ 70%|██████▉   | 698/1000 [01:35<00:44,  6.85it/s]
+ 70%|██████▉   | 699/1000 [01:35<00:50,  6.00it/s]
+ 70%|███████   | 701/1000 [01:35<00:39,  7.55it/s]
+ 70%|███████   | 702/1000 [01:35<00:53,  5.58it/s]
+ 70%|███████   | 704/1000 [01:36<00:41,  7.18it/s]
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+ 71%|███████   | 707/1000 [01:36<00:41,  7.01it/s]
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+ 71%|███████   | 710/1000 [01:37<00:44,  6.48it/s]
+ 71%|███████   | 712/1000 [01:37<00:37,  7.69it/s]
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.37it/s]
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.07it/s]
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.30it/s]
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.13it/s]
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.61it/s]
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.53it/s]
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.53it/s]
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.78it/s]
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.35it/s]
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.60it/s]
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.44it/s]
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.16it/s]
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.24it/s]
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.39it/s]
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.31it/s]
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.71it/s]
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.13it/s]
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.39it/s]
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.94it/s]
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.36it/s]
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.97it/s]
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.21it/s]
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+ 81%|████████  | 807/1000 [01:49<00:21,  8.96it/s]
+ 81%|████████  | 809/1000 [01:49<00:18, 10.19it/s]
+ 81%|████████  | 811/1000 [01:49<00:16, 11.22it/s]
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.06it/s]
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.32it/s]
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.12it/s]
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.17it/s]
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.74it/s]
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.98it/s]
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.78it/s]
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.00it/s]
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.74it/s]
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.22it/s]
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.61it/s]
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.15it/s]
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.76it/s]
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.56it/s]
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.86it/s]
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.45it/s]
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.04it/s]
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.18it/s]
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.50it/s]
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.74it/s]
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.60it/s]
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.66it/s]
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.30it/s]
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.26it/s]
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.27it/s]
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.77it/s]
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.83it/s]
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.04it/s]
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.30it/s]
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.76it/s]
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.38it/s]
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.16it/s]
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.64it/s]
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.09it/s]
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.13it/s]
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.33it/s]
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.07it/s]
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.38it/s]
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.70it/s]
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.82it/s]
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.39it/s]
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.04it/s]
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.44it/s]
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.79it/s]
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.81it/s]
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.43it/s]
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.43it/s]
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.43it/s]
+100%|█████████▉| 995/1000 [02:14<00:00,  5.47it/s]
+100%|█████████▉| 997/1000 [02:14<00:00,  6.99it/s]
+100%|█████████▉| 999/1000 [02:15<00:00,  6.74it/s]
+100%|██████████| 1000/1000 [02:15<00:00,  6.63it/s]
+100%|██████████| 1000/1000 [02:15<00:00,  7.39it/s]
+***** eval metrics *****
+  epoch                   =        1.0
+  eval_loss               =     0.8005
+  eval_runtime            = 0:02:15.43
+  eval_samples_per_second =      7.384
+  eval_steps_per_second   =      7.384
+  num_input_tokens_seen   =   12201816
+Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
diff --git a/optuna_30k_experiments/logs/trial_001_lr_5.000e-05.log b/optuna_30k_experiments/logs/trial_001_lr_5.000e-05.log
new file mode 100644
index 0000000..be44fe1
--- /dev/null
+++ b/optuna_30k_experiments/logs/trial_001_lr_5.000e-05.log
@@ -0,0 +1,12421 @@
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/torch/cuda/__init__.py:734: UserWarning: Can't initialize NVML
+  warnings.warn("Can't initialize NVML")
+[INFO|2026-10-12 23:28:06] llamafactory.hparams.parser:651 >> Process rank: 0, world size: 1, device: cuda:0, distributed training: False, compute dtype: torch.bfloat16
+Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
+[INFO|configuration_utils.py:765] 2026-10-12 23:28:07,192 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-12 23:28:07,201 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|configuration_utils.py:765] 2026-10-12 23:28:08,818 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-12 23:28:08,821 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|configuration_utils.py:765] 2026-10-12 23:28:08,965 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-12 23:28:08,969 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-12 23:28:09] llamafactory.data.loader:144 >> Loading dataset aya_slovak_hpo_30k.jsonl...
+training example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28828, 2176, 28711, 28863, 29114, 10268, 28709, 6612, 28714, 2892, 493, 363, 6294, 10719, 363, 3811, 28723, 7379, 626, 28889, 307, 14637, 311, 28768, 2218, 22885, 3748, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 284, 8341, 6729, 345, 28738, 28739, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28723, 524, 4634, 28711, 570, 1879, 261, 28863, 532, 613, 14483, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 345, 15962, 28739, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 1879, 15173, 537, 284, 8341, 705, 28723, 367, 280, 28879, 570, 4415, 28716, 28797, 9335, 3175, 29826, 626, 323, 412, 8514, 16986, 29826, 28708, 345, 15962, 28739, 1879, 15173, 537, 1221, 4566, 28718, 325, 28707, 28723, 461, 28723, 724, 494, 8891, 24160, 28725, 8582, 28862, 24160, 28725, 20532, 28764, 271, 28725, 438, 29365, 15745, 1319, 12035, 28862, 8269, 284, 28827, 2762, 28709, 264, 4798, 8359, 1450, 8325, 12121, 1798, 420, 762, 1089, 290, 28906, 18765, 724, 2331, 29114, 1221, 4304, 1780, 363, 13881, 29826, 316, 2085, 28708, 28723, 22627, 28873, 263, 335, 20208, 284, 28827, 2762, 28709, 264, 4798, 5424, 10056, 1286, 290, 28906, 18765, 724, 2331, 29114, 11323, 10719, 12490, 363, 13881, 29826, 316, 284, 8341, 2390, 28723, 22127, 280, 28879, 21022, 1150, 424, 724, 28827, 1144, 28873, 340, 17066, 270, 21022, 485, 284, 8341, 705, 28725, 446, 6909, 28797, 637, 28705, 29114, 28708, 28879, 4798, 6612, 28827, 1100, 28768, 28863, 28723, 320, 412, 28879, 637, 15612, 28716, 28889, 28726, 1150, 424, 19523, 412, 28873, 276, 28797, 2390, 284, 8341, 2390, 363, 23407, 11277, 9538, 2085, 28708, 28723, 9532, 2144, 686, 7865, 28718, 341, 14475, 1687, 28706, 18037, 452, 270, 8269, 284, 8341, 705, 28725, 290, 4827, 486, 2710, 2144, 332, 1800, 16344, 29114, 6588, 269, 28862, 28711, 28863, 686, 1653, 5388, 28718, 710, 261, 1296, 28709, 284, 8341, 705, 28723, 351, 28711, 1371, 28797, 1776, 29826, 8269, 284, 8341, 705, 268, 28863, 1776, 29826, 8269, 1879, 3497, 598, 8269, 8631, 28879, 279, 412, 28725, 12121, 7988, 268, 28863, 24270, 12115, 28797, 1879, 5626, 263, 28862, 8269, 8631, 28879, 279, 412, 28723, 318, 15612, 1473, 276, 15690, 2245, 626, 15690, 307, 14637, 311, 28768, 300, 15469, 28711, 570, 446, 450, 4798, 29826, 333, 28729, 1879, 319, 3976, 294, 405, 28768, 4882, 405, 28725, 534, 28724, 2710, 724, 313, 4827, 2245, 626, 28889, 20532, 28764, 271, 28723, 10650, 300, 12235, 28827, 28873, 424, 268, 27172, 2245, 28723, 1054, 15504, 28879, 424, 724, 313, 8768, 4545, 504, 270, 8269, 2124, 2245, 626, 12689, 4584, 10906, 28718, 710, 18782, 28879, 28715, 28797, 1135, 10356, 28723, 2645, 28764, 12275, 28820, 28873, 29114, 13067, 549, 2892, 5872, 3811, 8631, 25991, 3026, 29114, 408, 28906, 28764, 485, 2082, 1403, 12121, 1798, 1773, 29826, 28729, 22160, 710, 408, 28906, 28764, 485, 1135, 10874, 1776, 4536, 28873, 300, 23407, 11277, 9538, 2085, 28708, 28723, 22127, 3361, 13470, 1150, 317, 28873, 424, 2082, 7330, 511, 2085, 28708, 28723, 401, 492, 13591, 28820, 261, 2220, 28862, 10766, 381, 28827, 486, 29114, 363, 28879, 4689, 3418, 28879, 21775, 29114, 280, 28723, 418, 28820, 11277, 28716, 6294, 26177, 903, 12689, 6612, 20122, 28733, 12868, 291, 2124, 2085, 28708, 1879, 6670, 28862, 9348, 493, 686, 375, 5620, 386, 28862, 28827, 28725, 23374, 4536, 25306, 16388, 957, 450, 15612, 28764, 2126, 29114, 511, 2152, 363, 25769, 14110, 17413, 26177, 903, 28873, 15808, 19396, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 7379, 1879, 277, 7939, 1879, 724, 313, 8768, 2245, 28718, 363, 446, 551, 9538, 12121, 1798, 24473, 21694, 300, 261, 5869, 28723, 384, 28728, 491, 28712, 4566, 15469, 28711, 570, 1879, 2245, 28725, 534, 28724, 2710, 7463, 15612, 13335, 28728, 4371, 1144, 28723, 10650, 300, 15469, 28711, 570, 1879, 1160, 28709, 28879, 3907, 13288, 491, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 28725, 534, 28724, 637, 686, 598, 5237, 5945, 17984, 28718, 2117, 18904, 3146, 515, 13288, 25208, 28723, 367, 280, 28879, 570, 12472, 1006, 5872, 28863, 358, 446, 311, 4845, 1879, 6149, 783, 494, 28718, 2245, 28718, 363, 17984, 28718, 405, 13288, 491, 28723, 13343, 29826, 28729, 385, 29114, 284, 8341, 705, 28747, 399, 3711, 3175, 29826, 626, 12990, 17984, 28718, 2117, 268, 6125, 15690, 345, 28738, 28739, 16986, 29826, 28708, 1773, 29826, 10641, 2124, 345, 28738, 28739, 582, 20524, 12214, 1773, 29826, 28729, 385, 29114, 284, 8341, 705, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 15173, 537, 28718, 325, 849, 3748, 28725, 446, 6909, 28820, 3061, 28720, 300, 28827, 1520, 6612, 749, 1780, 20532, 28764, 271, 586, 28873, 28710, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28731, 1879, 686, 6729, 1773, 29826, 28729, 22160, 284, 8341, 705, 28723, 550, 11245, 412, 28747, 550, 11245, 412, 1683, 28827, 16032, 271, 1342, 4295, 3191, 316, 28729, 6449, 2245, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 345, 28741, 28739, 1879, 22954, 338, 28718, 297, 12689, 345, 28741, 28739, 686, 2129, 28710, 29114, 16986, 28863, 358, 28723, 17563, 971, 28747, 17563, 971, 582, 20524, 12214, 1342, 28764, 1193, 1342, 4295, 5765, 6175, 2959, 15416, 28710, 13288, 491, 3589, 28723, 9532, 484, 28717, 3811, 18904, 17101, 29114, 22399, 971, 28725, 15612, 13335, 28728, 4371, 28711, 570, 13288, 20996, 28725, 446, 6909, 28797, 484, 28717, 3811, 18904, 17101, 29114, 28725, 264, 2513, 300, 8631, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 268, 345, 28741, 28739, 264, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Zhrnúť tento článok v jednej vete. Textový nástroj je ikona, ktorá sa podobá písmenu "T" v paneli nástrojov vľavo. Kliknite na túto ikonu. Použite rozbaľovaciu ponuku vedľa "Character" v hornej časti obrazovky na výber písma. Použite druhé rozbaľovacie menu vedľa "Character" na výber formátu (t. j. pravidelný, tučný, kurzor, atď.). Tradičné písmo ako Times New Roman alebo Garamond môže pridať formálny vzhľad loga. Bezšerifové písmo ako Helvetica môže pridať modernejší vzhľad písmu. Nepoužívajte príliš dekoratívne písma, ktoré sa ťažko čítajú. Tiež sa vyhýbajte zmiešanému písmu v návrhu loga. Ak si z internetu stiahnete bezplatné písma, mali by ste si uvedomiť licenčnú zmluvu pre tieto písma. Mnohé voľné písma sú voľné na osobné použitie, ale nie sú povolené na komerčné použitie. S vybraným textovým nástrojom kliknite kdekoľvek na grafickej doske, aby ste pridali textový kurzor. Potom napíšte svoj text. Zvážte pridanie samostatného textového objektu pre každé slovo. Obzvlášť ak plánujete používať rôzne farby alebo veľkosti pre rôzne slová vo vašom návrhu loga. Nepridávaj ešte farbu do loga. Farebná tlač nemusí byť vždy možnosťou. Návrh jednoduchého čierno-bieleho loga na začiatok zabezpečí, že vaše logo bude vyzerať dobre v jeho najjednoduchšej forme. Môžete tiež použiť nástroj Text na ceste na pridanie textu v kruhu alebo zakrivenom tvare. Dvakrát kliknite na text, aby ste ho vyzdvihli. Potom kliknite na položku znak v hornej časti obrazovky, aby sa zobrazila ponuka nastavenia znakov. Použite nasledujúce kroky na úpravu textu v ponuke znak. Veľkosť písma: Rozbaľovacia ponuka s malým "T" vedľa veľkého "T" upravuje veľkosť písma. Môžete tiež použiť nástroj výberu (ikona, ktorá pripomína čierny kurzor myši v paneli nástrojov vľavo) na zmenu veľkosti písma. Vedenie: Vedenie mení priestor medzi riadkami textu. Použite rozbaľovaciu ponuku vedľa ikony, ktorá sa podobá "A" na vrchu iného "A" zmeniť vedúce. Kerning: Kerning upravuje medzery medzi konkrétnymi znakmi. Ak chcete nastaviť kerning, vyzdvihnite znaky, ktoré chcete nastaviť, a potom použite rozbaľovaciu ponuku vedľa ikony s "A" aVyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+labels:
+Vyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+eval example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28965, 28709, 637, 8631, 25991, 3026, 28725, 13067, 10531, 28729, 350, 2591, 686, 3023, 28862, 28827, 284, 28873, 269, 13096, 28711, 28863, 6149, 3399, 28804, 550, 13881, 29826, 316, 300, 1879, 7871, 338, 28820, 6936, 1150, 28863, 7340, 6957, 8905, 3907, 12235, 28827, 28873, 424, 5765, 772, 28725, 446, 6909, 28889, 14848, 912, 12214, 4321, 28720, 751, 29365, 28723, 351, 28906, 18765, 298, 486, 29114, 28705, 28740, 387, 28705, 28750, 28734, 22116, 28827, 28723, 8294, 772, 28747, 13, 13, 27332, 12107, 28747, 13, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Čo sa používa, ak škodca zničí pšeničnú úrodu? Vzhľadom na predchádzajúcu otázku napíšte kontext, ktorý obsahuje odpoveď. Môže to byť 1 - 20 vetí. Kontext:
+
+### Response:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+labels:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+[INFO|configuration_utils.py:765] 2026-10-12 23:28:10,443 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-12 23:28:10,447 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-12 23:28:10] llamafactory.model.model_utils.kv_cache:144 >> KV cache is disabled during training.
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/unsloth/__init__.py:165: UserWarning: WARNING: Unsloth should be imported before [trl, transformers, peft] to ensure all optimizations are applied. Your code may run slower or encounter memory issues without these optimizations.
+
+Please restructure your imports with 'import unsloth' at the top of your file.
+  from ._gpu_init import *
+🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/torch/cuda/__init__.py:734: UserWarning: Can't initialize NVML
+  warnings.warn("Can't initialize NVML")
+🦥 Unsloth Zoo will now patch everything to make training faster!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: WARNING `trust_remote_code` is True.
+Are you certain you want to do remote code execution?
+==((====))==  Unsloth 2026.6.1: Fast Mistral patching. Transformers: 5.5.0.
+   \\   /|    NVIDIA RTX A4500. Num GPUs = 1. Max memory: 19.698 GB. Platform: Linux.
+O^O/ \_/ \    Torch: 2.6.0+cu124. CUDA: 8.6. CUDA Toolkit: 12.4. Triton: 3.2.0
+\        /    Bfloat16 = TRUE. FA [Xformers = 0.0.29.post3. FA2 = False]
+ "-____-"     Free license: http://github.com/unslothai/unsloth
+Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading weights file model.safetensors from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/model.safetensors.index.json
+Will use dtype=torch.bfloat16 as defined in model's config object
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 2,
+  "use_cache": true
+}
+
+
+Loading weights:   0%|          | 0/291 [00:00<?, ?it/s]
+Loading weights:   2%|▏         | 5/291 [00:00<00:06, 43.71it/s]
+Loading weights:   5%|▌         | 16/291 [00:00<00:03, 80.52it/s]
+Loading weights:  11%|█         | 31/291 [00:00<00:02, 98.69it/s]
+Loading weights:  14%|█▍        | 41/291 [00:00<00:02, 99.07it/s]
+Loading weights:  19%|█▉        | 55/291 [00:00<00:02, 112.67it/s]
+Loading weights:  23%|██▎       | 67/291 [00:00<00:01, 113.60it/s]
+Loading weights:  27%|██▋       | 79/291 [00:00<00:02, 102.83it/s]
+Loading weights:  31%|███▏      | 91/291 [00:00<00:01, 106.07it/s]
+Loading weights:  35%|███▌      | 103/291 [00:00<00:01, 109.04it/s]
+Loading weights:  40%|███▉      | 115/291 [00:01<00:01, 96.83it/s] 
+Loading weights:  44%|████▍     | 128/291 [00:01<00:01, 105.22it/s]
+Loading weights:  48%|████▊     | 139/291 [00:01<00:01, 104.81it/s]
+Loading weights:  52%|█████▏    | 150/291 [00:01<00:01, 101.17it/s]
+Loading weights:  57%|█████▋    | 167/291 [00:01<00:01, 115.85it/s]
+Loading weights:  62%|██████▏   | 179/291 [00:01<00:00, 116.44it/s]
+Loading weights:  66%|██████▋   | 193/291 [00:01<00:00, 120.53it/s]
+Loading weights:  71%|███████   | 206/291 [00:01<00:00, 111.06it/s]
+Loading weights:  76%|███████▌  | 220/291 [00:02<00:00, 111.33it/s]
+Loading weights:  80%|███████▉  | 232/291 [00:02<00:00, 104.77it/s]
+Loading weights:  85%|████████▍ | 247/291 [00:02<00:00, 111.83it/s]
+Loading weights:  89%|████████▉ | 259/291 [00:02<00:00, 109.91it/s]
+Loading weights:  94%|█████████▍| 274/291 [00:02<00:00, 116.55it/s]
+Loading weights:  98%|█████████▊| 286/291 [00:02<00:00, 105.72it/s]
+Loading weights: 100%|██████████| 291/291 [00:02<00:00, 107.89it/s]
+loading configuration file generation_config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/generation_config.json
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2
+}
+
+Could not locate the custom_generate/generate.py inside slovak-nlp/mistral-sk-7b.
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: slovak-nlp/mistral-sk-7b had a bad pad_token (</s>). Using pad_token = <unk>.
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:68: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_fwd
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:85: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_bwd
+[INFO|2026-10-12 23:28:27] llamafactory.model.model_utils.checkpointing:144 >> Gradient checkpointing enabled.
+[INFO|2026-10-12 23:28:27] llamafactory.model.adapter:144 >> Upcasting trainable params to float32.
+[INFO|2026-10-12 23:28:27] llamafactory.model.adapter:144 >> Fine-tuning method: LoRA
+Unsloth 2026.6.1 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers.
+[INFO|2026-10-12 23:28:30] llamafactory.model.loader:144 >> trainable params: 41,943,040 || all params: 7,283,675,136 || trainable%: 0.5758
+The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'pad_token_id': 2}.
+==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 1
+   \\   /|    Num examples = 29,000 | Num Epochs = 1 | Total steps = 1,813
+O^O/ \_/ \    Batch size per device = 1 | Gradient accumulation steps = 16
+\        /    Data Parallel GPUs = 1 | Total batch size (1 x 16 x 1) = 16
+ "-____-"     Trainable parameters = 41,943,040 of 7,283,675,136 (0.58% trained)
+
+  0%|          | 0/1813 [00:00<?, ?it/s]`use_return_dict` is deprecated! Use `return_dict` instead!
+Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
+Unsloth: Will smartly offload gradients to save VRAM!
+
+  0%|          | 1/1813 [00:07<3:36:43,  7.18s/it]
+  0%|          | 2/1813 [00:13<3:29:50,  6.95s/it]
+  0%|          | 3/1813 [00:19<3:12:28,  6.38s/it]
+  0%|          | 4/1813 [00:23<2:46:02,  5.51s/it]
+  0%|          | 5/1813 [00:30<2:58:36,  5.93s/it]
+  0%|          | 6/1813 [00:38<3:15:00,  6.48s/it]
+  0%|          | 7/1813 [00:44<3:15:52,  6.51s/it]
+  0%|          | 8/1813 [00:49<3:02:42,  6.07s/it]
+  0%|          | 9/1813 [00:57<3:18:34,  6.60s/it]
+  1%|          | 10/1813 [01:03<3:09:26,  6.30s/it]
+                                                   
+{'loss': '1.802', 'grad_norm': '2.014', 'learning_rate': '3e-06', 'epoch': '0.005517', 'num_input_tokens_seen': 65928, 'train_runtime': '63.18', 'train_tokens_per_second': '1044'}
+
+  1%|          | 10/1813 [01:03<3:09:26,  6.30s/it]
+  1%|          | 11/1813 [01:09<3:10:26,  6.34s/it]
+  1%|          | 12/1813 [01:17<3:25:05,  6.83s/it]
+  1%|          | 13/1813 [01:22<3:04:06,  6.14s/it]
+  1%|          | 14/1813 [01:28<3:05:19,  6.18s/it]
+  1%|          | 15/1813 [01:33<2:55:34,  5.86s/it]
+  1%|          | 16/1813 [01:39<2:57:02,  5.91s/it]
+  1%|          | 17/1813 [01:45<3:01:09,  6.05s/it]
+  1%|          | 18/1813 [01:51<3:00:48,  6.04s/it]
+  1%|          | 19/1813 [01:58<3:06:25,  6.23s/it]
+  1%|          | 20/1813 [02:04<3:04:29,  6.17s/it]
+                                                   
+{'loss': '1.628', 'grad_norm': '2.526', 'learning_rate': '6.333e-06', 'epoch': '0.01103', 'num_input_tokens_seen': 131352, 'train_runtime': '124.6', 'train_tokens_per_second': '1054'}
+
+  1%|          | 20/1813 [02:04<3:04:29,  6.17s/it]
+  1%|          | 21/1813 [02:10<3:03:16,  6.14s/it]
+  1%|          | 22/1813 [02:15<2:54:08,  5.83s/it]
+  1%|▏         | 23/1813 [02:21<2:49:04,  5.67s/it]
+  1%|▏         | 24/1813 [02:26<2:50:39,  5.72s/it]
+  1%|▏         | 25/1813 [02:34<3:02:30,  6.12s/it]
+  1%|▏         | 26/1813 [02:41<3:12:38,  6.47s/it]
+  1%|▏         | 27/1813 [02:47<3:07:03,  6.28s/it]
+  2%|▏         | 28/1813 [02:53<3:11:00,  6.42s/it]
+  2%|▏         | 29/1813 [03:01<3:25:25,  6.91s/it]
+  2%|▏         | 30/1813 [03:06<3:07:50,  6.32s/it]
+                                                   
+{'loss': '1.523', 'grad_norm': '1.724', 'learning_rate': '9.667e-06', 'epoch': '0.01655', 'num_input_tokens_seen': 197376, 'train_runtime': '186.9', 'train_tokens_per_second': '1056'}
+
+  2%|▏         | 30/1813 [03:06<3:07:50,  6.32s/it]
+  2%|▏         | 31/1813 [03:13<3:07:30,  6.31s/it]
+  2%|▏         | 32/1813 [03:18<2:59:34,  6.05s/it]
+  2%|▏         | 33/1813 [03:24<2:59:21,  6.05s/it]
+  2%|▏         | 34/1813 [03:32<3:13:36,  6.53s/it]
+  2%|▏         | 35/1813 [03:38<3:14:57,  6.58s/it]
+  2%|▏         | 36/1813 [03:45<3:13:45,  6.54s/it]
+  2%|▏         | 37/1813 [03:50<3:01:42,  6.14s/it]
+  2%|▏         | 38/1813 [03:55<2:51:45,  5.81s/it]
+  2%|▏         | 39/1813 [04:01<2:50:05,  5.75s/it]
+  2%|▏         | 40/1813 [04:06<2:49:26,  5.73s/it]
+                                                   
+{'loss': '1.411', 'grad_norm': '1.019', 'learning_rate': '1.3e-05', 'epoch': '0.02207', 'num_input_tokens_seen': 261168, 'train_runtime': '247', 'train_tokens_per_second': '1057'}
+
+  2%|▏         | 40/1813 [04:06<2:49:26,  5.73s/it]
+  2%|▏         | 41/1813 [04:12<2:44:21,  5.57s/it]
+  2%|▏         | 42/1813 [04:18<2:46:51,  5.65s/it]
+  2%|▏         | 43/1813 [04:25<3:03:58,  6.24s/it]
+  2%|▏         | 44/1813 [04:32<3:06:06,  6.31s/it]
+  2%|▏         | 45/1813 [04:39<3:15:35,  6.64s/it]
+  3%|▎         | 46/1813 [04:45<3:08:35,  6.40s/it]
+  3%|▎         | 47/1813 [04:50<3:01:15,  6.16s/it]
+  3%|▎         | 48/1813 [04:57<3:05:56,  6.32s/it]
+  3%|▎         | 49/1813 [05:04<3:06:49,  6.35s/it]
+  3%|▎         | 50/1813 [05:10<3:07:51,  6.39s/it]
+                                                   
+{'loss': '1.412', 'grad_norm': '1.247', 'learning_rate': '1.633e-05', 'epoch': '0.02759', 'num_input_tokens_seen': 330104, 'train_runtime': '310.6', 'train_tokens_per_second': '1063'}
+
+  3%|▎         | 50/1813 [05:10<3:07:51,  6.39s/it]
+  3%|▎         | 51/1813 [05:18<3:18:43,  6.77s/it]
+  3%|▎         | 52/1813 [05:27<3:40:52,  7.53s/it]
+  3%|▎         | 53/1813 [05:33<3:26:37,  7.04s/it]
+  3%|▎         | 54/1813 [05:41<3:31:30,  7.21s/it]
+  3%|▎         | 55/1813 [05:47<3:24:55,  6.99s/it]
+  3%|▎         | 56/1813 [05:54<3:23:47,  6.96s/it]
+  3%|▎         | 57/1813 [05:59<3:09:43,  6.48s/it]
+  3%|▎         | 58/1813 [06:08<3:26:58,  7.08s/it]
+  3%|▎         | 59/1813 [06:15<3:27:03,  7.08s/it]
+  3%|▎         | 60/1813 [06:20<3:12:16,  6.58s/it]
+                                                   
+{'loss': '1.281', 'grad_norm': '1.398', 'learning_rate': '1.967e-05', 'epoch': '0.0331', 'num_input_tokens_seen': 407792, 'train_runtime': '380.7', 'train_tokens_per_second': '1071'}
+
+  3%|▎         | 60/1813 [06:20<3:12:16,  6.58s/it]
+  3%|▎         | 61/1813 [06:25<2:59:16,  6.14s/it]
+  3%|▎         | 62/1813 [06:31<2:53:58,  5.96s/it]
+  3%|▎         | 63/1813 [06:37<2:53:02,  5.93s/it]
+  4%|▎         | 64/1813 [06:43<2:51:52,  5.90s/it]
+  4%|▎         | 65/1813 [06:50<3:04:05,  6.32s/it]
+  4%|▎         | 66/1813 [06:56<3:05:42,  6.38s/it]
+  4%|▎         | 67/1813 [07:02<2:58:28,  6.13s/it]
+  4%|▍         | 68/1813 [07:08<2:59:49,  6.18s/it]
+  4%|▍         | 69/1813 [07:16<3:09:07,  6.51s/it]
+  4%|▍         | 70/1813 [07:23<3:18:01,  6.82s/it]
+                                                   
+{'loss': '1.15', 'grad_norm': '1.272', 'learning_rate': '2.3e-05', 'epoch': '0.03862', 'num_input_tokens_seen': 474368, 'train_runtime': '443.5', 'train_tokens_per_second': '1069'}
+
+  4%|▍         | 70/1813 [07:23<3:18:01,  6.82s/it]
+  4%|▍         | 71/1813 [07:29<3:08:05,  6.48s/it]
+  4%|▍         | 72/1813 [07:37<3:26:32,  7.12s/it]
+  4%|▍         | 73/1813 [07:43<3:11:59,  6.62s/it]
+  4%|▍         | 74/1813 [07:49<3:08:09,  6.49s/it]
+  4%|▍         | 75/1813 [07:56<3:08:43,  6.52s/it]
+  4%|▍         | 76/1813 [08:01<3:03:24,  6.34s/it]
+  4%|▍         | 77/1813 [08:08<3:07:56,  6.50s/it]
+  4%|▍         | 78/1813 [08:14<3:00:00,  6.23s/it]
+  4%|▍         | 79/1813 [08:19<2:53:35,  6.01s/it]
+  4%|▍         | 80/1813 [08:26<2:54:13,  6.03s/it]
+                                                   
+{'loss': '1.176', 'grad_norm': '1.318', 'learning_rate': '2.633e-05', 'epoch': '0.04414', 'num_input_tokens_seen': 541616, 'train_runtime': '506', 'train_tokens_per_second': '1070'}
+
+  4%|▍         | 80/1813 [08:26<2:54:13,  6.03s/it]
+  4%|▍         | 81/1813 [08:31<2:52:31,  5.98s/it]
+  5%|▍         | 82/1813 [08:38<3:01:31,  6.29s/it]
+  5%|▍         | 83/1813 [08:46<3:08:50,  6.55s/it]
+  5%|▍         | 84/1813 [08:53<3:12:42,  6.69s/it]
+  5%|▍         | 85/1813 [08:59<3:07:07,  6.50s/it]
+  5%|▍         | 86/1813 [09:05<3:06:31,  6.48s/it]
+  5%|▍         | 87/1813 [09:10<2:54:52,  6.08s/it]
+  5%|▍         | 88/1813 [09:16<2:55:05,  6.09s/it]
+  5%|▍         | 89/1813 [09:22<2:54:35,  6.08s/it]
+  5%|▍         | 90/1813 [09:29<2:58:25,  6.21s/it]
+                                                   
+{'loss': '1.097', 'grad_norm': '1.075', 'learning_rate': '2.967e-05', 'epoch': '0.04966', 'num_input_tokens_seen': 608792, 'train_runtime': '569.4', 'train_tokens_per_second': '1069'}
+
+  5%|▍         | 90/1813 [09:29<2:58:25,  6.21s/it]
+  5%|▌         | 91/1813 [09:35<2:54:17,  6.07s/it]
+  5%|▌         | 92/1813 [09:39<2:43:01,  5.68s/it]
+  5%|▌         | 93/1813 [09:46<2:46:29,  5.81s/it]
+  5%|▌         | 94/1813 [09:52<2:52:19,  6.01s/it]
+  5%|▌         | 95/1813 [09:58<2:55:17,  6.12s/it]
+  5%|▌         | 96/1813 [10:04<2:53:13,  6.05s/it]
+  5%|▌         | 97/1813 [10:11<2:57:23,  6.20s/it]
+  5%|▌         | 98/1813 [10:16<2:50:19,  5.96s/it]
+  5%|▌         | 99/1813 [10:22<2:48:49,  5.91s/it]
+  6%|▌         | 100/1813 [10:29<2:54:05,  6.10s/it]
+                                                    
+{'loss': '1.153', 'grad_norm': '1.116', 'learning_rate': '3.3e-05', 'epoch': '0.05517', 'num_input_tokens_seen': 670936, 'train_runtime': '629.1', 'train_tokens_per_second': '1067'}
+
+  6%|▌         | 100/1813 [10:29<2:54:05,  6.10s/it]
+  6%|▌         | 101/1813 [10:35<3:00:50,  6.34s/it]
+  6%|▌         | 102/1813 [10:42<3:04:37,  6.47s/it]
+  6%|▌         | 103/1813 [10:48<3:01:57,  6.38s/it]
+  6%|▌         | 104/1813 [10:55<3:02:14,  6.40s/it]
+  6%|▌         | 105/1813 [11:00<2:54:20,  6.12s/it]
+  6%|▌         | 106/1813 [11:06<2:51:25,  6.03s/it]
+  6%|▌         | 107/1813 [11:12<2:52:37,  6.07s/it]
+  6%|▌         | 108/1813 [11:19<2:54:55,  6.16s/it]
+  6%|▌         | 109/1813 [11:25<2:54:13,  6.13s/it]
+  6%|▌         | 110/1813 [11:32<3:00:47,  6.37s/it]
+                                                    
+{'loss': '1.143', 'grad_norm': '1.039', 'learning_rate': '3.633e-05', 'epoch': '0.06069', 'num_input_tokens_seen': 738632, 'train_runtime': '692.2', 'train_tokens_per_second': '1067'}
+
+  6%|▌         | 110/1813 [11:32<3:00:47,  6.37s/it]
+  6%|▌         | 111/1813 [11:39<3:05:32,  6.54s/it]
+  6%|▌         | 112/1813 [11:45<3:06:59,  6.60s/it]
+  6%|▌         | 113/1813 [11:52<3:05:38,  6.55s/it]
+  6%|▋         | 114/1813 [11:59<3:13:36,  6.84s/it]
+  6%|▋         | 115/1813 [12:05<3:06:35,  6.59s/it]
+  6%|▋         | 116/1813 [12:12<3:09:55,  6.72s/it]
+  6%|▋         | 117/1813 [12:18<2:59:26,  6.35s/it]
+  7%|▋         | 118/1813 [12:24<2:56:59,  6.27s/it]
+  7%|▋         | 119/1813 [12:30<2:58:13,  6.31s/it]
+  7%|▋         | 120/1813 [12:37<2:59:22,  6.36s/it]
+                                                    
+{'loss': '1.157', 'grad_norm': '1.584', 'learning_rate': '3.967e-05', 'epoch': '0.06621', 'num_input_tokens_seen': 808824, 'train_runtime': '757.3', 'train_tokens_per_second': '1068'}
+
+  7%|▋         | 120/1813 [12:37<2:59:22,  6.36s/it]
+  7%|▋         | 121/1813 [12:41<2:43:29,  5.80s/it]
+  7%|▋         | 122/1813 [12:47<2:39:08,  5.65s/it]
+  7%|▋         | 123/1813 [12:53<2:47:09,  5.93s/it]
+  7%|▋         | 124/1813 [12:59<2:43:45,  5.82s/it]
+  7%|▋         | 125/1813 [13:05<2:51:42,  6.10s/it]
+  7%|▋         | 126/1813 [13:12<2:54:01,  6.19s/it]
+  7%|▋         | 127/1813 [13:17<2:43:02,  5.80s/it]
+  7%|▋         | 128/1813 [13:22<2:34:22,  5.50s/it]
+  7%|▋         | 129/1813 [13:30<2:56:09,  6.28s/it]
+  7%|▋         | 130/1813 [13:36<3:00:26,  6.43s/it]
+                                                    
+{'loss': '1.094', 'grad_norm': '0.9586', 'learning_rate': '4.3e-05', 'epoch': '0.07172', 'num_input_tokens_seen': 869720, 'train_runtime': '816.9', 'train_tokens_per_second': '1065'}
+
+  7%|▋         | 130/1813 [13:36<3:00:26,  6.43s/it]
+  7%|▋         | 131/1813 [13:42<2:57:18,  6.32s/it]
+  7%|▋         | 132/1813 [13:48<2:50:44,  6.09s/it]
+  7%|▋         | 133/1813 [13:54<2:46:16,  5.94s/it]
+  7%|▋         | 134/1813 [13:59<2:37:52,  5.64s/it]
+  7%|▋         | 135/1813 [14:04<2:31:52,  5.43s/it]
+  8%|▊         | 136/1813 [14:09<2:35:41,  5.57s/it]
+  8%|▊         | 137/1813 [14:15<2:39:53,  5.72s/it]
+  8%|▊         | 138/1813 [14:23<2:51:28,  6.14s/it]
+  8%|▊         | 139/1813 [14:29<2:55:41,  6.30s/it]
+  8%|▊         | 140/1813 [14:36<3:01:20,  6.50s/it]
+                                                    
+{'loss': '1.085', 'grad_norm': '1.486', 'learning_rate': '4.633e-05', 'epoch': '0.07724', 'num_input_tokens_seen': 932000, 'train_runtime': '876.8', 'train_tokens_per_second': '1063'}
+
+  8%|▊         | 140/1813 [14:36<3:01:20,  6.50s/it]
+  8%|▊         | 141/1813 [14:42<2:54:39,  6.27s/it]
+  8%|▊         | 142/1813 [14:50<3:09:29,  6.80s/it]
+  8%|▊         | 143/1813 [14:56<2:58:29,  6.41s/it]
+  8%|▊         | 144/1813 [15:01<2:49:24,  6.09s/it]
+  8%|▊         | 145/1813 [15:08<2:58:45,  6.43s/it]
+  8%|▊         | 146/1813 [15:14<2:56:59,  6.37s/it]
+  8%|▊         | 147/1813 [15:20<2:48:19,  6.06s/it]
+  8%|▊         | 148/1813 [15:26<2:47:21,  6.03s/it]
+  8%|▊         | 149/1813 [15:32<2:46:14,  5.99s/it]
+  8%|▊         | 150/1813 [15:38<2:50:22,  6.15s/it]
+                                                    
+{'loss': '1.134', 'grad_norm': '1.329', 'learning_rate': '4.967e-05', 'epoch': '0.08276', 'num_input_tokens_seen': 997296, 'train_runtime': '938.5', 'train_tokens_per_second': '1063'}
+
+  8%|▊         | 150/1813 [15:38<2:50:22,  6.15s/it]
+  8%|▊         | 151/1813 [15:44<2:46:42,  6.02s/it]
+  8%|▊         | 152/1813 [15:49<2:43:01,  5.89s/it]
+  8%|▊         | 153/1813 [15:55<2:41:44,  5.85s/it]
+  8%|▊         | 154/1813 [16:02<2:46:42,  6.03s/it]
+  9%|▊         | 155/1813 [16:09<2:55:53,  6.37s/it]
+  9%|▊         | 156/1813 [16:14<2:50:59,  6.19s/it]
+  9%|▊         | 157/1813 [16:20<2:49:03,  6.13s/it]
+  9%|▊         | 158/1813 [16:25<2:39:30,  5.78s/it]
+  9%|▉         | 159/1813 [16:31<2:38:52,  5.76s/it]
+  9%|▉         | 160/1813 [16:37<2:37:20,  5.71s/it]
+                                                    
+{'loss': '1.06', 'grad_norm': '1.783', 'learning_rate': '5e-05', 'epoch': '0.08828', 'num_input_tokens_seen': 1060096, 'train_runtime': '997.2', 'train_tokens_per_second': '1063'}
+
+  9%|▉         | 160/1813 [16:37<2:37:20,  5.71s/it]
+  9%|▉         | 161/1813 [16:43<2:37:50,  5.73s/it]
+  9%|▉         | 162/1813 [16:47<2:25:13,  5.28s/it]
+  9%|▉         | 163/1813 [16:53<2:36:48,  5.70s/it]
+  9%|▉         | 164/1813 [17:01<2:49:40,  6.17s/it]
+  9%|▉         | 165/1813 [17:07<2:48:55,  6.15s/it]
+  9%|▉         | 166/1813 [17:15<3:01:55,  6.63s/it]
+  9%|▉         | 167/1813 [17:21<3:01:45,  6.63s/it]
+  9%|▉         | 168/1813 [17:28<3:06:06,  6.79s/it]
+  9%|▉         | 169/1813 [17:34<2:58:12,  6.50s/it]
+  9%|▉         | 170/1813 [17:41<3:01:12,  6.62s/it]
+                                                    
+{'loss': '1.077', 'grad_norm': '0.8104', 'learning_rate': '4.998e-05', 'epoch': '0.09379', 'num_input_tokens_seen': 1130368, 'train_runtime': '1062', 'train_tokens_per_second': '1065'}
+
+  9%|▉         | 170/1813 [17:41<3:01:12,  6.62s/it]
+  9%|▉         | 171/1813 [17:47<2:53:07,  6.33s/it]
+  9%|▉         | 172/1813 [17:52<2:47:07,  6.11s/it]
+ 10%|▉         | 173/1813 [17:59<2:49:25,  6.20s/it]
+ 10%|▉         | 174/1813 [18:05<2:46:52,  6.11s/it]
+ 10%|▉         | 175/1813 [18:11<2:49:18,  6.20s/it]
+ 10%|▉         | 176/1813 [18:16<2:42:21,  5.95s/it]
+ 10%|▉         | 177/1813 [18:24<2:55:52,  6.45s/it]
+ 10%|▉         | 178/1813 [18:29<2:45:55,  6.09s/it]
+ 10%|▉         | 179/1813 [18:35<2:46:21,  6.11s/it]
+ 10%|▉         | 180/1813 [18:42<2:46:57,  6.13s/it]
+                                                    
+{'loss': '1.063', 'grad_norm': '1.399', 'learning_rate': '4.996e-05', 'epoch': '0.09931', 'num_input_tokens_seen': 1193192, 'train_runtime': '1122', 'train_tokens_per_second': '1063'}
+
+ 10%|▉         | 180/1813 [18:42<2:46:57,  6.13s/it]
+ 10%|▉         | 181/1813 [18:47<2:43:11,  6.00s/it]
+ 10%|█         | 182/1813 [18:53<2:39:20,  5.86s/it]
+ 10%|█         | 183/1813 [18:59<2:38:35,  5.84s/it]
+ 10%|█         | 184/1813 [19:04<2:36:40,  5.77s/it]
+ 10%|█         | 185/1813 [19:12<2:55:03,  6.45s/it]
+ 10%|█         | 186/1813 [19:19<2:54:12,  6.42s/it]
+ 10%|█         | 187/1813 [19:25<2:51:14,  6.32s/it]
+ 10%|█         | 188/1813 [19:30<2:42:05,  5.98s/it]
+ 10%|█         | 189/1813 [19:36<2:46:56,  6.17s/it]
+ 10%|█         | 190/1813 [19:43<2:50:43,  6.31s/it]
+                                                    
+{'loss': '1.006', 'grad_norm': '1.279', 'learning_rate': '4.993e-05', 'epoch': '0.1048', 'num_input_tokens_seen': 1259336, 'train_runtime': '1184', 'train_tokens_per_second': '1064'}
+
+ 10%|█         | 190/1813 [19:43<2:50:43,  6.31s/it]
+ 11%|█         | 191/1813 [19:49<2:42:55,  6.03s/it]
+ 11%|█         | 192/1813 [19:55<2:46:51,  6.18s/it]
+ 11%|█         | 193/1813 [20:02<2:49:26,  6.28s/it]
+ 11%|█         | 194/1813 [20:07<2:46:09,  6.16s/it]
+ 11%|█         | 195/1813 [20:15<2:57:59,  6.60s/it]
+ 11%|█         | 196/1813 [20:20<2:48:06,  6.24s/it]
+ 11%|█         | 197/1813 [20:28<2:55:26,  6.51s/it]
+ 11%|█         | 198/1813 [20:35<3:02:37,  6.78s/it]
+ 11%|█         | 199/1813 [20:42<3:02:24,  6.78s/it]
+ 11%|█         | 200/1813 [20:48<3:00:53,  6.73s/it]
+                                                    
+{'loss': '1.111', 'grad_norm': '1.476', 'learning_rate': '4.989e-05', 'epoch': '0.1103', 'num_input_tokens_seen': 1331224, 'train_runtime': '1249', 'train_tokens_per_second': '1066'}
+
+ 11%|█         | 200/1813 [20:48<3:00:53,  6.73s/it]
+ 11%|█         | 201/1813 [20:54<2:51:05,  6.37s/it]
+ 11%|█         | 202/1813 [20:59<2:40:27,  5.98s/it]
+ 11%|█         | 203/1813 [21:04<2:32:24,  5.68s/it]
+ 11%|█▏        | 204/1813 [21:11<2:41:25,  6.02s/it]
+ 11%|█▏        | 205/1813 [21:16<2:37:38,  5.88s/it]
+ 11%|█▏        | 206/1813 [21:22<2:39:25,  5.95s/it]
+ 11%|█▏        | 207/1813 [21:28<2:39:53,  5.97s/it]
+ 11%|█▏        | 208/1813 [21:36<2:55:04,  6.55s/it]
+ 12%|█▏        | 209/1813 [21:42<2:44:55,  6.17s/it]
+ 12%|█▏        | 210/1813 [21:48<2:45:19,  6.19s/it]
+                                                    
+{'loss': '0.9885', 'grad_norm': '1.533', 'learning_rate': '4.984e-05', 'epoch': '0.1159', 'num_input_tokens_seen': 1392472, 'train_runtime': '1308', 'train_tokens_per_second': '1064'}
+
+ 12%|█▏        | 210/1813 [21:48<2:45:19,  6.19s/it]
+ 12%|█▏        | 211/1813 [21:57<3:05:47,  6.96s/it]
+ 12%|█▏        | 212/1813 [22:02<2:55:27,  6.58s/it]
+ 12%|█▏        | 213/1813 [22:08<2:46:27,  6.24s/it]
+ 12%|█▏        | 214/1813 [22:13<2:39:27,  5.98s/it]
+ 12%|█▏        | 215/1813 [22:19<2:35:01,  5.82s/it]
+ 12%|█▏        | 216/1813 [22:23<2:24:37,  5.43s/it]
+ 12%|█▏        | 217/1813 [22:28<2:22:17,  5.35s/it]
+ 12%|█▏        | 218/1813 [22:35<2:28:59,  5.60s/it]
+ 12%|█▏        | 219/1813 [22:40<2:31:16,  5.69s/it]
+ 12%|█▏        | 220/1813 [22:46<2:33:33,  5.78s/it]
+                                                    
+{'loss': '1.063', 'grad_norm': '1.046', 'learning_rate': '4.979e-05', 'epoch': '0.1214', 'num_input_tokens_seen': 1454040, 'train_runtime': '1367', 'train_tokens_per_second': '1064'}
+
+ 12%|█▏        | 220/1813 [22:46<2:33:33,  5.78s/it]
+ 12%|█▏        | 221/1813 [22:53<2:37:44,  5.95s/it]
+ 12%|█▏        | 222/1813 [22:58<2:30:11,  5.66s/it]
+ 12%|█▏        | 223/1813 [23:05<2:45:48,  6.26s/it]
+ 12%|█▏        | 224/1813 [23:10<2:35:59,  5.89s/it]
+ 12%|█▏        | 225/1813 [23:17<2:44:55,  6.23s/it]
+ 12%|█▏        | 226/1813 [23:24<2:51:14,  6.47s/it]
+ 13%|█▎        | 227/1813 [23:30<2:46:30,  6.30s/it]
+ 13%|█▎        | 228/1813 [23:37<2:49:32,  6.42s/it]
+ 13%|█▎        | 229/1813 [23:44<2:53:12,  6.56s/it]
+ 13%|█▎        | 230/1813 [23:49<2:39:32,  6.05s/it]
+                                                    
+{'loss': '1.031', 'grad_norm': '1.679', 'learning_rate': '4.972e-05', 'epoch': '0.1269', 'num_input_tokens_seen': 1522056, 'train_runtime': '1429', 'train_tokens_per_second': '1065'}
+
+ 13%|█▎        | 230/1813 [23:49<2:39:32,  6.05s/it]
+ 13%|█▎        | 231/1813 [23:54<2:31:47,  5.76s/it]
+ 13%|█▎        | 232/1813 [24:01<2:45:15,  6.27s/it]
+ 13%|█▎        | 233/1813 [24:07<2:37:01,  5.96s/it]
+ 13%|█▎        | 234/1813 [24:12<2:32:25,  5.79s/it]
+ 13%|█▎        | 235/1813 [24:19<2:41:17,  6.13s/it]
+ 13%|█▎        | 236/1813 [24:26<2:52:17,  6.55s/it]
+ 13%|█▎        | 237/1813 [24:33<2:54:52,  6.66s/it]
+ 13%|█▎        | 238/1813 [24:39<2:45:59,  6.32s/it]
+ 13%|█▎        | 239/1813 [24:46<2:55:13,  6.68s/it]
+ 13%|█▎        | 240/1813 [24:52<2:50:06,  6.49s/it]
+                                                    
+{'loss': '1.019', 'grad_norm': '1.024', 'learning_rate': '4.965e-05', 'epoch': '0.1324', 'num_input_tokens_seen': 1590200, 'train_runtime': '1493', 'train_tokens_per_second': '1065'}
+
+ 13%|█▎        | 240/1813 [24:52<2:50:06,  6.49s/it]
+ 13%|█▎        | 241/1813 [24:59<2:49:47,  6.48s/it]
+ 13%|█▎        | 242/1813 [25:05<2:47:12,  6.39s/it]
+ 13%|█▎        | 243/1813 [25:11<2:41:56,  6.19s/it]
+ 13%|█▎        | 244/1813 [25:18<2:49:10,  6.47s/it]
+ 14%|█▎        | 245/1813 [25:25<2:54:25,  6.67s/it]
+ 14%|█▎        | 246/1813 [25:31<2:45:40,  6.34s/it]
+ 14%|█▎        | 247/1813 [25:38<2:50:12,  6.52s/it]
+ 14%|█▎        | 248/1813 [25:45<3:00:35,  6.92s/it]
+ 14%|█▎        | 249/1813 [25:51<2:50:10,  6.53s/it]
+ 14%|█▍        | 250/1813 [25:57<2:44:05,  6.30s/it]
+                                                    
+{'loss': '1.109', 'grad_norm': '1.133', 'learning_rate': '4.956e-05', 'epoch': '0.1379', 'num_input_tokens_seen': 1658648, 'train_runtime': '1557', 'train_tokens_per_second': '1065'}
+
+ 14%|█▍        | 250/1813 [25:57<2:44:05,  6.30s/it]
+ 14%|█▍        | 251/1813 [26:02<2:33:34,  5.90s/it]
+ 14%|█▍        | 252/1813 [26:07<2:30:59,  5.80s/it]
+ 14%|█▍        | 253/1813 [26:14<2:33:52,  5.92s/it]
+ 14%|█▍        | 254/1813 [26:21<2:44:27,  6.33s/it]
+ 14%|█▍        | 255/1813 [26:28<2:47:06,  6.44s/it]
+ 14%|█▍        | 256/1813 [26:33<2:40:24,  6.18s/it]
+ 14%|█▍        | 257/1813 [26:40<2:45:22,  6.38s/it]
+ 14%|█▍        | 258/1813 [26:47<2:50:33,  6.58s/it]
+ 14%|█▍        | 259/1813 [26:52<2:36:06,  6.03s/it]
+ 14%|█▍        | 260/1813 [26:58<2:35:39,  6.01s/it]
+                                                    
+{'loss': '0.9163', 'grad_norm': '1.845', 'learning_rate': '4.947e-05', 'epoch': '0.1434', 'num_input_tokens_seen': 1722792, 'train_runtime': '1618', 'train_tokens_per_second': '1065'}
+
+ 14%|█▍        | 260/1813 [26:58<2:35:39,  6.01s/it]
+ 14%|█▍        | 261/1813 [27:05<2:45:46,  6.41s/it]
+ 14%|█▍        | 262/1813 [27:12<2:48:35,  6.52s/it]
+ 15%|█▍        | 263/1813 [27:18<2:49:13,  6.55s/it]
+ 15%|█▍        | 264/1813 [27:25<2:47:17,  6.48s/it]
+ 15%|█▍        | 265/1813 [27:31<2:44:35,  6.38s/it]
+ 15%|█▍        | 266/1813 [27:36<2:33:19,  5.95s/it]
+ 15%|█▍        | 267/1813 [27:41<2:29:13,  5.79s/it]
+ 15%|█▍        | 268/1813 [27:46<2:23:40,  5.58s/it]
+ 15%|█▍        | 269/1813 [27:53<2:28:14,  5.76s/it]
+ 15%|█▍        | 270/1813 [27:58<2:29:33,  5.82s/it]
+                                                    
+{'loss': '0.9424', 'grad_norm': '1.246', 'learning_rate': '4.937e-05', 'epoch': '0.149', 'num_input_tokens_seen': 1786808, 'train_runtime': '1679', 'train_tokens_per_second': '1064'}
+
+ 15%|█▍        | 270/1813 [27:58<2:29:33,  5.82s/it]
+ 15%|█▍        | 271/1813 [28:07<2:46:58,  6.50s/it]
+ 15%|█▌        | 272/1813 [28:12<2:38:56,  6.19s/it]
+ 15%|█▌        | 273/1813 [28:20<2:55:31,  6.84s/it]
+ 15%|█▌        | 274/1813 [28:27<2:53:54,  6.78s/it]
+ 15%|█▌        | 275/1813 [28:32<2:40:39,  6.27s/it]
+ 15%|█▌        | 276/1813 [28:37<2:30:53,  5.89s/it]
+ 15%|█▌        | 277/1813 [28:43<2:33:27,  5.99s/it]
+ 15%|█▌        | 278/1813 [28:50<2:34:48,  6.05s/it]
+ 15%|█▌        | 279/1813 [28:57<2:46:52,  6.53s/it]
+ 15%|█▌        | 280/1813 [29:04<2:47:45,  6.57s/it]
+                                                    
+{'loss': '1.018', 'grad_norm': '1.175', 'learning_rate': '4.926e-05', 'epoch': '0.1545', 'num_input_tokens_seen': 1858112, 'train_runtime': '1744', 'train_tokens_per_second': '1065'}
+
+ 15%|█▌        | 280/1813 [29:04<2:47:45,  6.57s/it]
+ 15%|█▌        | 281/1813 [29:10<2:42:06,  6.35s/it]
+ 16%|█▌        | 282/1813 [29:15<2:36:28,  6.13s/it]
+ 16%|█▌        | 283/1813 [29:21<2:29:11,  5.85s/it]
+ 16%|█▌        | 284/1813 [29:26<2:29:11,  5.85s/it]
+ 16%|█▌        | 285/1813 [29:33<2:36:00,  6.13s/it]
+ 16%|█▌        | 286/1813 [29:39<2:36:01,  6.13s/it]
+ 16%|█▌        | 287/1813 [29:46<2:37:26,  6.19s/it]
+ 16%|█▌        | 288/1813 [29:51<2:32:13,  5.99s/it]
+ 16%|█▌        | 289/1813 [29:58<2:35:33,  6.12s/it]
+ 16%|█▌        | 290/1813 [30:04<2:38:27,  6.24s/it]
+                                                    
+{'loss': '0.9902', 'grad_norm': '1.085', 'learning_rate': '4.914e-05', 'epoch': '0.16', 'num_input_tokens_seen': 1920600, 'train_runtime': '1805', 'train_tokens_per_second': '1064'}
+
+ 16%|█▌        | 290/1813 [30:04<2:38:27,  6.24s/it]
+ 16%|█▌        | 291/1813 [30:11<2:43:42,  6.45s/it]
+ 16%|█▌        | 292/1813 [30:18<2:49:17,  6.68s/it]
+ 16%|█▌        | 293/1813 [30:24<2:41:34,  6.38s/it]
+ 16%|█▌        | 294/1813 [30:31<2:47:42,  6.62s/it]
+ 16%|█▋        | 295/1813 [30:39<2:54:21,  6.89s/it]
+ 16%|█▋        | 296/1813 [30:44<2:45:15,  6.54s/it]
+ 16%|█▋        | 297/1813 [30:50<2:42:23,  6.43s/it]
+ 16%|█▋        | 298/1813 [30:56<2:32:03,  6.02s/it]
+ 16%|█▋        | 299/1813 [31:02<2:32:57,  6.06s/it]
+ 17%|█▋        | 300/1813 [31:07<2:30:30,  5.97s/it]
+                                                    
+{'loss': '1.057', 'grad_norm': '1.161', 'learning_rate': '4.902e-05', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312, 'train_runtime': '1868', 'train_tokens_per_second': '1064'}
+
+ 17%|█▋        | 300/1813 [31:07<2:30:30,  5.97s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.74it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.23it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.32it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.46it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.04it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.99it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.83it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  6.00it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.04it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.89it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.91it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.72it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.29it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.76it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.86it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.19it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.94it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:36,  9.34it/s]
+
+ 10%|█         | 103/1000 [00:14<01:57,  7.65it/s]
+
+ 10%|█         | 104/1000 [00:14<01:55,  7.74it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.26it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.38it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.14it/s]
+
+ 11%|█         | 112/1000 [00:14<01:14, 11.98it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.48it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.79it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.21it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.89it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:48,  8.07it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.13it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.27it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.08it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.69it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.13it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.62it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.95it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.88it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.07it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.92it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.05it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.41it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.98it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.62it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.15it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.14it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.71it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:42,  7.91it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:06,  6.38it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:07,  6.31it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:44,  7.69it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.14it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.59it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.74it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.62it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.93it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.51it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.67it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.68it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:26,  9.10it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.48it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.81it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.35it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.03it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.20it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.92it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.44it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:25,  8.95it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.01it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.46it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:55,  6.54it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:32,  8.14it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:50,  6.79it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:07,  5.91it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:08,  5.82it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  5.99it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.72it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.35it/s]
+
+ 26%|██▌       | 256/1000 [00:34<01:36,  7.67it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:25,  8.73it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.67it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.58it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.89it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.52it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.07it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.01it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.42it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.72it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.06it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.17it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.83it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.06it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.72it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.03it/s]
+
+ 30%|███       | 305/1000 [00:41<01:28,  7.86it/s]
+
+ 31%|███       | 306/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 307/1000 [00:41<01:43,  6.67it/s]
+
+ 31%|███       | 309/1000 [00:41<01:19,  8.70it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.16it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.22it/s]
+
+ 32%|███▏      | 315/1000 [00:42<01:07, 10.18it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.55it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:43<01:38,  6.88it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:31,  7.40it/s]
+
+ 33%|███▎      | 328/1000 [00:44<01:34,  7.15it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.89it/s]
+
+ 34%|███▍      | 344/1000 [00:46<01:26,  7.59it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.43it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:33,  6.93it/s]
+
+ 35%|███▌      | 350/1000 [00:47<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.37it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.97it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.58it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.42it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:58,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:26,  7.31it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:23,  7.53it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:18,  7.95it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:41,  6.16it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:54,  5.44it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:41,  6.15it/s]
+
+ 38%|███▊      | 379/1000 [00:51<01:16,  8.12it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.16it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.53it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.06it/s]
+
+ 39%|███▊      | 387/1000 [00:52<01:21,  7.55it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.29it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.83it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.49it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  9.01it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.52it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.83it/s]
+
+ 40%|███▉      | 398/1000 [00:53<01:23,  7.23it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:41,  5.94it/s]
+
+ 40%|████      | 400/1000 [00:54<01:51,  5.37it/s]
+
+ 40%|████      | 402/1000 [00:54<01:44,  5.75it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.21it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.56it/s]
+
+ 41%|████      | 407/1000 [00:54<01:24,  7.04it/s]
+
+ 41%|████      | 408/1000 [00:55<01:36,  6.14it/s]
+
+ 41%|████      | 410/1000 [00:55<01:33,  6.29it/s]
+
+ 41%|████      | 411/1000 [00:55<01:44,  5.63it/s]
+
+ 41%|████      | 412/1000 [00:55<01:53,  5.16it/s]
+
+ 41%|████▏     | 414/1000 [00:56<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.80it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:11,  8.10it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:09,  8.34it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.43it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.65it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.88it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.32it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.92it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:16,  7.51it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:13,  7.82it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.55it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.03it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.78it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.51it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.31it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.08it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.24it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.70it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.47it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.17it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.17it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.49it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.79it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.24it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.82it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.19it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.36it/s]
+
+ 47%|████▋     | 474/1000 [01:04<01:25,  6.17it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:25,  6.16it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:39,  5.28it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.06it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:20,  6.49it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.02it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.29it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:22,  6.28it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.98it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50, 10.00it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.40it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.91it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.53it/s]
+
+ 50%|█████     | 503/1000 [01:08<01:13,  6.79it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:55,  8.80it/s]
+
+ 51%|█████     | 512/1000 [01:09<00:49,  9.84it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:07,  7.25it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.50it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.07it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<01:00,  8.03it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.36it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.01it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.79it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.57it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:12<00:48,  9.72it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:15<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.17it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.94it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.75it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.01it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.84it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.04it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.36it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.24it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.59it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:21<00:59,  6.79it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.38it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.68it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.41it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:40,  9.68it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.76it/s]
+
+ 62%|██████▏   | 616/1000 [01:24<00:50,  7.59it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.73it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.09it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.05it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.22it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.44it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.62it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.20it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:39,  9.39it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.94it/s]
+
+ 64%|██████▍   | 640/1000 [01:27<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.94it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:28<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.55it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.91it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.11it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:30<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.47it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.60it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.69it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.65it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.70it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.98it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.92it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.87it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.83it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:47,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.59it/s]
+
+ 70%|███████   | 702/1000 [01:36<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.01it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.69it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.36it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.15it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:39<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.34it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.24it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.45it/s]
+
+ 74%|███████▎  | 735/1000 [01:40<00:26,  9.84it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.48it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.73it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.29it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.35it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.59it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.43it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.63it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.69it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.64it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.46it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.78it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.88it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.59it/s]
+
+ 78%|███████▊  | 780/1000 [01:46<00:31,  6.94it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.36it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.98it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.21it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.71it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.96it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.22it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.25it/s]
+
+ 81%|████████▏ | 813/1000 [01:50<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.30it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.13it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:51<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.81it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.74it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.00it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 833/1000 [01:53<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.52it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.10it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.44it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.15it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.46it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.04it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.19it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.50it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.76it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.63it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.70it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.26it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.28it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:59<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.76it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.75it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.85it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.71it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.07it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.59it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.79it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.38it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:04<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.57it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.18it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+
+ 93%|█████████▎| 927/1000 [02:05<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.64it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:06<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.84it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:08<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.39it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.40it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.18it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.79it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.83it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.37it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.14it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.97it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+                                                    
+
+                                                   
+{'eval_loss': '0.8623', 'eval_runtime': '135.6', 'eval_samples_per_second': '7.375', 'eval_steps_per_second': '7.375', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312}
+
+ 17%|█▋        | 300/1813 [33:23<2:30:30,  5.97s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+
+                                                   
+ 17%|█▋        | 301/1813 [33:28<19:30:25, 46.45s/it]
+ 17%|█▋        | 302/1813 [33:33<14:14:31, 33.93s/it]
+ 17%|█▋        | 303/1813 [33:40<10:49:12, 25.80s/it]
+ 17%|█▋        | 304/1813 [33:47<8:30:35, 20.30s/it] 
+ 17%|█▋        | 305/1813 [33:54<6:43:58, 16.07s/it]
+ 17%|█▋        | 306/1813 [33:59<5:26:24, 13.00s/it]
+ 17%|█▋        | 307/1813 [34:05<4:30:11, 10.76s/it]
+ 17%|█▋        | 308/1813 [34:10<3:50:10,  9.18s/it]
+ 17%|█▋        | 309/1813 [34:17<3:29:50,  8.37s/it]
+ 17%|█▋        | 310/1813 [34:22<3:03:38,  7.33s/it]
+                                                    
+{'loss': '0.9725', 'grad_norm': '1.431', 'learning_rate': '4.888e-05', 'epoch': '0.171', 'num_input_tokens_seen': 2050672, 'train_runtime': '2062', 'train_tokens_per_second': '994.3'}
+
+ 17%|█▋        | 310/1813 [34:22<3:03:38,  7.33s/it]
+ 17%|█▋        | 311/1813 [34:29<3:00:48,  7.22s/it]
+ 17%|█▋        | 312/1813 [34:36<3:01:46,  7.27s/it]
+ 17%|█▋        | 313/1813 [34:43<2:58:55,  7.16s/it]
+ 17%|█▋        | 314/1813 [34:49<2:47:48,  6.72s/it]
+ 17%|█▋        | 315/1813 [34:56<2:49:22,  6.78s/it]
+ 17%|█▋        | 316/1813 [35:02<2:45:51,  6.65s/it]
+ 17%|█▋        | 317/1813 [35:08<2:38:54,  6.37s/it]
+ 18%|█▊        | 318/1813 [35:13<2:31:08,  6.07s/it]
+ 18%|█▊        | 319/1813 [35:18<2:23:45,  5.77s/it]
+ 18%|█▊        | 320/1813 [35:24<2:23:52,  5.78s/it]
+                                                    
+{'loss': '1.063', 'grad_norm': '0.7571', 'learning_rate': '4.874e-05', 'epoch': '0.1766', 'num_input_tokens_seen': 2118000, 'train_runtime': '2125', 'train_tokens_per_second': '996.9'}
+
+ 18%|█▊        | 320/1813 [35:24<2:23:52,  5.78s/it]
+ 18%|█▊        | 321/1813 [35:30<2:24:00,  5.79s/it]
+ 18%|█▊        | 322/1813 [35:36<2:24:07,  5.80s/it]
+ 18%|█▊        | 323/1813 [35:42<2:27:12,  5.93s/it]
+ 18%|█▊        | 324/1813 [35:49<2:35:19,  6.26s/it]
+ 18%|█▊        | 325/1813 [35:56<2:40:09,  6.46s/it]
+ 18%|█▊        | 326/1813 [36:02<2:38:16,  6.39s/it]
+ 18%|█▊        | 327/1813 [36:08<2:35:02,  6.26s/it]
+ 18%|█▊        | 328/1813 [36:15<2:36:37,  6.33s/it]
+ 18%|█▊        | 329/1813 [36:20<2:32:45,  6.18s/it]
+ 18%|█▊        | 330/1813 [36:26<2:30:23,  6.08s/it]
+                                                    
+{'loss': '0.9757', 'grad_norm': '1.223', 'learning_rate': '4.858e-05', 'epoch': '0.1821', 'num_input_tokens_seen': 2184640, 'train_runtime': '2187', 'train_tokens_per_second': '999.1'}
+
+ 18%|█▊        | 330/1813 [36:26<2:30:23,  6.08s/it]
+ 18%|█▊        | 331/1813 [36:33<2:33:16,  6.21s/it]
+ 18%|█▊        | 332/1813 [36:38<2:26:30,  5.94s/it]
+ 18%|█▊        | 333/1813 [36:44<2:26:45,  5.95s/it]
+ 18%|█▊        | 334/1813 [36:50<2:23:59,  5.84s/it]
+ 18%|█▊        | 335/1813 [36:57<2:37:07,  6.38s/it]
+ 19%|█▊        | 336/1813 [37:04<2:40:57,  6.54s/it]
+ 19%|█▊        | 337/1813 [37:10<2:33:26,  6.24s/it]
+ 19%|█▊        | 338/1813 [37:16<2:32:37,  6.21s/it]
+ 19%|█▊        | 339/1813 [37:21<2:28:14,  6.03s/it]
+ 19%|█▉        | 340/1813 [37:28<2:35:23,  6.33s/it]
+                                                    
+{'loss': '0.955', 'grad_norm': '1.342', 'learning_rate': '4.842e-05', 'epoch': '0.1876', 'num_input_tokens_seen': 2251392, 'train_runtime': '2249', 'train_tokens_per_second': '1001'}
+
+ 19%|█▉        | 340/1813 [37:28<2:35:23,  6.33s/it]
+ 19%|█▉        | 341/1813 [37:35<2:35:51,  6.35s/it]
+ 19%|█▉        | 342/1813 [37:42<2:43:18,  6.66s/it]
+ 19%|█▉        | 343/1813 [37:48<2:35:26,  6.34s/it]
+ 19%|█▉        | 344/1813 [37:53<2:29:42,  6.11s/it]
+ 19%|█▉        | 345/1813 [38:01<2:37:27,  6.44s/it]
+ 19%|█▉        | 346/1813 [38:07<2:36:22,  6.40s/it]
+ 19%|█▉        | 347/1813 [38:12<2:25:46,  5.97s/it]
+ 19%|█▉        | 348/1813 [38:18<2:29:32,  6.12s/it]
+ 19%|█▉        | 349/1813 [38:27<2:46:04,  6.81s/it]
+ 19%|█▉        | 350/1813 [38:34<2:52:27,  7.07s/it]
+                                                    
+{'loss': '1.061', 'grad_norm': '0.9184', 'learning_rate': '4.825e-05', 'epoch': '0.1931', 'num_input_tokens_seen': 2324712, 'train_runtime': '2315', 'train_tokens_per_second': '1004'}
+
+ 19%|█▉        | 350/1813 [38:34<2:52:27,  7.07s/it]
+ 19%|█▉        | 351/1813 [38:41<2:51:32,  7.04s/it]
+ 19%|█▉        | 352/1813 [38:47<2:42:19,  6.67s/it]
+ 19%|█▉        | 353/1813 [38:53<2:33:16,  6.30s/it]
+ 20%|█▉        | 354/1813 [38:58<2:27:45,  6.08s/it]
+ 20%|█▉        | 355/1813 [39:04<2:26:23,  6.02s/it]
+ 20%|█▉        | 356/1813 [39:11<2:32:25,  6.28s/it]
+ 20%|█▉        | 357/1813 [39:17<2:27:47,  6.09s/it]
+ 20%|█▉        | 358/1813 [39:23<2:29:26,  6.16s/it]
+ 20%|█▉        | 359/1813 [39:29<2:25:32,  6.01s/it]
+ 20%|█▉        | 360/1813 [39:35<2:31:44,  6.27s/it]
+                                                    
+{'loss': '1.008', 'grad_norm': '1.069', 'learning_rate': '4.808e-05', 'epoch': '0.1986', 'num_input_tokens_seen': 2390776, 'train_runtime': '2376', 'train_tokens_per_second': '1006'}
+
+ 20%|█▉        | 360/1813 [39:35<2:31:44,  6.27s/it]
+ 20%|█▉        | 361/1813 [39:41<2:23:02,  5.91s/it]
+ 20%|█▉        | 362/1813 [39:46<2:21:41,  5.86s/it]
+ 20%|██        | 363/1813 [39:53<2:31:10,  6.26s/it]
+ 20%|██        | 364/1813 [40:00<2:34:45,  6.41s/it]
+ 20%|██        | 365/1813 [40:06<2:30:59,  6.26s/it]
+ 20%|██        | 366/1813 [40:13<2:36:12,  6.48s/it]
+ 20%|██        | 367/1813 [40:19<2:32:18,  6.32s/it]
+ 20%|██        | 368/1813 [40:25<2:25:52,  6.06s/it]
+ 20%|██        | 369/1813 [40:29<2:15:34,  5.63s/it]
+ 20%|██        | 370/1813 [40:35<2:17:06,  5.70s/it]
+                                                    
+{'loss': '1.001', 'grad_norm': '1.451', 'learning_rate': '4.789e-05', 'epoch': '0.2041', 'num_input_tokens_seen': 2454808, 'train_runtime': '2436', 'train_tokens_per_second': '1008'}
+
+ 20%|██        | 370/1813 [40:35<2:17:06,  5.70s/it]
+ 20%|██        | 371/1813 [40:40<2:12:47,  5.53s/it]
+ 21%|██        | 372/1813 [40:48<2:32:01,  6.33s/it]
+ 21%|██        | 373/1813 [40:54<2:30:22,  6.27s/it]
+ 21%|██        | 374/1813 [41:02<2:38:29,  6.61s/it]
+ 21%|██        | 375/1813 [41:09<2:39:50,  6.67s/it]
+ 21%|██        | 376/1813 [41:14<2:29:41,  6.25s/it]
+ 21%|██        | 377/1813 [41:20<2:28:11,  6.19s/it]
+ 21%|██        | 378/1813 [41:26<2:26:46,  6.14s/it]
+ 21%|██        | 379/1813 [41:31<2:19:29,  5.84s/it]
+ 21%|██        | 380/1813 [41:36<2:15:22,  5.67s/it]
+                                                    
+{'loss': '0.9748', 'grad_norm': '1.551', 'learning_rate': '4.77e-05', 'epoch': '0.2097', 'num_input_tokens_seen': 2520472, 'train_runtime': '2497', 'train_tokens_per_second': '1009'}
+
+ 21%|██        | 380/1813 [41:36<2:15:22,  5.67s/it]
+ 21%|██        | 381/1813 [41:43<2:24:40,  6.06s/it]
+ 21%|██        | 382/1813 [41:50<2:30:05,  6.29s/it]
+ 21%|██        | 383/1813 [41:56<2:28:27,  6.23s/it]
+ 21%|██        | 384/1813 [42:02<2:26:46,  6.16s/it]
+ 21%|██        | 385/1813 [42:08<2:26:32,  6.16s/it]
+ 21%|██▏       | 386/1813 [42:13<2:16:37,  5.74s/it]
+ 21%|██▏       | 387/1813 [42:20<2:20:33,  5.91s/it]
+ 21%|██▏       | 388/1813 [42:27<2:29:12,  6.28s/it]
+ 21%|██▏       | 389/1813 [42:34<2:39:46,  6.73s/it]
+ 22%|██▏       | 390/1813 [42:41<2:40:24,  6.76s/it]
+                                                    
+{'loss': '1.012', 'grad_norm': '0.9965', 'learning_rate': '4.749e-05', 'epoch': '0.2152', 'num_input_tokens_seen': 2591512, 'train_runtime': '2562', 'train_tokens_per_second': '1012'}
+
+ 22%|██▏       | 390/1813 [42:41<2:40:24,  6.76s/it]
+ 22%|██▏       | 391/1813 [42:48<2:36:45,  6.61s/it]
+ 22%|██▏       | 392/1813 [42:54<2:35:58,  6.59s/it]
+ 22%|██▏       | 393/1813 [43:01<2:36:59,  6.63s/it]
+ 22%|██▏       | 394/1813 [43:06<2:26:25,  6.19s/it]
+ 22%|██▏       | 395/1813 [43:12<2:22:27,  6.03s/it]
+ 22%|██▏       | 396/1813 [43:18<2:24:16,  6.11s/it]
+ 22%|██▏       | 397/1813 [43:23<2:16:06,  5.77s/it]
+ 22%|██▏       | 398/1813 [43:29<2:20:12,  5.95s/it]
+ 22%|██▏       | 399/1813 [43:37<2:32:39,  6.48s/it]
+ 22%|██▏       | 400/1813 [43:45<2:45:40,  7.04s/it]
+                                                    
+{'loss': '0.992', 'grad_norm': '1.038', 'learning_rate': '4.728e-05', 'epoch': '0.2207', 'num_input_tokens_seen': 2660336, 'train_runtime': '2626', 'train_tokens_per_second': '1013'}
+
+ 22%|██▏       | 400/1813 [43:45<2:45:40,  7.04s/it]
+ 22%|██▏       | 401/1813 [43:53<2:46:55,  7.09s/it]
+ 22%|██▏       | 402/1813 [43:57<2:31:13,  6.43s/it]
+ 22%|██▏       | 403/1813 [44:04<2:28:30,  6.32s/it]
+ 22%|██▏       | 404/1813 [44:10<2:32:00,  6.47s/it]
+ 22%|██▏       | 405/1813 [44:18<2:38:15,  6.74s/it]
+ 22%|██▏       | 406/1813 [44:23<2:25:22,  6.20s/it]
+ 22%|██▏       | 407/1813 [44:27<2:12:18,  5.65s/it]
+ 23%|██▎       | 408/1813 [44:33<2:14:29,  5.74s/it]
+ 23%|██▎       | 409/1813 [44:39<2:15:39,  5.80s/it]
+ 23%|██▎       | 410/1813 [44:44<2:12:23,  5.66s/it]
+                                                    
+{'loss': '1.014', 'grad_norm': '1.49', 'learning_rate': '4.707e-05', 'epoch': '0.2262', 'num_input_tokens_seen': 2722056, 'train_runtime': '2685', 'train_tokens_per_second': '1014'}
+
+ 23%|██▎       | 410/1813 [44:44<2:12:23,  5.66s/it]
+ 23%|██▎       | 411/1813 [44:50<2:15:21,  5.79s/it]
+ 23%|██▎       | 412/1813 [44:57<2:20:39,  6.02s/it]
+ 23%|██▎       | 413/1813 [45:04<2:29:23,  6.40s/it]
+ 23%|██▎       | 414/1813 [45:10<2:23:05,  6.14s/it]
+ 23%|██▎       | 415/1813 [45:16<2:24:46,  6.21s/it]
+ 23%|██▎       | 416/1813 [45:23<2:26:47,  6.30s/it]
+ 23%|██▎       | 417/1813 [45:30<2:36:25,  6.72s/it]
+ 23%|██▎       | 418/1813 [45:36<2:30:59,  6.49s/it]
+ 23%|██▎       | 419/1813 [45:43<2:31:51,  6.54s/it]
+ 23%|██▎       | 420/1813 [45:49<2:27:25,  6.35s/it]
+                                                    
+{'loss': '1.006', 'grad_norm': '0.8526', 'learning_rate': '4.684e-05', 'epoch': '0.2317', 'num_input_tokens_seen': 2790960, 'train_runtime': '2749', 'train_tokens_per_second': '1015'}
+
+ 23%|██▎       | 420/1813 [45:49<2:27:25,  6.35s/it]
+ 23%|██▎       | 421/1813 [45:55<2:27:18,  6.35s/it]
+ 23%|██▎       | 422/1813 [46:02<2:33:47,  6.63s/it]
+ 23%|██▎       | 423/1813 [46:08<2:29:22,  6.45s/it]
+ 23%|██▎       | 424/1813 [46:15<2:26:56,  6.35s/it]
+ 23%|██▎       | 425/1813 [46:20<2:22:09,  6.15s/it]
+ 23%|██▎       | 426/1813 [46:28<2:30:16,  6.50s/it]
+ 24%|██▎       | 427/1813 [46:33<2:25:53,  6.32s/it]
+ 24%|██▎       | 428/1813 [46:40<2:24:08,  6.24s/it]
+ 24%|██▎       | 429/1813 [46:45<2:16:57,  5.94s/it]
+ 24%|██▎       | 430/1813 [46:52<2:27:30,  6.40s/it]
+                                                    
+{'loss': '0.9937', 'grad_norm': '0.8889', 'learning_rate': '4.661e-05', 'epoch': '0.2372', 'num_input_tokens_seen': 2860096, 'train_runtime': '2813', 'train_tokens_per_second': '1017'}
+
+ 24%|██▎       | 430/1813 [46:52<2:27:30,  6.40s/it]
+ 24%|██▍       | 431/1813 [46:58<2:23:34,  6.23s/it]
+ 24%|██▍       | 432/1813 [47:04<2:19:44,  6.07s/it]
+ 24%|██▍       | 433/1813 [47:09<2:14:35,  5.85s/it]
+ 24%|██▍       | 434/1813 [47:16<2:18:51,  6.04s/it]
+ 24%|██▍       | 435/1813 [47:22<2:17:42,  6.00s/it]
+ 24%|██▍       | 436/1813 [47:28<2:22:23,  6.20s/it]
+ 24%|██▍       | 437/1813 [47:34<2:20:13,  6.11s/it]
+ 24%|██▍       | 438/1813 [47:41<2:23:10,  6.25s/it]
+ 24%|██▍       | 439/1813 [47:47<2:22:17,  6.21s/it]
+ 24%|██▍       | 440/1813 [47:53<2:23:26,  6.27s/it]
+                                                    
+{'loss': '0.9118', 'grad_norm': '1.509', 'learning_rate': '4.637e-05', 'epoch': '0.2428', 'num_input_tokens_seen': 2924856, 'train_runtime': '2874', 'train_tokens_per_second': '1018'}
+
+ 24%|██▍       | 440/1813 [47:53<2:23:26,  6.27s/it]
+ 24%|██▍       | 441/1813 [47:59<2:21:39,  6.20s/it]
+ 24%|██▍       | 442/1813 [48:05<2:21:48,  6.21s/it]
+ 24%|██▍       | 443/1813 [48:12<2:26:02,  6.40s/it]
+ 24%|██▍       | 444/1813 [48:18<2:21:04,  6.18s/it]
+ 25%|██▍       | 445/1813 [48:25<2:28:27,  6.51s/it]
+ 25%|██▍       | 446/1813 [48:32<2:28:33,  6.52s/it]
+ 25%|██▍       | 447/1813 [48:38<2:26:54,  6.45s/it]
+ 25%|██▍       | 448/1813 [48:44<2:26:10,  6.43s/it]
+ 25%|██▍       | 449/1813 [48:50<2:18:44,  6.10s/it]
+ 25%|██▍       | 450/1813 [48:55<2:15:39,  5.97s/it]
+                                                    
+{'loss': '1.043', 'grad_norm': '1.169', 'learning_rate': '4.612e-05', 'epoch': '0.2483', 'num_input_tokens_seen': 2990040, 'train_runtime': '2936', 'train_tokens_per_second': '1018'}
+
+ 25%|██▍       | 450/1813 [48:55<2:15:39,  5.97s/it]
+ 25%|██▍       | 451/1813 [49:03<2:26:31,  6.46s/it]
+ 25%|██▍       | 452/1813 [49:09<2:22:57,  6.30s/it]
+ 25%|██▍       | 453/1813 [49:14<2:16:15,  6.01s/it]
+ 25%|██▌       | 454/1813 [49:21<2:20:56,  6.22s/it]
+ 25%|██▌       | 455/1813 [49:27<2:21:45,  6.26s/it]
+ 25%|██▌       | 456/1813 [49:32<2:08:12,  5.67s/it]
+ 25%|██▌       | 457/1813 [49:38<2:10:58,  5.80s/it]
+ 25%|██▌       | 458/1813 [49:46<2:25:27,  6.44s/it]
+ 25%|██▌       | 459/1813 [49:53<2:28:14,  6.57s/it]
+ 25%|██▌       | 460/1813 [49:59<2:27:35,  6.55s/it]
+                                                    
+{'loss': '1.102', 'grad_norm': '0.8638', 'learning_rate': '4.586e-05', 'epoch': '0.2538', 'num_input_tokens_seen': 3058848, 'train_runtime': '3000', 'train_tokens_per_second': '1020'}
+
+ 25%|██▌       | 460/1813 [49:59<2:27:35,  6.55s/it]
+ 25%|██▌       | 461/1813 [50:05<2:25:44,  6.47s/it]
+ 25%|██▌       | 462/1813 [50:13<2:33:26,  6.81s/it]
+ 26%|██▌       | 463/1813 [50:19<2:27:50,  6.57s/it]
+ 26%|██▌       | 464/1813 [50:24<2:19:41,  6.21s/it]
+ 26%|██▌       | 465/1813 [50:30<2:16:26,  6.07s/it]
+ 26%|██▌       | 466/1813 [50:37<2:20:11,  6.24s/it]
+ 26%|██▌       | 467/1813 [50:43<2:16:44,  6.10s/it]
+ 26%|██▌       | 468/1813 [50:49<2:19:22,  6.22s/it]
+ 26%|██▌       | 469/1813 [50:55<2:20:56,  6.29s/it]
+ 26%|██▌       | 470/1813 [51:02<2:24:04,  6.44s/it]
+                                                    
+{'loss': '0.967', 'grad_norm': '0.8023', 'learning_rate': '4.56e-05', 'epoch': '0.2593', 'num_input_tokens_seen': 3126680, 'train_runtime': '3063', 'train_tokens_per_second': '1021'}
+
+ 26%|██▌       | 470/1813 [51:02<2:24:04,  6.44s/it]
+ 26%|██▌       | 471/1813 [51:08<2:18:01,  6.17s/it]
+ 26%|██▌       | 472/1813 [51:15<2:22:03,  6.36s/it]
+ 26%|██▌       | 473/1813 [51:21<2:25:05,  6.50s/it]
+ 26%|██▌       | 474/1813 [51:28<2:22:15,  6.37s/it]
+ 26%|██▌       | 475/1813 [51:33<2:15:29,  6.08s/it]
+ 26%|██▋       | 476/1813 [51:39<2:16:02,  6.11s/it]
+ 26%|██▋       | 477/1813 [51:46<2:20:14,  6.30s/it]
+ 26%|██▋       | 478/1813 [51:53<2:25:41,  6.55s/it]
+ 26%|██▋       | 479/1813 [51:59<2:21:22,  6.36s/it]
+ 26%|██▋       | 480/1813 [52:05<2:22:14,  6.40s/it]
+                                                    
+{'loss': '1.071', 'grad_norm': '1.093', 'learning_rate': '4.532e-05', 'epoch': '0.2648', 'num_input_tokens_seen': 3193704, 'train_runtime': '3126', 'train_tokens_per_second': '1022'}
+
+ 26%|██▋       | 480/1813 [52:05<2:22:14,  6.40s/it]
+ 27%|██▋       | 481/1813 [52:12<2:21:26,  6.37s/it]
+ 27%|██▋       | 482/1813 [52:18<2:23:51,  6.49s/it]
+ 27%|██▋       | 483/1813 [52:26<2:31:35,  6.84s/it]
+ 27%|██▋       | 484/1813 [52:32<2:23:55,  6.50s/it]
+ 27%|██▋       | 485/1813 [52:40<2:31:56,  6.87s/it]
+ 27%|██▋       | 486/1813 [52:46<2:27:57,  6.69s/it]
+ 27%|██▋       | 487/1813 [52:53<2:29:15,  6.75s/it]
+ 27%|██▋       | 488/1813 [52:59<2:22:55,  6.47s/it]
+ 27%|██▋       | 489/1813 [53:06<2:27:07,  6.67s/it]
+ 27%|██▋       | 490/1813 [53:12<2:24:06,  6.54s/it]
+                                                    
+{'loss': '1.025', 'grad_norm': '1.131', 'learning_rate': '4.505e-05', 'epoch': '0.2703', 'num_input_tokens_seen': 3268064, 'train_runtime': '3192', 'train_tokens_per_second': '1024'}
+
+ 27%|██▋       | 490/1813 [53:12<2:24:06,  6.54s/it]
+ 27%|██▋       | 491/1813 [53:19<2:25:38,  6.61s/it]
+ 27%|██▋       | 492/1813 [53:26<2:31:41,  6.89s/it]
+ 27%|██▋       | 493/1813 [53:32<2:21:28,  6.43s/it]
+ 27%|██▋       | 494/1813 [53:37<2:15:49,  6.18s/it]
+ 27%|██▋       | 495/1813 [53:44<2:21:26,  6.44s/it]
+ 27%|██▋       | 496/1813 [53:50<2:20:02,  6.38s/it]
+ 27%|██▋       | 497/1813 [53:56<2:13:50,  6.10s/it]
+ 27%|██▋       | 498/1813 [54:03<2:20:35,  6.42s/it]
+ 28%|██▊       | 499/1813 [54:09<2:17:36,  6.28s/it]
+ 28%|██▊       | 500/1813 [54:18<2:36:18,  7.14s/it]
+                                                    
+{'loss': '1.117', 'grad_norm': '0.7955', 'learning_rate': '4.476e-05', 'epoch': '0.2759', 'num_input_tokens_seen': 3341536, 'train_runtime': '3259', 'train_tokens_per_second': '1025'}
+
+ 28%|██▊       | 500/1813 [54:18<2:36:18,  7.14s/it]
+ 28%|██▊       | 501/1813 [54:24<2:30:24,  6.88s/it]
+ 28%|██▊       | 502/1813 [54:30<2:22:51,  6.54s/it]
+ 28%|██▊       | 503/1813 [54:35<2:13:32,  6.12s/it]
+ 28%|██▊       | 504/1813 [54:42<2:14:08,  6.15s/it]
+ 28%|██▊       | 505/1813 [54:49<2:21:39,  6.50s/it]
+ 28%|██▊       | 506/1813 [54:55<2:17:18,  6.30s/it]
+ 28%|██▊       | 507/1813 [55:02<2:23:18,  6.58s/it]
+ 28%|██▊       | 508/1813 [55:07<2:12:13,  6.08s/it]
+ 28%|██▊       | 509/1813 [55:12<2:04:43,  5.74s/it]
+ 28%|██▊       | 510/1813 [55:17<2:01:16,  5.58s/it]
+                                                    
+{'loss': '0.9408', 'grad_norm': '1.419', 'learning_rate': '4.447e-05', 'epoch': '0.2814', 'num_input_tokens_seen': 3402712, 'train_runtime': '3317', 'train_tokens_per_second': '1026'}
+
+ 28%|██▊       | 510/1813 [55:17<2:01:16,  5.58s/it]
+ 28%|██▊       | 511/1813 [55:22<1:59:52,  5.52s/it]
+ 28%|██▊       | 512/1813 [55:29<2:08:17,  5.92s/it]
+ 28%|██▊       | 513/1813 [55:36<2:15:56,  6.27s/it]
+ 28%|██▊       | 514/1813 [55:43<2:18:50,  6.41s/it]
+ 28%|██▊       | 515/1813 [55:48<2:09:48,  6.00s/it]
+ 28%|██▊       | 516/1813 [55:54<2:06:57,  5.87s/it]
+ 29%|██▊       | 517/1813 [56:00<2:11:00,  6.07s/it]
+ 29%|██▊       | 518/1813 [56:06<2:10:28,  6.04s/it]
+ 29%|██▊       | 519/1813 [56:12<2:10:48,  6.07s/it]
+ 29%|██▊       | 520/1813 [56:19<2:14:54,  6.26s/it]
+                                                    
+{'loss': '0.9838', 'grad_norm': '1.109', 'learning_rate': '4.417e-05', 'epoch': '0.2869', 'num_input_tokens_seen': 3469624, 'train_runtime': '3379', 'train_tokens_per_second': '1027'}
+
+ 29%|██▊       | 520/1813 [56:19<2:14:54,  6.26s/it]
+ 29%|██▊       | 521/1813 [56:23<2:00:00,  5.57s/it]
+ 29%|██▉       | 522/1813 [56:29<2:01:30,  5.65s/it]
+ 29%|██▉       | 523/1813 [56:35<2:05:42,  5.85s/it]
+ 29%|██▉       | 524/1813 [56:42<2:10:47,  6.09s/it]
+ 29%|██▉       | 525/1813 [56:47<2:05:23,  5.84s/it]
+ 29%|██▉       | 526/1813 [56:54<2:10:44,  6.10s/it]
+ 29%|██▉       | 527/1813 [57:01<2:15:14,  6.31s/it]
+ 29%|██▉       | 528/1813 [57:08<2:20:35,  6.56s/it]
+ 29%|██▉       | 529/1813 [57:15<2:24:18,  6.74s/it]
+ 29%|██▉       | 530/1813 [57:21<2:20:31,  6.57s/it]
+                                                    
+{'loss': '0.9603', 'grad_norm': '1.481', 'learning_rate': '4.386e-05', 'epoch': '0.2924', 'num_input_tokens_seen': 3534904, 'train_runtime': '3442', 'train_tokens_per_second': '1027'}
+
+ 29%|██▉       | 530/1813 [57:21<2:20:31,  6.57s/it]
+ 29%|██▉       | 531/1813 [57:29<2:31:20,  7.08s/it]
+ 29%|██▉       | 532/1813 [57:36<2:26:12,  6.85s/it]
+ 29%|██▉       | 533/1813 [57:43<2:27:38,  6.92s/it]
+ 29%|██▉       | 534/1813 [57:50<2:29:28,  7.01s/it]
+ 30%|██▉       | 535/1813 [57:54<2:12:29,  6.22s/it]
+ 30%|██▉       | 536/1813 [57:59<2:05:22,  5.89s/it]
+ 30%|██▉       | 537/1813 [58:06<2:09:37,  6.10s/it]
+ 30%|██▉       | 538/1813 [58:11<2:05:05,  5.89s/it]
+ 30%|██▉       | 539/1813 [58:18<2:06:39,  5.97s/it]
+ 30%|██▉       | 540/1813 [58:24<2:06:51,  5.98s/it]
+                                                    
+{'loss': '0.9769', 'grad_norm': '0.9603', 'learning_rate': '4.355e-05', 'epoch': '0.2979', 'num_input_tokens_seen': 3603488, 'train_runtime': '3504', 'train_tokens_per_second': '1028'}
+
+ 30%|██▉       | 540/1813 [58:24<2:06:51,  5.98s/it]
+ 30%|██▉       | 541/1813 [58:29<2:02:49,  5.79s/it]
+ 30%|██▉       | 542/1813 [58:35<2:02:35,  5.79s/it]
+ 30%|██▉       | 543/1813 [58:42<2:12:00,  6.24s/it]
+ 30%|███       | 544/1813 [58:50<2:24:55,  6.85s/it]
+ 30%|███       | 545/1813 [58:57<2:23:18,  6.78s/it]
+ 30%|███       | 546/1813 [59:03<2:21:36,  6.71s/it]
+ 30%|███       | 547/1813 [59:11<2:27:36,  7.00s/it]
+ 30%|███       | 548/1813 [59:19<2:31:53,  7.20s/it]
+ 30%|███       | 549/1813 [59:24<2:21:51,  6.73s/it]
+ 30%|███       | 550/1813 [59:30<2:16:12,  6.47s/it]
+                                                    
+{'loss': '0.923', 'grad_norm': '1.163', 'learning_rate': '4.323e-05', 'epoch': '0.3034', 'num_input_tokens_seen': 3676576, 'train_runtime': '3571', 'train_tokens_per_second': '1030'}
+
+ 30%|███       | 550/1813 [59:30<2:16:12,  6.47s/it]
+ 30%|███       | 551/1813 [59:36<2:12:30,  6.30s/it]
+ 30%|███       | 552/1813 [59:42<2:06:35,  6.02s/it]
+ 31%|███       | 553/1813 [59:46<1:59:07,  5.67s/it]
+ 31%|███       | 554/1813 [59:53<2:06:00,  6.00s/it]
+ 31%|███       | 555/1813 [59:58<2:00:07,  5.73s/it]
+ 31%|███       | 556/1813 [1:00:05<2:03:32,  5.90s/it]
+ 31%|███       | 557/1813 [1:00:12<2:10:47,  6.25s/it]
+ 31%|███       | 558/1813 [1:00:17<2:07:52,  6.11s/it]
+ 31%|███       | 559/1813 [1:00:22<2:01:12,  5.80s/it]
+ 31%|███       | 560/1813 [1:00:29<2:04:46,  5.97s/it]
+                                                      
+{'loss': '0.9923', 'grad_norm': '0.7128', 'learning_rate': '4.29e-05', 'epoch': '0.309', 'num_input_tokens_seen': 3738056, 'train_runtime': '3629', 'train_tokens_per_second': '1030'}
+
+ 31%|███       | 560/1813 [1:00:29<2:04:46,  5.97s/it]
+ 31%|███       | 561/1813 [1:00:35<2:04:19,  5.96s/it]
+ 31%|███       | 562/1813 [1:00:41<2:07:59,  6.14s/it]
+ 31%|███       | 563/1813 [1:00:48<2:12:12,  6.35s/it]
+ 31%|███       | 564/1813 [1:00:54<2:09:23,  6.22s/it]
+ 31%|███       | 565/1813 [1:00:59<2:04:27,  5.98s/it]
+ 31%|███       | 566/1813 [1:01:06<2:07:05,  6.11s/it]
+ 31%|███▏      | 567/1813 [1:01:11<2:03:11,  5.93s/it]
+ 31%|███▏      | 568/1813 [1:01:18<2:07:37,  6.15s/it]
+ 31%|███▏      | 569/1813 [1:01:26<2:15:58,  6.56s/it]
+ 31%|███▏      | 570/1813 [1:01:33<2:21:53,  6.85s/it]
+                                                      
+{'loss': '0.9906', 'grad_norm': '0.8182', 'learning_rate': '4.257e-05', 'epoch': '0.3145', 'num_input_tokens_seen': 3808208, 'train_runtime': '3694', 'train_tokens_per_second': '1031'}
+
+ 31%|███▏      | 570/1813 [1:01:33<2:21:53,  6.85s/it]
+ 31%|███▏      | 571/1813 [1:01:40<2:22:05,  6.86s/it]
+ 32%|███▏      | 572/1813 [1:01:46<2:16:43,  6.61s/it]
+ 32%|███▏      | 573/1813 [1:01:51<2:07:55,  6.19s/it]
+ 32%|███▏      | 574/1813 [1:01:57<2:05:46,  6.09s/it]
+ 32%|███▏      | 575/1813 [1:02:04<2:09:32,  6.28s/it]
+ 32%|███▏      | 576/1813 [1:02:09<2:00:06,  5.83s/it]
+ 32%|███▏      | 577/1813 [1:02:15<2:05:57,  6.11s/it]
+ 32%|███▏      | 578/1813 [1:02:22<2:06:29,  6.15s/it]
+ 32%|███▏      | 579/1813 [1:02:28<2:07:50,  6.22s/it]
+ 32%|███▏      | 580/1813 [1:02:34<2:08:39,  6.26s/it]
+                                                      
+{'loss': '0.9918', 'grad_norm': '1.012', 'learning_rate': '4.223e-05', 'epoch': '0.32', 'num_input_tokens_seen': 3872328, 'train_runtime': '3755', 'train_tokens_per_second': '1031'}
+
+ 32%|███▏      | 580/1813 [1:02:34<2:08:39,  6.26s/it]
+ 32%|███▏      | 581/1813 [1:02:40<2:06:13,  6.15s/it]
+ 32%|███▏      | 582/1813 [1:02:46<2:02:37,  5.98s/it]
+ 32%|███▏      | 583/1813 [1:02:52<2:01:17,  5.92s/it]
+ 32%|███▏      | 584/1813 [1:02:58<2:03:48,  6.04s/it]
+ 32%|███▏      | 585/1813 [1:03:04<2:03:50,  6.05s/it]
+ 32%|███▏      | 586/1813 [1:03:10<2:05:52,  6.16s/it]
+ 32%|███▏      | 587/1813 [1:03:16<2:05:15,  6.13s/it]
+ 32%|███▏      | 588/1813 [1:03:23<2:06:25,  6.19s/it]
+ 32%|███▏      | 589/1813 [1:03:29<2:08:17,  6.29s/it]
+ 33%|███▎      | 590/1813 [1:03:36<2:07:45,  6.27s/it]
+                                                      
+{'loss': '0.8963', 'grad_norm': '1.191', 'learning_rate': '4.188e-05', 'epoch': '0.3255', 'num_input_tokens_seen': 3936024, 'train_runtime': '3816', 'train_tokens_per_second': '1031'}
+
+ 33%|███▎      | 590/1813 [1:03:36<2:07:45,  6.27s/it]
+ 33%|███▎      | 591/1813 [1:03:41<2:03:01,  6.04s/it]
+ 33%|███▎      | 592/1813 [1:03:47<2:01:54,  5.99s/it]
+ 33%|███▎      | 593/1813 [1:03:54<2:05:27,  6.17s/it]
+ 33%|███▎      | 594/1813 [1:04:00<2:06:06,  6.21s/it]
+ 33%|███▎      | 595/1813 [1:04:06<2:03:45,  6.10s/it]
+ 33%|███▎      | 596/1813 [1:04:12<2:02:36,  6.04s/it]
+ 33%|███▎      | 597/1813 [1:04:17<2:00:17,  5.94s/it]
+ 33%|███▎      | 598/1813 [1:04:23<1:59:27,  5.90s/it]
+ 33%|███▎      | 599/1813 [1:04:29<2:01:15,  5.99s/it]
+ 33%|███▎      | 600/1813 [1:04:36<2:07:12,  6.29s/it]
+                                                      
+{'loss': '0.9582', 'grad_norm': '0.8625', 'learning_rate': '4.153e-05', 'epoch': '0.331', 'num_input_tokens_seen': 3999176, 'train_runtime': '3877', 'train_tokens_per_second': '1032'}
+
+ 33%|███▎      | 600/1813 [1:04:36<2:07:12,  6.29s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.73it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.21it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:32,  6.48it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.08it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.99it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.83it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.04it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.59it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.58it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.34it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.03it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.90it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.29it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:26,  6.31it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.85it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.20it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:15,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.94it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.16it/s]
+
+  9%|▉         | 93/1000 [00:12<02:37,  5.77it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.51it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.35it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.61it/s]
+
+ 10%|█         | 103/1000 [00:14<01:56,  7.73it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.30it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.42it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.17it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.01it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.49it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.80it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.22it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.90it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.07it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.13it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.14it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.89it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.95it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.88it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25, 10.05it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.42it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.48it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.90it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.02it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.41it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.96it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.66it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.85it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.15it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.14it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.46it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.51it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.73it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.87it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.37it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.12it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.76it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.66it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.77it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.62it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.91it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.51it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.68it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.73it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.51it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.79it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.22it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.95it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:25,  8.94it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:31,  8.32it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.01it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.46it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.64it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.24it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.84it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.76it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:55,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.05it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.74it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.17it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.84it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.11it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.75it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.07it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.94it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.79it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.26it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.57it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.05it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.02it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.86it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.59it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.21it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.54it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.58it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.43it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:19,  7.89it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:41,  6.13it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:55,  5.42it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:30,  6.85it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:11,  8.70it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.85it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.16it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.17it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.13it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.64it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.36it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.76it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.46it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.22it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.23it/s]
+
+ 41%|████      | 406/1000 [00:54<01:10,  8.48it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.44it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.49it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.68it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.53it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.83it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.53it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.01it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.79it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.51it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.49it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.80it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.25it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.81it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.20it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.38it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:25,  6.18it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.19it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.29it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.08it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.51it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.05it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.26it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.96it/s]
+
+ 50%|████▉     | 496/1000 [01:06<01:00,  8.39it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.81it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.57it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.07it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.38it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.04it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.11it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.68it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.68it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.35it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:19,  5.83it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:20,  5.69it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.22it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:06,  6.89it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.02it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:10,  6.47it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.68it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.91it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.50it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.25it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.17it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.86it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.96it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.71it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:06,  6.52it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:07,  6.42it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.42it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.29it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.33it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.82it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.72it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.58it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.79it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:02,  6.46it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.68it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.09it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.81it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.59it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.73it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.08it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.01it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.19it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.44it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.63it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.18it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.43it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.94it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.95it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.58it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.51it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.31it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.69it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.91it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.57it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.49it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.69it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:35,  9.11it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:30, 10.48it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:36,  8.69it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:32,  9.67it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:46,  6.79it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.09it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.77it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.81it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.24it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.40it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:48,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.59it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.01it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.49it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.69it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.36it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.79it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.15it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.30it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.61it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:27,  9.54it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.88it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.52it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.78it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.35it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.60it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.43it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.39it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.59it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:33,  7.30it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.70it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.45it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:30,  7.73it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.60it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.53it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.10it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.37it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.76it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.88it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.59it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.94it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.51it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.34it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.97it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.21it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.96it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.18it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.16it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.96it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.21it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.23it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.31it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.13it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.19it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.99it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.22it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.46it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.14it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.54it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.86it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.48it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.45it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.71it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.59it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.66it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.29it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.24it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.26it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.33it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.75it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.84it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.69it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.61it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  6.98it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.47it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  7.97it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.58it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:11,  6.34it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.72it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.56it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.13it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.26it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.73it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  7.97it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.09it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.32it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.05it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.33it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.67it/s]
+
+ 95%|█████████▌| 950/1000 [02:08<00:05,  8.82it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.40it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.48it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.43it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.33it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.18it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.79it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.63it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.8207', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.38', 'eval_steps_per_second': '7.38', 'epoch': '0.331', 'num_input_tokens_seen': 3999176}
+
+ 33%|███▎      | 600/1813 [1:06:52<2:07:12,  6.29s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.63it/s]
+
+                                                   
+ 33%|███▎      | 601/1813 [1:06:56<15:38:06, 46.44s/it]
+ 33%|███▎      | 602/1813 [1:07:01<11:25:56, 33.99s/it]
+ 33%|███▎      | 603/1813 [1:07:07<8:32:50, 25.43s/it] 
+ 33%|███▎      | 604/1813 [1:07:13<6:35:55, 19.65s/it]
+ 33%|███▎      | 605/1813 [1:07:21<5:25:45, 16.18s/it]
+ 33%|███▎      | 606/1813 [1:07:28<4:32:30, 13.55s/it]
+ 33%|███▎      | 607/1813 [1:07:35<3:49:08, 11.40s/it]
+ 34%|███▎      | 608/1813 [1:07:41<3:19:14,  9.92s/it]
+ 34%|███▎      | 609/1813 [1:07:48<2:57:42,  8.86s/it]
+ 34%|███▎      | 610/1813 [1:07:54<2:42:10,  8.09s/it]
+                                                      
+{'loss': '0.9928', 'grad_norm': '0.71', 'learning_rate': '4.118e-05', 'epoch': '0.3366', 'num_input_tokens_seen': 4066376, 'train_runtime': '4074', 'train_tokens_per_second': '998'}
+
+ 34%|███▎      | 610/1813 [1:07:54<2:42:10,  8.09s/it]
+ 34%|███▎      | 611/1813 [1:07:59<2:25:04,  7.24s/it]
+ 34%|███▍      | 612/1813 [1:08:03<2:06:29,  6.32s/it]
+ 34%|███▍      | 613/1813 [1:08:09<2:03:03,  6.15s/it]
+ 34%|███▍      | 614/1813 [1:08:15<2:03:23,  6.17s/it]
+ 34%|███▍      | 615/1813 [1:08:21<1:59:52,  6.00s/it]
+ 34%|███▍      | 616/1813 [1:08:26<1:55:20,  5.78s/it]
+ 34%|███▍      | 617/1813 [1:08:32<1:53:47,  5.71s/it]
+ 34%|███▍      | 618/1813 [1:08:39<2:02:00,  6.13s/it]
+ 34%|███▍      | 619/1813 [1:08:44<1:54:19,  5.75s/it]
+ 34%|███▍      | 620/1813 [1:08:50<1:58:54,  5.98s/it]
+                                                      
+{'loss': '0.8803', 'grad_norm': '1.018', 'learning_rate': '4.081e-05', 'epoch': '0.3421', 'num_input_tokens_seen': 4123976, 'train_runtime': '4131', 'train_tokens_per_second': '998.4'}
+
+ 34%|███▍      | 620/1813 [1:08:50<1:58:54,  5.98s/it]
+ 34%|███▍      | 621/1813 [1:08:57<2:03:57,  6.24s/it]
+ 34%|███▍      | 622/1813 [1:09:05<2:12:42,  6.69s/it]
+ 34%|███▍      | 623/1813 [1:09:10<2:03:51,  6.24s/it]
+ 34%|███▍      | 624/1813 [1:09:16<2:04:17,  6.27s/it]
+ 34%|███▍      | 625/1813 [1:09:21<1:56:07,  5.86s/it]
+ 35%|███▍      | 626/1813 [1:09:25<1:46:04,  5.36s/it]
+ 35%|███▍      | 627/1813 [1:09:32<1:50:16,  5.58s/it]
+ 35%|███▍      | 628/1813 [1:09:38<1:53:04,  5.73s/it]
+ 35%|███▍      | 629/1813 [1:09:46<2:08:24,  6.51s/it]
+ 35%|███▍      | 630/1813 [1:09:53<2:10:48,  6.63s/it]
+                                                      
+{'loss': '0.962', 'grad_norm': '0.778', 'learning_rate': '4.044e-05', 'epoch': '0.3476', 'num_input_tokens_seen': 4191992, 'train_runtime': '4193', 'train_tokens_per_second': '999.7'}
+
+ 35%|███▍      | 630/1813 [1:09:53<2:10:48,  6.63s/it]
+ 35%|███▍      | 631/1813 [1:10:00<2:10:54,  6.65s/it]
+ 35%|███▍      | 632/1813 [1:10:06<2:08:24,  6.52s/it]
+ 35%|███▍      | 633/1813 [1:10:12<2:04:46,  6.34s/it]
+ 35%|███▍      | 634/1813 [1:10:19<2:10:48,  6.66s/it]
+ 35%|███▌      | 635/1813 [1:10:26<2:11:50,  6.72s/it]
+ 35%|███▌      | 636/1813 [1:10:31<2:03:24,  6.29s/it]
+ 35%|███▌      | 637/1813 [1:10:38<2:08:08,  6.54s/it]
+ 35%|███▌      | 638/1813 [1:10:44<2:02:48,  6.27s/it]
+ 35%|███▌      | 639/1813 [1:10:49<1:56:24,  5.95s/it]
+ 35%|███▌      | 640/1813 [1:10:55<1:56:28,  5.96s/it]
+                                                      
+{'loss': '0.9794', 'grad_norm': '1.088', 'learning_rate': '4.007e-05', 'epoch': '0.3531', 'num_input_tokens_seen': 4257760, 'train_runtime': '4256', 'train_tokens_per_second': '1000'}
+
+ 35%|███▌      | 640/1813 [1:10:55<1:56:28,  5.96s/it]
+ 35%|███▌      | 641/1813 [1:11:00<1:51:19,  5.70s/it]
+ 35%|███▌      | 642/1813 [1:11:07<1:59:35,  6.13s/it]
+ 35%|███▌      | 643/1813 [1:11:14<2:01:30,  6.23s/it]
+ 36%|███▌      | 644/1813 [1:11:19<1:56:34,  5.98s/it]
+ 36%|███▌      | 645/1813 [1:11:25<1:56:37,  5.99s/it]
+ 36%|███▌      | 646/1813 [1:11:31<1:55:27,  5.94s/it]
+ 36%|███▌      | 647/1813 [1:11:38<1:58:25,  6.09s/it]
+ 36%|███▌      | 648/1813 [1:11:45<2:04:45,  6.43s/it]
+ 36%|███▌      | 649/1813 [1:11:51<2:01:17,  6.25s/it]
+ 36%|███▌      | 650/1813 [1:11:56<1:56:07,  5.99s/it]
+                                                      
+{'loss': '0.9807', 'grad_norm': '1.361', 'learning_rate': '3.969e-05', 'epoch': '0.3586', 'num_input_tokens_seen': 4322136, 'train_runtime': '4317', 'train_tokens_per_second': '1001'}
+
+ 36%|███▌      | 650/1813 [1:11:56<1:56:07,  5.99s/it]
+ 36%|███▌      | 651/1813 [1:12:03<2:00:00,  6.20s/it]
+ 36%|███▌      | 652/1813 [1:12:10<2:03:25,  6.38s/it]
+ 36%|███▌      | 653/1813 [1:12:15<1:58:12,  6.11s/it]
+ 36%|███▌      | 654/1813 [1:12:20<1:53:23,  5.87s/it]
+ 36%|███▌      | 655/1813 [1:12:27<1:55:28,  5.98s/it]
+ 36%|███▌      | 656/1813 [1:12:32<1:52:22,  5.83s/it]
+ 36%|███▌      | 657/1813 [1:12:37<1:49:47,  5.70s/it]
+ 36%|███▋      | 658/1813 [1:12:44<1:55:32,  6.00s/it]
+ 36%|███▋      | 659/1813 [1:12:51<1:58:44,  6.17s/it]
+ 36%|███▋      | 660/1813 [1:12:57<1:56:56,  6.09s/it]
+                                                      
+{'loss': '0.8999', 'grad_norm': '0.9594', 'learning_rate': '3.931e-05', 'epoch': '0.3641', 'num_input_tokens_seen': 4385688, 'train_runtime': '4377', 'train_tokens_per_second': '1002'}
+
+ 36%|███▋      | 660/1813 [1:12:57<1:56:56,  6.09s/it]
+ 36%|███▋      | 661/1813 [1:13:03<1:56:53,  6.09s/it]
+ 37%|███▋      | 662/1813 [1:13:09<1:56:43,  6.08s/it]
+ 37%|███▋      | 663/1813 [1:13:16<2:00:29,  6.29s/it]
+ 37%|███▋      | 664/1813 [1:13:22<2:01:24,  6.34s/it]
+ 37%|███▋      | 665/1813 [1:13:27<1:54:48,  6.00s/it]
+ 37%|███▋      | 666/1813 [1:13:33<1:51:44,  5.85s/it]
+ 37%|███▋      | 667/1813 [1:13:40<1:57:32,  6.15s/it]
+ 37%|███▋      | 668/1813 [1:13:46<1:59:22,  6.26s/it]
+ 37%|███▋      | 669/1813 [1:13:52<1:58:35,  6.22s/it]
+ 37%|███▋      | 670/1813 [1:13:58<1:55:26,  6.06s/it]
+                                                      
+{'loss': '0.9984', 'grad_norm': '1.088', 'learning_rate': '3.892e-05', 'epoch': '0.3697', 'num_input_tokens_seen': 4449456, 'train_runtime': '4438', 'train_tokens_per_second': '1003'}
+
+ 37%|███▋      | 670/1813 [1:13:58<1:55:26,  6.06s/it]
+ 37%|███▋      | 671/1813 [1:14:04<1:57:16,  6.16s/it]
+ 37%|███▋      | 672/1813 [1:14:11<2:01:23,  6.38s/it]
+ 37%|███▋      | 673/1813 [1:14:18<2:01:40,  6.40s/it]
+ 37%|███▋      | 674/1813 [1:14:24<2:02:50,  6.47s/it]
+ 37%|███▋      | 675/1813 [1:14:30<1:59:22,  6.29s/it]
+ 37%|███▋      | 676/1813 [1:14:36<1:54:54,  6.06s/it]
+ 37%|███▋      | 677/1813 [1:14:44<2:06:18,  6.67s/it]
+ 37%|███▋      | 678/1813 [1:14:49<1:56:59,  6.18s/it]
+ 37%|███▋      | 679/1813 [1:14:54<1:51:11,  5.88s/it]
+ 38%|███▊      | 680/1813 [1:15:00<1:50:23,  5.85s/it]
+                                                      
+{'loss': '0.9608', 'grad_norm': '0.9286', 'learning_rate': '3.852e-05', 'epoch': '0.3752', 'num_input_tokens_seen': 4514984, 'train_runtime': '4500', 'train_tokens_per_second': '1003'}
+
+ 38%|███▊      | 680/1813 [1:15:00<1:50:23,  5.85s/it]
+ 38%|███▊      | 681/1813 [1:15:06<1:53:33,  6.02s/it]
+ 38%|███▊      | 682/1813 [1:15:11<1:47:18,  5.69s/it]
+ 38%|███▊      | 683/1813 [1:15:17<1:49:17,  5.80s/it]
+ 38%|███▊      | 684/1813 [1:15:24<1:53:05,  6.01s/it]
+ 38%|███▊      | 685/1813 [1:15:31<2:02:55,  6.54s/it]
+ 38%|███▊      | 686/1813 [1:15:38<2:02:21,  6.51s/it]
+ 38%|███▊      | 687/1813 [1:15:46<2:08:58,  6.87s/it]
+ 38%|███▊      | 688/1813 [1:15:50<1:56:02,  6.19s/it]
+ 38%|███▊      | 689/1813 [1:15:59<2:10:33,  6.97s/it]
+ 38%|███▊      | 690/1813 [1:16:06<2:11:46,  7.04s/it]
+                                                      
+{'loss': '0.9922', 'grad_norm': '1.09', 'learning_rate': '3.812e-05', 'epoch': '0.3807', 'num_input_tokens_seen': 4587840, 'train_runtime': '4567', 'train_tokens_per_second': '1005'}
+
+ 38%|███▊      | 690/1813 [1:16:06<2:11:46,  7.04s/it]
+ 38%|███▊      | 691/1813 [1:16:12<2:04:09,  6.64s/it]
+ 38%|███▊      | 692/1813 [1:16:18<1:59:54,  6.42s/it]
+ 38%|███▊      | 693/1813 [1:16:23<1:53:51,  6.10s/it]
+ 38%|███▊      | 694/1813 [1:16:29<1:55:11,  6.18s/it]
+ 38%|███▊      | 695/1813 [1:16:35<1:54:14,  6.13s/it]
+ 38%|███▊      | 696/1813 [1:16:43<2:03:40,  6.64s/it]
+ 38%|███▊      | 697/1813 [1:16:48<1:54:03,  6.13s/it]
+ 38%|███▊      | 698/1813 [1:16:54<1:49:58,  5.92s/it]
+ 39%|███▊      | 699/1813 [1:17:01<1:55:29,  6.22s/it]
+ 39%|███▊      | 700/1813 [1:17:07<1:57:30,  6.33s/it]
+                                                      
+{'loss': '1.007', 'grad_norm': '1.052', 'learning_rate': '3.772e-05', 'epoch': '0.3862', 'num_input_tokens_seen': 4653312, 'train_runtime': '4628', 'train_tokens_per_second': '1006'}
+
+ 39%|███▊      | 700/1813 [1:17:07<1:57:30,  6.33s/it]
+ 39%|███▊      | 701/1813 [1:17:15<2:02:47,  6.63s/it]
+ 39%|███▊      | 702/1813 [1:17:21<2:00:35,  6.51s/it]
+ 39%|███▉      | 703/1813 [1:17:29<2:08:08,  6.93s/it]
+ 39%|███▉      | 704/1813 [1:17:34<2:01:00,  6.55s/it]
+ 39%|███▉      | 705/1813 [1:17:41<2:03:11,  6.67s/it]
+ 39%|███▉      | 706/1813 [1:17:48<2:01:21,  6.58s/it]
+ 39%|███▉      | 707/1813 [1:17:57<2:17:42,  7.47s/it]
+ 39%|███▉      | 708/1813 [1:18:05<2:16:49,  7.43s/it]
+ 39%|███▉      | 709/1813 [1:18:11<2:11:47,  7.16s/it]
+ 39%|███▉      | 710/1813 [1:18:17<2:03:11,  6.70s/it]
+                                                      
+{'loss': '0.9655', 'grad_norm': '0.7867', 'learning_rate': '3.731e-05', 'epoch': '0.3917', 'num_input_tokens_seen': 4732080, 'train_runtime': '4697', 'train_tokens_per_second': '1007'}
+
+ 39%|███▉      | 710/1813 [1:18:17<2:03:11,  6.70s/it]
+ 39%|███▉      | 711/1813 [1:18:23<2:03:13,  6.71s/it]
+ 39%|███▉      | 712/1813 [1:18:30<2:02:33,  6.68s/it]
+ 39%|███▉      | 713/1813 [1:18:36<1:56:19,  6.35s/it]
+ 39%|███▉      | 714/1813 [1:18:42<1:54:32,  6.25s/it]
+ 39%|███▉      | 715/1813 [1:18:48<1:56:07,  6.35s/it]
+ 39%|███▉      | 716/1813 [1:18:55<1:59:24,  6.53s/it]
+ 40%|███▉      | 717/1813 [1:19:01<1:57:06,  6.41s/it]
+ 40%|███▉      | 718/1813 [1:19:08<1:57:22,  6.43s/it]
+ 40%|███▉      | 719/1813 [1:19:13<1:50:58,  6.09s/it]
+ 40%|███▉      | 720/1813 [1:19:19<1:51:02,  6.10s/it]
+                                                      
+{'loss': '1.011', 'grad_norm': '1.093', 'learning_rate': '3.69e-05', 'epoch': '0.3972', 'num_input_tokens_seen': 4799552, 'train_runtime': '4760', 'train_tokens_per_second': '1008'}
+
+ 40%|███▉      | 720/1813 [1:19:19<1:51:02,  6.10s/it]
+ 40%|███▉      | 721/1813 [1:19:26<1:53:14,  6.22s/it]
+ 40%|███▉      | 722/1813 [1:19:31<1:46:50,  5.88s/it]
+ 40%|███▉      | 723/1813 [1:19:36<1:42:54,  5.66s/it]
+ 40%|███▉      | 724/1813 [1:19:44<1:55:09,  6.35s/it]
+ 40%|███▉      | 725/1813 [1:19:52<2:07:03,  7.01s/it]
+ 40%|████      | 726/1813 [1:19:59<2:03:59,  6.84s/it]
+ 40%|████      | 727/1813 [1:20:05<1:58:45,  6.56s/it]
+ 40%|████      | 728/1813 [1:20:10<1:50:21,  6.10s/it]
+ 40%|████      | 729/1813 [1:20:15<1:47:52,  5.97s/it]
+ 40%|████      | 730/1813 [1:20:21<1:44:30,  5.79s/it]
+                                                      
+{'loss': '0.9826', 'grad_norm': '1.259', 'learning_rate': '3.648e-05', 'epoch': '0.4028', 'num_input_tokens_seen': 4866864, 'train_runtime': '4821', 'train_tokens_per_second': '1009'}
+
+ 40%|████      | 730/1813 [1:20:21<1:44:30,  5.79s/it]
+ 40%|████      | 731/1813 [1:20:28<1:50:54,  6.15s/it]
+ 40%|████      | 732/1813 [1:20:34<1:52:27,  6.24s/it]
+ 40%|████      | 733/1813 [1:20:42<1:59:03,  6.61s/it]
+ 40%|████      | 734/1813 [1:20:49<2:01:48,  6.77s/it]
+ 41%|████      | 735/1813 [1:20:56<2:01:27,  6.76s/it]
+ 41%|████      | 736/1813 [1:21:02<1:56:40,  6.50s/it]
+ 41%|████      | 737/1813 [1:21:07<1:53:34,  6.33s/it]
+ 41%|████      | 738/1813 [1:21:13<1:47:57,  6.03s/it]
+ 41%|████      | 739/1813 [1:21:19<1:46:45,  5.96s/it]
+ 41%|████      | 740/1813 [1:21:25<1:50:45,  6.19s/it]
+                                                      
+{'loss': '0.9153', 'grad_norm': '0.7697', 'learning_rate': '3.606e-05', 'epoch': '0.4083', 'num_input_tokens_seen': 4936352, 'train_runtime': '4886', 'train_tokens_per_second': '1010'}
+
+ 41%|████      | 740/1813 [1:21:25<1:50:45,  6.19s/it]
+ 41%|████      | 741/1813 [1:21:32<1:53:22,  6.35s/it]
+ 41%|████      | 742/1813 [1:21:40<1:59:33,  6.70s/it]
+ 41%|████      | 743/1813 [1:21:47<2:03:07,  6.90s/it]
+ 41%|████      | 744/1813 [1:21:54<2:01:40,  6.83s/it]
+ 41%|████      | 745/1813 [1:22:01<2:03:24,  6.93s/it]
+ 41%|████      | 746/1813 [1:22:07<1:59:55,  6.74s/it]
+ 41%|████      | 747/1813 [1:22:15<2:06:27,  7.12s/it]
+ 41%|████▏     | 748/1813 [1:22:21<1:59:10,  6.71s/it]
+ 41%|████▏     | 749/1813 [1:22:27<1:58:21,  6.67s/it]
+ 41%|████▏     | 750/1813 [1:22:33<1:54:38,  6.47s/it]
+                                                      
+{'loss': '0.9939', 'grad_norm': '0.8564', 'learning_rate': '3.563e-05', 'epoch': '0.4138', 'num_input_tokens_seen': 5011064, 'train_runtime': '4954', 'train_tokens_per_second': '1012'}
+
+ 41%|████▏     | 750/1813 [1:22:33<1:54:38,  6.47s/it]
+ 41%|████▏     | 751/1813 [1:22:40<1:53:36,  6.42s/it]
+ 41%|████▏     | 752/1813 [1:22:46<1:51:39,  6.31s/it]
+ 42%|████▏     | 753/1813 [1:22:52<1:53:37,  6.43s/it]
+ 42%|████▏     | 754/1813 [1:23:01<2:04:50,  7.07s/it]
+ 42%|████▏     | 755/1813 [1:23:06<1:54:43,  6.51s/it]
+ 42%|████▏     | 756/1813 [1:23:12<1:50:34,  6.28s/it]
+ 42%|████▏     | 757/1813 [1:23:18<1:48:54,  6.19s/it]
+ 42%|████▏     | 758/1813 [1:23:25<1:52:10,  6.38s/it]
+ 42%|████▏     | 759/1813 [1:23:32<1:55:26,  6.57s/it]
+ 42%|████▏     | 760/1813 [1:23:38<1:52:44,  6.42s/it]
+                                                      
+{'loss': '1.004', 'grad_norm': '1.146', 'learning_rate': '3.52e-05', 'epoch': '0.4193', 'num_input_tokens_seen': 5078960, 'train_runtime': '5018', 'train_tokens_per_second': '1012'}
+
+ 42%|████▏     | 760/1813 [1:23:38<1:52:44,  6.42s/it]
+ 42%|████▏     | 761/1813 [1:23:45<1:54:41,  6.54s/it]
+ 42%|████▏     | 762/1813 [1:23:51<1:55:29,  6.59s/it]
+ 42%|████▏     | 763/1813 [1:23:59<1:58:18,  6.76s/it]
+ 42%|████▏     | 764/1813 [1:24:06<1:59:26,  6.83s/it]
+ 42%|████▏     | 765/1813 [1:24:13<2:02:12,  7.00s/it]
+ 42%|████▏     | 766/1813 [1:24:19<1:58:46,  6.81s/it]
+ 42%|████▏     | 767/1813 [1:24:25<1:51:16,  6.38s/it]
+ 42%|████▏     | 768/1813 [1:24:29<1:39:27,  5.71s/it]
+ 42%|████▏     | 769/1813 [1:24:36<1:44:23,  6.00s/it]
+ 42%|████▏     | 770/1813 [1:24:42<1:45:04,  6.04s/it]
+                                                      
+{'loss': '0.9399', 'grad_norm': '0.9321', 'learning_rate': '3.477e-05', 'epoch': '0.4248', 'num_input_tokens_seen': 5148504, 'train_runtime': '5082', 'train_tokens_per_second': '1013'}
+
+ 42%|████▏     | 770/1813 [1:24:42<1:45:04,  6.04s/it]
+ 43%|████▎     | 771/1813 [1:24:48<1:47:13,  6.17s/it]
+ 43%|████▎     | 772/1813 [1:24:55<1:52:47,  6.50s/it]
+ 43%|████▎     | 773/1813 [1:25:01<1:50:25,  6.37s/it]
+ 43%|████▎     | 774/1813 [1:25:07<1:44:10,  6.02s/it]
+ 43%|████▎     | 775/1813 [1:25:13<1:44:01,  6.01s/it]
+ 43%|████▎     | 776/1813 [1:25:20<1:48:38,  6.29s/it]
+ 43%|████▎     | 777/1813 [1:25:27<1:56:06,  6.72s/it]
+ 43%|████▎     | 778/1813 [1:25:32<1:47:07,  6.21s/it]
+ 43%|████▎     | 779/1813 [1:25:37<1:41:10,  5.87s/it]
+ 43%|████▎     | 780/1813 [1:25:44<1:45:00,  6.10s/it]
+                                                      
+{'loss': '0.8935', 'grad_norm': '1.005', 'learning_rate': '3.433e-05', 'epoch': '0.4303', 'num_input_tokens_seen': 5215784, 'train_runtime': '5145', 'train_tokens_per_second': '1014'}
+
+ 43%|████▎     | 780/1813 [1:25:44<1:45:00,  6.10s/it]
+ 43%|████▎     | 781/1813 [1:25:51<1:48:44,  6.32s/it]
+ 43%|████▎     | 782/1813 [1:25:59<1:58:31,  6.90s/it]
+ 43%|████▎     | 783/1813 [1:26:06<1:59:44,  6.98s/it]
+ 43%|████▎     | 784/1813 [1:26:12<1:54:27,  6.67s/it]
+ 43%|████▎     | 785/1813 [1:26:19<1:56:34,  6.80s/it]
+ 43%|████▎     | 786/1813 [1:26:27<1:58:22,  6.92s/it]
+ 43%|████▎     | 787/1813 [1:26:32<1:49:05,  6.38s/it]
+ 43%|████▎     | 788/1813 [1:26:37<1:44:54,  6.14s/it]
+ 44%|████▎     | 789/1813 [1:26:43<1:40:33,  5.89s/it]
+ 44%|████▎     | 790/1813 [1:26:49<1:40:43,  5.91s/it]
+                                                      
+{'loss': '1.012', 'grad_norm': '1.129', 'learning_rate': '3.389e-05', 'epoch': '0.4359', 'num_input_tokens_seen': 5285160, 'train_runtime': '5209', 'train_tokens_per_second': '1015'}
+
+ 44%|████▎     | 790/1813 [1:26:49<1:40:43,  5.91s/it]
+ 44%|████▎     | 791/1813 [1:26:55<1:42:35,  6.02s/it]
+ 44%|████▎     | 792/1813 [1:27:02<1:49:19,  6.42s/it]
+ 44%|████▎     | 793/1813 [1:27:10<1:54:20,  6.73s/it]
+ 44%|████▍     | 794/1813 [1:27:16<1:54:07,  6.72s/it]
+ 44%|████▍     | 795/1813 [1:27:22<1:50:28,  6.51s/it]
+ 44%|████▍     | 796/1813 [1:27:29<1:52:54,  6.66s/it]
+ 44%|████▍     | 797/1813 [1:27:37<1:58:08,  6.98s/it]
+ 44%|████▍     | 798/1813 [1:27:43<1:53:34,  6.71s/it]
+ 44%|████▍     | 799/1813 [1:27:49<1:49:30,  6.48s/it]
+ 44%|████▍     | 800/1813 [1:27:55<1:46:24,  6.30s/it]
+                                                      
+{'loss': '0.9528', 'grad_norm': '0.8776', 'learning_rate': '3.345e-05', 'epoch': '0.4414', 'num_input_tokens_seen': 5357720, 'train_runtime': '5275', 'train_tokens_per_second': '1016'}
+
+ 44%|████▍     | 800/1813 [1:27:55<1:46:24,  6.30s/it]
+ 44%|████▍     | 801/1813 [1:28:00<1:41:40,  6.03s/it]
+ 44%|████▍     | 802/1813 [1:28:06<1:42:03,  6.06s/it]
+ 44%|████▍     | 803/1813 [1:28:14<1:51:12,  6.61s/it]
+ 44%|████▍     | 804/1813 [1:28:21<1:48:41,  6.46s/it]
+ 44%|████▍     | 805/1813 [1:28:26<1:45:09,  6.26s/it]
+ 44%|████▍     | 806/1813 [1:28:32<1:43:58,  6.20s/it]
+ 45%|████▍     | 807/1813 [1:28:38<1:42:06,  6.09s/it]
+ 45%|████▍     | 808/1813 [1:28:46<1:50:48,  6.61s/it]
+ 45%|████▍     | 809/1813 [1:28:53<1:51:50,  6.68s/it]
+ 45%|████▍     | 810/1813 [1:29:01<1:57:48,  7.05s/it]
+                                                      
+{'loss': '0.9651', 'grad_norm': '5.31', 'learning_rate': '3.3e-05', 'epoch': '0.4469', 'num_input_tokens_seen': 5430288, 'train_runtime': '5341', 'train_tokens_per_second': '1017'}
+
+ 45%|████▍     | 810/1813 [1:29:01<1:57:48,  7.05s/it]
+ 45%|████▍     | 811/1813 [1:29:08<1:56:08,  6.95s/it]
+ 45%|████▍     | 812/1813 [1:29:13<1:50:24,  6.62s/it]
+ 45%|████▍     | 813/1813 [1:29:19<1:45:54,  6.35s/it]
+ 45%|████▍     | 814/1813 [1:29:25<1:45:19,  6.33s/it]
+ 45%|████▍     | 815/1813 [1:29:30<1:38:24,  5.92s/it]
+ 45%|████▌     | 816/1813 [1:29:37<1:42:57,  6.20s/it]
+ 45%|████▌     | 817/1813 [1:29:43<1:39:47,  6.01s/it]
+ 45%|████▌     | 818/1813 [1:29:50<1:45:09,  6.34s/it]
+ 45%|████▌     | 819/1813 [1:29:56<1:45:23,  6.36s/it]
+ 45%|████▌     | 820/1813 [1:30:04<1:50:00,  6.65s/it]
+                                                      
+{'loss': '0.917', 'grad_norm': '1.147', 'learning_rate': '3.255e-05', 'epoch': '0.4524', 'num_input_tokens_seen': 5497552, 'train_runtime': '5404', 'train_tokens_per_second': '1017'}
+
+ 45%|████▌     | 820/1813 [1:30:04<1:50:00,  6.65s/it]
+ 45%|████▌     | 821/1813 [1:30:10<1:48:47,  6.58s/it]
+ 45%|████▌     | 822/1813 [1:30:15<1:40:06,  6.06s/it]
+ 45%|████▌     | 823/1813 [1:30:20<1:33:23,  5.66s/it]
+ 45%|████▌     | 824/1813 [1:30:26<1:36:14,  5.84s/it]
+ 46%|████▌     | 825/1813 [1:30:32<1:38:40,  5.99s/it]
+ 46%|████▌     | 826/1813 [1:30:39<1:44:17,  6.34s/it]
+ 46%|████▌     | 827/1813 [1:30:45<1:42:41,  6.25s/it]
+ 46%|████▌     | 828/1813 [1:30:52<1:42:27,  6.24s/it]
+ 46%|████▌     | 829/1813 [1:30:57<1:40:19,  6.12s/it]
+ 46%|████▌     | 830/1813 [1:31:05<1:47:22,  6.55s/it]
+                                                      
+{'loss': '0.8098', 'grad_norm': '0.9319', 'learning_rate': '3.21e-05', 'epoch': '0.4579', 'num_input_tokens_seen': 5563144, 'train_runtime': '5465', 'train_tokens_per_second': '1018'}
+
+ 46%|████▌     | 830/1813 [1:31:05<1:47:22,  6.55s/it]
+ 46%|████▌     | 831/1813 [1:31:11<1:45:33,  6.45s/it]
+ 46%|████▌     | 832/1813 [1:31:19<1:50:10,  6.74s/it]
+ 46%|████▌     | 833/1813 [1:31:25<1:48:32,  6.65s/it]
+ 46%|████▌     | 834/1813 [1:31:32<1:49:01,  6.68s/it]
+ 46%|████▌     | 835/1813 [1:31:38<1:46:29,  6.53s/it]
+ 46%|████▌     | 836/1813 [1:31:45<1:48:08,  6.64s/it]
+ 46%|████▌     | 837/1813 [1:31:53<1:53:54,  7.00s/it]
+ 46%|████▌     | 838/1813 [1:31:59<1:48:54,  6.70s/it]
+ 46%|████▋     | 839/1813 [1:32:05<1:46:26,  6.56s/it]
+ 46%|████▋     | 840/1813 [1:32:11<1:42:41,  6.33s/it]
+                                                      
+{'loss': '0.9797', 'grad_norm': '1.098', 'learning_rate': '3.165e-05', 'epoch': '0.4634', 'num_input_tokens_seen': 5634104, 'train_runtime': '5531', 'train_tokens_per_second': '1019'}
+
+ 46%|████▋     | 840/1813 [1:32:11<1:42:41,  6.33s/it]
+ 46%|████▋     | 841/1813 [1:32:17<1:41:10,  6.25s/it]
+ 46%|████▋     | 842/1813 [1:32:24<1:43:44,  6.41s/it]
+ 46%|████▋     | 843/1813 [1:32:30<1:42:09,  6.32s/it]
+ 47%|████▋     | 844/1813 [1:32:35<1:39:07,  6.14s/it]
+ 47%|████▋     | 845/1813 [1:32:42<1:40:38,  6.24s/it]
+ 47%|████▋     | 846/1813 [1:32:49<1:43:05,  6.40s/it]
+ 47%|████▋     | 847/1813 [1:32:56<1:46:52,  6.64s/it]
+ 47%|████▋     | 848/1813 [1:33:03<1:50:26,  6.87s/it]
+ 47%|████▋     | 849/1813 [1:33:09<1:43:33,  6.45s/it]
+ 47%|████▋     | 850/1813 [1:33:17<1:52:24,  7.00s/it]
+                                                      
+{'loss': '0.9366', 'grad_norm': '0.8263', 'learning_rate': '3.119e-05', 'epoch': '0.469', 'num_input_tokens_seen': 5704744, 'train_runtime': '5598', 'train_tokens_per_second': '1019'}
+
+ 47%|████▋     | 850/1813 [1:33:17<1:52:24,  7.00s/it]
+ 47%|████▋     | 851/1813 [1:33:24<1:51:33,  6.96s/it]
+ 47%|████▋     | 852/1813 [1:33:29<1:42:13,  6.38s/it]
+ 47%|████▋     | 853/1813 [1:33:35<1:41:47,  6.36s/it]
+ 47%|████▋     | 854/1813 [1:33:42<1:44:37,  6.55s/it]
+ 47%|████▋     | 855/1813 [1:33:48<1:43:02,  6.45s/it]
+ 47%|████▋     | 856/1813 [1:33:55<1:42:07,  6.40s/it]
+ 47%|████▋     | 857/1813 [1:34:00<1:37:34,  6.12s/it]
+ 47%|████▋     | 858/1813 [1:34:04<1:26:34,  5.44s/it]
+ 47%|████▋     | 859/1813 [1:34:10<1:29:48,  5.65s/it]
+ 47%|████▋     | 860/1813 [1:34:17<1:33:35,  5.89s/it]
+                                                      
+{'loss': '0.9563', 'grad_norm': '1.009', 'learning_rate': '3.073e-05', 'epoch': '0.4745', 'num_input_tokens_seen': 5766424, 'train_runtime': '5657', 'train_tokens_per_second': '1019'}
+
+ 47%|████▋     | 860/1813 [1:34:17<1:33:35,  5.89s/it]
+ 47%|████▋     | 861/1813 [1:34:22<1:29:00,  5.61s/it]
+ 48%|████▊     | 862/1813 [1:34:28<1:34:18,  5.95s/it]
+ 48%|████▊     | 863/1813 [1:34:34<1:34:48,  5.99s/it]
+ 48%|████▊     | 864/1813 [1:34:39<1:26:10,  5.45s/it]
+ 48%|████▊     | 865/1813 [1:34:45<1:32:34,  5.86s/it]
+ 48%|████▊     | 866/1813 [1:34:51<1:32:21,  5.85s/it]
+ 48%|████▊     | 867/1813 [1:34:57<1:30:20,  5.73s/it]
+ 48%|████▊     | 868/1813 [1:35:02<1:29:39,  5.69s/it]
+ 48%|████▊     | 869/1813 [1:35:10<1:37:14,  6.18s/it]
+ 48%|████▊     | 870/1813 [1:35:17<1:44:10,  6.63s/it]
+                                                      
+{'loss': '0.9456', 'grad_norm': '0.8068', 'learning_rate': '3.027e-05', 'epoch': '0.48', 'num_input_tokens_seen': 5831352, 'train_runtime': '5718', 'train_tokens_per_second': '1020'}
+
+ 48%|████▊     | 870/1813 [1:35:17<1:44:10,  6.63s/it]
+ 48%|████▊     | 871/1813 [1:35:23<1:37:46,  6.23s/it]
+ 48%|████▊     | 872/1813 [1:35:30<1:42:11,  6.52s/it]
+ 48%|████▊     | 873/1813 [1:35:36<1:42:21,  6.53s/it]
+ 48%|████▊     | 874/1813 [1:35:44<1:46:24,  6.80s/it]
+ 48%|████▊     | 875/1813 [1:35:51<1:50:34,  7.07s/it]
+ 48%|████▊     | 876/1813 [1:35:58<1:49:01,  6.98s/it]
+ 48%|████▊     | 877/1813 [1:36:06<1:53:36,  7.28s/it]
+ 48%|████▊     | 878/1813 [1:36:13<1:49:14,  7.01s/it]
+ 48%|████▊     | 879/1813 [1:36:20<1:51:13,  7.14s/it]
+ 49%|████▊     | 880/1813 [1:36:24<1:38:20,  6.32s/it]
+                                                      
+{'loss': '0.9691', 'grad_norm': '1.457', 'learning_rate': '2.981e-05', 'epoch': '0.4855', 'num_input_tokens_seen': 5907208, 'train_runtime': '5785', 'train_tokens_per_second': '1021'}
+
+ 49%|████▊     | 880/1813 [1:36:24<1:38:20,  6.32s/it]
+ 49%|████▊     | 881/1813 [1:36:30<1:33:12,  6.00s/it]
+ 49%|████▊     | 882/1813 [1:36:36<1:33:43,  6.04s/it]
+ 49%|████▊     | 883/1813 [1:36:43<1:36:52,  6.25s/it]
+ 49%|████▉     | 884/1813 [1:36:49<1:35:38,  6.18s/it]
+ 49%|████▉     | 885/1813 [1:36:55<1:34:24,  6.10s/it]
+ 49%|████▉     | 886/1813 [1:37:01<1:34:51,  6.14s/it]
+ 49%|████▉     | 887/1813 [1:37:09<1:42:48,  6.66s/it]
+ 49%|████▉     | 888/1813 [1:37:15<1:42:42,  6.66s/it]
+ 49%|████▉     | 889/1813 [1:37:22<1:44:51,  6.81s/it]
+ 49%|████▉     | 890/1813 [1:37:29<1:41:26,  6.59s/it]
+                                                      
+{'loss': '0.9384', 'grad_norm': '0.8358', 'learning_rate': '2.935e-05', 'epoch': '0.491', 'num_input_tokens_seen': 5978048, 'train_runtime': '5849', 'train_tokens_per_second': '1022'}
+
+ 49%|████▉     | 890/1813 [1:37:29<1:41:26,  6.59s/it]
+ 49%|████▉     | 891/1813 [1:37:34<1:37:56,  6.37s/it]
+ 49%|████▉     | 892/1813 [1:37:42<1:42:11,  6.66s/it]
+ 49%|████▉     | 893/1813 [1:37:48<1:38:59,  6.46s/it]
+ 49%|████▉     | 894/1813 [1:37:53<1:35:03,  6.21s/it]
+ 49%|████▉     | 895/1813 [1:38:01<1:40:02,  6.54s/it]
+ 49%|████▉     | 896/1813 [1:38:06<1:34:16,  6.17s/it]
+ 49%|████▉     | 897/1813 [1:38:12<1:32:10,  6.04s/it]
+ 50%|████▉     | 898/1813 [1:38:19<1:37:29,  6.39s/it]
+ 50%|████▉     | 899/1813 [1:38:26<1:41:19,  6.65s/it]
+ 50%|████▉     | 900/1813 [1:38:34<1:45:47,  6.95s/it]
+                                                      
+{'loss': '0.9989', 'grad_norm': '9.544', 'learning_rate': '2.888e-05', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856, 'train_runtime': '5914', 'train_tokens_per_second': '1023'}
+
+ 50%|████▉     | 900/1813 [1:38:34<1:45:47,  6.95s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:02, 15.83it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.22it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.31it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.07it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.99it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.82it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.04it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.59it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:55,  5.58it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.34it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.98it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.09it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.09it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.32it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.04it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:49,  8.72it/s]
+
+  5%|▌         | 51/1000 [00:07<01:36,  9.88it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.90it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.30it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.20it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  6.99it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:37,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.51it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.35it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.62it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.31it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.44it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.17it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.00it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.50it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.80it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.20it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.90it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.68it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.12it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.62it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.88it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25, 10.06it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.42it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.92it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.05it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.39it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.60it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.17it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.77it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:47,  7.71it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.10it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:15,  6.12it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.19it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:03,  6.69it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:38,  8.39it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.79it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.56it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.74it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.70it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:48,  7.55it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:08,  6.30it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:41,  8.02it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.59it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:05,  6.45it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.36it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.73it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.18it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.65it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.75it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.63it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.91it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.51it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.47it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.66it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.72it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.13it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.49it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:29,  8.77it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.54it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.81it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.22it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.95it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.98it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.36it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.73it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.35it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.68it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.05it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.72it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.75it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.99it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.06it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.77it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.80it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.21it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.28it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.24it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.58it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.16it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:36,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.37it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.03it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.87it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.63it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.54it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.60it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.43it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.59it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.54it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.71it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.19it/s]
+
+ 38%|███▊      | 379/1000 [00:51<01:16,  8.17it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.19it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.56it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.08it/s]
+
+ 39%|███▊      | 387/1000 [00:52<01:20,  7.57it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.31it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.85it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.49it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  9.01it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.53it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.85it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.26it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.44it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.67it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.92it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.35it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.93it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.53it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.83it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.53it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.02it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.80it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.51it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.71it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.46it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.17it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.49it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.80it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.82it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.50it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.04it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.98it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.98it/s]
+
+ 50%|████▉     | 496/1000 [01:06<01:00,  8.39it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.42it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.79it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.57it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.96it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.10it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.06it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.79it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.58it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.72it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.91it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.50it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.75it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.01it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.84it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.03it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.82it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.58it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.38it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:02,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 611/1000 [01:23<00:44,  8.68it/s]
+
+ 61%|██████▏   | 613/1000 [01:23<00:38, 10.03it/s]
+
+ 62%|██████▏   | 615/1000 [01:23<00:45,  8.51it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.59it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.27it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.44it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.65it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.17it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.20it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.46it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.70it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.05it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.94it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.16it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.58it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.09it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.55it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.31it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:45,  7.68it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:39,  8.74it/s]
+
+ 66%|██████▌   | 657/1000 [01:29<00:46,  7.32it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:55,  6.17it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.66it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:51,  6.57it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:57,  5.84it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.38it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:58,  5.64it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.22it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.72it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:57,  5.75it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:50,  6.52it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:45,  7.18it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.57it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.60it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.90it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.90it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.87it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.85it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.15it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.25it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.41it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:48,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:36<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.70it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.37it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.15it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.12it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.61it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.53it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.52it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.78it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.44it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.16it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.19it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.37it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.57it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:33,  7.29it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.68it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.64it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.45it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.77it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.77it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.61it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.98it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.16it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:27,  7.69it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:26,  7.93it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.17it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.85it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.00it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.16it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.96it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.20it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.23it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.04it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.30it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.13it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.81it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:53<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.38it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.59it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.12it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.40it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.76it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.24it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.86it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.22it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14,  9.83it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.07it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.46it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.72it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.57it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.60it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.28it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.23it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.25it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.76it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.62it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.86it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.86it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.59it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.53it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.95it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.16it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.83it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.71it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.07it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.32it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.59it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.78it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.93it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.09it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.62it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.84it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.07it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.38it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.71it/s]
+
+ 95%|█████████▌| 950/1000 [02:08<00:05,  8.84it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.39it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.33it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.19it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.79it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.81it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.52it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.04it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.74it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.7972', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.38', 'eval_steps_per_second': '7.38', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856}
+
+ 50%|████▉     | 900/1813 [1:40:49<1:45:47,  6.95s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+
+                                                   
+ 50%|████▉     | 901/1813 [1:40:56<12:01:15, 47.45s/it]
+ 50%|████▉     | 902/1813 [1:41:02<8:52:10, 35.05s/it] 
+ 50%|████▉     | 903/1813 [1:41:08<6:39:55, 26.37s/it]
+ 50%|████▉     | 904/1813 [1:41:14<5:06:49, 20.25s/it]
+ 50%|████▉     | 905/1813 [1:41:22<4:10:17, 16.54s/it]
+ 50%|████▉     | 906/1813 [1:41:27<3:19:02, 13.17s/it]
+ 50%|█████     | 907/1813 [1:41:32<2:40:04, 10.60s/it]
+ 50%|█████     | 908/1813 [1:41:37<2:17:24,  9.11s/it]
+ 50%|█████     | 909/1813 [1:41:44<2:06:14,  8.38s/it]
+ 50%|█████     | 910/1813 [1:41:50<1:56:53,  7.77s/it]
+                                                      
+{'loss': '0.9311', 'grad_norm': '1.186', 'learning_rate': '2.841e-05', 'epoch': '0.5021', 'num_input_tokens_seen': 6115160, 'train_runtime': '6111', 'train_tokens_per_second': '1001'}
+
+ 50%|█████     | 910/1813 [1:41:50<1:56:53,  7.77s/it]
+ 50%|█████     | 911/1813 [1:41:56<1:47:26,  7.15s/it]
+ 50%|█████     | 912/1813 [1:42:02<1:40:43,  6.71s/it]
+ 50%|█████     | 913/1813 [1:42:07<1:35:00,  6.33s/it]
+ 50%|█████     | 914/1813 [1:42:12<1:29:15,  5.96s/it]
+ 50%|█████     | 915/1813 [1:42:20<1:37:50,  6.54s/it]
+ 51%|█████     | 916/1813 [1:42:27<1:41:00,  6.76s/it]
+ 51%|█████     | 917/1813 [1:42:34<1:41:43,  6.81s/it]
+ 51%|█████     | 918/1813 [1:42:40<1:36:38,  6.48s/it]
+ 51%|█████     | 919/1813 [1:42:46<1:36:06,  6.45s/it]
+ 51%|█████     | 920/1813 [1:42:52<1:32:06,  6.19s/it]
+                                                      
+{'loss': '0.9677', 'grad_norm': '0.802', 'learning_rate': '2.794e-05', 'epoch': '0.5076', 'num_input_tokens_seen': 6178904, 'train_runtime': '6173', 'train_tokens_per_second': '1001'}
+
+ 51%|█████     | 920/1813 [1:42:52<1:32:06,  6.19s/it]
+ 51%|█████     | 921/1813 [1:42:58<1:29:22,  6.01s/it]
+ 51%|█████     | 922/1813 [1:43:03<1:27:31,  5.89s/it]
+ 51%|█████     | 923/1813 [1:43:09<1:27:58,  5.93s/it]
+ 51%|█████     | 924/1813 [1:43:16<1:32:00,  6.21s/it]
+ 51%|█████     | 925/1813 [1:43:22<1:30:01,  6.08s/it]
+ 51%|█████     | 926/1813 [1:43:29<1:32:20,  6.25s/it]
+ 51%|█████     | 927/1813 [1:43:35<1:31:46,  6.21s/it]
+ 51%|█████     | 928/1813 [1:43:41<1:29:48,  6.09s/it]
+ 51%|█████     | 929/1813 [1:43:47<1:30:27,  6.14s/it]
+ 51%|█████▏    | 930/1813 [1:43:54<1:34:09,  6.40s/it]
+                                                      
+{'loss': '0.9298', 'grad_norm': '1.013', 'learning_rate': '2.748e-05', 'epoch': '0.5131', 'num_input_tokens_seen': 6244360, 'train_runtime': '6234', 'train_tokens_per_second': '1002'}
+
+ 51%|█████▏    | 930/1813 [1:43:54<1:34:09,  6.40s/it]
+ 51%|█████▏    | 931/1813 [1:44:01<1:37:06,  6.61s/it]
+ 51%|█████▏    | 932/1813 [1:44:07<1:33:10,  6.35s/it]
+ 51%|█████▏    | 933/1813 [1:44:13<1:31:52,  6.26s/it]
+ 52%|█████▏    | 934/1813 [1:44:20<1:34:25,  6.44s/it]
+ 52%|█████▏    | 935/1813 [1:44:29<1:45:18,  7.20s/it]
+ 52%|█████▏    | 936/1813 [1:44:35<1:41:46,  6.96s/it]
+ 52%|█████▏    | 937/1813 [1:44:41<1:37:52,  6.70s/it]
+ 52%|█████▏    | 938/1813 [1:44:48<1:39:18,  6.81s/it]
+ 52%|█████▏    | 939/1813 [1:44:54<1:34:54,  6.52s/it]
+ 52%|█████▏    | 940/1813 [1:45:01<1:37:05,  6.67s/it]
+                                                      
+{'loss': '0.9125', 'grad_norm': '1.258', 'learning_rate': '2.701e-05', 'epoch': '0.5186', 'num_input_tokens_seen': 6318968, 'train_runtime': '6301', 'train_tokens_per_second': '1003'}
+
+ 52%|█████▏    | 940/1813 [1:45:01<1:37:05,  6.67s/it]
+ 52%|█████▏    | 941/1813 [1:45:06<1:30:05,  6.20s/it]
+ 52%|█████▏    | 942/1813 [1:45:12<1:26:59,  5.99s/it]
+ 52%|█████▏    | 943/1813 [1:45:17<1:24:54,  5.86s/it]
+ 52%|█████▏    | 944/1813 [1:45:23<1:25:15,  5.89s/it]
+ 52%|█████▏    | 945/1813 [1:45:29<1:23:20,  5.76s/it]
+ 52%|█████▏    | 946/1813 [1:45:35<1:24:56,  5.88s/it]
+ 52%|█████▏    | 947/1813 [1:45:41<1:25:57,  5.96s/it]
+ 52%|█████▏    | 948/1813 [1:45:47<1:27:38,  6.08s/it]
+ 52%|█████▏    | 949/1813 [1:45:53<1:24:20,  5.86s/it]
+ 52%|█████▏    | 950/1813 [1:45:59<1:26:41,  6.03s/it]
+                                                      
+{'loss': '0.9627', 'grad_norm': '1.126', 'learning_rate': '2.653e-05', 'epoch': '0.5241', 'num_input_tokens_seen': 6379704, 'train_runtime': '6359', 'train_tokens_per_second': '1003'}
+
+ 52%|█████▏    | 950/1813 [1:45:59<1:26:41,  6.03s/it]
+ 52%|█████▏    | 951/1813 [1:46:05<1:25:29,  5.95s/it]
+ 53%|█████▎    | 952/1813 [1:46:10<1:24:19,  5.88s/it]
+ 53%|█████▎    | 953/1813 [1:46:16<1:24:27,  5.89s/it]
+ 53%|█████▎    | 954/1813 [1:46:22<1:23:25,  5.83s/it]
+ 53%|█████▎    | 955/1813 [1:46:28<1:23:44,  5.86s/it]
+ 53%|█████▎    | 956/1813 [1:46:34<1:23:34,  5.85s/it]
+ 53%|█████▎    | 957/1813 [1:46:39<1:22:37,  5.79s/it]
+ 53%|█████▎    | 958/1813 [1:46:44<1:17:06,  5.41s/it]
+ 53%|█████▎    | 959/1813 [1:46:50<1:19:05,  5.56s/it]
+ 53%|█████▎    | 960/1813 [1:46:58<1:30:43,  6.38s/it]
+                                                      
+{'loss': '0.9726', 'grad_norm': '0.929', 'learning_rate': '2.606e-05', 'epoch': '0.5297', 'num_input_tokens_seen': 6441656, 'train_runtime': '6419', 'train_tokens_per_second': '1004'}
+
+ 53%|█████▎    | 960/1813 [1:46:58<1:30:43,  6.38s/it]
+ 53%|█████▎    | 961/1813 [1:47:05<1:32:00,  6.48s/it]
+ 53%|█████▎    | 962/1813 [1:47:13<1:37:24,  6.87s/it]
+ 53%|█████▎    | 963/1813 [1:47:20<1:37:22,  6.87s/it]
+ 53%|█████▎    | 964/1813 [1:47:26<1:33:44,  6.62s/it]
+ 53%|█████▎    | 965/1813 [1:47:32<1:31:21,  6.46s/it]
+ 53%|█████▎    | 966/1813 [1:47:39<1:36:23,  6.83s/it]
+ 53%|█████▎    | 967/1813 [1:47:47<1:39:09,  7.03s/it]
+ 53%|█████▎    | 968/1813 [1:47:53<1:33:41,  6.65s/it]
+ 53%|█████▎    | 969/1813 [1:47:59<1:31:48,  6.53s/it]
+ 54%|█████▎    | 970/1813 [1:48:04<1:25:24,  6.08s/it]
+                                                      
+{'loss': '1.034', 'grad_norm': '1.225', 'learning_rate': '2.559e-05', 'epoch': '0.5352', 'num_input_tokens_seen': 6512320, 'train_runtime': '6484', 'train_tokens_per_second': '1004'}
+
+ 54%|█████▎    | 970/1813 [1:48:04<1:25:24,  6.08s/it]
+ 54%|█████▎    | 971/1813 [1:48:11<1:28:23,  6.30s/it]
+ 54%|█████▎    | 972/1813 [1:48:17<1:28:18,  6.30s/it]
+ 54%|█████▎    | 973/1813 [1:48:23<1:27:17,  6.24s/it]
+ 54%|█████▎    | 974/1813 [1:48:32<1:36:37,  6.91s/it]
+ 54%|█████▍    | 975/1813 [1:48:38<1:34:22,  6.76s/it]
+ 54%|█████▍    | 976/1813 [1:48:44<1:29:42,  6.43s/it]
+ 54%|█████▍    | 977/1813 [1:48:51<1:35:20,  6.84s/it]
+ 54%|█████▍    | 978/1813 [1:48:58<1:33:36,  6.73s/it]
+ 54%|█████▍    | 979/1813 [1:49:05<1:35:15,  6.85s/it]
+ 54%|█████▍    | 980/1813 [1:49:12<1:34:23,  6.80s/it]
+                                                      
+{'loss': '0.9438', 'grad_norm': '1.97', 'learning_rate': '2.512e-05', 'epoch': '0.5407', 'num_input_tokens_seen': 6586672, 'train_runtime': '6552', 'train_tokens_per_second': '1005'}
+
+ 54%|█████▍    | 980/1813 [1:49:12<1:34:23,  6.80s/it]
+ 54%|█████▍    | 981/1813 [1:49:18<1:32:43,  6.69s/it]
+ 54%|█████▍    | 982/1813 [1:49:23<1:26:25,  6.24s/it]
+ 54%|█████▍    | 983/1813 [1:49:29<1:23:00,  6.00s/it]
+ 54%|█████▍    | 984/1813 [1:49:34<1:20:24,  5.82s/it]
+ 54%|█████▍    | 985/1813 [1:49:41<1:24:09,  6.10s/it]
+ 54%|█████▍    | 986/1813 [1:49:49<1:30:22,  6.56s/it]
+ 54%|█████▍    | 987/1813 [1:49:55<1:29:01,  6.47s/it]
+ 54%|█████▍    | 988/1813 [1:50:01<1:25:48,  6.24s/it]
+ 55%|█████▍    | 989/1813 [1:50:06<1:20:50,  5.89s/it]
+ 55%|█████▍    | 990/1813 [1:50:14<1:29:39,  6.54s/it]
+                                                      
+{'loss': '0.9229', 'grad_norm': '1.036', 'learning_rate': '2.465e-05', 'epoch': '0.5462', 'num_input_tokens_seen': 6653096, 'train_runtime': '6614', 'train_tokens_per_second': '1006'}
+
+ 55%|█████▍    | 990/1813 [1:50:14<1:29:39,  6.54s/it]
+ 55%|█████▍    | 991/1813 [1:50:19<1:24:01,  6.13s/it]
+ 55%|█████▍    | 992/1813 [1:50:26<1:29:06,  6.51s/it]
+ 55%|█████▍    | 993/1813 [1:50:32<1:26:00,  6.29s/it]
+ 55%|█████▍    | 994/1813 [1:50:40<1:34:31,  6.93s/it]
+ 55%|█████▍    | 995/1813 [1:50:46<1:30:25,  6.63s/it]
+ 55%|█████▍    | 996/1813 [1:50:53<1:28:35,  6.51s/it]
+ 55%|█████▍    | 997/1813 [1:50:58<1:24:42,  6.23s/it]
+ 55%|█████▌    | 998/1813 [1:51:03<1:19:20,  5.84s/it]
+ 55%|█████▌    | 999/1813 [1:51:09<1:18:04,  5.76s/it]
+ 55%|█████▌    | 1000/1813 [1:51:14<1:14:36,  5.51s/it]
+                                                       
+{'loss': '1.016', 'grad_norm': '0.8968', 'learning_rate': '2.417e-05', 'epoch': '0.5517', 'num_input_tokens_seen': 6716784, 'train_runtime': '6674', 'train_tokens_per_second': '1006'}
+
+ 55%|█████▌    | 1000/1813 [1:51:14<1:14:36,  5.51s/it]
+ 55%|█████▌    | 1001/1813 [1:51:20<1:18:30,  5.80s/it]
+ 55%|█████▌    | 1002/1813 [1:51:26<1:19:55,  5.91s/it]
+ 55%|█████▌    | 1003/1813 [1:51:33<1:23:31,  6.19s/it]
+ 55%|█████▌    | 1004/1813 [1:51:38<1:19:30,  5.90s/it]
+ 55%|█████▌    | 1005/1813 [1:51:45<1:24:36,  6.28s/it]
+ 55%|█████▌    | 1006/1813 [1:51:52<1:25:19,  6.34s/it]
+ 56%|█████▌    | 1007/1813 [1:51:58<1:22:11,  6.12s/it]
+ 56%|█████▌    | 1008/1813 [1:52:03<1:20:35,  6.01s/it]
+ 56%|█████▌    | 1009/1813 [1:52:09<1:20:34,  6.01s/it]
+ 56%|█████▌    | 1010/1813 [1:52:15<1:18:45,  5.88s/it]
+                                                       
+{'loss': '0.9655', 'grad_norm': '1.3', 'learning_rate': '2.37e-05', 'epoch': '0.5572', 'num_input_tokens_seen': 6783768, 'train_runtime': '6735', 'train_tokens_per_second': '1007'}
+
+ 56%|█████▌    | 1010/1813 [1:52:15<1:18:45,  5.88s/it]
+ 56%|█████▌    | 1011/1813 [1:52:22<1:22:13,  6.15s/it]
+ 56%|█████▌    | 1012/1813 [1:52:26<1:16:32,  5.73s/it]
+ 56%|█████▌    | 1013/1813 [1:52:31<1:10:52,  5.32s/it]
+ 56%|█████▌    | 1014/1813 [1:52:39<1:22:51,  6.22s/it]
+ 56%|█████▌    | 1015/1813 [1:52:44<1:18:25,  5.90s/it]
+ 56%|█████▌    | 1016/1813 [1:52:49<1:15:37,  5.69s/it]
+ 56%|█████▌    | 1017/1813 [1:52:56<1:19:59,  6.03s/it]
+ 56%|█████▌    | 1018/1813 [1:53:03<1:20:49,  6.10s/it]
+ 56%|█████▌    | 1019/1813 [1:53:07<1:15:56,  5.74s/it]
+ 56%|█████▋    | 1020/1813 [1:53:12<1:12:56,  5.52s/it]
+                                                       
+{'loss': '0.8823', 'grad_norm': '1.139', 'learning_rate': '2.323e-05', 'epoch': '0.5628', 'num_input_tokens_seen': 6844992, 'train_runtime': '6793', 'train_tokens_per_second': '1008'}
+
+ 56%|█████▋    | 1020/1813 [1:53:12<1:12:56,  5.52s/it]
+ 56%|█████▋    | 1021/1813 [1:53:19<1:18:21,  5.94s/it]
+ 56%|█████▋    | 1022/1813 [1:53:25<1:18:57,  5.99s/it]
+ 56%|█████▋    | 1023/1813 [1:53:33<1:24:42,  6.43s/it]
+ 56%|█████▋    | 1024/1813 [1:53:39<1:23:39,  6.36s/it]
+ 57%|█████▋    | 1025/1813 [1:53:48<1:34:22,  7.19s/it]
+ 57%|█████▋    | 1026/1813 [1:53:54<1:26:59,  6.63s/it]
+ 57%|█████▋    | 1027/1813 [1:54:00<1:24:47,  6.47s/it]
+ 57%|█████▋    | 1028/1813 [1:54:07<1:28:53,  6.79s/it]
+ 57%|█████▋    | 1029/1813 [1:54:14<1:27:00,  6.66s/it]
+ 57%|█████▋    | 1030/1813 [1:54:20<1:26:53,  6.66s/it]
+                                                       
+{'loss': '1.015', 'grad_norm': '0.7937', 'learning_rate': '2.276e-05', 'epoch': '0.5683', 'num_input_tokens_seen': 6920856, 'train_runtime': '6861', 'train_tokens_per_second': '1009'}
+
+ 57%|█████▋    | 1030/1813 [1:54:20<1:26:53,  6.66s/it]
+ 57%|█████▋    | 1031/1813 [1:54:26<1:22:08,  6.30s/it]
+ 57%|█████▋    | 1032/1813 [1:54:32<1:23:03,  6.38s/it]
+ 57%|█████▋    | 1033/1813 [1:54:38<1:21:41,  6.28s/it]
+ 57%|█████▋    | 1034/1813 [1:54:44<1:18:58,  6.08s/it]
+ 57%|█████▋    | 1035/1813 [1:54:50<1:18:48,  6.08s/it]
+ 57%|█████▋    | 1036/1813 [1:54:56<1:20:16,  6.20s/it]
+ 57%|█████▋    | 1037/1813 [1:55:02<1:18:23,  6.06s/it]
+ 57%|█████▋    | 1038/1813 [1:55:08<1:18:29,  6.08s/it]
+ 57%|█████▋    | 1039/1813 [1:55:15<1:21:43,  6.34s/it]
+ 57%|█████▋    | 1040/1813 [1:55:22<1:23:39,  6.49s/it]
+                                                       
+{'loss': '0.95', 'grad_norm': '0.8464', 'learning_rate': '2.229e-05', 'epoch': '0.5738', 'num_input_tokens_seen': 6988568, 'train_runtime': '6923', 'train_tokens_per_second': '1010'}
+
+ 57%|█████▋    | 1040/1813 [1:55:22<1:23:39,  6.49s/it]
+ 57%|█████▋    | 1041/1813 [1:55:28<1:22:09,  6.39s/it]
+ 57%|█████▋    | 1042/1813 [1:55:35<1:22:33,  6.43s/it]
+ 58%|█████▊    | 1043/1813 [1:55:40<1:19:35,  6.20s/it]
+ 58%|█████▊    | 1044/1813 [1:55:46<1:16:43,  5.99s/it]
+ 58%|█████▊    | 1045/1813 [1:55:52<1:15:46,  5.92s/it]
+ 58%|█████▊    | 1046/1813 [1:55:58<1:15:22,  5.90s/it]
+ 58%|█████▊    | 1047/1813 [1:56:04<1:16:57,  6.03s/it]
+ 58%|█████▊    | 1048/1813 [1:56:11<1:19:53,  6.27s/it]
+ 58%|█████▊    | 1049/1813 [1:56:16<1:16:32,  6.01s/it]
+ 58%|█████▊    | 1050/1813 [1:56:22<1:15:44,  5.96s/it]
+                                                       
+{'loss': '0.9108', 'grad_norm': '1.21', 'learning_rate': '2.182e-05', 'epoch': '0.5793', 'num_input_tokens_seen': 7051808, 'train_runtime': '6982', 'train_tokens_per_second': '1010'}
+
+ 58%|█████▊    | 1050/1813 [1:56:22<1:15:44,  5.96s/it]
+ 58%|█████▊    | 1051/1813 [1:56:27<1:11:50,  5.66s/it]
+ 58%|█████▊    | 1052/1813 [1:56:33<1:11:48,  5.66s/it]
+ 58%|█████▊    | 1053/1813 [1:56:38<1:12:12,  5.70s/it]
+ 58%|█████▊    | 1054/1813 [1:56:44<1:12:08,  5.70s/it]
+ 58%|█████▊    | 1055/1813 [1:56:49<1:10:23,  5.57s/it]
+ 58%|█████▊    | 1056/1813 [1:56:55<1:11:03,  5.63s/it]
+ 58%|█████▊    | 1057/1813 [1:57:02<1:15:10,  5.97s/it]
+ 58%|█████▊    | 1058/1813 [1:57:07<1:11:47,  5.70s/it]
+ 58%|█████▊    | 1059/1813 [1:57:12<1:08:38,  5.46s/it]
+ 58%|█████▊    | 1060/1813 [1:57:19<1:14:37,  5.95s/it]
+                                                       
+{'loss': '0.9343', 'grad_norm': '0.7663', 'learning_rate': '2.135e-05', 'epoch': '0.5848', 'num_input_tokens_seen': 7110624, 'train_runtime': '7039', 'train_tokens_per_second': '1010'}
+
+ 58%|█████▊    | 1060/1813 [1:57:19<1:14:37,  5.95s/it]
+ 59%|█████▊    | 1061/1813 [1:57:24<1:11:52,  5.73s/it]
+ 59%|█████▊    | 1062/1813 [1:57:30<1:10:18,  5.62s/it]
+ 59%|█████▊    | 1063/1813 [1:57:36<1:14:05,  5.93s/it]
+ 59%|█████▊    | 1064/1813 [1:57:44<1:19:47,  6.39s/it]
+ 59%|█████▊    | 1065/1813 [1:57:50<1:20:10,  6.43s/it]
+ 59%|█████▉    | 1066/1813 [1:57:58<1:23:51,  6.74s/it]
+ 59%|█████▉    | 1067/1813 [1:58:04<1:22:52,  6.67s/it]
+ 59%|█████▉    | 1068/1813 [1:58:09<1:16:37,  6.17s/it]
+ 59%|█████▉    | 1069/1813 [1:58:17<1:23:06,  6.70s/it]
+ 59%|█████▉    | 1070/1813 [1:58:23<1:18:55,  6.37s/it]
+                                                       
+{'loss': '0.9316', 'grad_norm': '1.077', 'learning_rate': '2.089e-05', 'epoch': '0.5903', 'num_input_tokens_seen': 7179728, 'train_runtime': '7103', 'train_tokens_per_second': '1011'}
+
+ 59%|█████▉    | 1070/1813 [1:58:23<1:18:55,  6.37s/it]
+ 59%|█████▉    | 1071/1813 [1:58:28<1:14:02,  5.99s/it]
+ 59%|█████▉    | 1072/1813 [1:58:34<1:14:31,  6.03s/it]
+ 59%|█████▉    | 1073/1813 [1:58:40<1:13:09,  5.93s/it]
+ 59%|█████▉    | 1074/1813 [1:58:45<1:11:22,  5.80s/it]
+ 59%|█████▉    | 1075/1813 [1:58:51<1:13:28,  5.97s/it]
+ 59%|█████▉    | 1076/1813 [1:58:58<1:15:04,  6.11s/it]
+ 59%|█████▉    | 1077/1813 [1:59:03<1:09:28,  5.66s/it]
+ 59%|█████▉    | 1078/1813 [1:59:08<1:07:04,  5.48s/it]
+ 60%|█████▉    | 1079/1813 [1:59:14<1:10:38,  5.77s/it]
+ 60%|█████▉    | 1080/1813 [1:59:21<1:13:38,  6.03s/it]
+                                                       
+{'loss': '0.9274', 'grad_norm': '1.06', 'learning_rate': '2.042e-05', 'epoch': '0.5959', 'num_input_tokens_seen': 7240984, 'train_runtime': '7161', 'train_tokens_per_second': '1011'}
+
+ 60%|█████▉    | 1080/1813 [1:59:21<1:13:38,  6.03s/it]
+ 60%|█████▉    | 1081/1813 [1:59:27<1:14:34,  6.11s/it]
+ 60%|█████▉    | 1082/1813 [1:59:33<1:13:35,  6.04s/it]
+ 60%|█████▉    | 1083/1813 [1:59:39<1:14:10,  6.10s/it]
+ 60%|█████▉    | 1084/1813 [1:59:45<1:14:06,  6.10s/it]
+ 60%|█████▉    | 1085/1813 [1:59:51<1:13:45,  6.08s/it]
+ 60%|█████▉    | 1086/1813 [1:59:55<1:06:58,  5.53s/it]
+ 60%|█████▉    | 1087/1813 [2:00:01<1:07:58,  5.62s/it]
+ 60%|██████    | 1088/1813 [2:00:07<1:08:51,  5.70s/it]
+ 60%|██████    | 1089/1813 [2:00:14<1:12:28,  6.01s/it]
+ 60%|██████    | 1090/1813 [2:00:20<1:12:59,  6.06s/it]
+                                                       
+{'loss': '0.8672', 'grad_norm': '1.124', 'learning_rate': '1.996e-05', 'epoch': '0.6014', 'num_input_tokens_seen': 7301824, 'train_runtime': '7221', 'train_tokens_per_second': '1011'}
+
+ 60%|██████    | 1090/1813 [2:00:20<1:12:59,  6.06s/it]
+ 60%|██████    | 1091/1813 [2:00:26<1:13:24,  6.10s/it]
+ 60%|██████    | 1092/1813 [2:00:32<1:12:54,  6.07s/it]
+ 60%|██████    | 1093/1813 [2:00:38<1:10:58,  5.91s/it]
+ 60%|██████    | 1094/1813 [2:00:44<1:13:05,  6.10s/it]
+ 60%|██████    | 1095/1813 [2:00:51<1:13:59,  6.18s/it]
+ 60%|██████    | 1096/1813 [2:00:56<1:12:03,  6.03s/it]
+ 61%|██████    | 1097/1813 [2:01:02<1:12:04,  6.04s/it]
+ 61%|██████    | 1098/1813 [2:01:10<1:15:58,  6.38s/it]
+ 61%|██████    | 1099/1813 [2:01:16<1:15:39,  6.36s/it]
+ 61%|██████    | 1100/1813 [2:01:24<1:20:50,  6.80s/it]
+                                                       
+{'loss': '0.9085', 'grad_norm': '1.134', 'learning_rate': '1.95e-05', 'epoch': '0.6069', 'num_input_tokens_seen': 7369528, 'train_runtime': '7284', 'train_tokens_per_second': '1012'}
+
+ 61%|██████    | 1100/1813 [2:01:24<1:20:50,  6.80s/it]
+ 61%|██████    | 1101/1813 [2:01:30<1:19:12,  6.67s/it]
+ 61%|██████    | 1102/1813 [2:01:37<1:20:09,  6.76s/it]
+ 61%|██████    | 1103/1813 [2:01:43<1:15:26,  6.38s/it]
+ 61%|██████    | 1104/1813 [2:01:50<1:19:12,  6.70s/it]
+ 61%|██████    | 1105/1813 [2:01:55<1:13:07,  6.20s/it]
+ 61%|██████    | 1106/1813 [2:02:01<1:11:33,  6.07s/it]
+ 61%|██████    | 1107/1813 [2:02:09<1:20:19,  6.83s/it]
+ 61%|██████    | 1108/1813 [2:02:15<1:15:49,  6.45s/it]
+ 61%|██████    | 1109/1813 [2:02:22<1:16:03,  6.48s/it]
+ 61%|██████    | 1110/1813 [2:02:29<1:20:14,  6.85s/it]
+                                                       
+{'loss': '0.9508', 'grad_norm': '1.065', 'learning_rate': '1.904e-05', 'epoch': '0.6124', 'num_input_tokens_seen': 7441040, 'train_runtime': '7350', 'train_tokens_per_second': '1012'}
+
+ 61%|██████    | 1110/1813 [2:02:29<1:20:14,  6.85s/it]
+ 61%|██████▏   | 1111/1813 [2:02:34<1:13:10,  6.25s/it]
+ 61%|██████▏   | 1112/1813 [2:02:40<1:11:54,  6.15s/it]
+ 61%|██████▏   | 1113/1813 [2:02:47<1:13:02,  6.26s/it]
+ 61%|██████▏   | 1114/1813 [2:02:54<1:15:54,  6.52s/it]
+ 62%|██████▏   | 1115/1813 [2:03:00<1:16:20,  6.56s/it]
+ 62%|██████▏   | 1116/1813 [2:03:07<1:14:56,  6.45s/it]
+ 62%|██████▏   | 1117/1813 [2:03:12<1:11:06,  6.13s/it]
+ 62%|██████▏   | 1118/1813 [2:03:17<1:06:19,  5.73s/it]
+ 62%|██████▏   | 1119/1813 [2:03:23<1:08:01,  5.88s/it]
+ 62%|██████▏   | 1120/1813 [2:03:29<1:10:10,  6.08s/it]
+                                                       
+{'loss': '0.9087', 'grad_norm': '1.613', 'learning_rate': '1.858e-05', 'epoch': '0.6179', 'num_input_tokens_seen': 7504976, 'train_runtime': '7410', 'train_tokens_per_second': '1013'}
+
+ 62%|██████▏   | 1120/1813 [2:03:29<1:10:10,  6.08s/it]
+ 62%|██████▏   | 1121/1813 [2:03:37<1:16:05,  6.60s/it]
+ 62%|██████▏   | 1122/1813 [2:03:42<1:09:30,  6.04s/it]
+ 62%|██████▏   | 1123/1813 [2:03:50<1:15:05,  6.53s/it]
+ 62%|██████▏   | 1124/1813 [2:03:55<1:11:53,  6.26s/it]
+ 62%|██████▏   | 1125/1813 [2:04:02<1:13:56,  6.45s/it]
+ 62%|██████▏   | 1126/1813 [2:04:08<1:12:59,  6.37s/it]
+ 62%|██████▏   | 1127/1813 [2:04:14<1:09:29,  6.08s/it]
+ 62%|██████▏   | 1128/1813 [2:04:19<1:06:59,  5.87s/it]
+ 62%|██████▏   | 1129/1813 [2:04:25<1:05:28,  5.74s/it]
+ 62%|██████▏   | 1130/1813 [2:04:30<1:03:47,  5.60s/it]
+                                                       
+{'loss': '0.9097', 'grad_norm': '0.8762', 'learning_rate': '1.812e-05', 'epoch': '0.6234', 'num_input_tokens_seen': 7568576, 'train_runtime': '7470', 'train_tokens_per_second': '1013'}
+
+ 62%|██████▏   | 1130/1813 [2:04:30<1:03:47,  5.60s/it]
+ 62%|██████▏   | 1131/1813 [2:04:36<1:04:02,  5.63s/it]
+ 62%|██████▏   | 1132/1813 [2:04:42<1:05:37,  5.78s/it]
+ 62%|██████▏   | 1133/1813 [2:04:48<1:08:30,  6.05s/it]
+ 63%|██████▎   | 1134/1813 [2:04:56<1:15:02,  6.63s/it]
+ 63%|██████▎   | 1135/1813 [2:05:03<1:15:37,  6.69s/it]
+ 63%|██████▎   | 1136/1813 [2:05:10<1:17:03,  6.83s/it]
+ 63%|██████▎   | 1137/1813 [2:05:17<1:15:54,  6.74s/it]
+ 63%|██████▎   | 1138/1813 [2:05:24<1:16:39,  6.81s/it]
+ 63%|██████▎   | 1139/1813 [2:05:29<1:11:47,  6.39s/it]
+ 63%|██████▎   | 1140/1813 [2:05:35<1:08:35,  6.12s/it]
+                                                       
+{'loss': '0.9738', 'grad_norm': '0.9971', 'learning_rate': '1.767e-05', 'epoch': '0.629', 'num_input_tokens_seen': 7639168, 'train_runtime': '7535', 'train_tokens_per_second': '1014'}
+
+ 63%|██████▎   | 1140/1813 [2:05:35<1:08:35,  6.12s/it]
+ 63%|██████▎   | 1141/1813 [2:05:41<1:08:40,  6.13s/it]
+ 63%|██████▎   | 1142/1813 [2:05:47<1:08:12,  6.10s/it]
+ 63%|██████▎   | 1143/1813 [2:05:53<1:08:03,  6.09s/it]
+ 63%|██████▎   | 1144/1813 [2:06:00<1:10:40,  6.34s/it]
+ 63%|██████▎   | 1145/1813 [2:06:07<1:11:35,  6.43s/it]
+ 63%|██████▎   | 1146/1813 [2:06:12<1:08:04,  6.12s/it]
+ 63%|██████▎   | 1147/1813 [2:06:20<1:12:34,  6.54s/it]
+ 63%|██████▎   | 1148/1813 [2:06:26<1:13:52,  6.66s/it]
+ 63%|██████▎   | 1149/1813 [2:06:32<1:10:31,  6.37s/it]
+ 63%|██████▎   | 1150/1813 [2:06:40<1:14:16,  6.72s/it]
+                                                       
+{'loss': '0.9193', 'grad_norm': '0.885', 'learning_rate': '1.722e-05', 'epoch': '0.6345', 'num_input_tokens_seen': 7707832, 'train_runtime': '7600', 'train_tokens_per_second': '1014'}
+
+ 63%|██████▎   | 1150/1813 [2:06:40<1:14:16,  6.72s/it]
+ 63%|██████▎   | 1151/1813 [2:06:44<1:05:43,  5.96s/it]
+ 64%|██████▎   | 1152/1813 [2:06:49<1:04:16,  5.83s/it]
+ 64%|██████▎   | 1153/1813 [2:06:57<1:08:24,  6.22s/it]
+ 64%|██████▎   | 1154/1813 [2:07:03<1:08:00,  6.19s/it]
+ 64%|██████▎   | 1155/1813 [2:07:09<1:09:26,  6.33s/it]
+ 64%|██████▍   | 1156/1813 [2:07:16<1:09:36,  6.36s/it]
+ 64%|██████▍   | 1157/1813 [2:07:23<1:13:41,  6.74s/it]
+ 64%|██████▍   | 1158/1813 [2:07:30<1:14:01,  6.78s/it]
+ 64%|██████▍   | 1159/1813 [2:07:38<1:15:59,  6.97s/it]
+ 64%|██████▍   | 1160/1813 [2:07:44<1:13:41,  6.77s/it]
+                                                       
+{'loss': '0.9417', 'grad_norm': '0.7144', 'learning_rate': '1.677e-05', 'epoch': '0.64', 'num_input_tokens_seen': 7775872, 'train_runtime': '7664', 'train_tokens_per_second': '1015'}
+
+ 64%|██████▍   | 1160/1813 [2:07:44<1:13:41,  6.77s/it]
+ 64%|██████▍   | 1161/1813 [2:07:51<1:15:42,  6.97s/it]
+ 64%|██████▍   | 1162/1813 [2:07:58<1:15:17,  6.94s/it]
+ 64%|██████▍   | 1163/1813 [2:08:05<1:13:47,  6.81s/it]
+ 64%|██████▍   | 1164/1813 [2:08:11<1:12:05,  6.67s/it]
+ 64%|██████▍   | 1165/1813 [2:08:18<1:11:44,  6.64s/it]
+ 64%|██████▍   | 1166/1813 [2:08:24<1:10:02,  6.50s/it]
+ 64%|██████▍   | 1167/1813 [2:08:31<1:12:10,  6.70s/it]
+ 64%|██████▍   | 1168/1813 [2:08:38<1:11:59,  6.70s/it]
+ 64%|██████▍   | 1169/1813 [2:08:44<1:10:42,  6.59s/it]
+ 65%|██████▍   | 1170/1813 [2:08:50<1:07:57,  6.34s/it]
+                                                       
+{'loss': '0.9299', 'grad_norm': '0.9908', 'learning_rate': '1.633e-05', 'epoch': '0.6455', 'num_input_tokens_seen': 7846688, 'train_runtime': '7730', 'train_tokens_per_second': '1015'}
+
+ 65%|██████▍   | 1170/1813 [2:08:50<1:07:57,  6.34s/it]
+ 65%|██████▍   | 1171/1813 [2:08:57<1:11:05,  6.64s/it]
+ 65%|██████▍   | 1172/1813 [2:09:04<1:12:28,  6.78s/it]
+ 65%|██████▍   | 1173/1813 [2:09:10<1:08:15,  6.40s/it]
+ 65%|██████▍   | 1174/1813 [2:09:15<1:04:19,  6.04s/it]
+ 65%|██████▍   | 1175/1813 [2:09:21<1:03:43,  5.99s/it]
+ 65%|██████▍   | 1176/1813 [2:09:27<1:04:28,  6.07s/it]
+ 65%|██████▍   | 1177/1813 [2:09:33<1:03:35,  6.00s/it]
+ 65%|██████▍   | 1178/1813 [2:09:39<1:03:22,  5.99s/it]
+ 65%|██████▌   | 1179/1813 [2:09:45<1:03:49,  6.04s/it]
+ 65%|██████▌   | 1180/1813 [2:09:52<1:06:02,  6.26s/it]
+                                                       
+{'loss': '0.9153', 'grad_norm': '1.007', 'learning_rate': '1.589e-05', 'epoch': '0.651', 'num_input_tokens_seen': 7914072, 'train_runtime': '7792', 'train_tokens_per_second': '1016'}
+
+ 65%|██████▌   | 1180/1813 [2:09:52<1:06:02,  6.26s/it]
+ 65%|██████▌   | 1181/1813 [2:10:00<1:10:45,  6.72s/it]
+ 65%|██████▌   | 1182/1813 [2:10:07<1:13:48,  7.02s/it]
+ 65%|██████▌   | 1183/1813 [2:10:14<1:13:58,  7.05s/it]
+ 65%|██████▌   | 1184/1813 [2:10:21<1:10:43,  6.75s/it]
+ 65%|██████▌   | 1185/1813 [2:10:28<1:13:54,  7.06s/it]
+ 65%|██████▌   | 1186/1813 [2:10:36<1:15:10,  7.19s/it]
+ 65%|██████▌   | 1187/1813 [2:10:43<1:14:25,  7.13s/it]
+ 66%|██████▌   | 1188/1813 [2:10:48<1:08:39,  6.59s/it]
+ 66%|██████▌   | 1189/1813 [2:10:53<1:04:03,  6.16s/it]
+ 66%|██████▌   | 1190/1813 [2:11:00<1:04:38,  6.23s/it]
+                                                       
+{'loss': '0.965', 'grad_norm': '1.112', 'learning_rate': '1.545e-05', 'epoch': '0.6566', 'num_input_tokens_seen': 7989064, 'train_runtime': '7860', 'train_tokens_per_second': '1016'}
+
+ 66%|██████▌   | 1190/1813 [2:11:00<1:04:38,  6.23s/it]
+ 66%|██████▌   | 1191/1813 [2:11:05<1:02:54,  6.07s/it]
+ 66%|██████▌   | 1192/1813 [2:11:12<1:04:13,  6.20s/it]
+ 66%|██████▌   | 1193/1813 [2:11:18<1:03:24,  6.14s/it]
+ 66%|██████▌   | 1194/1813 [2:11:24<1:04:27,  6.25s/it]
+ 66%|██████▌   | 1195/1813 [2:11:29<1:00:06,  5.84s/it]
+ 66%|██████▌   | 1196/1813 [2:11:36<1:02:10,  6.05s/it]
+ 66%|██████▌   | 1197/1813 [2:11:42<1:03:14,  6.16s/it]
+ 66%|██████▌   | 1198/1813 [2:11:48<1:02:49,  6.13s/it]
+ 66%|██████▌   | 1199/1813 [2:11:54<1:02:13,  6.08s/it]
+ 66%|██████▌   | 1200/1813 [2:12:02<1:06:03,  6.46s/it]
+                                                       
+{'loss': '0.8738', 'grad_norm': '0.9894', 'learning_rate': '1.501e-05', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352, 'train_runtime': '7922', 'train_tokens_per_second': '1017'}
+
+ 66%|██████▌   | 1200/1813 [2:12:02<1:06:03,  6.46s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.73it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.23it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.07it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  7.00it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.81it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.95it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.71it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.04it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.35it/s]
+
+  3%|▎         | 26/1000 [00:04<02:36,  6.24it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 29/1000 [00:04<02:33,  6.34it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.33it/s]
+
+  3%|▎         | 32/1000 [00:05<02:42,  5.98it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.74it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.32it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.04it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:49,  8.72it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.89it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.91it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.29it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.98it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.86it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.21it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:15,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.16it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.83it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.30it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.43it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.18it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.01it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.49it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.78it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.20it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.07it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.69it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.14it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.89it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.07it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.48it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.91it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.03it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.38it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.95it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.59it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.79it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.66it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.17it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.77it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.83it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.15it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.51it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.13it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.47it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:45,  7.61it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:28,  9.07it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.54it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.70it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.59it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.87it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.47it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.66it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.72it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.14it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.49it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.54it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.36it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.96it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.70it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:55,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:44,  7.01it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.68it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.04it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.84it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.76it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.13it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.77it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.13it/s]
+
+ 31%|███       | 306/1000 [00:41<01:40,  6.93it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.77it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.77it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.18it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.25it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.23it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.57it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.02it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.87it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.59it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.20it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.39it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.90it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:25,  7.62it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.53it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.57it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.41it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:58,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.54it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.21it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:30,  6.91it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.76it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.90it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.19it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.17it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.13it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.64it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.37it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.79it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.46it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.74it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.50it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.68it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.92it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.53it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.83it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.55it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.02it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.80it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.52it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.80it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:50, 10.75it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.60it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.73it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:29,  6.00it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.30it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.20it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.37it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:25,  6.18it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.19it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.29it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.08it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.50it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.04it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.29it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.98it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.61it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.42it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.53it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.78it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.55it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.97it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.06it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.58it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.70it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:20,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.52it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.17it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.94it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.82it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.02it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.35it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:06,  6.54it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.43it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.81it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.72it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.24it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.96it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.54it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.57it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.74it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.40it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.68it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.72it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.78it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.61it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.74it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.08it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.02it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.20it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.84it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.42it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.94it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.16it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.95it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.57it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.33it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.90it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.11it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.71it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 11.00it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.92it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.90it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:46,  6.71it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.75it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.21it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.37it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:48,  6.29it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.91it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.03it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.58it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.60it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.20it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.49it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.69it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.84it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.29it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.02it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.78it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.91it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.45it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.15it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:27, 10.11it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.57it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.31it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.50it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.87it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.51it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.76it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.60it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.43it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.67it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.39it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.59it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:33,  7.30it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.67it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.63it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.45it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.77it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.77it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.88it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.60it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.94it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.99it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.98it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.20it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.73it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.84it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.16it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.95it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.21it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.24it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.04it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.29it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.12it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.19it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.99it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.46it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.86it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.46it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.04it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.51it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.75it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.62it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.68it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.30it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.25it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.27it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.95it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.83it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.69it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.38it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.09it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.13it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.33it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.07it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.38it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.70it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.41it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.00it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.17it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.78it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.59it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.35it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.02it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.41it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.90it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.35it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.14it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+                                                       
+
+                                                   
+{'eval_loss': '0.7821', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.383', 'eval_steps_per_second': '7.383', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352}
+
+ 66%|██████▌   | 1200/1813 [2:14:17<1:06:03,  6.46s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+
+                                                   
+ 66%|██████▌   | 1201/1813 [2:14:22<7:56:50, 46.75s/it]
+ 66%|██████▋   | 1202/1813 [2:14:29<5:53:21, 34.70s/it]
+ 66%|██████▋   | 1203/1813 [2:14:35<4:26:22, 26.20s/it]
+ 66%|██████▋   | 1204/1813 [2:14:42<3:26:06, 20.31s/it]
+ 66%|██████▋   | 1205/1813 [2:14:48<2:43:45, 16.16s/it]
+ 67%|██████▋   | 1206/1813 [2:14:55<2:13:15, 13.17s/it]
+ 67%|██████▋   | 1207/1813 [2:15:01<1:52:29, 11.14s/it]
+ 67%|██████▋   | 1208/1813 [2:15:08<1:41:08, 10.03s/it]
+ 67%|██████▋   | 1209/1813 [2:15:15<1:30:59,  9.04s/it]
+ 67%|██████▋   | 1210/1813 [2:15:21<1:21:07,  8.07s/it]
+                                                       
+{'loss': '0.9444', 'grad_norm': '1.167', 'learning_rate': '1.458e-05', 'epoch': '0.6676', 'num_input_tokens_seen': 8125224, 'train_runtime': '8121', 'train_tokens_per_second': '1000'}
+
+ 67%|██████▋   | 1210/1813 [2:15:21<1:21:07,  8.07s/it]
+ 67%|██████▋   | 1211/1813 [2:15:28<1:17:22,  7.71s/it]
+ 67%|██████▋   | 1212/1813 [2:15:32<1:07:40,  6.76s/it]
+ 67%|██████▋   | 1213/1813 [2:15:39<1:06:52,  6.69s/it]
+ 67%|██████▋   | 1214/1813 [2:15:47<1:12:04,  7.22s/it]
+ 67%|██████▋   | 1215/1813 [2:15:54<1:09:00,  6.92s/it]
+ 67%|██████▋   | 1216/1813 [2:16:01<1:09:16,  6.96s/it]
+ 67%|██████▋   | 1217/1813 [2:16:07<1:06:57,  6.74s/it]
+ 67%|██████▋   | 1218/1813 [2:16:13<1:05:11,  6.57s/it]
+ 67%|██████▋   | 1219/1813 [2:16:19<1:02:13,  6.29s/it]
+ 67%|██████▋   | 1220/1813 [2:16:27<1:07:35,  6.84s/it]
+                                                       
+{'loss': '0.9811', 'grad_norm': '8.715', 'learning_rate': '1.416e-05', 'epoch': '0.6731', 'num_input_tokens_seen': 8197488, 'train_runtime': '8187', 'train_tokens_per_second': '1001'}
+
+ 67%|██████▋   | 1220/1813 [2:16:27<1:07:35,  6.84s/it]
+ 67%|██████▋   | 1221/1813 [2:16:32<1:03:42,  6.46s/it]
+ 67%|██████▋   | 1222/1813 [2:16:37<59:37,  6.05s/it]  
+ 67%|██████▋   | 1223/1813 [2:16:43<57:49,  5.88s/it]
+ 68%|██████▊   | 1224/1813 [2:16:48<54:58,  5.60s/it]
+ 68%|██████▊   | 1225/1813 [2:16:54<56:09,  5.73s/it]
+ 68%|██████▊   | 1226/1813 [2:17:00<57:27,  5.87s/it]
+ 68%|██████▊   | 1227/1813 [2:17:07<59:43,  6.12s/it]
+ 68%|██████▊   | 1228/1813 [2:17:12<57:06,  5.86s/it]
+ 68%|██████▊   | 1229/1813 [2:17:18<58:33,  6.02s/it]
+ 68%|██████▊   | 1230/1813 [2:17:24<58:30,  6.02s/it]
+                                                     
+{'loss': '0.8382', 'grad_norm': '0.898', 'learning_rate': '1.373e-05', 'epoch': '0.6786', 'num_input_tokens_seen': 8257288, 'train_runtime': '8245', 'train_tokens_per_second': '1001'}
+
+ 68%|██████▊   | 1230/1813 [2:17:24<58:30,  6.02s/it]
+ 68%|██████▊   | 1231/1813 [2:17:32<1:02:56,  6.49s/it]
+ 68%|██████▊   | 1232/1813 [2:17:40<1:05:43,  6.79s/it]
+ 68%|██████▊   | 1233/1813 [2:17:46<1:04:24,  6.66s/it]
+ 68%|██████▊   | 1234/1813 [2:17:52<1:02:03,  6.43s/it]
+ 68%|██████▊   | 1235/1813 [2:17:58<1:00:59,  6.33s/it]
+ 68%|██████▊   | 1236/1813 [2:18:06<1:07:20,  7.00s/it]
+ 68%|██████▊   | 1237/1813 [2:18:13<1:05:08,  6.79s/it]
+ 68%|██████▊   | 1238/1813 [2:18:18<1:00:59,  6.36s/it]
+ 68%|██████▊   | 1239/1813 [2:18:23<57:18,  5.99s/it]  
+ 68%|██████▊   | 1240/1813 [2:18:28<54:38,  5.72s/it]
+                                                     
+{'loss': '0.846', 'grad_norm': '0.939', 'learning_rate': '1.331e-05', 'epoch': '0.6841', 'num_input_tokens_seen': 8327600, 'train_runtime': '8309', 'train_tokens_per_second': '1002'}
+
+ 68%|██████▊   | 1240/1813 [2:18:28<54:38,  5.72s/it]
+ 68%|██████▊   | 1241/1813 [2:18:34<53:53,  5.65s/it]
+ 69%|██████▊   | 1242/1813 [2:18:40<54:18,  5.71s/it]
+ 69%|██████▊   | 1243/1813 [2:18:46<54:52,  5.78s/it]
+ 69%|██████▊   | 1244/1813 [2:18:52<57:39,  6.08s/it]
+ 69%|██████▊   | 1245/1813 [2:19:00<1:00:40,  6.41s/it]
+ 69%|██████▊   | 1246/1813 [2:19:05<58:47,  6.22s/it]  
+ 69%|██████▉   | 1247/1813 [2:19:12<1:00:47,  6.44s/it]
+ 69%|██████▉   | 1248/1813 [2:19:19<1:02:27,  6.63s/it]
+ 69%|██████▉   | 1249/1813 [2:19:26<1:01:51,  6.58s/it]
+ 69%|██████▉   | 1250/1813 [2:19:31<56:32,  6.03s/it]  
+                                                     
+{'loss': '0.981', 'grad_norm': '1.238', 'learning_rate': '1.29e-05', 'epoch': '0.6897', 'num_input_tokens_seen': 8394824, 'train_runtime': '8371', 'train_tokens_per_second': '1003'}
+
+ 69%|██████▉   | 1250/1813 [2:19:31<56:32,  6.03s/it]
+ 69%|██████▉   | 1251/1813 [2:19:37<56:15,  6.01s/it]
+ 69%|██████▉   | 1252/1813 [2:19:42<55:01,  5.88s/it]
+ 69%|██████▉   | 1253/1813 [2:19:48<54:53,  5.88s/it]
+ 69%|██████▉   | 1254/1813 [2:19:55<56:36,  6.08s/it]
+ 69%|██████▉   | 1255/1813 [2:20:01<56:45,  6.10s/it]
+ 69%|██████▉   | 1256/1813 [2:20:07<56:56,  6.13s/it]
+ 69%|██████▉   | 1257/1813 [2:20:13<55:49,  6.02s/it]
+ 69%|██████▉   | 1258/1813 [2:20:20<58:00,  6.27s/it]
+ 69%|██████▉   | 1259/1813 [2:20:26<58:47,  6.37s/it]
+ 69%|██████▉   | 1260/1813 [2:20:33<1:00:59,  6.62s/it]
+                                                       
+{'loss': '0.9285', 'grad_norm': '1.281', 'learning_rate': '1.249e-05', 'epoch': '0.6952', 'num_input_tokens_seen': 8461096, 'train_runtime': '8434', 'train_tokens_per_second': '1003'}
+
+ 69%|██████▉   | 1260/1813 [2:20:33<1:00:59,  6.62s/it]
+ 70%|██████▉   | 1261/1813 [2:20:41<1:05:05,  7.07s/it]
+ 70%|██████▉   | 1262/1813 [2:20:49<1:06:22,  7.23s/it]
+ 70%|██████▉   | 1263/1813 [2:20:56<1:06:44,  7.28s/it]
+ 70%|██████▉   | 1264/1813 [2:21:02<1:02:58,  6.88s/it]
+ 70%|██████▉   | 1265/1813 [2:21:08<58:16,  6.38s/it]  
+ 70%|██████▉   | 1266/1813 [2:21:15<1:01:32,  6.75s/it]
+ 70%|██████▉   | 1267/1813 [2:21:21<57:39,  6.34s/it]  
+ 70%|██████▉   | 1268/1813 [2:21:26<54:14,  5.97s/it]
+ 70%|██████▉   | 1269/1813 [2:21:31<52:22,  5.78s/it]
+ 70%|███████   | 1270/1813 [2:21:37<53:32,  5.92s/it]
+                                                     
+{'loss': '0.9328', 'grad_norm': '1.17', 'learning_rate': '1.208e-05', 'epoch': '0.7007', 'num_input_tokens_seen': 8529224, 'train_runtime': '8498', 'train_tokens_per_second': '1004'}
+
+ 70%|███████   | 1270/1813 [2:21:37<53:32,  5.92s/it]
+ 70%|███████   | 1271/1813 [2:21:42<50:56,  5.64s/it]
+ 70%|███████   | 1272/1813 [2:21:49<54:53,  6.09s/it]
+ 70%|███████   | 1273/1813 [2:21:56<57:03,  6.34s/it]
+ 70%|███████   | 1274/1813 [2:22:04<59:26,  6.62s/it]
+ 70%|███████   | 1275/1813 [2:22:10<59:33,  6.64s/it]
+ 70%|███████   | 1276/1813 [2:22:16<56:33,  6.32s/it]
+ 70%|███████   | 1277/1813 [2:22:23<58:20,  6.53s/it]
+ 70%|███████   | 1278/1813 [2:22:28<55:17,  6.20s/it]
+ 71%|███████   | 1279/1813 [2:22:33<51:07,  5.74s/it]
+ 71%|███████   | 1280/1813 [2:22:40<54:09,  6.10s/it]
+                                                     
+{'loss': '0.959', 'grad_norm': '1.066', 'learning_rate': '1.168e-05', 'epoch': '0.7062', 'num_input_tokens_seen': 8596456, 'train_runtime': '8560', 'train_tokens_per_second': '1004'}
+
+ 71%|███████   | 1280/1813 [2:22:40<54:09,  6.10s/it]
+ 71%|███████   | 1281/1813 [2:22:45<52:01,  5.87s/it]
+ 71%|███████   | 1282/1813 [2:22:52<53:28,  6.04s/it]
+ 71%|███████   | 1283/1813 [2:22:58<55:20,  6.26s/it]
+ 71%|███████   | 1284/1813 [2:23:04<54:05,  6.14s/it]
+ 71%|███████   | 1285/1813 [2:23:10<52:05,  5.92s/it]
+ 71%|███████   | 1286/1813 [2:23:17<55:28,  6.32s/it]
+ 71%|███████   | 1287/1813 [2:23:23<54:22,  6.20s/it]
+ 71%|███████   | 1288/1813 [2:23:28<51:37,  5.90s/it]
+ 71%|███████   | 1289/1813 [2:23:35<53:36,  6.14s/it]
+ 71%|███████   | 1290/1813 [2:23:42<55:58,  6.42s/it]
+                                                     
+{'loss': '1.005', 'grad_norm': '0.7443', 'learning_rate': '1.128e-05', 'epoch': '0.7117', 'num_input_tokens_seen': 8662472, 'train_runtime': '8622', 'train_tokens_per_second': '1005'}
+
+ 71%|███████   | 1290/1813 [2:23:42<55:58,  6.42s/it]
+ 71%|███████   | 1291/1813 [2:23:48<55:25,  6.37s/it]
+ 71%|███████▏  | 1292/1813 [2:23:55<56:54,  6.55s/it]
+ 71%|███████▏  | 1293/1813 [2:24:02<58:35,  6.76s/it]
+ 71%|███████▏  | 1294/1813 [2:24:09<57:07,  6.60s/it]
+ 71%|███████▏  | 1295/1813 [2:24:14<54:50,  6.35s/it]
+ 71%|███████▏  | 1296/1813 [2:24:20<51:51,  6.02s/it]
+ 72%|███████▏  | 1297/1813 [2:24:25<50:30,  5.87s/it]
+ 72%|███████▏  | 1298/1813 [2:24:32<52:39,  6.13s/it]
+ 72%|███████▏  | 1299/1813 [2:24:39<54:53,  6.41s/it]
+ 72%|███████▏  | 1300/1813 [2:24:45<54:27,  6.37s/it]
+                                                     
+{'loss': '0.899', 'grad_norm': '1.331', 'learning_rate': '1.089e-05', 'epoch': '0.7172', 'num_input_tokens_seen': 8730600, 'train_runtime': '8686', 'train_tokens_per_second': '1005'}
+
+ 72%|███████▏  | 1300/1813 [2:24:45<54:27,  6.37s/it]
+ 72%|███████▏  | 1301/1813 [2:24:51<53:19,  6.25s/it]
+ 72%|███████▏  | 1302/1813 [2:24:58<54:45,  6.43s/it]
+ 72%|███████▏  | 1303/1813 [2:25:06<58:14,  6.85s/it]
+ 72%|███████▏  | 1304/1813 [2:25:12<55:12,  6.51s/it]
+ 72%|███████▏  | 1305/1813 [2:25:19<57:38,  6.81s/it]
+ 72%|███████▏  | 1306/1813 [2:25:25<55:08,  6.53s/it]
+ 72%|███████▏  | 1307/1813 [2:25:31<54:54,  6.51s/it]
+ 72%|███████▏  | 1308/1813 [2:25:38<55:20,  6.58s/it]
+ 72%|███████▏  | 1309/1813 [2:25:44<54:39,  6.51s/it]
+ 72%|███████▏  | 1310/1813 [2:25:52<56:07,  6.69s/it]
+                                                     
+{'loss': '0.9832', 'grad_norm': '0.9035', 'learning_rate': '1.05e-05', 'epoch': '0.7228', 'num_input_tokens_seen': 8802000, 'train_runtime': '8752', 'train_tokens_per_second': '1006'}
+
+ 72%|███████▏  | 1310/1813 [2:25:52<56:07,  6.69s/it]
+ 72%|███████▏  | 1311/1813 [2:25:58<55:12,  6.60s/it]
+ 72%|███████▏  | 1312/1813 [2:26:04<53:04,  6.36s/it]
+ 72%|███████▏  | 1313/1813 [2:26:10<52:23,  6.29s/it]
+ 72%|███████▏  | 1314/1813 [2:26:16<52:48,  6.35s/it]
+ 73%|███████▎  | 1315/1813 [2:26:22<51:20,  6.19s/it]
+ 73%|███████▎  | 1316/1813 [2:26:28<50:53,  6.14s/it]
+ 73%|███████▎  | 1317/1813 [2:26:35<51:15,  6.20s/it]
+ 73%|███████▎  | 1318/1813 [2:26:40<49:52,  6.04s/it]
+ 73%|███████▎  | 1319/1813 [2:26:47<51:11,  6.22s/it]
+ 73%|███████▎  | 1320/1813 [2:26:54<52:26,  6.38s/it]
+                                                     
+{'loss': '0.9757', 'grad_norm': '0.9107', 'learning_rate': '1.012e-05', 'epoch': '0.7283', 'num_input_tokens_seen': 8866824, 'train_runtime': '8814', 'train_tokens_per_second': '1006'}
+
+ 73%|███████▎  | 1320/1813 [2:26:54<52:26,  6.38s/it]
+ 73%|███████▎  | 1321/1813 [2:27:00<53:16,  6.50s/it]
+ 73%|███████▎  | 1322/1813 [2:27:07<52:30,  6.42s/it]
+ 73%|███████▎  | 1323/1813 [2:27:13<53:12,  6.52s/it]
+ 73%|███████▎  | 1324/1813 [2:27:19<51:20,  6.30s/it]
+ 73%|███████▎  | 1325/1813 [2:27:25<49:48,  6.12s/it]
+ 73%|███████▎  | 1326/1813 [2:27:31<49:57,  6.16s/it]
+ 73%|███████▎  | 1327/1813 [2:27:39<53:30,  6.61s/it]
+ 73%|███████▎  | 1328/1813 [2:27:46<53:47,  6.65s/it]
+ 73%|███████▎  | 1329/1813 [2:27:52<52:08,  6.46s/it]
+ 73%|███████▎  | 1330/1813 [2:27:57<49:46,  6.18s/it]
+                                                     
+{'loss': '0.9145', 'grad_norm': '1.141', 'learning_rate': '9.742e-06', 'epoch': '0.7338', 'num_input_tokens_seen': 8935584, 'train_runtime': '8878', 'train_tokens_per_second': '1007'}
+
+ 73%|███████▎  | 1330/1813 [2:27:57<49:46,  6.18s/it]
+ 73%|███████▎  | 1331/1813 [2:28:03<48:35,  6.05s/it]
+ 73%|███████▎  | 1332/1813 [2:28:09<48:17,  6.02s/it]
+ 74%|███████▎  | 1333/1813 [2:28:14<47:22,  5.92s/it]
+ 74%|███████▎  | 1334/1813 [2:28:20<45:59,  5.76s/it]
+ 74%|███████▎  | 1335/1813 [2:28:25<44:52,  5.63s/it]
+ 74%|███████▎  | 1336/1813 [2:28:31<45:59,  5.79s/it]
+ 74%|███████▎  | 1337/1813 [2:28:39<49:49,  6.28s/it]
+ 74%|███████▍  | 1338/1813 [2:28:46<51:45,  6.54s/it]
+ 74%|███████▍  | 1339/1813 [2:28:51<48:07,  6.09s/it]
+ 74%|███████▍  | 1340/1813 [2:28:58<50:15,  6.38s/it]
+                                                     
+{'loss': '0.9998', 'grad_norm': '0.933', 'learning_rate': '9.371e-06', 'epoch': '0.7393', 'num_input_tokens_seen': 9000416, 'train_runtime': '8939', 'train_tokens_per_second': '1007'}
+
+ 74%|███████▍  | 1340/1813 [2:28:58<50:15,  6.38s/it]
+ 74%|███████▍  | 1341/1813 [2:29:05<50:40,  6.44s/it]
+ 74%|███████▍  | 1342/1813 [2:29:10<47:41,  6.07s/it]
+ 74%|███████▍  | 1343/1813 [2:29:15<46:11,  5.90s/it]
+ 74%|███████▍  | 1344/1813 [2:29:21<46:20,  5.93s/it]
+ 74%|███████▍  | 1345/1813 [2:29:29<50:29,  6.47s/it]
+ 74%|███████▍  | 1346/1813 [2:29:36<50:47,  6.53s/it]
+ 74%|███████▍  | 1347/1813 [2:29:42<49:31,  6.38s/it]
+ 74%|███████▍  | 1348/1813 [2:29:48<49:33,  6.39s/it]
+ 74%|███████▍  | 1349/1813 [2:29:55<50:23,  6.52s/it]
+ 74%|███████▍  | 1350/1813 [2:30:01<48:44,  6.32s/it]
+                                                     
+{'loss': '0.9551', 'grad_norm': '1.223', 'learning_rate': '9.005e-06', 'epoch': '0.7448', 'num_input_tokens_seen': 9066864, 'train_runtime': '9001', 'train_tokens_per_second': '1007'}
+
+ 74%|███████▍  | 1350/1813 [2:30:01<48:44,  6.32s/it]
+ 75%|███████▍  | 1351/1813 [2:30:09<52:36,  6.83s/it]
+ 75%|███████▍  | 1352/1813 [2:30:15<51:26,  6.69s/it]
+ 75%|███████▍  | 1353/1813 [2:30:22<51:35,  6.73s/it]
+ 75%|███████▍  | 1354/1813 [2:30:29<51:37,  6.75s/it]
+ 75%|███████▍  | 1355/1813 [2:30:37<55:29,  7.27s/it]
+ 75%|███████▍  | 1356/1813 [2:30:43<51:49,  6.80s/it]
+ 75%|███████▍  | 1357/1813 [2:30:49<49:52,  6.56s/it]
+ 75%|███████▍  | 1358/1813 [2:30:55<49:26,  6.52s/it]
+ 75%|███████▍  | 1359/1813 [2:31:01<47:50,  6.32s/it]
+ 75%|███████▌  | 1360/1813 [2:31:07<47:17,  6.26s/it]
+                                                     
+{'loss': '0.9284', 'grad_norm': '1.376', 'learning_rate': '8.645e-06', 'epoch': '0.7503', 'num_input_tokens_seen': 9140520, 'train_runtime': '9068', 'train_tokens_per_second': '1008'}
+
+ 75%|███████▌  | 1360/1813 [2:31:07<47:17,  6.26s/it]
+ 75%|███████▌  | 1361/1813 [2:31:14<48:32,  6.44s/it]
+ 75%|███████▌  | 1362/1813 [2:31:21<48:24,  6.44s/it]
+ 75%|███████▌  | 1363/1813 [2:31:28<50:24,  6.72s/it]
+ 75%|███████▌  | 1364/1813 [2:31:34<48:36,  6.50s/it]
+ 75%|███████▌  | 1365/1813 [2:31:42<50:45,  6.80s/it]
+ 75%|███████▌  | 1366/1813 [2:31:48<48:54,  6.56s/it]
+ 75%|███████▌  | 1367/1813 [2:31:54<48:47,  6.56s/it]
+ 75%|███████▌  | 1368/1813 [2:31:59<45:15,  6.10s/it]
+ 76%|███████▌  | 1369/1813 [2:32:07<49:32,  6.69s/it]
+ 76%|███████▌  | 1370/1813 [2:32:13<47:55,  6.49s/it]
+                                                     
+{'loss': '0.9155', 'grad_norm': '1.081', 'learning_rate': '8.29e-06', 'epoch': '0.7559', 'num_input_tokens_seen': 9213048, 'train_runtime': '9134', 'train_tokens_per_second': '1009'}
+
+ 76%|███████▌  | 1370/1813 [2:32:13<47:55,  6.49s/it]
+ 76%|███████▌  | 1371/1813 [2:32:21<51:10,  6.95s/it]
+ 76%|███████▌  | 1372/1813 [2:32:27<47:38,  6.48s/it]
+ 76%|███████▌  | 1373/1813 [2:32:33<47:16,  6.45s/it]
+ 76%|███████▌  | 1374/1813 [2:32:39<46:25,  6.34s/it]
+ 76%|███████▌  | 1375/1813 [2:32:45<45:20,  6.21s/it]
+ 76%|███████▌  | 1376/1813 [2:32:51<43:46,  6.01s/it]
+ 76%|███████▌  | 1377/1813 [2:32:57<43:54,  6.04s/it]
+ 76%|███████▌  | 1378/1813 [2:33:02<42:47,  5.90s/it]
+ 76%|███████▌  | 1379/1813 [2:33:08<42:59,  5.94s/it]
+ 76%|███████▌  | 1380/1813 [2:33:15<45:22,  6.29s/it]
+                                                     
+{'loss': '0.9053', 'grad_norm': '1.157', 'learning_rate': '7.942e-06', 'epoch': '0.7614', 'num_input_tokens_seen': 9279088, 'train_runtime': '9196', 'train_tokens_per_second': '1009'}
+
+ 76%|███████▌  | 1380/1813 [2:33:15<45:22,  6.29s/it]
+ 76%|███████▌  | 1381/1813 [2:33:22<46:54,  6.52s/it]
+ 76%|███████▌  | 1382/1813 [2:33:28<44:44,  6.23s/it]
+ 76%|███████▋  | 1383/1813 [2:33:34<44:23,  6.19s/it]
+ 76%|███████▋  | 1384/1813 [2:33:40<43:33,  6.09s/it]
+ 76%|███████▋  | 1385/1813 [2:33:47<45:00,  6.31s/it]
+ 76%|███████▋  | 1386/1813 [2:33:53<45:35,  6.41s/it]
+ 77%|███████▋  | 1387/1813 [2:34:01<47:10,  6.64s/it]
+ 77%|███████▋  | 1388/1813 [2:34:06<44:58,  6.35s/it]
+ 77%|███████▋  | 1389/1813 [2:34:11<41:04,  5.81s/it]
+ 77%|███████▋  | 1390/1813 [2:34:18<42:51,  6.08s/it]
+                                                     
+{'loss': '0.9649', 'grad_norm': '1.116', 'learning_rate': '7.6e-06', 'epoch': '0.7669', 'num_input_tokens_seen': 9344880, 'train_runtime': '9258', 'train_tokens_per_second': '1009'}
+
+ 77%|███████▋  | 1390/1813 [2:34:18<42:51,  6.08s/it]
+ 77%|███████▋  | 1391/1813 [2:34:23<40:55,  5.82s/it]
+ 77%|███████▋  | 1392/1813 [2:34:29<42:24,  6.04s/it]
+ 77%|███████▋  | 1393/1813 [2:34:35<40:51,  5.84s/it]
+ 77%|███████▋  | 1394/1813 [2:34:42<42:47,  6.13s/it]
+ 77%|███████▋  | 1395/1813 [2:34:48<43:20,  6.22s/it]
+ 77%|███████▋  | 1396/1813 [2:34:54<43:16,  6.23s/it]
+ 77%|███████▋  | 1397/1813 [2:35:01<44:38,  6.44s/it]
+ 77%|███████▋  | 1398/1813 [2:35:06<41:09,  5.95s/it]
+ 77%|███████▋  | 1399/1813 [2:35:11<39:21,  5.70s/it]
+ 77%|███████▋  | 1400/1813 [2:35:18<41:48,  6.07s/it]
+                                                     
+{'loss': '0.8944', 'grad_norm': '0.8213', 'learning_rate': '7.264e-06', 'epoch': '0.7724', 'num_input_tokens_seen': 9408280, 'train_runtime': '9319', 'train_tokens_per_second': '1010'}
+
+ 77%|███████▋  | 1400/1813 [2:35:18<41:48,  6.07s/it]
+ 77%|███████▋  | 1401/1813 [2:35:24<40:38,  5.92s/it]
+ 77%|███████▋  | 1402/1813 [2:35:30<41:10,  6.01s/it]
+ 77%|███████▋  | 1403/1813 [2:35:37<42:36,  6.24s/it]
+ 77%|███████▋  | 1404/1813 [2:35:43<42:24,  6.22s/it]
+ 77%|███████▋  | 1405/1813 [2:35:48<39:30,  5.81s/it]
+ 78%|███████▊  | 1406/1813 [2:35:53<39:27,  5.82s/it]
+ 78%|███████▊  | 1407/1813 [2:36:00<41:12,  6.09s/it]
+ 78%|███████▊  | 1408/1813 [2:36:05<39:17,  5.82s/it]
+ 78%|███████▊  | 1409/1813 [2:36:10<37:25,  5.56s/it]
+ 78%|███████▊  | 1410/1813 [2:36:16<38:00,  5.66s/it]
+                                                     
+{'loss': '0.9596', 'grad_norm': '1.202', 'learning_rate': '6.934e-06', 'epoch': '0.7779', 'num_input_tokens_seen': 9467408, 'train_runtime': '9377', 'train_tokens_per_second': '1010'}
+
+ 78%|███████▊  | 1410/1813 [2:36:16<38:00,  5.66s/it]
+ 78%|███████▊  | 1411/1813 [2:36:22<37:27,  5.59s/it]
+ 78%|███████▊  | 1412/1813 [2:36:28<38:48,  5.81s/it]
+ 78%|███████▊  | 1413/1813 [2:36:34<38:17,  5.74s/it]
+ 78%|███████▊  | 1414/1813 [2:36:40<39:07,  5.88s/it]
+ 78%|███████▊  | 1415/1813 [2:36:47<41:39,  6.28s/it]
+ 78%|███████▊  | 1416/1813 [2:36:53<41:38,  6.29s/it]
+ 78%|███████▊  | 1417/1813 [2:36:59<40:08,  6.08s/it]
+ 78%|███████▊  | 1418/1813 [2:37:05<39:29,  6.00s/it]
+ 78%|███████▊  | 1419/1813 [2:37:13<44:33,  6.79s/it]
+ 78%|███████▊  | 1420/1813 [2:37:18<40:27,  6.18s/it]
+                                                     
+{'loss': '0.907', 'grad_norm': '1.094', 'learning_rate': '6.611e-06', 'epoch': '0.7834', 'num_input_tokens_seen': 9532720, 'train_runtime': '9439', 'train_tokens_per_second': '1010'}
+
+ 78%|███████▊  | 1420/1813 [2:37:18<40:27,  6.18s/it]
+ 78%|███████▊  | 1421/1813 [2:37:25<41:32,  6.36s/it]
+ 78%|███████▊  | 1422/1813 [2:37:30<39:36,  6.08s/it]
+ 78%|███████▊  | 1423/1813 [2:37:36<38:22,  5.91s/it]
+ 79%|███████▊  | 1424/1813 [2:37:43<41:11,  6.35s/it]
+ 79%|███████▊  | 1425/1813 [2:37:48<37:32,  5.80s/it]
+ 79%|███████▊  | 1426/1813 [2:37:56<42:43,  6.62s/it]
+ 79%|███████▊  | 1427/1813 [2:38:02<40:26,  6.29s/it]
+ 79%|███████▉  | 1428/1813 [2:38:08<40:53,  6.37s/it]
+ 79%|███████▉  | 1429/1813 [2:38:14<40:19,  6.30s/it]
+ 79%|███████▉  | 1430/1813 [2:38:21<41:23,  6.49s/it]
+                                                     
+{'loss': '0.906', 'grad_norm': '1.043', 'learning_rate': '6.294e-06', 'epoch': '0.789', 'num_input_tokens_seen': 9602112, 'train_runtime': '9502', 'train_tokens_per_second': '1011'}
+
+ 79%|███████▉  | 1430/1813 [2:38:21<41:23,  6.49s/it]
+ 79%|███████▉  | 1431/1813 [2:38:27<40:18,  6.33s/it]
+ 79%|███████▉  | 1432/1813 [2:38:33<39:14,  6.18s/it]
+ 79%|███████▉  | 1433/1813 [2:38:40<39:36,  6.25s/it]
+ 79%|███████▉  | 1434/1813 [2:38:46<39:32,  6.26s/it]
+ 79%|███████▉  | 1435/1813 [2:38:51<37:58,  6.03s/it]
+ 79%|███████▉  | 1436/1813 [2:38:58<39:18,  6.25s/it]
+ 79%|███████▉  | 1437/1813 [2:39:04<38:06,  6.08s/it]
+ 79%|███████▉  | 1438/1813 [2:39:11<40:12,  6.43s/it]
+ 79%|███████▉  | 1439/1813 [2:39:17<38:43,  6.21s/it]
+ 79%|███████▉  | 1440/1813 [2:39:22<36:44,  5.91s/it]
+                                                     
+{'loss': '0.909', 'grad_norm': '1.193', 'learning_rate': '5.984e-06', 'epoch': '0.7945', 'num_input_tokens_seen': 9667112, 'train_runtime': '9562', 'train_tokens_per_second': '1011'}
+
+ 79%|███████▉  | 1440/1813 [2:39:22<36:44,  5.91s/it]
+ 79%|███████▉  | 1441/1813 [2:39:28<37:45,  6.09s/it]
+ 80%|███████▉  | 1442/1813 [2:39:34<36:10,  5.85s/it]
+ 80%|███████▉  | 1443/1813 [2:39:39<34:27,  5.59s/it]
+ 80%|███████▉  | 1444/1813 [2:39:45<36:29,  5.93s/it]
+ 80%|███████▉  | 1445/1813 [2:39:51<35:39,  5.81s/it]
+ 80%|███████▉  | 1446/1813 [2:39:56<34:48,  5.69s/it]
+ 80%|███████▉  | 1447/1813 [2:40:05<39:27,  6.47s/it]
+ 80%|███████▉  | 1448/1813 [2:40:12<40:55,  6.73s/it]
+ 80%|███████▉  | 1449/1813 [2:40:18<40:25,  6.66s/it]
+ 80%|███████▉  | 1450/1813 [2:40:25<39:54,  6.60s/it]
+                                                     
+{'loss': '0.9943', 'grad_norm': '1.054', 'learning_rate': '5.681e-06', 'epoch': '0.8', 'num_input_tokens_seen': 9735792, 'train_runtime': '9625', 'train_tokens_per_second': '1011'}
+
+ 80%|███████▉  | 1450/1813 [2:40:25<39:54,  6.60s/it]
+ 80%|████████  | 1451/1813 [2:40:32<40:28,  6.71s/it]
+ 80%|████████  | 1452/1813 [2:40:36<36:22,  6.05s/it]
+ 80%|████████  | 1453/1813 [2:40:45<40:13,  6.70s/it]
+ 80%|████████  | 1454/1813 [2:40:50<37:36,  6.29s/it]
+ 80%|████████  | 1455/1813 [2:40:56<37:17,  6.25s/it]
+ 80%|████████  | 1456/1813 [2:41:03<38:38,  6.50s/it]
+ 80%|████████  | 1457/1813 [2:41:07<33:22,  5.63s/it]
+ 80%|████████  | 1458/1813 [2:41:12<33:05,  5.59s/it]
+ 80%|████████  | 1459/1813 [2:41:19<35:33,  6.03s/it]
+ 81%|████████  | 1460/1813 [2:41:27<37:57,  6.45s/it]
+                                                     
+{'loss': '0.9361', 'grad_norm': '0.9657', 'learning_rate': '5.385e-06', 'epoch': '0.8055', 'num_input_tokens_seen': 9802968, 'train_runtime': '9687', 'train_tokens_per_second': '1012'}
+
+ 81%|████████  | 1460/1813 [2:41:27<37:57,  6.45s/it]
+ 81%|████████  | 1461/1813 [2:41:33<37:17,  6.36s/it]
+ 81%|████████  | 1462/1813 [2:41:38<35:36,  6.09s/it]
+ 81%|████████  | 1463/1813 [2:41:46<37:59,  6.51s/it]
+ 81%|████████  | 1464/1813 [2:41:52<38:00,  6.53s/it]
+ 81%|████████  | 1465/1813 [2:42:01<40:51,  7.04s/it]
+ 81%|████████  | 1466/1813 [2:42:06<38:27,  6.65s/it]
+ 81%|████████  | 1467/1813 [2:42:14<39:50,  6.91s/it]
+ 81%|████████  | 1468/1813 [2:42:21<39:41,  6.90s/it]
+ 81%|████████  | 1469/1813 [2:42:30<42:45,  7.46s/it]
+ 81%|████████  | 1470/1813 [2:42:36<41:08,  7.20s/it]
+                                                     
+{'loss': '0.931', 'grad_norm': '1.027', 'learning_rate': '5.096e-06', 'epoch': '0.811', 'num_input_tokens_seen': 9879232, 'train_runtime': '9757', 'train_tokens_per_second': '1013'}
+
+ 81%|████████  | 1470/1813 [2:42:36<41:08,  7.20s/it]
+ 81%|████████  | 1471/1813 [2:42:42<39:27,  6.92s/it]
+ 81%|████████  | 1472/1813 [2:42:49<39:08,  6.89s/it]
+ 81%|████████  | 1473/1813 [2:42:55<37:21,  6.59s/it]
+ 81%|████████▏ | 1474/1813 [2:43:02<37:33,  6.65s/it]
+ 81%|████████▏ | 1475/1813 [2:43:08<37:10,  6.60s/it]
+ 81%|████████▏ | 1476/1813 [2:43:15<36:44,  6.54s/it]
+ 81%|████████▏ | 1477/1813 [2:43:21<36:26,  6.51s/it]
+ 82%|████████▏ | 1478/1813 [2:43:27<35:05,  6.28s/it]
+ 82%|████████▏ | 1479/1813 [2:43:33<35:04,  6.30s/it]
+ 82%|████████▏ | 1480/1813 [2:43:41<36:23,  6.56s/it]
+                                                     
+{'loss': '0.9109', 'grad_norm': '0.781', 'learning_rate': '4.814e-06', 'epoch': '0.8166', 'num_input_tokens_seen': 9948032, 'train_runtime': '9821', 'train_tokens_per_second': '1013'}
+
+ 82%|████████▏ | 1480/1813 [2:43:41<36:23,  6.56s/it]
+ 82%|████████▏ | 1481/1813 [2:43:47<36:25,  6.58s/it]
+ 82%|████████▏ | 1482/1813 [2:43:53<34:53,  6.33s/it]
+ 82%|████████▏ | 1483/1813 [2:44:00<35:53,  6.53s/it]
+ 82%|████████▏ | 1484/1813 [2:44:06<35:51,  6.54s/it]
+ 82%|████████▏ | 1485/1813 [2:44:13<36:32,  6.68s/it]
+ 82%|████████▏ | 1486/1813 [2:44:19<34:17,  6.29s/it]
+ 82%|████████▏ | 1487/1813 [2:44:27<36:55,  6.79s/it]
+ 82%|████████▏ | 1488/1813 [2:44:33<36:33,  6.75s/it]
+ 82%|████████▏ | 1489/1813 [2:44:41<37:38,  6.97s/it]
+ 82%|████████▏ | 1490/1813 [2:44:48<37:36,  6.99s/it]
+                                                     
+{'loss': '0.9736', 'grad_norm': '1.211', 'learning_rate': '4.539e-06', 'epoch': '0.8221', 'num_input_tokens_seen': 10021400, 'train_runtime': '9888', 'train_tokens_per_second': '1013'}
+
+ 82%|████████▏ | 1490/1813 [2:44:48<37:36,  6.99s/it]
+ 82%|████████▏ | 1491/1813 [2:44:54<35:48,  6.67s/it]
+ 82%|████████▏ | 1492/1813 [2:45:02<38:10,  7.14s/it]
+ 82%|████████▏ | 1493/1813 [2:45:08<35:59,  6.75s/it]
+ 82%|████████▏ | 1494/1813 [2:45:14<34:46,  6.54s/it]
+ 82%|████████▏ | 1495/1813 [2:45:22<37:04,  7.00s/it]
+ 83%|████████▎ | 1496/1813 [2:45:27<34:12,  6.48s/it]
+ 83%|████████▎ | 1497/1813 [2:45:32<31:55,  6.06s/it]
+ 83%|████████▎ | 1498/1813 [2:45:39<32:34,  6.21s/it]
+ 83%|████████▎ | 1499/1813 [2:45:44<30:01,  5.74s/it]
+ 83%|████████▎ | 1500/1813 [2:45:51<32:42,  6.27s/it]
+                                                     
+{'loss': '0.8796', 'grad_norm': '1.042', 'learning_rate': '4.271e-06', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904, 'train_runtime': '9952', 'train_tokens_per_second': '1014'}
+
+ 83%|████████▎ | 1500/1813 [2:45:51<32:42,  6.27s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.78it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.23it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.32it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.08it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.00it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.83it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  6.00it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.11it/s]
+
+  4%|▍         | 39/1000 [00:05<02:07,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:44,  9.12it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.34it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.75it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.31it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.76it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.11it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.66it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.86it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.19it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.18it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.83it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.31it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.43it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.20it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.04it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.52it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.81it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.23it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.69it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.13it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.89it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.07it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.92it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.05it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.35it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.96it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.59it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.66it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.77it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.83it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.15it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.23it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.46it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.51it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.12it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.18it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.60it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.75it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.63it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.93it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.50it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.68it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.80it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.92it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.38it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.03it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.22it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.97it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.47it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.70it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.74it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.92it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:36,  7.55it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.05it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.74it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.79it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.84it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.09it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.16it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.80it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.21it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.26it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.24it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.56it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.97it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.39it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.06it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.73it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.21it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.40it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.65it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.56it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.41it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.93it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.54it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.71it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.78it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.22it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.18it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.66it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.39it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.79it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:21,  7.45it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.74it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.24it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.50it/s]
+
+ 41%|████      | 408/1000 [00:54<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.46it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.96it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.83it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.51it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 10.99it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:48, 11.77it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.51it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.51it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:51, 10.47it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:11,  7.52it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.69it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:29,  5.97it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.80it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.28it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.82it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.18it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.35it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:25,  6.17it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.18it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.29it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.08it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.51it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:05,  7.90it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:11,  7.24it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:22,  6.26it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:22,  6.27it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.96it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.60it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.19it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.44it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.42it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.95it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.97it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.10it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.07it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.69it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.57it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.13it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:50,  9.32it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:10,  6.57it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.28it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:22,  5.60it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:18,  5.89it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:20,  5.73it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:06,  6.91it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.04it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:10,  6.49it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.91it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.87it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.32it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.81it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.72it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.24it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.96it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.54it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.57it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.90it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.14it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.52it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.74it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.40it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:02,  6.46it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.68it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.09it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:45,  8.48it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.74it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.79it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.61it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.75it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.09it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.02it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.19it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.45it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.43it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.55it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.91it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.11it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.47it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.64it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.70it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.98it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.89it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.82it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.93it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.71it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.34it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.15it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.62it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.31it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.50it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.87it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.52it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.76it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.29it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.34it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.59it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.44it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.69it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.24it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.39it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.72it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.64it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.79it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.99it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:24,  7.84it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.18it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.93it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.18it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.19it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.04it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.30it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.12it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:20,  8.99it/s]
+
+ 82%|████████▏ | 820/1000 [01:50<00:24,  7.42it/s]
+
+ 82%|████████▏ | 822/1000 [01:50<00:20,  8.69it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:23,  7.66it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:27,  6.45it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.77it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:31,  5.57it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:33,  5.10it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:30,  5.63it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  6.96it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.70it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.20it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.00it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.52it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.10it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.44it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.88it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.19it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14,  9.81it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.06it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.44it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.60it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:13, 10.31it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.41it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.19it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:13,  9.96it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.00it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.70it/s]
+
+ 87%|████████▋ | 874/1000 [01:58<00:17,  7.03it/s]
+
+ 88%|████████▊ | 876/1000 [01:58<00:14,  8.48it/s]
+
+ 88%|████████▊ | 878/1000 [01:58<00:16,  7.60it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.93it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.79it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.53it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  8.09it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.36it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.55it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.89it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.71it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:14,  7.56it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.82it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.52it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.69it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.85it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.05it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.03it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.16it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.63it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.09it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.60it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.06it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.31it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.06it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.20it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.55it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.75it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.38it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.14it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.38it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.38it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.76it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.71it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  5.99it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.43it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.41it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.29it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:04,  7.97it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.37it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.19it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.78it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.79it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.43it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.75it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.99it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.56it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7754', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.381', 'eval_steps_per_second': '7.381', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904}
+
+ 83%|████████▎ | 1500/1813 [2:48:07<32:42,  6.27s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 83%|████████▎ | 1501/1813 [2:48:13<4:04:18, 46.98s/it]
+ 83%|████████▎ | 1502/1813 [2:48:18<2:57:53, 34.32s/it]
+ 83%|████████▎ | 1503/1813 [2:48:24<2:14:04, 25.95s/it]
+ 83%|████████▎ | 1504/1813 [2:48:30<1:41:46, 19.76s/it]
+ 83%|████████▎ | 1505/1813 [2:48:35<1:19:11, 15.43s/it]
+ 83%|████████▎ | 1506/1813 [2:48:42<1:05:51, 12.87s/it]
+ 83%|████████▎ | 1507/1813 [2:48:47<53:27, 10.48s/it]  
+ 83%|████████▎ | 1508/1813 [2:48:54<47:55,  9.43s/it]
+ 83%|████████▎ | 1509/1813 [2:49:00<42:21,  8.36s/it]
+ 83%|████████▎ | 1510/1813 [2:49:06<38:45,  7.68s/it]
+                                                     
+{'loss': '0.8889', 'grad_norm': '1.203', 'learning_rate': '4.011e-06', 'epoch': '0.8331', 'num_input_tokens_seen': 10151928, 'train_runtime': '1.015e+04', 'train_tokens_per_second': '1001'}
+
+ 83%|████████▎ | 1510/1813 [2:49:06<38:45,  7.68s/it]
+ 83%|████████▎ | 1511/1813 [2:49:12<36:52,  7.32s/it]
+ 83%|████████▎ | 1512/1813 [2:49:18<34:18,  6.84s/it]
+ 83%|████████▎ | 1513/1813 [2:49:23<32:11,  6.44s/it]
+ 84%|████████▎ | 1514/1813 [2:49:30<32:15,  6.47s/it]
+ 84%|████████▎ | 1515/1813 [2:49:37<33:37,  6.77s/it]
+ 84%|████████▎ | 1516/1813 [2:49:45<34:05,  6.89s/it]
+ 84%|████████▎ | 1517/1813 [2:49:51<33:46,  6.85s/it]
+ 84%|████████▎ | 1518/1813 [2:49:59<34:55,  7.10s/it]
+ 84%|████████▍ | 1519/1813 [2:50:06<35:04,  7.16s/it]
+ 84%|████████▍ | 1520/1813 [2:50:13<33:48,  6.92s/it]
+                                                     
+{'loss': '0.9119', 'grad_norm': '1.284', 'learning_rate': '3.758e-06', 'epoch': '0.8386', 'num_input_tokens_seen': 10225024, 'train_runtime': '1.021e+04', 'train_tokens_per_second': '1001'}
+
+ 84%|████████▍ | 1520/1813 [2:50:13<33:48,  6.92s/it]
+ 84%|████████▍ | 1521/1813 [2:50:20<33:46,  6.94s/it]
+ 84%|████████▍ | 1522/1813 [2:50:26<32:52,  6.78s/it]
+ 84%|████████▍ | 1523/1813 [2:50:34<33:50,  7.00s/it]
+ 84%|████████▍ | 1524/1813 [2:50:40<33:05,  6.87s/it]
+ 84%|████████▍ | 1525/1813 [2:50:45<30:24,  6.34s/it]
+ 84%|████████▍ | 1526/1813 [2:50:52<30:38,  6.41s/it]
+ 84%|████████▍ | 1527/1813 [2:50:59<31:15,  6.56s/it]
+ 84%|████████▍ | 1528/1813 [2:51:05<31:10,  6.56s/it]
+ 84%|████████▍ | 1529/1813 [2:51:12<31:53,  6.74s/it]
+ 84%|████████▍ | 1530/1813 [2:51:18<30:32,  6.48s/it]
+                                                     
+{'loss': '0.9531', 'grad_norm': '0.8593', 'learning_rate': '3.513e-06', 'epoch': '0.8441', 'num_input_tokens_seen': 10294368, 'train_runtime': '1.028e+04', 'train_tokens_per_second': '1002'}
+
+ 84%|████████▍ | 1530/1813 [2:51:18<30:32,  6.48s/it]
+ 84%|████████▍ | 1531/1813 [2:51:24<28:58,  6.16s/it]
+ 85%|████████▍ | 1532/1813 [2:51:31<30:12,  6.45s/it]
+ 85%|████████▍ | 1533/1813 [2:51:36<28:30,  6.11s/it]
+ 85%|████████▍ | 1534/1813 [2:51:44<31:15,  6.72s/it]
+ 85%|████████▍ | 1535/1813 [2:51:50<29:36,  6.39s/it]
+ 85%|████████▍ | 1536/1813 [2:51:58<31:14,  6.77s/it]
+ 85%|████████▍ | 1537/1813 [2:52:04<31:01,  6.75s/it]
+ 85%|████████▍ | 1538/1813 [2:52:09<28:20,  6.18s/it]
+ 85%|████████▍ | 1539/1813 [2:52:15<27:45,  6.08s/it]
+ 85%|████████▍ | 1540/1813 [2:52:21<27:08,  5.97s/it]
+                                                     
+{'loss': '0.9309', 'grad_norm': '1.347', 'learning_rate': '3.275e-06', 'epoch': '0.8497', 'num_input_tokens_seen': 10359416, 'train_runtime': '1.034e+04', 'train_tokens_per_second': '1002'}
+
+ 85%|████████▍ | 1540/1813 [2:52:21<27:08,  5.97s/it]
+ 85%|████████▍ | 1541/1813 [2:52:27<28:09,  6.21s/it]
+ 85%|████████▌ | 1542/1813 [2:52:34<28:24,  6.29s/it]
+ 85%|████████▌ | 1543/1813 [2:52:39<26:54,  5.98s/it]
+ 85%|████████▌ | 1544/1813 [2:52:44<25:42,  5.74s/it]
+ 85%|████████▌ | 1545/1813 [2:52:52<28:36,  6.40s/it]
+ 85%|████████▌ | 1546/1813 [2:52:58<27:51,  6.26s/it]
+ 85%|████████▌ | 1547/1813 [2:53:03<26:19,  5.94s/it]
+ 85%|████████▌ | 1548/1813 [2:53:12<29:09,  6.60s/it]
+ 85%|████████▌ | 1549/1813 [2:53:17<27:16,  6.20s/it]
+ 85%|████████▌ | 1550/1813 [2:53:24<28:35,  6.52s/it]
+                                                     
+{'loss': '1.029', 'grad_norm': '1.285', 'learning_rate': '3.045e-06', 'epoch': '0.8552', 'num_input_tokens_seen': 10427368, 'train_runtime': '1.04e+04', 'train_tokens_per_second': '1002'}
+
+ 85%|████████▌ | 1550/1813 [2:53:24<28:35,  6.52s/it]
+ 86%|████████▌ | 1551/1813 [2:53:32<30:08,  6.90s/it]
+ 86%|████████▌ | 1552/1813 [2:53:38<28:54,  6.64s/it]
+ 86%|████████▌ | 1553/1813 [2:53:44<27:47,  6.41s/it]
+ 86%|████████▌ | 1554/1813 [2:53:51<28:10,  6.53s/it]
+ 86%|████████▌ | 1555/1813 [2:53:57<28:15,  6.57s/it]
+ 86%|████████▌ | 1556/1813 [2:54:03<27:38,  6.45s/it]
+ 86%|████████▌ | 1557/1813 [2:54:11<29:18,  6.87s/it]
+ 86%|████████▌ | 1558/1813 [2:54:18<28:26,  6.69s/it]
+ 86%|████████▌ | 1559/1813 [2:54:24<27:54,  6.59s/it]
+ 86%|████████▌ | 1560/1813 [2:54:29<25:56,  6.15s/it]
+                                                     
+{'loss': '0.9987', 'grad_norm': '0.9392', 'learning_rate': '2.823e-06', 'epoch': '0.8607', 'num_input_tokens_seen': 10497472, 'train_runtime': '1.047e+04', 'train_tokens_per_second': '1003'}
+
+ 86%|████████▌ | 1560/1813 [2:54:29<25:56,  6.15s/it]
+ 86%|████████▌ | 1561/1813 [2:54:36<26:47,  6.38s/it]
+ 86%|████████▌ | 1562/1813 [2:54:42<26:41,  6.38s/it]
+ 86%|████████▌ | 1563/1813 [2:54:49<26:24,  6.34s/it]
+ 86%|████████▋ | 1564/1813 [2:54:55<26:01,  6.27s/it]
+ 86%|████████▋ | 1565/1813 [2:54:59<23:29,  5.69s/it]
+ 86%|████████▋ | 1566/1813 [2:55:05<23:47,  5.78s/it]
+ 86%|████████▋ | 1567/1813 [2:55:11<24:16,  5.92s/it]
+ 86%|████████▋ | 1568/1813 [2:55:16<22:58,  5.63s/it]
+ 87%|████████▋ | 1569/1813 [2:55:24<25:21,  6.24s/it]
+ 87%|████████▋ | 1570/1813 [2:55:31<26:44,  6.60s/it]
+                                                     
+{'loss': '0.8632', 'grad_norm': '1.126', 'learning_rate': '2.609e-06', 'epoch': '0.8662', 'num_input_tokens_seen': 10564336, 'train_runtime': '1.053e+04', 'train_tokens_per_second': '1003'}
+
+ 87%|████████▋ | 1570/1813 [2:55:31<26:44,  6.60s/it]
+ 87%|████████▋ | 1571/1813 [2:55:37<25:15,  6.26s/it]
+ 87%|████████▋ | 1572/1813 [2:55:43<24:32,  6.11s/it]
+ 87%|████████▋ | 1573/1813 [2:55:48<23:45,  5.94s/it]
+ 87%|████████▋ | 1574/1813 [2:55:55<24:23,  6.12s/it]
+ 87%|████████▋ | 1575/1813 [2:56:01<24:04,  6.07s/it]
+ 87%|████████▋ | 1576/1813 [2:56:08<25:26,  6.44s/it]
+ 87%|████████▋ | 1577/1813 [2:56:15<25:42,  6.53s/it]
+ 87%|████████▋ | 1578/1813 [2:56:21<25:12,  6.44s/it]
+ 87%|████████▋ | 1579/1813 [2:56:27<24:44,  6.34s/it]
+ 87%|████████▋ | 1580/1813 [2:56:32<22:53,  5.89s/it]
+                                                     
+{'loss': '0.951', 'grad_norm': '1.44', 'learning_rate': '2.403e-06', 'epoch': '0.8717', 'num_input_tokens_seen': 10628400, 'train_runtime': '1.059e+04', 'train_tokens_per_second': '1003'}
+
+ 87%|████████▋ | 1580/1813 [2:56:32<22:53,  5.89s/it]
+ 87%|████████▋ | 1581/1813 [2:56:38<22:44,  5.88s/it]
+ 87%|████████▋ | 1582/1813 [2:56:42<20:43,  5.38s/it]
+ 87%|████████▋ | 1583/1813 [2:56:48<21:34,  5.63s/it]
+ 87%|████████▋ | 1584/1813 [2:56:54<22:14,  5.83s/it]
+ 87%|████████▋ | 1585/1813 [2:57:02<23:41,  6.23s/it]
+ 87%|████████▋ | 1586/1813 [2:57:09<24:50,  6.56s/it]
+ 88%|████████▊ | 1587/1813 [2:57:16<25:44,  6.83s/it]
+ 88%|████████▊ | 1588/1813 [2:57:22<24:18,  6.48s/it]
+ 88%|████████▊ | 1589/1813 [2:57:27<22:30,  6.03s/it]
+ 88%|████████▊ | 1590/1813 [2:57:34<23:00,  6.19s/it]
+                                                     
+{'loss': '0.9113', 'grad_norm': '1.015', 'learning_rate': '2.205e-06', 'epoch': '0.8772', 'num_input_tokens_seen': 10695288, 'train_runtime': '1.065e+04', 'train_tokens_per_second': '1004'}
+
+ 88%|████████▊ | 1590/1813 [2:57:34<23:00,  6.19s/it]
+ 88%|████████▊ | 1591/1813 [2:57:40<22:43,  6.14s/it]
+ 88%|████████▊ | 1592/1813 [2:57:46<22:49,  6.20s/it]
+ 88%|████████▊ | 1593/1813 [2:57:53<23:10,  6.32s/it]
+ 88%|████████▊ | 1594/1813 [2:57:59<23:24,  6.41s/it]
+ 88%|████████▊ | 1595/1813 [2:58:06<24:15,  6.67s/it]
+ 88%|████████▊ | 1596/1813 [2:58:15<26:23,  7.30s/it]
+ 88%|████████▊ | 1597/1813 [2:58:22<25:34,  7.10s/it]
+ 88%|████████▊ | 1598/1813 [2:58:28<24:44,  6.90s/it]
+ 88%|████████▊ | 1599/1813 [2:58:34<23:05,  6.47s/it]
+ 88%|████████▊ | 1600/1813 [2:58:39<21:19,  6.01s/it]
+                                                     
+{'loss': '0.9497', 'grad_norm': '1.03', 'learning_rate': '2.015e-06', 'epoch': '0.8828', 'num_input_tokens_seen': 10767512, 'train_runtime': '1.072e+04', 'train_tokens_per_second': '1005'}
+
+ 88%|████████▊ | 1600/1813 [2:58:39<21:19,  6.01s/it]
+ 88%|████████▊ | 1601/1813 [2:58:47<23:25,  6.63s/it]
+ 88%|████████▊ | 1602/1813 [2:58:53<22:34,  6.42s/it]
+ 88%|████████▊ | 1603/1813 [2:58:59<22:49,  6.52s/it]
+ 88%|████████▊ | 1604/1813 [2:59:06<22:54,  6.58s/it]
+ 89%|████████▊ | 1605/1813 [2:59:15<24:37,  7.11s/it]
+ 89%|████████▊ | 1606/1813 [2:59:20<22:27,  6.51s/it]
+ 89%|████████▊ | 1607/1813 [2:59:26<22:37,  6.59s/it]
+ 89%|████████▊ | 1608/1813 [2:59:34<23:26,  6.86s/it]
+ 89%|████████▊ | 1609/1813 [2:59:39<21:56,  6.45s/it]
+ 89%|████████▉ | 1610/1813 [2:59:46<21:31,  6.36s/it]
+                                                     
+{'loss': '0.9349', 'grad_norm': '1.381', 'learning_rate': '1.834e-06', 'epoch': '0.8883', 'num_input_tokens_seen': 10840800, 'train_runtime': '1.079e+04', 'train_tokens_per_second': '1005'}
+
+ 89%|████████▉ | 1610/1813 [2:59:46<21:31,  6.36s/it]
+ 89%|████████▉ | 1611/1813 [2:59:51<20:18,  6.03s/it]
+ 89%|████████▉ | 1612/1813 [2:59:57<19:56,  5.95s/it]
+ 89%|████████▉ | 1613/1813 [3:00:01<18:18,  5.49s/it]
+ 89%|████████▉ | 1614/1813 [3:00:07<18:28,  5.57s/it]
+ 89%|████████▉ | 1615/1813 [3:00:13<19:14,  5.83s/it]
+ 89%|████████▉ | 1616/1813 [3:00:20<19:49,  6.04s/it]
+ 89%|████████▉ | 1617/1813 [3:00:26<20:19,  6.22s/it]
+ 89%|████████▉ | 1618/1813 [3:00:32<19:10,  5.90s/it]
+ 89%|████████▉ | 1619/1813 [3:00:37<19:01,  5.89s/it]
+ 89%|████████▉ | 1620/1813 [3:00:42<17:54,  5.57s/it]
+                                                     
+{'loss': '0.9403', 'grad_norm': '1.12', 'learning_rate': '1.66e-06', 'epoch': '0.8938', 'num_input_tokens_seen': 10899640, 'train_runtime': '1.084e+04', 'train_tokens_per_second': '1005'}
+
+ 89%|████████▉ | 1620/1813 [3:00:42<17:54,  5.57s/it]
+ 89%|████████▉ | 1621/1813 [3:00:48<17:33,  5.49s/it]
+ 89%|████████▉ | 1622/1813 [3:00:55<19:35,  6.15s/it]
+ 90%|████████▉ | 1623/1813 [3:01:02<20:08,  6.36s/it]
+ 90%|████████▉ | 1624/1813 [3:01:08<19:31,  6.20s/it]
+ 90%|████████▉ | 1625/1813 [3:01:15<20:27,  6.53s/it]
+ 90%|████████▉ | 1626/1813 [3:01:21<19:50,  6.37s/it]
+ 90%|████████▉ | 1627/1813 [3:01:27<19:18,  6.23s/it]
+ 90%|████████▉ | 1628/1813 [3:01:33<18:37,  6.04s/it]
+ 90%|████████▉ | 1629/1813 [3:01:40<19:32,  6.37s/it]
+ 90%|████████▉ | 1630/1813 [3:01:45<18:24,  6.03s/it]
+                                                     
+{'loss': '0.9479', 'grad_norm': '1.739', 'learning_rate': '1.495e-06', 'epoch': '0.8993', 'num_input_tokens_seen': 10968528, 'train_runtime': '1.091e+04', 'train_tokens_per_second': '1006'}
+
+ 90%|████████▉ | 1630/1813 [3:01:45<18:24,  6.03s/it]
+ 90%|████████▉ | 1631/1813 [3:01:50<17:42,  5.84s/it]
+ 90%|█████████ | 1632/1813 [3:01:55<16:38,  5.52s/it]
+ 90%|█████████ | 1633/1813 [3:02:03<18:12,  6.07s/it]
+ 90%|█████████ | 1634/1813 [3:02:07<17:01,  5.71s/it]
+ 90%|█████████ | 1635/1813 [3:02:14<17:49,  6.01s/it]
+ 90%|█████████ | 1636/1813 [3:02:20<17:17,  5.86s/it]
+ 90%|█████████ | 1637/1813 [3:02:26<17:34,  5.99s/it]
+ 90%|█████████ | 1638/1813 [3:02:31<16:44,  5.74s/it]
+ 90%|█████████ | 1639/1813 [3:02:38<17:40,  6.10s/it]
+ 90%|█████████ | 1640/1813 [3:02:43<16:57,  5.88s/it]
+                                                     
+{'loss': '0.9517', 'grad_norm': '0.8701', 'learning_rate': '1.338e-06', 'epoch': '0.9048', 'num_input_tokens_seen': 11030048, 'train_runtime': '1.096e+04', 'train_tokens_per_second': '1006'}
+
+ 90%|█████████ | 1640/1813 [3:02:43<16:57,  5.88s/it]
+ 91%|█████████ | 1641/1813 [3:02:49<16:55,  5.91s/it]
+ 91%|█████████ | 1642/1813 [3:02:56<17:26,  6.12s/it]
+ 91%|█████████ | 1643/1813 [3:03:02<17:27,  6.16s/it]
+ 91%|█████████ | 1644/1813 [3:03:09<17:57,  6.38s/it]
+ 91%|█████████ | 1645/1813 [3:03:19<20:28,  7.31s/it]
+ 91%|█████████ | 1646/1813 [3:03:25<19:51,  7.14s/it]
+ 91%|█████████ | 1647/1813 [3:03:31<18:54,  6.83s/it]
+ 91%|█████████ | 1648/1813 [3:03:38<18:37,  6.78s/it]
+ 91%|█████████ | 1649/1813 [3:03:43<17:11,  6.29s/it]
+ 91%|█████████ | 1650/1813 [3:03:48<16:11,  5.96s/it]
+                                                     
+{'loss': '0.9149', 'grad_norm': '1.061', 'learning_rate': '1.19e-06', 'epoch': '0.9103', 'num_input_tokens_seen': 11098768, 'train_runtime': '1.103e+04', 'train_tokens_per_second': '1006'}
+
+ 91%|█████████ | 1650/1813 [3:03:48<16:11,  5.96s/it]
+ 91%|█████████ | 1651/1813 [3:03:55<16:25,  6.08s/it]
+ 91%|█████████ | 1652/1813 [3:04:01<16:31,  6.16s/it]
+ 91%|█████████ | 1653/1813 [3:04:08<16:59,  6.37s/it]
+ 91%|█████████ | 1654/1813 [3:04:14<16:39,  6.29s/it]
+ 91%|█████████▏| 1655/1813 [3:04:21<16:37,  6.31s/it]
+ 91%|█████████▏| 1656/1813 [3:04:26<16:08,  6.17s/it]
+ 91%|█████████▏| 1657/1813 [3:04:34<17:16,  6.64s/it]
+ 91%|█████████▏| 1658/1813 [3:04:41<17:06,  6.62s/it]
+ 92%|█████████▏| 1659/1813 [3:04:48<17:16,  6.73s/it]
+ 92%|█████████▏| 1660/1813 [3:04:54<16:47,  6.58s/it]
+                                                     
+{'loss': '0.966', 'grad_norm': '1.297', 'learning_rate': '1.051e-06', 'epoch': '0.9159', 'num_input_tokens_seen': 11168848, 'train_runtime': '1.109e+04', 'train_tokens_per_second': '1007'}
+
+ 92%|█████████▏| 1660/1813 [3:04:54<16:47,  6.58s/it]
+ 92%|█████████▏| 1661/1813 [3:05:01<17:01,  6.72s/it]
+ 92%|█████████▏| 1662/1813 [3:05:07<16:45,  6.66s/it]
+ 92%|█████████▏| 1663/1813 [3:05:14<16:47,  6.72s/it]
+ 92%|█████████▏| 1664/1813 [3:05:21<16:46,  6.76s/it]
+ 92%|█████████▏| 1665/1813 [3:05:29<17:18,  7.01s/it]
+ 92%|█████████▏| 1666/1813 [3:05:33<15:16,  6.23s/it]
+ 92%|█████████▏| 1667/1813 [3:05:40<15:25,  6.34s/it]
+ 92%|█████████▏| 1668/1813 [3:05:46<15:03,  6.23s/it]
+ 92%|█████████▏| 1669/1813 [3:05:56<17:37,  7.34s/it]
+ 92%|█████████▏| 1670/1813 [3:06:01<16:06,  6.76s/it]
+                                                     
+{'loss': '0.9285', 'grad_norm': '1.257', 'learning_rate': '9.193e-07', 'epoch': '0.9214', 'num_input_tokens_seen': 11244288, 'train_runtime': '1.116e+04', 'train_tokens_per_second': '1007'}
+
+ 92%|█████████▏| 1670/1813 [3:06:01<16:06,  6.76s/it]
+ 92%|█████████▏| 1671/1813 [3:06:06<14:47,  6.25s/it]
+ 92%|█████████▏| 1672/1813 [3:06:12<14:18,  6.09s/it]
+ 92%|█████████▏| 1673/1813 [3:06:18<13:59,  6.00s/it]
+ 92%|█████████▏| 1674/1813 [3:06:25<14:31,  6.27s/it]
+ 92%|█████████▏| 1675/1813 [3:06:32<14:59,  6.52s/it]
+ 92%|█████████▏| 1676/1813 [3:06:39<15:25,  6.76s/it]
+ 92%|█████████▏| 1677/1813 [3:06:45<15:02,  6.64s/it]
+ 93%|█████████▎| 1678/1813 [3:06:52<14:46,  6.56s/it]
+ 93%|█████████▎| 1679/1813 [3:06:58<14:20,  6.42s/it]
+ 93%|█████████▎| 1680/1813 [3:07:07<15:47,  7.12s/it]
+                                                     
+{'loss': '0.9868', 'grad_norm': '0.9997', 'learning_rate': '7.967e-07', 'epoch': '0.9269', 'num_input_tokens_seen': 11317056, 'train_runtime': '1.123e+04', 'train_tokens_per_second': '1008'}
+
+ 93%|█████████▎| 1680/1813 [3:07:07<15:47,  7.12s/it]
+ 93%|█████████▎| 1681/1813 [3:07:12<14:18,  6.50s/it]
+ 93%|█████████▎| 1682/1813 [3:07:17<13:28,  6.17s/it]
+ 93%|█████████▎| 1683/1813 [3:07:22<12:32,  5.79s/it]
+ 93%|█████████▎| 1684/1813 [3:07:29<13:14,  6.16s/it]
+ 93%|█████████▎| 1685/1813 [3:07:36<13:36,  6.38s/it]
+ 93%|█████████▎| 1686/1813 [3:07:43<13:53,  6.57s/it]
+ 93%|█████████▎| 1687/1813 [3:07:49<13:49,  6.59s/it]
+ 93%|█████████▎| 1688/1813 [3:07:56<13:56,  6.69s/it]
+ 93%|█████████▎| 1689/1813 [3:08:01<12:40,  6.13s/it]
+ 93%|█████████▎| 1690/1813 [3:08:08<12:56,  6.31s/it]
+                                                     
+{'loss': '0.908', 'grad_norm': '0.8963', 'learning_rate': '6.828e-07', 'epoch': '0.9324', 'num_input_tokens_seen': 11380560, 'train_runtime': '1.129e+04', 'train_tokens_per_second': '1008'}
+
+ 93%|█████████▎| 1690/1813 [3:08:08<12:56,  6.31s/it]
+ 93%|█████████▎| 1691/1813 [3:08:14<12:52,  6.33s/it]
+ 93%|█████████▎| 1692/1813 [3:08:22<13:34,  6.73s/it]
+ 93%|█████████▎| 1693/1813 [3:08:28<13:05,  6.55s/it]
+ 93%|█████████▎| 1694/1813 [3:08:36<14:04,  7.09s/it]
+ 93%|█████████▎| 1695/1813 [3:08:43<13:20,  6.78s/it]
+ 94%|█████████▎| 1696/1813 [3:08:48<12:34,  6.45s/it]
+ 94%|█████████▎| 1697/1813 [3:08:55<12:24,  6.41s/it]
+ 94%|█████████▎| 1698/1813 [3:09:00<11:33,  6.03s/it]
+ 94%|█████████▎| 1699/1813 [3:09:06<11:35,  6.10s/it]
+ 94%|█████████▍| 1700/1813 [3:09:12<11:34,  6.15s/it]
+                                                     
+{'loss': '0.9315', 'grad_norm': '1.47', 'learning_rate': '5.775e-07', 'epoch': '0.9379', 'num_input_tokens_seen': 11450712, 'train_runtime': '1.135e+04', 'train_tokens_per_second': '1009'}
+
+ 94%|█████████▍| 1700/1813 [3:09:12<11:34,  6.15s/it]
+ 94%|█████████▍| 1701/1813 [3:09:17<10:43,  5.74s/it]
+ 94%|█████████▍| 1702/1813 [3:09:22<10:25,  5.64s/it]
+ 94%|█████████▍| 1703/1813 [3:09:28<10:28,  5.71s/it]
+ 94%|█████████▍| 1704/1813 [3:09:36<11:28,  6.32s/it]
+ 94%|█████████▍| 1705/1813 [3:09:43<11:33,  6.42s/it]
+ 94%|█████████▍| 1706/1813 [3:09:49<11:29,  6.44s/it]
+ 94%|█████████▍| 1707/1813 [3:09:55<10:53,  6.16s/it]
+ 94%|█████████▍| 1708/1813 [3:10:00<10:10,  5.81s/it]
+ 94%|█████████▍| 1709/1813 [3:10:06<10:24,  6.00s/it]
+ 94%|█████████▍| 1710/1813 [3:10:12<10:09,  5.92s/it]
+                                                     
+{'loss': '0.9534', 'grad_norm': '1.313', 'learning_rate': '4.809e-07', 'epoch': '0.9434', 'num_input_tokens_seen': 11511984, 'train_runtime': '1.141e+04', 'train_tokens_per_second': '1009'}
+
+ 94%|█████████▍| 1710/1813 [3:10:12<10:09,  5.92s/it]
+ 94%|█████████▍| 1711/1813 [3:10:17<09:42,  5.72s/it]
+ 94%|█████████▍| 1712/1813 [3:10:23<09:34,  5.69s/it]
+ 94%|█████████▍| 1713/1813 [3:10:28<09:31,  5.72s/it]
+ 95%|█████████▍| 1714/1813 [3:10:34<09:26,  5.72s/it]
+ 95%|█████████▍| 1715/1813 [3:10:40<09:19,  5.71s/it]
+ 95%|█████████▍| 1716/1813 [3:10:45<08:59,  5.56s/it]
+ 95%|█████████▍| 1717/1813 [3:10:51<08:51,  5.54s/it]
+ 95%|█████████▍| 1718/1813 [3:10:57<09:09,  5.78s/it]
+ 95%|█████████▍| 1719/1813 [3:11:03<09:19,  5.96s/it]
+ 95%|█████████▍| 1720/1813 [3:11:09<09:15,  5.97s/it]
+                                                     
+{'loss': '0.9314', 'grad_norm': '0.9809', 'learning_rate': '3.931e-07', 'epoch': '0.949', 'num_input_tokens_seen': 11571368, 'train_runtime': '1.147e+04', 'train_tokens_per_second': '1009'}
+
+ 95%|█████████▍| 1720/1813 [3:11:09<09:15,  5.97s/it]
+ 95%|█████████▍| 1721/1813 [3:11:17<09:46,  6.37s/it]
+ 95%|█████████▍| 1722/1813 [3:11:23<09:30,  6.27s/it]
+ 95%|█████████▌| 1723/1813 [3:11:29<09:17,  6.20s/it]
+ 95%|█████████▌| 1724/1813 [3:11:35<09:08,  6.16s/it]
+ 95%|█████████▌| 1725/1813 [3:11:42<09:31,  6.49s/it]
+ 95%|█████████▌| 1726/1813 [3:11:50<09:50,  6.79s/it]
+ 95%|█████████▌| 1727/1813 [3:11:56<09:23,  6.56s/it]
+ 95%|█████████▌| 1728/1813 [3:12:01<08:46,  6.20s/it]
+ 95%|█████████▌| 1729/1813 [3:12:06<08:22,  5.99s/it]
+ 95%|█████████▌| 1730/1813 [3:12:13<08:36,  6.22s/it]
+                                                     
+{'loss': '0.9531', 'grad_norm': '0.7799', 'learning_rate': '3.141e-07', 'epoch': '0.9545', 'num_input_tokens_seen': 11640392, 'train_runtime': '1.153e+04', 'train_tokens_per_second': '1009'}
+
+ 95%|█████████▌| 1730/1813 [3:12:13<08:36,  6.22s/it]
+ 95%|█████████▌| 1731/1813 [3:12:19<08:32,  6.25s/it]
+ 96%|█████████▌| 1732/1813 [3:12:26<08:42,  6.46s/it]
+ 96%|█████████▌| 1733/1813 [3:12:32<08:05,  6.06s/it]
+ 96%|█████████▌| 1734/1813 [3:12:37<07:53,  5.99s/it]
+ 96%|█████████▌| 1735/1813 [3:12:44<07:56,  6.11s/it]
+ 96%|█████████▌| 1736/1813 [3:12:51<08:07,  6.33s/it]
+ 96%|█████████▌| 1737/1813 [3:12:57<07:56,  6.27s/it]
+ 96%|█████████▌| 1738/1813 [3:13:02<07:21,  5.88s/it]
+ 96%|█████████▌| 1739/1813 [3:13:10<08:02,  6.53s/it]
+ 96%|█████████▌| 1740/1813 [3:13:17<08:02,  6.61s/it]
+                                                     
+{'loss': '0.9412', 'grad_norm': '1.207', 'learning_rate': '2.439e-07', 'epoch': '0.96', 'num_input_tokens_seen': 11709120, 'train_runtime': '1.16e+04', 'train_tokens_per_second': '1010'}
+
+ 96%|█████████▌| 1740/1813 [3:13:17<08:02,  6.61s/it]
+ 96%|█████████▌| 1741/1813 [3:13:23<07:42,  6.42s/it]
+ 96%|█████████▌| 1742/1813 [3:13:29<07:45,  6.56s/it]
+ 96%|█████████▌| 1743/1813 [3:13:37<07:51,  6.74s/it]
+ 96%|█████████▌| 1744/1813 [3:13:44<07:54,  6.87s/it]
+ 96%|█████████▌| 1745/1813 [3:13:50<07:33,  6.68s/it]
+ 96%|█████████▋| 1746/1813 [3:13:57<07:40,  6.88s/it]
+ 96%|█████████▋| 1747/1813 [3:14:02<06:51,  6.24s/it]
+ 96%|█████████▋| 1748/1813 [3:14:10<07:11,  6.63s/it]
+ 96%|█████████▋| 1749/1813 [3:14:15<06:43,  6.31s/it]
+ 97%|█████████▋| 1750/1813 [3:14:21<06:25,  6.12s/it]
+                                                     
+{'loss': '0.9197', 'grad_norm': '0.9061', 'learning_rate': '1.825e-07', 'epoch': '0.9655', 'num_input_tokens_seen': 11777768, 'train_runtime': '1.166e+04', 'train_tokens_per_second': '1010'}
+
+ 97%|█████████▋| 1750/1813 [3:14:21<06:25,  6.12s/it]
+ 97%|█████████▋| 1751/1813 [3:14:27<06:18,  6.11s/it]
+ 97%|█████████▋| 1752/1813 [3:14:33<06:12,  6.10s/it]
+ 97%|█████████▋| 1753/1813 [3:14:41<06:35,  6.59s/it]
+ 97%|█████████▋| 1754/1813 [3:14:48<06:42,  6.82s/it]
+ 97%|█████████▋| 1755/1813 [3:14:55<06:30,  6.73s/it]
+ 97%|█████████▋| 1756/1813 [3:15:00<06:02,  6.36s/it]
+ 97%|█████████▋| 1757/1813 [3:15:06<05:45,  6.17s/it]
+ 97%|█████████▋| 1758/1813 [3:15:12<05:34,  6.09s/it]
+ 97%|█████████▋| 1759/1813 [3:15:18<05:29,  6.10s/it]
+ 97%|█████████▋| 1760/1813 [3:15:24<05:16,  5.97s/it]
+                                                     
+{'loss': '0.9303', 'grad_norm': '1.212', 'learning_rate': '1.3e-07', 'epoch': '0.971', 'num_input_tokens_seen': 11843192, 'train_runtime': '1.172e+04', 'train_tokens_per_second': '1010'}
+
+ 97%|█████████▋| 1760/1813 [3:15:24<05:16,  5.97s/it]
+ 97%|█████████▋| 1761/1813 [3:15:29<05:05,  5.87s/it]
+ 97%|█████████▋| 1762/1813 [3:15:34<04:45,  5.60s/it]
+ 97%|█████████▋| 1763/1813 [3:15:42<05:11,  6.24s/it]
+ 97%|█████████▋| 1764/1813 [3:15:47<04:53,  5.98s/it]
+ 97%|█████████▋| 1765/1813 [3:15:53<04:40,  5.84s/it]
+ 97%|█████████▋| 1766/1813 [3:15:58<04:28,  5.72s/it]
+ 97%|█████████▋| 1767/1813 [3:16:04<04:23,  5.72s/it]
+ 98%|█████████▊| 1768/1813 [3:16:10<04:21,  5.81s/it]
+ 98%|█████████▊| 1769/1813 [3:16:16<04:17,  5.85s/it]
+ 98%|█████████▊| 1770/1813 [3:16:23<04:23,  6.12s/it]
+                                                     
+{'loss': '0.9433', 'grad_norm': '0.9712', 'learning_rate': '8.631e-08', 'epoch': '0.9766', 'num_input_tokens_seen': 11903728, 'train_runtime': '1.178e+04', 'train_tokens_per_second': '1010'}
+
+ 98%|█████████▊| 1770/1813 [3:16:23<04:23,  6.12s/it]
+ 98%|█████████▊| 1771/1813 [3:16:29<04:19,  6.17s/it]
+ 98%|█████████▊| 1772/1813 [3:16:35<04:14,  6.22s/it]
+ 98%|█████████▊| 1773/1813 [3:16:42<04:17,  6.45s/it]
+ 98%|█████████▊| 1774/1813 [3:16:47<03:49,  5.89s/it]
+ 98%|█████████▊| 1775/1813 [3:16:54<04:02,  6.39s/it]
+ 98%|█████████▊| 1776/1813 [3:17:02<04:06,  6.67s/it]
+ 98%|█████████▊| 1777/1813 [3:17:09<04:08,  6.90s/it]
+ 98%|█████████▊| 1778/1813 [3:17:15<03:46,  6.48s/it]
+ 98%|█████████▊| 1779/1813 [3:17:20<03:30,  6.18s/it]
+ 98%|█████████▊| 1780/1813 [3:17:26<03:20,  6.07s/it]
+                                                     
+{'loss': '0.9469', 'grad_norm': '0.8247', 'learning_rate': '5.155e-08', 'epoch': '0.9821', 'num_input_tokens_seen': 11972864, 'train_runtime': '1.185e+04', 'train_tokens_per_second': '1011'}
+
+ 98%|█████████▊| 1780/1813 [3:17:26<03:20,  6.07s/it]
+ 98%|█████████▊| 1781/1813 [3:17:32<03:15,  6.09s/it]
+ 98%|█████████▊| 1782/1813 [3:17:38<03:06,  6.01s/it]
+ 98%|█████████▊| 1783/1813 [3:17:44<02:59,  5.99s/it]
+ 98%|█████████▊| 1784/1813 [3:17:51<02:59,  6.21s/it]
+ 98%|█████████▊| 1785/1813 [3:17:56<02:51,  6.12s/it]
+ 99%|█████████▊| 1786/1813 [3:18:03<02:45,  6.14s/it]
+ 99%|█████████▊| 1787/1813 [3:18:08<02:33,  5.90s/it]
+ 99%|█████████▊| 1788/1813 [3:18:15<02:32,  6.10s/it]
+ 99%|█████████▊| 1789/1813 [3:18:23<02:43,  6.81s/it]
+ 99%|█████████▊| 1790/1813 [3:18:30<02:38,  6.90s/it]
+                                                     
+{'loss': '0.9332', 'grad_norm': '0.9594', 'learning_rate': '2.569e-08', 'epoch': '0.9876', 'num_input_tokens_seen': 12043496, 'train_runtime': '1.191e+04', 'train_tokens_per_second': '1011'}
+
+ 99%|█████████▊| 1790/1813 [3:18:30<02:38,  6.90s/it]
+ 99%|█████████▉| 1791/1813 [3:18:35<02:20,  6.40s/it]
+ 99%|█████████▉| 1792/1813 [3:18:41<02:12,  6.32s/it]
+ 99%|█████████▉| 1793/1813 [3:18:48<02:09,  6.46s/it]
+ 99%|█████████▉| 1794/1813 [3:18:55<02:06,  6.67s/it]
+ 99%|█████████▉| 1795/1813 [3:19:02<02:00,  6.68s/it]
+ 99%|█████████▉| 1796/1813 [3:19:09<01:53,  6.70s/it]
+ 99%|█████████▉| 1797/1813 [3:19:15<01:44,  6.56s/it]
+ 99%|█████████▉| 1798/1813 [3:19:24<01:50,  7.37s/it]
+ 99%|█████████▉| 1799/1813 [3:19:31<01:40,  7.15s/it]
+ 99%|█████████▉| 1800/1813 [3:19:36<01:22,  6.37s/it]
+                                                     
+{'loss': '0.9137', 'grad_norm': '0.9823', 'learning_rate': '8.743e-09', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448, 'train_runtime': '1.198e+04', 'train_tokens_per_second': '1012'}
+
+ 99%|█████████▉| 1800/1813 [3:19:36<01:22,  6.37s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.79it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:05,  7.88it/s]
+
+  1%|          | 12/1000 [00:01<02:03,  8.00it/s]
+
+  1%|▏         | 13/1000 [00:01<02:29,  6.61it/s]
+
+  1%|▏         | 14/1000 [00:01<02:31,  6.51it/s]
+
+  2%|▏         | 15/1000 [00:02<02:54,  5.64it/s]
+
+  2%|▏         | 16/1000 [00:02<03:00,  5.46it/s]
+
+  2%|▏         | 18/1000 [00:02<02:46,  5.91it/s]
+
+  2%|▏         | 19/1000 [00:02<03:15,  5.03it/s]
+
+  2%|▏         | 20/1000 [00:03<03:37,  4.50it/s]
+
+  2%|▏         | 22/1000 [00:03<02:37,  6.20it/s]
+
+  2%|▏         | 23/1000 [00:03<02:56,  5.55it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.35it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.35it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.09it/s]
+
+  4%|▍         | 39/1000 [00:05<02:07,  7.56it/s]
+
+  4%|▍         | 40/1000 [00:06<02:04,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.04it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.34it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.73it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.90it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.91it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.30it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:35,  9.76it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.86it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.20it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.77it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.83it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.33it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.45it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.20it/s]
+
+ 11%|█         | 112/1000 [00:14<01:14, 12.00it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.49it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.77it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.18it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.90it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:48,  8.07it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.08it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.69it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.07it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.92it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.05it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.41it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.44it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.56it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.49it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:22,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.89it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.10it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.73it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.18it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.67it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.78it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.64it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.94it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.69it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.73it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.14it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.96it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.96it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.36it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.46it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.73it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.74it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.71it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.99it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.06it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.90it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.84it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.07it/s]
+
+ 30%|███       | 303/1000 [00:40<01:20,  8.70it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.04it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.13it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.94it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.78it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.78it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.18it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.25it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.59it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.45it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:28,  7.66it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:59,  5.66it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:31,  7.36it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:34,  7.14it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.77it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.37it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.81it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.02it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.87it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.59it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.40it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:58,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:27,  7.20it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:24,  7.44it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:19,  7.89it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:42,  6.13it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:55,  5.42it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:30,  6.87it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:11,  8.72it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.88it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.09it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:07,  9.10it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:26,  7.10it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:20,  7.62it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.36it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.77it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.46it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:09,  8.73it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+
+ 40%|████      | 404/1000 [00:54<01:23,  7.15it/s]
+
+ 41%|████      | 406/1000 [00:54<01:10,  8.42it/s]
+
+ 41%|████      | 407/1000 [00:54<01:24,  7.04it/s]
+
+ 41%|████      | 408/1000 [00:55<01:36,  6.16it/s]
+
+ 41%|████      | 410/1000 [00:55<01:34,  6.22it/s]
+
+ 41%|████      | 411/1000 [00:55<01:45,  5.61it/s]
+
+ 41%|████      | 412/1000 [00:55<01:53,  5.16it/s]
+
+ 41%|████▏     | 414/1000 [00:56<01:23,  7.01it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:26,  6.77it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.28it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.47it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.51it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.68it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.92it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:16,  7.49it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:13,  7.80it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.52it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.00it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.77it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.52it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.09it/s]
+
+ 44%|████▍     | 441/1000 [00:59<00:59,  9.32it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:11,  7.83it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:18,  7.09it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:04,  8.56it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:14,  7.42it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:19,  6.95it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:29,  6.15it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.57it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.79it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.24it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.80it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.20it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.37it/s]
+
+ 47%|████▋     | 474/1000 [01:04<01:25,  6.18it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.19it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.30it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.09it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.51it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.04it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.29it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:22,  6.27it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.98it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.44it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:08<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.97it/s]
+
+ 51%|█████     | 512/1000 [01:09<00:48,  9.97it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.07it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.11it/s]
+
+ 53%|█████▎    | 533/1000 [01:12<00:48,  9.72it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.68it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.51it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:15<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.86it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.84it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.57it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.58it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.16it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.79it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.38it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:50,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.81it/s]
+
+ 62%|██████▏   | 616/1000 [01:24<00:50,  7.62it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.75it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.09it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.04it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.22it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:39,  9.40it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.94it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.95it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.57it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.65it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.70it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.98it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.92it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.87it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.23it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.49it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.68it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.84it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.35it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.26it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.15it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.62it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.53it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.54it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.78it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.35it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:28,  8.66it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.71it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.66it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.62it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.13it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.60it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  9.00it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.21it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.71it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:26,  7.95it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:24,  7.85it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.19it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.20it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.23it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.07it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.32it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.83it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.00it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.22it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.46it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:13, 10.22it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.63it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.72it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.26it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.27it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.79it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.56it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:14,  8.64it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.77it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.52it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.48it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.89it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.11it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.31it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.77it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.59it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.83it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.07it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.06it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.33it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.78it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.66it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.77it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.58it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.16it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.82it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.42it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.41it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.19it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.80it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.84it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.44it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.37it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7742', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.382', 'eval_steps_per_second': '7.382', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448}
+
+ 99%|█████████▉| 1800/1813 [3:21:51<01:22,  6.37s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_001_lr_5.000e-05/checkpoint-1800
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+ 99%|█████████▉| 1801/1813 [3:22:00<09:33, 47.77s/it]
+ 99%|█████████▉| 1802/1813 [3:22:05<06:26, 35.10s/it]
+ 99%|█████████▉| 1803/1813 [3:22:11<04:22, 26.30s/it]
+100%|█████████▉| 1804/1813 [3:22:19<03:06, 20.68s/it]
+100%|█████████▉| 1805/1813 [3:22:25<02:10, 16.26s/it]
+100%|█████████▉| 1806/1813 [3:22:31<01:33, 13.29s/it]
+100%|█████████▉| 1807/1813 [3:22:38<01:08, 11.40s/it]
+100%|█████████▉| 1808/1813 [3:22:46<00:51, 10.20s/it]
+100%|█████████▉| 1809/1813 [3:22:52<00:36,  9.02s/it]
+100%|█████████▉| 1810/1813 [3:22:58<00:24,  8.03s/it]
+                                                     
+{'loss': '0.8849', 'grad_norm': '1.105', 'learning_rate': '7.137e-10', 'epoch': '0.9986', 'num_input_tokens_seen': 12187432, 'train_runtime': '1.218e+04', 'train_tokens_per_second': '1001'}
+
+100%|█████████▉| 1810/1813 [3:22:58<00:24,  8.03s/it]
+100%|█████████▉| 1811/1813 [3:23:03<00:14,  7.30s/it]
+100%|█████████▉| 1812/1813 [3:23:08<00:06,  6.67s/it]
+100%|██████████| 1813/1813 [3:23:11<00:00,  5.56s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.81it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.23it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.34it/s]
+
+  1%|          | 9/1000 [00:01<02:32,  6.48it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.09it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.00it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.27it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  6.00it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  6.00it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.77it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.68it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.12it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.34it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.75it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.74it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.30it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.88it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.22it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.03it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.18it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.51it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.61it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.30it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.44it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.20it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.03it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.51it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.82it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.23it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.09it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.44it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.07it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.48it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.90it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.01it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.38it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.60it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.15it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:22,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.73it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.87it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.76it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.21it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.64it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.75it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.63it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.93it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.67it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.73it/s]
+
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.51it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.81it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.36it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.95it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.47it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.36it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.58it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.89it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.52it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.73it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.84it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.11it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.15it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.96it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.80it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.19it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.26it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.58it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.87it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:31,  7.39it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.16it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.05it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.83it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.87it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.59it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.21it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.56it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.60it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.43it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.18it/s]
+
+ 38%|███▊      | 379/1000 [00:50<01:16,  8.16it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.19it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.56it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.07it/s]
+
+ 39%|███▊      | 387/1000 [00:51<01:21,  7.56it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.30it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.85it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.49it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  9.01it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.52it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.84it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.52it/s]
+
+ 41%|████      | 408/1000 [00:54<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.92it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.54it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.02it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.80it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.52it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.09it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.31it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.24it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.19it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.82it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.58it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.32it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.42it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.55it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.97it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.51it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.09it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.06it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.01it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:12,  6.64it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.79it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.57it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.13it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.68it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.67it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.35it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.87it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.07it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.43it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.59it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.76it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.19it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.79it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.09it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:50,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.46it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.72it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.79it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.62it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.75it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.08it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.02it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.20it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.63it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.18it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.44it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.51it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.55it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.31it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.69it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.91it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:43,  7.94it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:57,  5.94it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:45,  7.50it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.45it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.75it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:03,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:04,  5.17it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.67it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:56,  5.87it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.62it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.25it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.64it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.98it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.88it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.83it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.25it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:48,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.72it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.36it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.14it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.21it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.29it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.12it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.60it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.31it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.50it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.87it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.67it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.24it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.62it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.33it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.72it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.67it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.62it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.78it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.13it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.59it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.94it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.52it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.00it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:26,  7.96it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:24,  7.84it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.96it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.07it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:17, 11.11it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.00it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.29it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.12it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.18it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.83it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.03it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.76it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.56it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.46it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.04it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.21it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.62it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.69it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.26it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.27it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.34it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.75it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.74it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.76it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.51it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.68it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.83it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.03it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.32it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.16it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.64it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.09it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.81it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.60it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.16it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.29it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.82it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.03it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.13it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.07it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.70it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.38it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.15it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.39it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.44it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.33it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.41it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.80it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.44it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.75it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7741', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.387', 'eval_steps_per_second': '7.387', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:25:27<00:00,  5.56s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_001_lr_5.000e-05/checkpoint-1813
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+
+Training completed. Do not forget to share your model on huggingface.co/models =)
+
+
+
+                                                     
+{'train_runtime': '1.233e+04', 'train_samples_per_second': '2.352', 'train_steps_per_second': '0.147', 'train_loss': '0.9868', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:25:28<00:00,  5.56s/it]
+100%|██████████| 1813/1813 [3:25:28<00:00,  6.80s/it]
+Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_001_lr_5.000e-05
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+***** train metrics *****
+  epoch                    =         1.0
+  num_input_tokens_seen    =    12201816
+  total_flos               = 487685654GF
+  train_loss               =      0.9868
+  train_runtime            =  3:25:28.08
+  train_samples_per_second =       2.352
+  train_steps_per_second   =       0.147
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_001_lr_5.000e-05/training_loss.png
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_001_lr_5.000e-05/training_eval_loss.png
+[WARNING|2026-10-13 02:54:00] llamafactory.extras.ploting:149 >> No metric eval_accuracy to plot.
+
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+  0%|          | 3/1000 [00:00<01:03, 15.81it/s]
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+  1%|          | 7/1000 [00:00<01:59,  8.34it/s]
+  1%|          | 9/1000 [00:01<02:32,  6.48it/s]
+  1%|          | 11/1000 [00:01<02:02,  8.10it/s]
+  1%|▏         | 13/1000 [00:01<02:20,  7.02it/s]
+  1%|▏         | 14/1000 [00:01<02:23,  6.85it/s]
+  2%|▏         | 15/1000 [00:02<02:44,  5.98it/s]
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+  2%|▏         | 18/1000 [00:02<02:42,  6.06it/s]
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+  2%|▏         | 20/1000 [00:03<03:32,  4.60it/s]
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+  2%|▎         | 25/1000 [00:03<02:12,  7.37it/s]
+  3%|▎         | 26/1000 [00:03<02:35,  6.27it/s]
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+  4%|▎         | 37/1000 [00:05<01:45,  9.12it/s]
+  4%|▍         | 39/1000 [00:05<02:06,  7.59it/s]
+  4%|▍         | 40/1000 [00:05<02:03,  7.76it/s]
+  4%|▍         | 42/1000 [00:06<01:44,  9.13it/s]
+  4%|▍         | 44/1000 [00:06<01:32, 10.35it/s]
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+  5%|▍         | 47/1000 [00:06<02:09,  7.38it/s]
+  5%|▍         | 49/1000 [00:06<01:48,  8.76it/s]
+  5%|▌         | 51/1000 [00:07<01:35,  9.93it/s]
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+  6%|▌         | 55/1000 [00:07<01:27, 10.74it/s]
+  6%|▌         | 57/1000 [00:07<01:41,  9.29it/s]
+  6%|▌         | 59/1000 [00:08<01:54,  8.21it/s]
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+  7%|▋         | 67/1000 [00:08<02:02,  7.60it/s]
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+  7%|▋         | 74/1000 [00:09<01:53,  8.13it/s]
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+  8%|▊         | 80/1000 [00:10<01:57,  7.86it/s]
+  8%|▊         | 82/1000 [00:10<01:39,  9.20it/s]
+  8%|▊         | 84/1000 [00:11<01:53,  8.04it/s]
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+  9%|▊         | 87/1000 [00:11<02:49,  5.38it/s]
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+  9%|▉         | 90/1000 [00:12<02:05,  7.26it/s]
+  9%|▉         | 91/1000 [00:12<02:29,  6.09it/s]
+  9%|▉         | 92/1000 [00:12<02:26,  6.18it/s]
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+ 10%|▉         | 97/1000 [00:13<02:12,  6.79it/s]
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+ 10%|█         | 101/1000 [00:13<01:33,  9.64it/s]
+ 10%|█         | 103/1000 [00:14<01:55,  7.76it/s]
+ 10%|█         | 104/1000 [00:14<01:54,  7.84it/s]
+ 11%|█         | 106/1000 [00:14<01:35,  9.32it/s]
+ 11%|█         | 108/1000 [00:14<01:25, 10.44it/s]
+ 11%|█         | 110/1000 [00:14<01:23, 10.70it/s]
+ 11%|█         | 112/1000 [00:14<01:16, 11.65it/s]
+ 11%|█▏        | 114/1000 [00:14<01:12, 12.23it/s]
+ 12%|█▏        | 116/1000 [00:15<01:10, 12.59it/s]
+ 12%|█▏        | 118/1000 [00:15<01:07, 13.09it/s]
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.89it/s]
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.06it/s]
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.72it/s]
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.11it/s]
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.14it/s]
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.89it/s]
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.03it/s]
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.07it/s]
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.91it/s]
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.05it/s]
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.39it/s]
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.96it/s]
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.60it/s]
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.18it/s]
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.85it/s]
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.23it/s]
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.46it/s]
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.80it/s]
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.51it/s]
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+ 18%|█▊        | 182/1000 [00:24<02:22,  5.73it/s]
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.12it/s]
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.64it/s]
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.20it/s]
+ 20%|██        | 200/1000 [00:26<01:14, 10.69it/s]
+ 20%|██        | 202/1000 [00:26<01:30,  8.79it/s]
+ 20%|██        | 204/1000 [00:26<01:31,  8.65it/s]
+ 21%|██        | 206/1000 [00:26<01:19,  9.95it/s]
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+ 21%|██        | 212/1000 [00:27<01:21,  9.69it/s]
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.75it/s]
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.16it/s]
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.49it/s]
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.79it/s]
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.31it/s]
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.43it/s]
+ 23%|██▎       | 228/1000 [00:30<01:52,  6.84it/s]
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.20it/s]
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.95it/s]
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.00it/s]
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.46it/s]
+ 24%|██▍       | 245/1000 [00:32<01:54,  6.59it/s]
+ 25%|██▍       | 247/1000 [00:32<01:32,  8.18it/s]
+ 25%|██▍       | 248/1000 [00:32<01:50,  6.81it/s]
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.93it/s]
+ 25%|██▌       | 250/1000 [00:33<02:08,  5.85it/s]
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.73it/s]
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.35it/s]
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.74it/s]
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.70it/s]
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.02it/s]
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.06it/s]
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.82it/s]
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.44it/s]
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.73it/s]
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+ 30%|██▉       | 298/1000 [00:39<01:26,  8.16it/s]
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+ 30%|███       | 300/1000 [00:40<01:42,  6.84it/s]
+ 30%|███       | 301/1000 [00:40<01:38,  7.11it/s]
+ 30%|███       | 303/1000 [00:40<01:19,  8.75it/s]
+ 30%|███       | 304/1000 [00:40<01:26,  8.07it/s]
+ 30%|███       | 305/1000 [00:40<01:25,  8.13it/s]
+ 31%|███       | 306/1000 [00:41<01:40,  6.93it/s]
+ 31%|███       | 307/1000 [00:41<01:42,  6.77it/s]
+ 31%|███       | 309/1000 [00:41<01:18,  8.78it/s]
+ 31%|███       | 311/1000 [00:41<01:07, 10.21it/s]
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.27it/s]
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.24it/s]
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.59it/s]
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.42it/s]
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.18it/s]
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.59it/s]
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.39it/s]
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.05it/s]
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.83it/s]
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.04it/s]
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.62it/s]
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.73it/s]
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.58it/s]
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.42it/s]
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.92it/s]
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.21it/s]
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.93it/s]
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.78it/s]
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.21it/s]
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.18it/s]
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.64it/s]
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.36it/s]
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.78it/s]
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.47it/s]
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+ 41%|████      | 408/1000 [00:55<01:32,  6.40it/s]
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+ 41%|████      | 411/1000 [00:55<01:41,  5.83it/s]
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.06it/s]
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.27it/s]
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.46it/s]
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.83it/s]
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.55it/s]
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.03it/s]
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.80it/s]
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.52it/s]
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.09it/s]
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.31it/s]
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.51it/s]
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.27it/s]
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.84it/s]
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.50it/s]
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.40it/s]
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.58it/s]
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.32it/s]
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+ 49%|████▉     | 488/1000 [01:05<01:04,  8.00it/s]
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.29it/s]
+ 49%|████▉     | 494/1000 [01:06<00:50, 10.00it/s]
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.44it/s]
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+ 50%|█████     | 503/1000 [01:07<01:13,  6.79it/s]
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+ 51%|█████     | 512/1000 [01:08<00:48,  9.97it/s]
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.38it/s]
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.71it/s]
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.91it/s]
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.86it/s]
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.46it/s]
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.82it/s]
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.24it/s]
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.55it/s]
+ 58%|█████▊    | 584/1000 [01:19<00:55,  7.56it/s]
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.89it/s]
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.14it/s]
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.51it/s]
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.74it/s]
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+ 60%|██████    | 601/1000 [01:22<01:02,  6.40it/s]
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+ 61%|██████    | 608/1000 [01:22<00:49,  7.86it/s]
+ 61%|██████    | 610/1000 [01:23<00:45,  8.48it/s]
+ 61%|██████    | 612/1000 [01:23<00:39,  9.74it/s]
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.81it/s]
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.62it/s]
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.75it/s]
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.09it/s]
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.20it/s]
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.45it/s]
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.44it/s]
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.08it/s]
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.96it/s]
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.19it/s]
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.97it/s]
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.60it/s]
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.69it/s]
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.90it/s]
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.09it/s]
+ 66%|██████▌   | 659/1000 [01:29<00:56,  5.99it/s]
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.55it/s]
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.47it/s]
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.64it/s]
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.65it/s]
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.96it/s]
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.93it/s]
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.89it/s]
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.83it/s]
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.25it/s]
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.41it/s]
+ 70%|██████▉   | 696/1000 [01:34<00:48,  6.33it/s]
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.93it/s]
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+ 70%|███████   | 704/1000 [01:36<00:40,  7.23it/s]
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+ 71%|███████   | 709/1000 [01:36<00:38,  7.61it/s]
+ 71%|███████   | 710/1000 [01:36<00:44,  6.51it/s]
+ 71%|███████   | 712/1000 [01:37<00:37,  7.71it/s]
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.37it/s]
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.81it/s]
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.12it/s]
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.59it/s]
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.31it/s]
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.51it/s]
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.87it/s]
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.51it/s]
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.61it/s]
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.69it/s]
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.25it/s]
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.39it/s]
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.59it/s]
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.31it/s]
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.71it/s]
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.47it/s]
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.38it/s]
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.77it/s]
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.61it/s]
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.96it/s]
+ 78%|███████▊  | 782/1000 [01:45<00:26,  8.38it/s]
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.00it/s]
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.74it/s]
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.98it/s]
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+ 80%|████████  | 804/1000 [01:48<00:25,  7.84it/s]
+ 81%|████████  | 806/1000 [01:49<00:22,  8.72it/s]
+ 81%|████████  | 807/1000 [01:49<00:22,  8.66it/s]
+ 81%|████████  | 809/1000 [01:49<00:19,  9.96it/s]
+ 81%|████████  | 811/1000 [01:49<00:17, 11.03it/s]
+ 81%|████████▏ | 813/1000 [01:49<00:20,  8.99it/s]
+ 81%|████████▏ | 814/1000 [01:49<00:21,  8.81it/s]
+ 82%|████████▏ | 816/1000 [01:50<00:18,  9.83it/s]
+ 82%|████████▏ | 818/1000 [01:50<00:21,  8.31it/s]
+ 82%|████████▏ | 820/1000 [01:50<00:23,  7.52it/s]
+ 82%|████████▏ | 822/1000 [01:50<00:20,  8.67it/s]
+ 82%|████████▏ | 823/1000 [01:51<00:23,  7.69it/s]
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.53it/s]
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.85it/s]
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.62it/s]
+ 83%|████████▎ | 828/1000 [01:52<00:33,  5.13it/s]
+ 83%|████████▎ | 829/1000 [01:52<00:30,  5.66it/s]
+ 83%|████████▎ | 831/1000 [01:52<00:24,  6.98it/s]
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.71it/s]
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.20it/s]
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.00it/s]
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.52it/s]
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.10it/s]
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.54it/s]
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.85it/s]
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.46it/s]
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.04it/s]
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.64it/s]
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.73it/s]
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.30it/s]
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.26it/s]
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.28it/s]
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.88it/s]
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.95it/s]
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.16it/s]
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.78it/s]
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.77it/s]
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.83it/s]
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.53it/s]
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.69it/s]
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.05it/s]
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.32it/s]
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.59it/s]
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.61it/s]
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.11it/s]
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.84it/s]
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.38it/s]
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.59it/s]
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.72it/s]
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.37it/s]
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.14it/s]
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.37it/s]
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.39it/s]
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.78it/s]
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.02it/s]
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.48it/s]
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.43it/s]
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.33it/s]
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.00it/s]
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.40it/s]
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.18it/s]
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.78it/s]
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.79it/s]
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.43it/s]
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.14it/s]
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+100%|██████████| 1000/1000 [02:15<00:00,  7.40it/s]
+***** eval metrics *****
+  epoch                   =        1.0
+  eval_loss               =     0.7741
+  eval_runtime            = 0:02:15.36
+  eval_samples_per_second =      7.387
+  eval_steps_per_second   =      7.387
+  num_input_tokens_seen   =   12201816
+Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
diff --git a/optuna_30k_experiments/logs/trial_002_lr_1.000e-04.log b/optuna_30k_experiments/logs/trial_002_lr_1.000e-04.log
new file mode 100644
index 0000000..675e3c5
--- /dev/null
+++ b/optuna_30k_experiments/logs/trial_002_lr_1.000e-04.log
@@ -0,0 +1,12418 @@
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/torch/cuda/__init__.py:734: UserWarning: Can't initialize NVML
+  warnings.warn("Can't initialize NVML")
+[INFO|2026-10-13 02:56:22] llamafactory.hparams.parser:651 >> Process rank: 0, world size: 1, device: cuda:0, distributed training: False, compute dtype: torch.bfloat16
+[INFO|configuration_utils.py:765] 2026-10-13 02:56:22,232 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-13 02:56:22,241 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
+[INFO|configuration_utils.py:765] 2026-10-13 02:56:23,843 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-13 02:56:23,846 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|configuration_utils.py:765] 2026-10-13 02:56:23,991 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-13 02:56:23,995 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-13 02:56:24] llamafactory.data.loader:144 >> Loading dataset aya_slovak_hpo_30k.jsonl...
+training example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28828, 2176, 28711, 28863, 29114, 10268, 28709, 6612, 28714, 2892, 493, 363, 6294, 10719, 363, 3811, 28723, 7379, 626, 28889, 307, 14637, 311, 28768, 2218, 22885, 3748, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 284, 8341, 6729, 345, 28738, 28739, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28723, 524, 4634, 28711, 570, 1879, 261, 28863, 532, 613, 14483, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 345, 15962, 28739, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 1879, 15173, 537, 284, 8341, 705, 28723, 367, 280, 28879, 570, 4415, 28716, 28797, 9335, 3175, 29826, 626, 323, 412, 8514, 16986, 29826, 28708, 345, 15962, 28739, 1879, 15173, 537, 1221, 4566, 28718, 325, 28707, 28723, 461, 28723, 724, 494, 8891, 24160, 28725, 8582, 28862, 24160, 28725, 20532, 28764, 271, 28725, 438, 29365, 15745, 1319, 12035, 28862, 8269, 284, 28827, 2762, 28709, 264, 4798, 8359, 1450, 8325, 12121, 1798, 420, 762, 1089, 290, 28906, 18765, 724, 2331, 29114, 1221, 4304, 1780, 363, 13881, 29826, 316, 2085, 28708, 28723, 22627, 28873, 263, 335, 20208, 284, 28827, 2762, 28709, 264, 4798, 5424, 10056, 1286, 290, 28906, 18765, 724, 2331, 29114, 11323, 10719, 12490, 363, 13881, 29826, 316, 284, 8341, 2390, 28723, 22127, 280, 28879, 21022, 1150, 424, 724, 28827, 1144, 28873, 340, 17066, 270, 21022, 485, 284, 8341, 705, 28725, 446, 6909, 28797, 637, 28705, 29114, 28708, 28879, 4798, 6612, 28827, 1100, 28768, 28863, 28723, 320, 412, 28879, 637, 15612, 28716, 28889, 28726, 1150, 424, 19523, 412, 28873, 276, 28797, 2390, 284, 8341, 2390, 363, 23407, 11277, 9538, 2085, 28708, 28723, 9532, 2144, 686, 7865, 28718, 341, 14475, 1687, 28706, 18037, 452, 270, 8269, 284, 8341, 705, 28725, 290, 4827, 486, 2710, 2144, 332, 1800, 16344, 29114, 6588, 269, 28862, 28711, 28863, 686, 1653, 5388, 28718, 710, 261, 1296, 28709, 284, 8341, 705, 28723, 351, 28711, 1371, 28797, 1776, 29826, 8269, 284, 8341, 705, 268, 28863, 1776, 29826, 8269, 1879, 3497, 598, 8269, 8631, 28879, 279, 412, 28725, 12121, 7988, 268, 28863, 24270, 12115, 28797, 1879, 5626, 263, 28862, 8269, 8631, 28879, 279, 412, 28723, 318, 15612, 1473, 276, 15690, 2245, 626, 15690, 307, 14637, 311, 28768, 300, 15469, 28711, 570, 446, 450, 4798, 29826, 333, 28729, 1879, 319, 3976, 294, 405, 28768, 4882, 405, 28725, 534, 28724, 2710, 724, 313, 4827, 2245, 626, 28889, 20532, 28764, 271, 28723, 10650, 300, 12235, 28827, 28873, 424, 268, 27172, 2245, 28723, 1054, 15504, 28879, 424, 724, 313, 8768, 4545, 504, 270, 8269, 2124, 2245, 626, 12689, 4584, 10906, 28718, 710, 18782, 28879, 28715, 28797, 1135, 10356, 28723, 2645, 28764, 12275, 28820, 28873, 29114, 13067, 549, 2892, 5872, 3811, 8631, 25991, 3026, 29114, 408, 28906, 28764, 485, 2082, 1403, 12121, 1798, 1773, 29826, 28729, 22160, 710, 408, 28906, 28764, 485, 1135, 10874, 1776, 4536, 28873, 300, 23407, 11277, 9538, 2085, 28708, 28723, 22127, 3361, 13470, 1150, 317, 28873, 424, 2082, 7330, 511, 2085, 28708, 28723, 401, 492, 13591, 28820, 261, 2220, 28862, 10766, 381, 28827, 486, 29114, 363, 28879, 4689, 3418, 28879, 21775, 29114, 280, 28723, 418, 28820, 11277, 28716, 6294, 26177, 903, 12689, 6612, 20122, 28733, 12868, 291, 2124, 2085, 28708, 1879, 6670, 28862, 9348, 493, 686, 375, 5620, 386, 28862, 28827, 28725, 23374, 4536, 25306, 16388, 957, 450, 15612, 28764, 2126, 29114, 511, 2152, 363, 25769, 14110, 17413, 26177, 903, 28873, 15808, 19396, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 7379, 1879, 277, 7939, 1879, 724, 313, 8768, 2245, 28718, 363, 446, 551, 9538, 12121, 1798, 24473, 21694, 300, 261, 5869, 28723, 384, 28728, 491, 28712, 4566, 15469, 28711, 570, 1879, 2245, 28725, 534, 28724, 2710, 7463, 15612, 13335, 28728, 4371, 1144, 28723, 10650, 300, 15469, 28711, 570, 1879, 1160, 28709, 28879, 3907, 13288, 491, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 28725, 534, 28724, 637, 686, 598, 5237, 5945, 17984, 28718, 2117, 18904, 3146, 515, 13288, 25208, 28723, 367, 280, 28879, 570, 12472, 1006, 5872, 28863, 358, 446, 311, 4845, 1879, 6149, 783, 494, 28718, 2245, 28718, 363, 17984, 28718, 405, 13288, 491, 28723, 13343, 29826, 28729, 385, 29114, 284, 8341, 705, 28747, 399, 3711, 3175, 29826, 626, 12990, 17984, 28718, 2117, 268, 6125, 15690, 345, 28738, 28739, 16986, 29826, 28708, 1773, 29826, 10641, 2124, 345, 28738, 28739, 582, 20524, 12214, 1773, 29826, 28729, 385, 29114, 284, 8341, 705, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 15173, 537, 28718, 325, 849, 3748, 28725, 446, 6909, 28820, 3061, 28720, 300, 28827, 1520, 6612, 749, 1780, 20532, 28764, 271, 586, 28873, 28710, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28731, 1879, 686, 6729, 1773, 29826, 28729, 22160, 284, 8341, 705, 28723, 550, 11245, 412, 28747, 550, 11245, 412, 1683, 28827, 16032, 271, 1342, 4295, 3191, 316, 28729, 6449, 2245, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 345, 28741, 28739, 1879, 22954, 338, 28718, 297, 12689, 345, 28741, 28739, 686, 2129, 28710, 29114, 16986, 28863, 358, 28723, 17563, 971, 28747, 17563, 971, 582, 20524, 12214, 1342, 28764, 1193, 1342, 4295, 5765, 6175, 2959, 15416, 28710, 13288, 491, 3589, 28723, 9532, 484, 28717, 3811, 18904, 17101, 29114, 22399, 971, 28725, 15612, 13335, 28728, 4371, 28711, 570, 13288, 20996, 28725, 446, 6909, 28797, 484, 28717, 3811, 18904, 17101, 29114, 28725, 264, 2513, 300, 8631, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 268, 345, 28741, 28739, 264, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Zhrnúť tento článok v jednej vete. Textový nástroj je ikona, ktorá sa podobá písmenu "T" v paneli nástrojov vľavo. Kliknite na túto ikonu. Použite rozbaľovaciu ponuku vedľa "Character" v hornej časti obrazovky na výber písma. Použite druhé rozbaľovacie menu vedľa "Character" na výber formátu (t. j. pravidelný, tučný, kurzor, atď.). Tradičné písmo ako Times New Roman alebo Garamond môže pridať formálny vzhľad loga. Bezšerifové písmo ako Helvetica môže pridať modernejší vzhľad písmu. Nepoužívajte príliš dekoratívne písma, ktoré sa ťažko čítajú. Tiež sa vyhýbajte zmiešanému písmu v návrhu loga. Ak si z internetu stiahnete bezplatné písma, mali by ste si uvedomiť licenčnú zmluvu pre tieto písma. Mnohé voľné písma sú voľné na osobné použitie, ale nie sú povolené na komerčné použitie. S vybraným textovým nástrojom kliknite kdekoľvek na grafickej doske, aby ste pridali textový kurzor. Potom napíšte svoj text. Zvážte pridanie samostatného textového objektu pre každé slovo. Obzvlášť ak plánujete používať rôzne farby alebo veľkosti pre rôzne slová vo vašom návrhu loga. Nepridávaj ešte farbu do loga. Farebná tlač nemusí byť vždy možnosťou. Návrh jednoduchého čierno-bieleho loga na začiatok zabezpečí, že vaše logo bude vyzerať dobre v jeho najjednoduchšej forme. Môžete tiež použiť nástroj Text na ceste na pridanie textu v kruhu alebo zakrivenom tvare. Dvakrát kliknite na text, aby ste ho vyzdvihli. Potom kliknite na položku znak v hornej časti obrazovky, aby sa zobrazila ponuka nastavenia znakov. Použite nasledujúce kroky na úpravu textu v ponuke znak. Veľkosť písma: Rozbaľovacia ponuka s malým "T" vedľa veľkého "T" upravuje veľkosť písma. Môžete tiež použiť nástroj výberu (ikona, ktorá pripomína čierny kurzor myši v paneli nástrojov vľavo) na zmenu veľkosti písma. Vedenie: Vedenie mení priestor medzi riadkami textu. Použite rozbaľovaciu ponuku vedľa ikony, ktorá sa podobá "A" na vrchu iného "A" zmeniť vedúce. Kerning: Kerning upravuje medzery medzi konkrétnymi znakmi. Ak chcete nastaviť kerning, vyzdvihnite znaky, ktoré chcete nastaviť, a potom použite rozbaľovaciu ponuku vedľa ikony s "A" aVyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+labels:
+Vyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+eval example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28965, 28709, 637, 8631, 25991, 3026, 28725, 13067, 10531, 28729, 350, 2591, 686, 3023, 28862, 28827, 284, 28873, 269, 13096, 28711, 28863, 6149, 3399, 28804, 550, 13881, 29826, 316, 300, 1879, 7871, 338, 28820, 6936, 1150, 28863, 7340, 6957, 8905, 3907, 12235, 28827, 28873, 424, 5765, 772, 28725, 446, 6909, 28889, 14848, 912, 12214, 4321, 28720, 751, 29365, 28723, 351, 28906, 18765, 298, 486, 29114, 28705, 28740, 387, 28705, 28750, 28734, 22116, 28827, 28723, 8294, 772, 28747, 13, 13, 27332, 12107, 28747, 13, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Čo sa používa, ak škodca zničí pšeničnú úrodu? Vzhľadom na predchádzajúcu otázku napíšte kontext, ktorý obsahuje odpoveď. Môže to byť 1 - 20 vetí. Kontext:
+
+### Response:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+labels:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+[INFO|configuration_utils.py:765] 2026-10-13 02:56:25,469 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-13 02:56:25,473 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-13 02:56:25] llamafactory.model.model_utils.kv_cache:144 >> KV cache is disabled during training.
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/unsloth/__init__.py:165: UserWarning: WARNING: Unsloth should be imported before [trl, transformers, peft] to ensure all optimizations are applied. Your code may run slower or encounter memory issues without these optimizations.
+
+Please restructure your imports with 'import unsloth' at the top of your file.
+  from ._gpu_init import *
+🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/torch/cuda/__init__.py:734: UserWarning: Can't initialize NVML
+  warnings.warn("Can't initialize NVML")
+🦥 Unsloth Zoo will now patch everything to make training faster!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: WARNING `trust_remote_code` is True.
+Are you certain you want to do remote code execution?
+==((====))==  Unsloth 2026.6.1: Fast Mistral patching. Transformers: 5.5.0.
+   \\   /|    NVIDIA RTX A4500. Num GPUs = 1. Max memory: 19.698 GB. Platform: Linux.
+O^O/ \_/ \    Torch: 2.6.0+cu124. CUDA: 8.6. CUDA Toolkit: 12.4. Triton: 3.2.0
+\        /    Bfloat16 = TRUE. FA [Xformers = 0.0.29.post3. FA2 = False]
+ "-____-"     Free license: http://github.com/unslothai/unsloth
+Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading weights file model.safetensors from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/model.safetensors.index.json
+Will use dtype=torch.bfloat16 as defined in model's config object
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 2,
+  "use_cache": true
+}
+
+
+Loading weights:   0%|          | 0/291 [00:00<?, ?it/s]
+Loading weights:   2%|▏         | 5/291 [00:00<00:06, 43.02it/s]
+Loading weights:   6%|▌         | 18/291 [00:00<00:03, 87.41it/s]
+Loading weights:  11%|█         | 31/291 [00:00<00:02, 100.85it/s]
+Loading weights:  14%|█▍        | 42/291 [00:00<00:02, 95.56it/s] 
+Loading weights:  19%|█▉        | 56/291 [00:00<00:02, 109.46it/s]
+Loading weights:  23%|██▎       | 68/291 [00:00<00:02, 101.69it/s]
+Loading weights:  29%|██▉       | 85/291 [00:00<00:01, 114.14it/s]
+Loading weights:  33%|███▎      | 97/291 [00:00<00:01, 103.52it/s]
+Loading weights:  37%|███▋      | 108/291 [00:01<00:01, 105.19it/s]
+Loading weights:  41%|████      | 119/291 [00:01<00:01, 104.64it/s]
+Loading weights:  45%|████▌     | 131/291 [00:01<00:01, 104.75it/s]
+Loading weights:  49%|████▉     | 142/291 [00:01<00:01, 104.44it/s]
+Loading weights:  54%|█████▍    | 158/291 [00:01<00:01, 118.00it/s]
+Loading weights:  58%|█████▊    | 170/291 [00:01<00:01, 113.59it/s]
+Loading weights:  63%|██████▎   | 182/291 [00:01<00:00, 112.12it/s]
+Loading weights:  67%|██████▋   | 194/291 [00:01<00:00, 106.45it/s]
+Loading weights:  70%|███████   | 205/291 [00:01<00:00, 99.80it/s] 
+Loading weights:  74%|███████▍  | 216/291 [00:02<00:00, 100.64it/s]
+Loading weights:  79%|███████▊  | 229/291 [00:02<00:00, 102.21it/s]
+Loading weights:  82%|████████▏ | 240/291 [00:02<00:00, 93.54it/s] 
+Loading weights:  87%|████████▋ | 254/291 [00:02<00:00, 105.02it/s]
+Loading weights:  91%|█████████ | 265/291 [00:02<00:00, 101.77it/s]
+Loading weights:  95%|█████████▍| 276/291 [00:02<00:00, 90.57it/s] 
+Loading weights:  99%|█████████▉| 288/291 [00:02<00:00, 96.37it/s]
+Loading weights: 100%|██████████| 291/291 [00:02<00:00, 102.62it/s]
+loading configuration file generation_config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/generation_config.json
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2
+}
+
+Could not locate the custom_generate/generate.py inside slovak-nlp/mistral-sk-7b.
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: slovak-nlp/mistral-sk-7b had a bad pad_token (</s>). Using pad_token = <unk>.
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:68: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_fwd
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:85: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_bwd
+[INFO|2026-10-13 02:56:42] llamafactory.model.model_utils.checkpointing:144 >> Gradient checkpointing enabled.
+[INFO|2026-10-13 02:56:42] llamafactory.model.adapter:144 >> Upcasting trainable params to float32.
+[INFO|2026-10-13 02:56:42] llamafactory.model.adapter:144 >> Fine-tuning method: LoRA
+Unsloth 2026.6.1 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers.
+[INFO|2026-10-13 02:56:45] llamafactory.model.loader:144 >> trainable params: 41,943,040 || all params: 7,283,675,136 || trainable%: 0.5758
+The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'pad_token_id': 2}.
+==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 1
+   \\   /|    Num examples = 29,000 | Num Epochs = 1 | Total steps = 1,813
+O^O/ \_/ \    Batch size per device = 1 | Gradient accumulation steps = 16
+\        /    Data Parallel GPUs = 1 | Total batch size (1 x 16 x 1) = 16
+ "-____-"     Trainable parameters = 41,943,040 of 7,283,675,136 (0.58% trained)
+
+  0%|          | 0/1813 [00:00<?, ?it/s]`use_return_dict` is deprecated! Use `return_dict` instead!
+Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
+Unsloth: Will smartly offload gradients to save VRAM!
+
+  0%|          | 1/1813 [00:07<3:36:20,  7.16s/it]
+  0%|          | 2/1813 [00:13<3:25:26,  6.81s/it]
+  0%|          | 3/1813 [00:19<3:05:06,  6.14s/it]
+  0%|          | 4/1813 [00:23<2:40:13,  5.31s/it]
+  0%|          | 5/1813 [00:29<2:53:22,  5.75s/it]
+  0%|          | 6/1813 [00:37<3:11:14,  6.35s/it]
+  0%|          | 7/1813 [00:43<3:14:43,  6.47s/it]
+  0%|          | 8/1813 [00:49<3:04:56,  6.15s/it]
+  0%|          | 9/1813 [00:57<3:19:45,  6.64s/it]
+  1%|          | 10/1813 [01:02<3:10:40,  6.35s/it]
+                                                   
+{'loss': '1.797', 'grad_norm': '2.267', 'learning_rate': '6e-06', 'epoch': '0.005517', 'num_input_tokens_seen': 65928, 'train_runtime': '62.74', 'train_tokens_per_second': '1051'}
+
+  1%|          | 10/1813 [01:02<3:10:40,  6.35s/it]
+  1%|          | 11/1813 [01:09<3:11:15,  6.37s/it]
+  1%|          | 12/1813 [01:17<3:25:28,  6.85s/it]
+  1%|          | 13/1813 [01:21<3:03:18,  6.11s/it]
+  1%|          | 14/1813 [01:27<3:03:08,  6.11s/it]
+  1%|          | 15/1813 [01:32<2:53:14,  5.78s/it]
+  1%|          | 16/1813 [01:38<2:55:32,  5.86s/it]
+  1%|          | 17/1813 [01:45<3:00:08,  6.02s/it]
+  1%|          | 18/1813 [01:51<3:00:41,  6.04s/it]
+  1%|          | 19/1813 [01:57<3:07:00,  6.25s/it]
+  1%|          | 20/1813 [02:04<3:05:29,  6.21s/it]
+                                                   
+{'loss': '1.533', 'grad_norm': '1.407', 'learning_rate': '1.267e-05', 'epoch': '0.01103', 'num_input_tokens_seen': 131352, 'train_runtime': '124', 'train_tokens_per_second': '1059'}
+
+  1%|          | 20/1813 [02:04<3:05:29,  6.21s/it]
+  1%|          | 21/1813 [02:10<3:03:51,  6.16s/it]
+  1%|          | 22/1813 [02:15<2:54:24,  5.84s/it]
+  1%|▏         | 23/1813 [02:20<2:51:10,  5.74s/it]
+  1%|▏         | 24/1813 [02:26<2:52:59,  5.80s/it]
+  1%|▏         | 25/1813 [02:33<3:05:31,  6.23s/it]
+  1%|▏         | 26/1813 [02:41<3:15:17,  6.56s/it]
+  1%|▏         | 27/1813 [02:47<3:10:34,  6.40s/it]
+  2%|▏         | 28/1813 [02:54<3:15:36,  6.57s/it]
+  2%|▏         | 29/1813 [03:02<3:28:43,  7.02s/it]
+  2%|▏         | 30/1813 [03:07<3:09:51,  6.39s/it]
+                                                   
+{'loss': '1.443', 'grad_norm': '1.26', 'learning_rate': '1.933e-05', 'epoch': '0.01655', 'num_input_tokens_seen': 197376, 'train_runtime': '187.1', 'train_tokens_per_second': '1055'}
+
+  2%|▏         | 30/1813 [03:07<3:09:51,  6.39s/it]
+  2%|▏         | 31/1813 [03:13<3:10:27,  6.41s/it]
+  2%|▏         | 32/1813 [03:19<3:01:57,  6.13s/it]
+  2%|▏         | 33/1813 [03:25<3:01:37,  6.12s/it]
+  2%|▏         | 34/1813 [03:32<3:15:21,  6.59s/it]
+  2%|▏         | 35/1813 [03:39<3:15:38,  6.60s/it]
+  2%|▏         | 36/1813 [03:45<3:13:41,  6.54s/it]
+  2%|▏         | 37/1813 [03:51<3:00:56,  6.11s/it]
+  2%|▏         | 38/1813 [03:56<2:51:17,  5.79s/it]
+  2%|▏         | 39/1813 [04:01<2:51:43,  5.81s/it]
+  2%|▏         | 40/1813 [04:07<2:53:21,  5.87s/it]
+                                                   
+{'loss': '1.301', 'grad_norm': '1.084', 'learning_rate': '2.6e-05', 'epoch': '0.02207', 'num_input_tokens_seen': 261168, 'train_runtime': '247.9', 'train_tokens_per_second': '1054'}
+
+  2%|▏         | 40/1813 [04:07<2:53:21,  5.87s/it]
+  2%|▏         | 41/1813 [04:13<2:49:31,  5.74s/it]
+  2%|▏         | 42/1813 [04:19<2:52:17,  5.84s/it]
+  2%|▏         | 43/1813 [04:27<3:08:30,  6.39s/it]
+  2%|▏         | 44/1813 [04:33<3:11:22,  6.49s/it]
+  2%|▏         | 45/1813 [04:41<3:21:03,  6.82s/it]
+  3%|▎         | 46/1813 [04:47<3:13:08,  6.56s/it]
+  3%|▎         | 47/1813 [04:52<3:04:15,  6.26s/it]
+  3%|▎         | 48/1813 [04:59<3:07:53,  6.39s/it]
+  3%|▎         | 49/1813 [05:05<3:06:59,  6.36s/it]
+  3%|▎         | 50/1813 [05:12<3:05:33,  6.31s/it]
+                                                   
+{'loss': '1.326', 'grad_norm': '1.65', 'learning_rate': '3.267e-05', 'epoch': '0.02759', 'num_input_tokens_seen': 330104, 'train_runtime': '312.1', 'train_tokens_per_second': '1058'}
+
+  3%|▎         | 50/1813 [05:12<3:05:33,  6.31s/it]
+  3%|▎         | 51/1813 [05:19<3:15:37,  6.66s/it]
+  3%|▎         | 52/1813 [05:28<3:38:59,  7.46s/it]
+  3%|▎         | 53/1813 [05:34<3:25:19,  7.00s/it]
+  3%|▎         | 54/1813 [05:42<3:30:30,  7.18s/it]
+  3%|▎         | 55/1813 [05:48<3:23:50,  6.96s/it]
+  3%|▎         | 56/1813 [05:55<3:22:37,  6.92s/it]
+  3%|▎         | 57/1813 [06:01<3:08:41,  6.45s/it]
+  3%|▎         | 58/1813 [06:09<3:25:49,  7.04s/it]
+  3%|▎         | 59/1813 [06:16<3:26:35,  7.07s/it]
+  3%|▎         | 60/1813 [06:22<3:12:28,  6.59s/it]
+                                                   
+{'loss': '1.219', 'grad_norm': '1.516', 'learning_rate': '3.933e-05', 'epoch': '0.0331', 'num_input_tokens_seen': 407792, 'train_runtime': '382.1', 'train_tokens_per_second': '1067'}
+
+  3%|▎         | 60/1813 [06:22<3:12:28,  6.59s/it]
+  3%|▎         | 61/1813 [06:27<2:58:58,  6.13s/it]
+  3%|▎         | 62/1813 [06:32<2:53:30,  5.95s/it]
+  3%|▎         | 63/1813 [06:38<2:51:15,  5.87s/it]
+  4%|▎         | 64/1813 [06:44<2:49:43,  5.82s/it]
+  4%|▎         | 65/1813 [06:51<3:02:19,  6.26s/it]
+  4%|▎         | 66/1813 [06:57<3:04:42,  6.34s/it]
+  4%|▎         | 67/1813 [07:03<2:59:37,  6.17s/it]
+  4%|▍         | 68/1813 [07:10<3:01:38,  6.25s/it]
+  4%|▍         | 69/1813 [07:17<3:10:55,  6.57s/it]
+  4%|▍         | 70/1813 [07:24<3:19:32,  6.87s/it]
+                                                   
+{'loss': '1.098', 'grad_norm': '1.199', 'learning_rate': '4.6e-05', 'epoch': '0.03862', 'num_input_tokens_seen': 474368, 'train_runtime': '444.9', 'train_tokens_per_second': '1066'}
+
+  4%|▍         | 70/1813 [07:24<3:19:32,  6.87s/it]
+  4%|▍         | 71/1813 [07:30<3:09:12,  6.52s/it]
+  4%|▍         | 72/1813 [07:39<3:29:12,  7.21s/it]
+  4%|▍         | 73/1813 [07:45<3:15:03,  6.73s/it]
+  4%|▍         | 74/1813 [07:51<3:11:53,  6.62s/it]
+  4%|▍         | 75/1813 [07:58<3:13:42,  6.69s/it]
+  4%|▍         | 76/1813 [08:04<3:08:19,  6.51s/it]
+  4%|▍         | 77/1813 [08:11<3:12:07,  6.64s/it]
+  4%|▍         | 78/1813 [08:16<3:01:45,  6.29s/it]
+  4%|▍         | 79/1813 [08:22<2:54:40,  6.04s/it]
+  4%|▍         | 80/1813 [08:28<2:54:55,  6.06s/it]
+                                                   
+{'loss': '1.126', 'grad_norm': '1.306', 'learning_rate': '5.267e-05', 'epoch': '0.04414', 'num_input_tokens_seen': 541616, 'train_runtime': '508.3', 'train_tokens_per_second': '1065'}
+
+  4%|▍         | 80/1813 [08:28<2:54:55,  6.06s/it]
+  4%|▍         | 81/1813 [08:34<2:52:51,  5.99s/it]
+  5%|▍         | 82/1813 [08:41<3:02:19,  6.32s/it]
+  5%|▍         | 83/1813 [08:48<3:08:55,  6.55s/it]
+  5%|▍         | 84/1813 [08:55<3:12:58,  6.70s/it]
+  5%|▍         | 85/1813 [09:01<3:07:28,  6.51s/it]
+  5%|▍         | 86/1813 [09:07<3:06:01,  6.46s/it]
+  5%|▍         | 87/1813 [09:12<2:51:57,  5.98s/it]
+  5%|▍         | 88/1813 [09:18<2:51:33,  5.97s/it]
+  5%|▍         | 89/1813 [09:24<2:49:58,  5.92s/it]
+  5%|▍         | 90/1813 [09:30<2:53:33,  6.04s/it]
+                                                   
+{'loss': '1.05', 'grad_norm': '1.049', 'learning_rate': '5.933e-05', 'epoch': '0.04966', 'num_input_tokens_seen': 608792, 'train_runtime': '570.7', 'train_tokens_per_second': '1067'}
+
+  5%|▍         | 90/1813 [09:30<2:53:33,  6.04s/it]
+  5%|▌         | 91/1813 [09:36<2:49:44,  5.91s/it]
+  5%|▌         | 92/1813 [09:41<2:39:43,  5.57s/it]
+  5%|▌         | 93/1813 [09:47<2:42:39,  5.67s/it]
+  5%|▌         | 94/1813 [09:53<2:48:56,  5.90s/it]
+  5%|▌         | 95/1813 [09:59<2:53:21,  6.05s/it]
+  5%|▌         | 96/1813 [10:05<2:52:28,  6.03s/it]
+  5%|▌         | 97/1813 [10:12<2:56:41,  6.18s/it]
+  5%|▌         | 98/1813 [10:17<2:47:12,  5.85s/it]
+  5%|▌         | 99/1813 [10:22<2:44:17,  5.75s/it]
+  6%|▌         | 100/1813 [10:29<2:48:45,  5.91s/it]
+                                                    
+{'loss': '1.1', 'grad_norm': '1.002', 'learning_rate': '6.6e-05', 'epoch': '0.05517', 'num_input_tokens_seen': 670936, 'train_runtime': '629.3', 'train_tokens_per_second': '1066'}
+
+  6%|▌         | 100/1813 [10:29<2:48:45,  5.91s/it]
+  6%|▌         | 101/1813 [10:35<2:54:45,  6.12s/it]
+  6%|▌         | 102/1813 [10:42<2:59:17,  6.29s/it]
+  6%|▌         | 103/1813 [10:48<2:56:49,  6.20s/it]
+  6%|▌         | 104/1813 [10:55<2:59:00,  6.28s/it]
+  6%|▌         | 105/1813 [11:00<2:51:53,  6.04s/it]
+  6%|▌         | 106/1813 [11:06<2:51:31,  6.03s/it]
+  6%|▌         | 107/1813 [11:12<2:54:21,  6.13s/it]
+  6%|▌         | 108/1813 [11:19<2:56:11,  6.20s/it]
+  6%|▌         | 109/1813 [11:25<2:55:09,  6.17s/it]
+  6%|▌         | 110/1813 [11:32<2:59:47,  6.33s/it]
+                                                    
+{'loss': '1.11', 'grad_norm': '0.9462', 'learning_rate': '7.267e-05', 'epoch': '0.06069', 'num_input_tokens_seen': 738632, 'train_runtime': '692.1', 'train_tokens_per_second': '1067'}
+
+  6%|▌         | 110/1813 [11:32<2:59:47,  6.33s/it]
+  6%|▌         | 111/1813 [11:38<3:02:52,  6.45s/it]
+  6%|▌         | 112/1813 [11:45<3:05:05,  6.53s/it]
+  6%|▌         | 113/1813 [11:51<3:04:21,  6.51s/it]
+  6%|▋         | 114/1813 [11:59<3:13:35,  6.84s/it]
+  6%|▋         | 115/1813 [12:05<3:07:00,  6.61s/it]
+  6%|▋         | 116/1813 [12:12<3:10:12,  6.73s/it]
+  6%|▋         | 117/1813 [12:18<3:01:30,  6.42s/it]
+  7%|▋         | 118/1813 [12:24<2:57:56,  6.30s/it]
+  7%|▋         | 119/1813 [12:30<2:59:08,  6.34s/it]
+  7%|▋         | 120/1813 [12:37<3:00:19,  6.39s/it]
+                                                    
+{'loss': '1.124', 'grad_norm': '1.452', 'learning_rate': '7.933e-05', 'epoch': '0.06621', 'num_input_tokens_seen': 808824, 'train_runtime': '757.3', 'train_tokens_per_second': '1068'}
+
+  7%|▋         | 120/1813 [12:37<3:00:19,  6.39s/it]
+  7%|▋         | 121/1813 [12:41<2:43:41,  5.80s/it]
+  7%|▋         | 122/1813 [12:47<2:39:05,  5.64s/it]
+  7%|▋         | 123/1813 [12:53<2:46:59,  5.93s/it]
+  7%|▋         | 124/1813 [12:58<2:40:51,  5.71s/it]
+  7%|▋         | 125/1813 [13:05<2:49:11,  6.01s/it]
+  7%|▋         | 126/1813 [13:11<2:51:41,  6.11s/it]
+  7%|▋         | 127/1813 [13:16<2:39:11,  5.67s/it]
+  7%|▋         | 128/1813 [13:20<2:29:28,  5.32s/it]
+  7%|▋         | 129/1813 [13:29<2:52:06,  6.13s/it]
+  7%|▋         | 130/1813 [13:35<2:57:22,  6.32s/it]
+                                                    
+{'loss': '1.061', 'grad_norm': '0.8902', 'learning_rate': '8.6e-05', 'epoch': '0.07172', 'num_input_tokens_seen': 869720, 'train_runtime': '815.8', 'train_tokens_per_second': '1066'}
+
+  7%|▋         | 130/1813 [13:35<2:57:22,  6.32s/it]
+  7%|▋         | 131/1813 [13:41<2:53:58,  6.21s/it]
+  7%|▋         | 132/1813 [13:47<2:48:13,  6.00s/it]
+  7%|▋         | 133/1813 [13:52<2:44:28,  5.87s/it]
+  7%|▋         | 134/1813 [13:57<2:36:34,  5.60s/it]
+  7%|▋         | 135/1813 [14:02<2:31:44,  5.43s/it]
+  8%|▊         | 136/1813 [14:08<2:34:53,  5.54s/it]
+  8%|▊         | 137/1813 [14:14<2:38:19,  5.67s/it]
+  8%|▊         | 138/1813 [14:21<2:50:23,  6.10s/it]
+  8%|▊         | 139/1813 [14:28<2:54:57,  6.27s/it]
+  8%|▊         | 140/1813 [14:35<3:01:33,  6.51s/it]
+                                                    
+{'loss': '1.051', 'grad_norm': '1.229', 'learning_rate': '9.267e-05', 'epoch': '0.07724', 'num_input_tokens_seen': 932000, 'train_runtime': '875.4', 'train_tokens_per_second': '1065'}
+
+  8%|▊         | 140/1813 [14:35<3:01:33,  6.51s/it]
+  8%|▊         | 141/1813 [14:41<2:54:08,  6.25s/it]
+  8%|▊         | 142/1813 [14:48<3:07:40,  6.74s/it]
+  8%|▊         | 143/1813 [14:54<2:55:48,  6.32s/it]
+  8%|▊         | 144/1813 [14:59<2:45:31,  5.95s/it]
+  8%|▊         | 145/1813 [15:06<2:54:22,  6.27s/it]
+  8%|▊         | 146/1813 [15:12<2:52:44,  6.22s/it]
+  8%|▊         | 147/1813 [15:17<2:45:16,  5.95s/it]
+  8%|▊         | 148/1813 [15:23<2:45:10,  5.95s/it]
+  8%|▊         | 149/1813 [15:29<2:44:12,  5.92s/it]
+  8%|▊         | 150/1813 [15:35<2:46:37,  6.01s/it]
+                                                    
+{'loss': '1.109', 'grad_norm': '1.101', 'learning_rate': '9.933e-05', 'epoch': '0.08276', 'num_input_tokens_seen': 997296, 'train_runtime': '935.9', 'train_tokens_per_second': '1066'}
+
+  8%|▊         | 150/1813 [15:35<2:46:37,  6.01s/it]
+  8%|▊         | 151/1813 [15:41<2:43:45,  5.91s/it]
+  8%|▊         | 152/1813 [15:47<2:41:31,  5.83s/it]
+  8%|▊         | 153/1813 [15:52<2:41:09,  5.82s/it]
+  8%|▊         | 154/1813 [15:59<2:46:22,  6.02s/it]
+  9%|▊         | 155/1813 [16:06<2:55:39,  6.36s/it]
+  9%|▊         | 156/1813 [16:12<2:51:46,  6.22s/it]
+  9%|▊         | 157/1813 [16:18<2:51:23,  6.21s/it]
+  9%|▊         | 158/1813 [16:23<2:42:30,  5.89s/it]
+  9%|▉         | 159/1813 [16:29<2:42:57,  5.91s/it]
+  9%|▉         | 160/1813 [16:35<2:40:14,  5.82s/it]
+                                                    
+{'loss': '1.037', 'grad_norm': '1.749', 'learning_rate': '9.999e-05', 'epoch': '0.08828', 'num_input_tokens_seen': 1060096, 'train_runtime': '995.4', 'train_tokens_per_second': '1065'}
+
+  9%|▉         | 160/1813 [16:35<2:40:14,  5.82s/it]
+  9%|▉         | 161/1813 [16:41<2:39:26,  5.79s/it]
+  9%|▉         | 162/1813 [16:45<2:26:32,  5.33s/it]
+  9%|▉         | 163/1813 [16:52<2:38:31,  5.76s/it]
+  9%|▉         | 164/1813 [16:59<2:52:21,  6.27s/it]
+  9%|▉         | 165/1813 [17:06<2:53:20,  6.31s/it]
+  9%|▉         | 166/1813 [17:13<3:06:02,  6.78s/it]
+  9%|▉         | 167/1813 [17:20<3:07:08,  6.82s/it]
+  9%|▉         | 168/1813 [17:28<3:11:05,  6.97s/it]
+  9%|▉         | 169/1813 [17:33<3:01:53,  6.64s/it]
+  9%|▉         | 170/1813 [17:40<3:03:27,  6.70s/it]
+                                                    
+{'loss': '1.061', 'grad_norm': '0.6994', 'learning_rate': '9.997e-05', 'epoch': '0.09379', 'num_input_tokens_seen': 1130368, 'train_runtime': '1061', 'train_tokens_per_second': '1066'}
+
+  9%|▉         | 170/1813 [17:40<3:03:27,  6.70s/it]
+  9%|▉         | 171/1813 [17:46<2:54:24,  6.37s/it]
+  9%|▉         | 172/1813 [17:52<2:48:39,  6.17s/it]
+ 10%|▉         | 173/1813 [17:58<2:51:39,  6.28s/it]
+ 10%|▉         | 174/1813 [18:04<2:48:50,  6.18s/it]
+ 10%|▉         | 175/1813 [18:11<2:50:30,  6.25s/it]
+ 10%|▉         | 176/1813 [18:16<2:42:51,  5.97s/it]
+ 10%|▉         | 177/1813 [18:23<2:55:27,  6.44s/it]
+ 10%|▉         | 178/1813 [18:28<2:42:51,  5.98s/it]
+ 10%|▉         | 179/1813 [18:34<2:42:07,  5.95s/it]
+ 10%|▉         | 180/1813 [18:40<2:45:03,  6.06s/it]
+                                                    
+{'loss': '1.044', 'grad_norm': '1.103', 'learning_rate': '9.992e-05', 'epoch': '0.09931', 'num_input_tokens_seen': 1193192, 'train_runtime': '1121', 'train_tokens_per_second': '1064'}
+
+ 10%|▉         | 180/1813 [18:40<2:45:03,  6.06s/it]
+ 10%|▉         | 181/1813 [18:46<2:42:47,  5.99s/it]
+ 10%|█         | 182/1813 [18:52<2:38:53,  5.85s/it]
+ 10%|█         | 183/1813 [18:58<2:38:23,  5.83s/it]
+ 10%|█         | 184/1813 [19:03<2:37:17,  5.79s/it]
+ 10%|█         | 185/1813 [19:11<2:56:02,  6.49s/it]
+ 10%|█         | 186/1813 [19:18<2:54:24,  6.43s/it]
+ 10%|█         | 187/1813 [19:24<2:51:18,  6.32s/it]
+ 10%|█         | 188/1813 [19:29<2:42:12,  5.99s/it]
+ 10%|█         | 189/1813 [19:36<2:47:50,  6.20s/it]
+ 10%|█         | 190/1813 [19:43<2:52:51,  6.39s/it]
+                                                    
+{'loss': '0.9859', 'grad_norm': '1.041', 'learning_rate': '9.986e-05', 'epoch': '0.1048', 'num_input_tokens_seen': 1259336, 'train_runtime': '1183', 'train_tokens_per_second': '1065'}
+
+ 10%|█         | 190/1813 [19:43<2:52:51,  6.39s/it]
+ 11%|█         | 191/1813 [19:48<2:46:34,  6.16s/it]
+ 11%|█         | 192/1813 [19:55<2:49:30,  6.27s/it]
+ 11%|█         | 193/1813 [20:01<2:51:13,  6.34s/it]
+ 11%|█         | 194/1813 [20:07<2:47:19,  6.20s/it]
+ 11%|█         | 195/1813 [20:15<2:57:52,  6.60s/it]
+ 11%|█         | 196/1813 [20:20<2:46:44,  6.19s/it]
+ 11%|█         | 197/1813 [20:27<2:54:05,  6.46s/it]
+ 11%|█         | 198/1813 [20:34<3:00:57,  6.72s/it]
+ 11%|█         | 199/1813 [20:41<3:00:42,  6.72s/it]
+ 11%|█         | 200/1813 [20:48<2:59:31,  6.68s/it]
+                                                    
+{'loss': '1.094', 'grad_norm': '1.185', 'learning_rate': '9.979e-05', 'epoch': '0.1103', 'num_input_tokens_seen': 1331224, 'train_runtime': '1248', 'train_tokens_per_second': '1067'}
+
+ 11%|█         | 200/1813 [20:48<2:59:31,  6.68s/it]
+ 11%|█         | 201/1813 [20:53<2:51:21,  6.38s/it]
+ 11%|█         | 202/1813 [20:58<2:41:35,  6.02s/it]
+ 11%|█         | 203/1813 [21:03<2:31:41,  5.65s/it]
+ 11%|█▏        | 204/1813 [21:10<2:41:20,  6.02s/it]
+ 11%|█▏        | 205/1813 [21:16<2:37:27,  5.88s/it]
+ 11%|█▏        | 206/1813 [21:22<2:38:46,  5.93s/it]
+ 11%|█▏        | 207/1813 [21:27<2:37:48,  5.90s/it]
+ 11%|█▏        | 208/1813 [21:35<2:53:51,  6.50s/it]
+ 12%|█▏        | 209/1813 [21:41<2:42:54,  6.09s/it]
+ 12%|█▏        | 210/1813 [21:47<2:43:49,  6.13s/it]
+                                                    
+{'loss': '0.9715', 'grad_norm': '1.195', 'learning_rate': '9.969e-05', 'epoch': '0.1159', 'num_input_tokens_seen': 1392472, 'train_runtime': '1307', 'train_tokens_per_second': '1065'}
+
+ 12%|█▏        | 210/1813 [21:47<2:43:49,  6.13s/it]
+ 12%|█▏        | 211/1813 [21:56<3:04:46,  6.92s/it]
+ 12%|█▏        | 212/1813 [22:01<2:53:30,  6.50s/it]
+ 12%|█▏        | 213/1813 [22:07<2:45:24,  6.20s/it]
+ 12%|█▏        | 214/1813 [22:12<2:38:15,  5.94s/it]
+ 12%|█▏        | 215/1813 [22:18<2:35:59,  5.86s/it]
+ 12%|█▏        | 216/1813 [22:22<2:27:57,  5.56s/it]
+ 12%|█▏        | 217/1813 [22:28<2:25:28,  5.47s/it]
+ 12%|█▏        | 218/1813 [22:34<2:32:23,  5.73s/it]
+ 12%|█▏        | 219/1813 [22:40<2:35:48,  5.87s/it]
+ 12%|█▏        | 220/1813 [22:46<2:38:22,  5.97s/it]
+                                                    
+{'loss': '1.041', 'grad_norm': '0.8222', 'learning_rate': '9.958e-05', 'epoch': '0.1214', 'num_input_tokens_seen': 1454040, 'train_runtime': '1367', 'train_tokens_per_second': '1064'}
+
+ 12%|█▏        | 220/1813 [22:46<2:38:22,  5.97s/it]
+ 12%|█▏        | 221/1813 [22:53<2:42:49,  6.14s/it]
+ 12%|█▏        | 222/1813 [22:58<2:36:13,  5.89s/it]
+ 12%|█▏        | 223/1813 [23:06<2:50:41,  6.44s/it]
+ 12%|█▏        | 224/1813 [23:11<2:39:37,  6.03s/it]
+ 12%|█▏        | 225/1813 [23:18<2:49:07,  6.39s/it]
+ 12%|█▏        | 226/1813 [23:25<2:52:47,  6.53s/it]
+ 13%|█▎        | 227/1813 [23:31<2:47:35,  6.34s/it]
+ 13%|█▎        | 228/1813 [23:38<2:49:57,  6.43s/it]
+ 13%|█▎        | 229/1813 [23:45<2:54:55,  6.63s/it]
+ 13%|█▎        | 230/1813 [23:50<2:43:20,  6.19s/it]
+                                                    
+{'loss': '1.012', 'grad_norm': '1.35', 'learning_rate': '9.944e-05', 'epoch': '0.1269', 'num_input_tokens_seen': 1522056, 'train_runtime': '1430', 'train_tokens_per_second': '1064'}
+
+ 13%|█▎        | 230/1813 [23:50<2:43:20,  6.19s/it]
+ 13%|█▎        | 231/1813 [23:55<2:34:26,  5.86s/it]
+ 13%|█▎        | 232/1813 [24:02<2:47:20,  6.35s/it]
+ 13%|█▎        | 233/1813 [24:08<2:38:38,  6.02s/it]
+ 13%|█▎        | 234/1813 [24:13<2:34:05,  5.86s/it]
+ 13%|█▎        | 235/1813 [24:20<2:43:05,  6.20s/it]
+ 13%|█▎        | 236/1813 [24:28<2:54:34,  6.64s/it]
+ 13%|█▎        | 237/1813 [24:35<2:55:25,  6.68s/it]
+ 13%|█▎        | 238/1813 [24:40<2:46:21,  6.34s/it]
+ 13%|█▎        | 239/1813 [24:48<2:56:06,  6.71s/it]
+ 13%|█▎        | 240/1813 [24:54<2:51:33,  6.54s/it]
+                                                    
+{'loss': '1.003', 'grad_norm': '0.8033', 'learning_rate': '9.929e-05', 'epoch': '0.1324', 'num_input_tokens_seen': 1590200, 'train_runtime': '1494', 'train_tokens_per_second': '1064'}
+
+ 13%|█▎        | 240/1813 [24:54<2:51:33,  6.54s/it]
+ 13%|█▎        | 241/1813 [25:00<2:50:21,  6.50s/it]
+ 13%|█▎        | 242/1813 [25:06<2:46:49,  6.37s/it]
+ 13%|█▎        | 243/1813 [25:12<2:39:19,  6.09s/it]
+ 13%|█▎        | 244/1813 [25:19<2:46:12,  6.36s/it]
+ 14%|█▎        | 245/1813 [25:26<2:50:56,  6.54s/it]
+ 14%|█▎        | 246/1813 [25:31<2:41:26,  6.18s/it]
+ 14%|█▎        | 247/1813 [25:38<2:46:39,  6.39s/it]
+ 14%|█▎        | 248/1813 [25:46<2:56:43,  6.78s/it]
+ 14%|█▎        | 249/1813 [25:51<2:44:35,  6.31s/it]
+ 14%|█▍        | 250/1813 [25:56<2:38:23,  6.08s/it]
+                                                    
+{'loss': '1.092', 'grad_norm': '0.9446', 'learning_rate': '9.913e-05', 'epoch': '0.1379', 'num_input_tokens_seen': 1658648, 'train_runtime': '1557', 'train_tokens_per_second': '1065'}
+
+ 14%|█▍        | 250/1813 [25:56<2:38:23,  6.08s/it]
+ 14%|█▍        | 251/1813 [26:01<2:26:16,  5.62s/it]
+ 14%|█▍        | 252/1813 [26:06<2:23:52,  5.53s/it]
+ 14%|█▍        | 253/1813 [26:13<2:29:53,  5.77s/it]
+ 14%|█▍        | 254/1813 [26:20<2:41:42,  6.22s/it]
+ 14%|█▍        | 255/1813 [26:27<2:44:42,  6.34s/it]
+ 14%|█▍        | 256/1813 [26:32<2:36:54,  6.05s/it]
+ 14%|█▍        | 257/1813 [26:39<2:43:02,  6.29s/it]
+ 14%|█▍        | 258/1813 [26:46<2:47:58,  6.48s/it]
+ 14%|█▍        | 259/1813 [26:50<2:34:30,  5.97s/it]
+ 14%|█▍        | 260/1813 [26:57<2:36:20,  6.04s/it]
+                                                    
+{'loss': '0.8911', 'grad_norm': '1.303', 'learning_rate': '9.894e-05', 'epoch': '0.1434', 'num_input_tokens_seen': 1722792, 'train_runtime': '1617', 'train_tokens_per_second': '1065'}
+
+ 14%|█▍        | 260/1813 [26:57<2:36:20,  6.04s/it]
+ 14%|█▍        | 261/1813 [27:04<2:46:09,  6.42s/it]
+ 14%|█▍        | 262/1813 [27:11<2:49:02,  6.54s/it]
+ 15%|█▍        | 263/1813 [27:17<2:49:50,  6.57s/it]
+ 15%|█▍        | 264/1813 [27:24<2:47:12,  6.48s/it]
+ 15%|█▍        | 265/1813 [27:30<2:45:34,  6.42s/it]
+ 15%|█▍        | 266/1813 [27:35<2:34:18,  5.98s/it]
+ 15%|█▍        | 267/1813 [27:41<2:31:16,  5.87s/it]
+ 15%|█▍        | 268/1813 [27:46<2:26:58,  5.71s/it]
+ 15%|█▍        | 269/1813 [27:52<2:30:55,  5.87s/it]
+ 15%|█▍        | 270/1813 [27:58<2:31:25,  5.89s/it]
+                                                    
+{'loss': '0.9272', 'grad_norm': '0.9646', 'learning_rate': '9.874e-05', 'epoch': '0.149', 'num_input_tokens_seen': 1786808, 'train_runtime': '1679', 'train_tokens_per_second': '1064'}
+
+ 15%|█▍        | 270/1813 [27:58<2:31:25,  5.89s/it]
+ 15%|█▍        | 271/1813 [28:06<2:48:29,  6.56s/it]
+ 15%|█▌        | 272/1813 [28:12<2:39:05,  6.19s/it]
+ 15%|█▌        | 273/1813 [28:20<2:55:46,  6.85s/it]
+ 15%|█▌        | 274/1813 [28:27<2:55:24,  6.84s/it]
+ 15%|█▌        | 275/1813 [28:32<2:42:56,  6.36s/it]
+ 15%|█▌        | 276/1813 [28:37<2:35:29,  6.07s/it]
+ 15%|█▌        | 277/1813 [28:44<2:37:36,  6.16s/it]
+ 15%|█▌        | 278/1813 [28:50<2:38:13,  6.18s/it]
+ 15%|█▌        | 279/1813 [28:58<2:50:07,  6.65s/it]
+ 15%|█▌        | 280/1813 [29:04<2:50:09,  6.66s/it]
+                                                    
+{'loss': '1.002', 'grad_norm': '0.9154', 'learning_rate': '9.852e-05', 'epoch': '0.1545', 'num_input_tokens_seen': 1858112, 'train_runtime': '1745', 'train_tokens_per_second': '1065'}
+
+ 15%|█▌        | 280/1813 [29:04<2:50:09,  6.66s/it]
+ 15%|█▌        | 281/1813 [29:10<2:43:54,  6.42s/it]
+ 16%|█▌        | 282/1813 [29:16<2:38:27,  6.21s/it]
+ 16%|█▌        | 283/1813 [29:21<2:32:27,  5.98s/it]
+ 16%|█▌        | 284/1813 [29:27<2:30:24,  5.90s/it]
+ 16%|█▌        | 285/1813 [29:34<2:37:01,  6.17s/it]
+ 16%|█▌        | 286/1813 [29:40<2:35:25,  6.11s/it]
+ 16%|█▌        | 287/1813 [29:46<2:35:30,  6.11s/it]
+ 16%|█▌        | 288/1813 [29:51<2:29:35,  5.89s/it]
+ 16%|█▌        | 289/1813 [29:58<2:33:08,  6.03s/it]
+ 16%|█▌        | 290/1813 [30:04<2:36:40,  6.17s/it]
+                                                    
+{'loss': '0.973', 'grad_norm': '0.8369', 'learning_rate': '9.829e-05', 'epoch': '0.16', 'num_input_tokens_seen': 1920600, 'train_runtime': '1805', 'train_tokens_per_second': '1064'}
+
+ 16%|█▌        | 290/1813 [30:04<2:36:40,  6.17s/it]
+ 16%|█▌        | 291/1813 [30:11<2:42:21,  6.40s/it]
+ 16%|█▌        | 292/1813 [30:18<2:48:28,  6.65s/it]
+ 16%|█▌        | 293/1813 [30:24<2:42:43,  6.42s/it]
+ 16%|█▌        | 294/1813 [30:31<2:48:06,  6.64s/it]
+ 16%|█▋        | 295/1813 [30:39<2:54:26,  6.89s/it]
+ 16%|█▋        | 296/1813 [30:45<2:45:40,  6.55s/it]
+ 16%|█▋        | 297/1813 [30:51<2:43:17,  6.46s/it]
+ 16%|█▋        | 298/1813 [30:56<2:32:44,  6.05s/it]
+ 16%|█▋        | 299/1813 [31:02<2:33:08,  6.07s/it]
+ 17%|█▋        | 300/1813 [31:08<2:32:20,  6.04s/it]
+                                                    
+{'loss': '1.042', 'grad_norm': '0.9333', 'learning_rate': '9.803e-05', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312, 'train_runtime': '1869', 'train_tokens_per_second': '1064'}
+
+ 17%|█▋        | 300/1813 [31:08<2:32:20,  6.04s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.79it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.46it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.07it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.00it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.27it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.65it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.76it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.92it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.95it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.74it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.29it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.27it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.66it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.86it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.19it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.02it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.53it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.64it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.32it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.45it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.21it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.05it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.53it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.81it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.19it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.90it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.06it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.71it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.13it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.69it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.13it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.90it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.08it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.04it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.40it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.60it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.17it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.46it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.52it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.74it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.73it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.88it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.47it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.67it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.77it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.63it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.92it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.51it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.65it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.73it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:30,  8.63it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:04,  6.25it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:04,  6.26it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.52it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:09,  6.00it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.06it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.81it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.35it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.01it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.20it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.97it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:31,  8.34it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.01it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.24it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.70it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.05it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.74it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.79it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.01it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.90it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.84it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.11it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.15it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.94it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.78it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.79it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.27it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.26it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.59it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.45it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.76it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.42it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.18it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.80it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.39it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.05it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.84it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.04it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.89it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.73it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.23it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.42it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:33,  6.93it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.65it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.94it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.55it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.41it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.19it/s]
+
+ 38%|███▊      | 379/1000 [00:50<01:15,  8.18it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.20it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:11,  8.57it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.08it/s]
+
+ 39%|███▊      | 387/1000 [00:51<01:21,  7.56it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.30it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.85it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.50it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  9.02it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.51it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.82it/s]
+
+ 40%|███▉      | 398/1000 [00:53<01:23,  7.25it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:40,  5.95it/s]
+
+ 40%|████      | 400/1000 [00:53<01:51,  5.37it/s]
+
+ 40%|████      | 402/1000 [00:54<01:43,  5.76it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.26it/s]
+
+ 41%|████      | 406/1000 [00:54<01:08,  8.62it/s]
+
+ 41%|████      | 407/1000 [00:54<01:23,  7.07it/s]
+
+ 41%|████      | 408/1000 [00:55<01:36,  6.15it/s]
+
+ 41%|████      | 410/1000 [00:55<01:33,  6.31it/s]
+
+ 41%|████      | 411/1000 [00:55<01:44,  5.64it/s]
+
+ 41%|████      | 412/1000 [00:55<01:53,  5.17it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:26,  6.79it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.31it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.50it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:16,  7.52it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.68it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.92it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.97it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.55it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.85it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.53it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.00it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:48, 11.76it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.51it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.09it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.31it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.26it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.74it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.80it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.83it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.20it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.37it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:25,  6.17it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.19it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.29it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.09it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.51it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.05it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.39it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.64it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.53it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.79it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.95it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.97it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.27it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.10it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.07it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.58it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.14it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:47,  9.74it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.67it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.49it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.17it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.94it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.87it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.07it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.99it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.57it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.90it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.14it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.81it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.40it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.40it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:45,  8.49it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.75it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.84it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.11it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.05it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.22it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.63it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.87it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.44it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.18it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.97it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.71it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.11it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.69it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.97it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.89it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.83it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.25it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.59it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.71it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.34it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.05it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.14it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.61it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.51it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.87it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.52it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.78it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.35it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.60it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.43it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.63it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.72it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.66it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.13it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.79it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.60it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.94it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.36it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  8.99it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.18it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.98it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.22it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.24it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.31it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.46it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.49it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.46it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.84it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.45it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.03it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.19it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.51it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.75it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.61it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.63it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.28it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.25it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.27it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.76it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.84it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.30it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.85it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.38it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.69it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.82it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.41it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.44it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.03it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.21it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.81it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.81it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.42it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.05it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.43it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.91it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                    
+
+                                                   
+{'eval_loss': '0.8473', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.384', 'eval_steps_per_second': '7.384', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312}
+
+ 17%|█▋        | 300/1813 [33:24<2:32:20,  6.04s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 17%|█▋        | 301/1813 [33:29<19:32:14, 46.52s/it]
+ 17%|█▋        | 302/1813 [33:34<14:17:15, 34.04s/it]
+ 17%|█▋        | 303/1813 [33:41<10:51:02, 25.87s/it]
+ 17%|█▋        | 304/1813 [33:48<8:31:47, 20.35s/it] 
+ 17%|█▋        | 305/1813 [33:54<6:44:44, 16.10s/it]
+ 17%|█▋        | 306/1813 [34:00<5:26:49, 13.01s/it]
+ 17%|█▋        | 307/1813 [34:06<4:29:24, 10.73s/it]
+ 17%|█▋        | 308/1813 [34:11<3:49:36,  9.15s/it]
+ 17%|█▋        | 309/1813 [34:17<3:28:26,  8.32s/it]
+ 17%|█▋        | 310/1813 [34:22<3:02:28,  7.28s/it]
+                                                    
+{'loss': '0.9655', 'grad_norm': '1.182', 'learning_rate': '9.776e-05', 'epoch': '0.171', 'num_input_tokens_seen': 2050672, 'train_runtime': '2063', 'train_tokens_per_second': '994.1'}
+
+ 17%|█▋        | 310/1813 [34:22<3:02:28,  7.28s/it]
+ 17%|█▋        | 311/1813 [34:30<3:01:29,  7.25s/it]
+ 17%|█▋        | 312/1813 [34:37<3:02:06,  7.28s/it]
+ 17%|█▋        | 313/1813 [34:44<2:57:49,  7.11s/it]
+ 17%|█▋        | 314/1813 [34:49<2:46:38,  6.67s/it]
+ 17%|█▋        | 315/1813 [34:56<2:48:44,  6.76s/it]
+ 17%|█▋        | 316/1813 [35:03<2:46:41,  6.68s/it]
+ 17%|█▋        | 317/1813 [35:09<2:41:28,  6.48s/it]
+ 18%|█▊        | 318/1813 [35:14<2:34:29,  6.20s/it]
+ 18%|█▊        | 319/1813 [35:19<2:25:34,  5.85s/it]
+ 18%|█▊        | 320/1813 [35:25<2:25:59,  5.87s/it]
+                                                    
+{'loss': '1.054', 'grad_norm': '0.6159', 'learning_rate': '9.747e-05', 'epoch': '0.1766', 'num_input_tokens_seen': 2118000, 'train_runtime': '2126', 'train_tokens_per_second': '996.4'}
+
+ 18%|█▊        | 320/1813 [35:25<2:25:59,  5.87s/it]
+ 18%|█▊        | 321/1813 [35:31<2:26:50,  5.91s/it]
+ 18%|█▊        | 322/1813 [35:37<2:27:57,  5.95s/it]
+ 18%|█▊        | 323/1813 [35:44<2:31:31,  6.10s/it]
+ 18%|█▊        | 324/1813 [35:51<2:37:40,  6.35s/it]
+ 18%|█▊        | 325/1813 [35:58<2:41:37,  6.52s/it]
+ 18%|█▊        | 326/1813 [36:04<2:37:48,  6.37s/it]
+ 18%|█▊        | 327/1813 [36:09<2:32:44,  6.17s/it]
+ 18%|█▊        | 328/1813 [36:16<2:36:09,  6.31s/it]
+ 18%|█▊        | 329/1813 [36:22<2:31:17,  6.12s/it]
+ 18%|█▊        | 330/1813 [36:27<2:27:41,  5.98s/it]
+                                                    
+{'loss': '0.9614', 'grad_norm': '0.9737', 'learning_rate': '9.717e-05', 'epoch': '0.1821', 'num_input_tokens_seen': 2184640, 'train_runtime': '2188', 'train_tokens_per_second': '998.6'}
+
+ 18%|█▊        | 330/1813 [36:27<2:27:41,  5.98s/it]
+ 18%|█▊        | 331/1813 [36:33<2:29:12,  6.04s/it]
+ 18%|█▊        | 332/1813 [36:38<2:21:18,  5.72s/it]
+ 18%|█▊        | 333/1813 [36:44<2:22:39,  5.78s/it]
+ 18%|█▊        | 334/1813 [36:50<2:22:18,  5.77s/it]
+ 18%|█▊        | 335/1813 [36:58<2:36:20,  6.35s/it]
+ 19%|█▊        | 336/1813 [37:05<2:41:12,  6.55s/it]
+ 19%|█▊        | 337/1813 [37:11<2:35:28,  6.32s/it]
+ 19%|█▊        | 338/1813 [37:17<2:35:16,  6.32s/it]
+ 19%|█▊        | 339/1813 [37:23<2:31:47,  6.18s/it]
+ 19%|█▉        | 340/1813 [37:30<2:39:15,  6.49s/it]
+                                                    
+{'loss': '0.9404', 'grad_norm': '1.043', 'learning_rate': '9.685e-05', 'epoch': '0.1876', 'num_input_tokens_seen': 2251392, 'train_runtime': '2250', 'train_tokens_per_second': '1000'}
+
+ 19%|█▉        | 340/1813 [37:30<2:39:15,  6.49s/it]
+ 19%|█▉        | 341/1813 [37:36<2:38:28,  6.46s/it]
+ 19%|█▉        | 342/1813 [37:44<2:45:27,  6.75s/it]
+ 19%|█▉        | 343/1813 [37:49<2:37:48,  6.44s/it]
+ 19%|█▉        | 344/1813 [37:55<2:32:53,  6.24s/it]
+ 19%|█▉        | 345/1813 [38:03<2:40:47,  6.57s/it]
+ 19%|█▉        | 346/1813 [38:09<2:39:34,  6.53s/it]
+ 19%|█▉        | 347/1813 [38:14<2:28:45,  6.09s/it]
+ 19%|█▉        | 348/1813 [38:21<2:32:25,  6.24s/it]
+ 19%|█▉        | 349/1813 [38:29<2:48:21,  6.90s/it]
+ 19%|█▉        | 350/1813 [38:37<2:55:03,  7.18s/it]
+                                                    
+{'loss': '1.055', 'grad_norm': '0.817', 'learning_rate': '9.651e-05', 'epoch': '0.1931', 'num_input_tokens_seen': 2324712, 'train_runtime': '2317', 'train_tokens_per_second': '1003'}
+
+ 19%|█▉        | 350/1813 [38:37<2:55:03,  7.18s/it]
+ 19%|█▉        | 351/1813 [38:44<2:53:43,  7.13s/it]
+ 19%|█▉        | 352/1813 [38:50<2:45:46,  6.81s/it]
+ 19%|█▉        | 353/1813 [38:56<2:37:54,  6.49s/it]
+ 20%|█▉        | 354/1813 [39:02<2:33:32,  6.31s/it]
+ 20%|█▉        | 355/1813 [39:08<2:32:35,  6.28s/it]
+ 20%|█▉        | 356/1813 [39:15<2:37:58,  6.51s/it]
+ 20%|█▉        | 357/1813 [39:21<2:33:44,  6.34s/it]
+ 20%|█▉        | 358/1813 [39:27<2:34:41,  6.38s/it]
+ 20%|█▉        | 359/1813 [39:33<2:31:38,  6.26s/it]
+ 20%|█▉        | 360/1813 [39:40<2:35:54,  6.44s/it]
+                                                    
+{'loss': '0.9962', 'grad_norm': '1.841', 'learning_rate': '9.615e-05', 'epoch': '0.1986', 'num_input_tokens_seen': 2390776, 'train_runtime': '2381', 'train_tokens_per_second': '1004'}
+
+ 20%|█▉        | 360/1813 [39:40<2:35:54,  6.44s/it]
+ 20%|█▉        | 361/1813 [39:45<2:25:53,  6.03s/it]
+ 20%|█▉        | 362/1813 [39:51<2:23:05,  5.92s/it]
+ 20%|██        | 363/1813 [39:58<2:29:58,  6.21s/it]
+ 20%|██        | 364/1813 [40:04<2:32:40,  6.32s/it]
+ 20%|██        | 365/1813 [40:10<2:29:31,  6.20s/it]
+ 20%|██        | 366/1813 [40:17<2:35:08,  6.43s/it]
+ 20%|██        | 367/1813 [40:23<2:32:05,  6.31s/it]
+ 20%|██        | 368/1813 [40:29<2:25:52,  6.06s/it]
+ 20%|██        | 369/1813 [40:33<2:15:27,  5.63s/it]
+ 20%|██        | 370/1813 [40:39<2:16:58,  5.70s/it]
+                                                    
+{'loss': '0.9922', 'grad_norm': '1.217', 'learning_rate': '9.578e-05', 'epoch': '0.2041', 'num_input_tokens_seen': 2454808, 'train_runtime': '2440', 'train_tokens_per_second': '1006'}
+
+ 20%|██        | 370/1813 [40:39<2:16:58,  5.70s/it]
+ 20%|██        | 371/1813 [40:44<2:12:04,  5.50s/it]
+ 21%|██        | 372/1813 [40:52<2:31:20,  6.30s/it]
+ 21%|██        | 373/1813 [40:59<2:29:50,  6.24s/it]
+ 21%|██        | 374/1813 [41:06<2:38:04,  6.59s/it]
+ 21%|██        | 375/1813 [41:13<2:39:30,  6.66s/it]
+ 21%|██        | 376/1813 [41:18<2:29:23,  6.24s/it]
+ 21%|██        | 377/1813 [41:24<2:28:12,  6.19s/it]
+ 21%|██        | 378/1813 [41:30<2:26:21,  6.12s/it]
+ 21%|██        | 379/1813 [41:35<2:18:08,  5.78s/it]
+ 21%|██        | 380/1813 [41:40<2:12:59,  5.57s/it]
+                                                    
+{'loss': '0.9639', 'grad_norm': '1.139', 'learning_rate': '9.539e-05', 'epoch': '0.2097', 'num_input_tokens_seen': 2520472, 'train_runtime': '2501', 'train_tokens_per_second': '1008'}
+
+ 21%|██        | 380/1813 [41:40<2:12:59,  5.57s/it]
+ 21%|██        | 381/1813 [41:47<2:22:48,  5.98s/it]
+ 21%|██        | 382/1813 [41:54<2:28:42,  6.24s/it]
+ 21%|██        | 383/1813 [42:00<2:27:29,  6.19s/it]
+ 21%|██        | 384/1813 [42:06<2:26:11,  6.14s/it]
+ 21%|██        | 385/1813 [42:12<2:26:06,  6.14s/it]
+ 21%|██▏       | 386/1813 [42:17<2:14:55,  5.67s/it]
+ 21%|██▏       | 387/1813 [42:23<2:17:28,  5.78s/it]
+ 21%|██▏       | 388/1813 [42:30<2:26:33,  6.17s/it]
+ 21%|██▏       | 389/1813 [42:38<2:37:26,  6.63s/it]
+ 22%|██▏       | 390/1813 [42:44<2:37:12,  6.63s/it]
+                                                    
+{'loss': '1.002', 'grad_norm': '0.7518', 'learning_rate': '9.499e-05', 'epoch': '0.2152', 'num_input_tokens_seen': 2591512, 'train_runtime': '2565', 'train_tokens_per_second': '1010'}
+
+ 22%|██▏       | 390/1813 [42:44<2:37:12,  6.63s/it]
+ 22%|██▏       | 391/1813 [42:50<2:33:27,  6.47s/it]
+ 22%|██▏       | 392/1813 [42:57<2:32:32,  6.44s/it]
+ 22%|██▏       | 393/1813 [43:03<2:34:58,  6.55s/it]
+ 22%|██▏       | 394/1813 [43:08<2:22:52,  6.04s/it]
+ 22%|██▏       | 395/1813 [43:14<2:18:36,  5.86s/it]
+ 22%|██▏       | 396/1813 [43:20<2:23:04,  6.06s/it]
+ 22%|██▏       | 397/1813 [43:25<2:17:06,  5.81s/it]
+ 22%|██▏       | 398/1813 [43:32<2:21:34,  6.00s/it]
+ 22%|██▏       | 399/1813 [43:40<2:33:32,  6.52s/it]
+ 22%|██▏       | 400/1813 [43:48<2:46:13,  7.06s/it]
+                                                    
+{'loss': '0.9768', 'grad_norm': '0.8913', 'learning_rate': '9.457e-05', 'epoch': '0.2207', 'num_input_tokens_seen': 2660336, 'train_runtime': '2628', 'train_tokens_per_second': '1012'}
+
+ 22%|██▏       | 400/1813 [43:48<2:46:13,  7.06s/it]
+ 22%|██▏       | 401/1813 [43:55<2:48:11,  7.15s/it]
+ 22%|██▏       | 402/1813 [44:00<2:32:49,  6.50s/it]
+ 22%|██▏       | 403/1813 [44:07<2:31:08,  6.43s/it]
+ 22%|██▏       | 404/1813 [44:14<2:34:45,  6.59s/it]
+ 22%|██▏       | 405/1813 [44:21<2:39:34,  6.80s/it]
+ 22%|██▏       | 406/1813 [44:26<2:25:54,  6.22s/it]
+ 22%|██▏       | 407/1813 [44:30<2:13:42,  5.71s/it]
+ 23%|██▎       | 408/1813 [44:36<2:15:24,  5.78s/it]
+ 23%|██▎       | 409/1813 [44:42<2:16:54,  5.85s/it]
+ 23%|██▎       | 410/1813 [44:48<2:14:53,  5.77s/it]
+                                                    
+{'loss': '1.009', 'grad_norm': '1.12', 'learning_rate': '9.413e-05', 'epoch': '0.2262', 'num_input_tokens_seen': 2722056, 'train_runtime': '2688', 'train_tokens_per_second': '1013'}
+
+ 23%|██▎       | 410/1813 [44:48<2:14:53,  5.77s/it]
+ 23%|██▎       | 411/1813 [44:54<2:18:39,  5.93s/it]
+ 23%|██▎       | 412/1813 [45:00<2:21:49,  6.07s/it]
+ 23%|██▎       | 413/1813 [45:08<2:28:47,  6.38s/it]
+ 23%|██▎       | 414/1813 [45:13<2:20:33,  6.03s/it]
+ 23%|██▎       | 415/1813 [45:19<2:21:37,  6.08s/it]
+ 23%|██▎       | 416/1813 [45:26<2:25:04,  6.23s/it]
+ 23%|██▎       | 417/1813 [45:33<2:34:17,  6.63s/it]
+ 23%|██▎       | 418/1813 [45:39<2:27:20,  6.34s/it]
+ 23%|██▎       | 419/1813 [45:45<2:28:53,  6.41s/it]
+ 23%|██▎       | 420/1813 [45:51<2:24:52,  6.24s/it]
+                                                    
+{'loss': '0.9971', 'grad_norm': '0.6842', 'learning_rate': '9.368e-05', 'epoch': '0.2317', 'num_input_tokens_seen': 2790960, 'train_runtime': '2752', 'train_tokens_per_second': '1014'}
+
+ 23%|██▎       | 420/1813 [45:51<2:24:52,  6.24s/it]
+ 23%|██▎       | 421/1813 [45:58<2:25:41,  6.28s/it]
+ 23%|██▎       | 422/1813 [46:05<2:32:41,  6.59s/it]
+ 23%|██▎       | 423/1813 [46:11<2:27:24,  6.36s/it]
+ 23%|██▎       | 424/1813 [46:17<2:24:34,  6.25s/it]
+ 23%|██▎       | 425/1813 [46:22<2:19:06,  6.01s/it]
+ 23%|██▎       | 426/1813 [46:29<2:27:26,  6.38s/it]
+ 24%|██▎       | 427/1813 [46:35<2:24:30,  6.26s/it]
+ 24%|██▎       | 428/1813 [46:41<2:23:18,  6.21s/it]
+ 24%|██▎       | 429/1813 [46:47<2:16:24,  5.91s/it]
+ 24%|██▎       | 430/1813 [46:54<2:27:45,  6.41s/it]
+                                                    
+{'loss': '0.986', 'grad_norm': '0.6879', 'learning_rate': '9.321e-05', 'epoch': '0.2372', 'num_input_tokens_seen': 2860096, 'train_runtime': '2815', 'train_tokens_per_second': '1016'}
+
+ 24%|██▎       | 430/1813 [46:54<2:27:45,  6.41s/it]
+ 24%|██▍       | 431/1813 [47:00<2:25:02,  6.30s/it]
+ 24%|██▍       | 432/1813 [47:06<2:20:56,  6.12s/it]
+ 24%|██▍       | 433/1813 [47:11<2:15:26,  5.89s/it]
+ 24%|██▍       | 434/1813 [47:18<2:19:26,  6.07s/it]
+ 24%|██▍       | 435/1813 [47:24<2:18:04,  6.01s/it]
+ 24%|██▍       | 436/1813 [47:30<2:20:31,  6.12s/it]
+ 24%|██▍       | 437/1813 [47:36<2:17:36,  6.00s/it]
+ 24%|██▍       | 438/1813 [47:42<2:19:36,  6.09s/it]
+ 24%|██▍       | 439/1813 [47:48<2:18:31,  6.05s/it]
+ 24%|██▍       | 440/1813 [47:54<2:19:28,  6.10s/it]
+                                                    
+{'loss': '0.8981', 'grad_norm': '1.202', 'learning_rate': '9.273e-05', 'epoch': '0.2428', 'num_input_tokens_seen': 2924856, 'train_runtime': '2875', 'train_tokens_per_second': '1017'}
+
+ 24%|██▍       | 440/1813 [47:54<2:19:28,  6.10s/it]
+ 24%|██▍       | 441/1813 [48:00<2:18:12,  6.04s/it]
+ 24%|██▍       | 442/1813 [48:06<2:19:22,  6.10s/it]
+ 24%|██▍       | 443/1813 [48:13<2:24:00,  6.31s/it]
+ 24%|██▍       | 444/1813 [48:19<2:18:22,  6.07s/it]
+ 25%|██▍       | 445/1813 [48:26<2:25:36,  6.39s/it]
+ 25%|██▍       | 446/1813 [48:32<2:25:10,  6.37s/it]
+ 25%|██▍       | 447/1813 [48:38<2:23:03,  6.28s/it]
+ 25%|██▍       | 448/1813 [48:44<2:22:00,  6.24s/it]
+ 25%|██▍       | 449/1813 [48:50<2:14:42,  5.93s/it]
+ 25%|██▍       | 450/1813 [48:55<2:10:33,  5.75s/it]
+                                                    
+{'loss': '1.027', 'grad_norm': '0.9296', 'learning_rate': '9.223e-05', 'epoch': '0.2483', 'num_input_tokens_seen': 2990040, 'train_runtime': '2935', 'train_tokens_per_second': '1019'}
+
+ 25%|██▍       | 450/1813 [48:55<2:10:33,  5.75s/it]
+ 25%|██▍       | 451/1813 [49:02<2:22:07,  6.26s/it]
+ 25%|██▍       | 452/1813 [49:08<2:18:14,  6.09s/it]
+ 25%|██▍       | 453/1813 [49:13<2:10:49,  5.77s/it]
+ 25%|██▌       | 454/1813 [49:20<2:17:15,  6.06s/it]
+ 25%|██▌       | 455/1813 [49:26<2:19:48,  6.18s/it]
+ 25%|██▌       | 456/1813 [49:31<2:06:45,  5.60s/it]
+ 25%|██▌       | 457/1813 [49:37<2:10:00,  5.75s/it]
+ 25%|██▌       | 458/1813 [49:45<2:24:41,  6.41s/it]
+ 25%|██▌       | 459/1813 [49:52<2:28:00,  6.56s/it]
+ 25%|██▌       | 460/1813 [49:58<2:26:23,  6.49s/it]
+                                                    
+{'loss': '1.088', 'grad_norm': '0.7194', 'learning_rate': '9.172e-05', 'epoch': '0.2538', 'num_input_tokens_seen': 3058848, 'train_runtime': '2998', 'train_tokens_per_second': '1020'}
+
+ 25%|██▌       | 460/1813 [49:58<2:26:23,  6.49s/it]
+ 25%|██▌       | 461/1813 [50:04<2:22:40,  6.33s/it]
+ 25%|██▌       | 462/1813 [50:11<2:29:43,  6.65s/it]
+ 26%|██▌       | 463/1813 [50:17<2:24:26,  6.42s/it]
+ 26%|██▌       | 464/1813 [50:22<2:14:54,  6.00s/it]
+ 26%|██▌       | 465/1813 [50:28<2:11:58,  5.87s/it]
+ 26%|██▌       | 466/1813 [50:34<2:16:52,  6.10s/it]
+ 26%|██▌       | 467/1813 [50:40<2:14:22,  5.99s/it]
+ 26%|██▌       | 468/1813 [50:47<2:17:38,  6.14s/it]
+ 26%|██▌       | 469/1813 [50:53<2:19:39,  6.23s/it]
+ 26%|██▌       | 470/1813 [51:00<2:21:55,  6.34s/it]
+                                                    
+{'loss': '0.9541', 'grad_norm': '0.6546', 'learning_rate': '9.119e-05', 'epoch': '0.2593', 'num_input_tokens_seen': 3126680, 'train_runtime': '3060', 'train_tokens_per_second': '1022'}
+
+ 26%|██▌       | 470/1813 [51:00<2:21:55,  6.34s/it]
+ 26%|██▌       | 471/1813 [51:05<2:16:46,  6.12s/it]
+ 26%|██▌       | 472/1813 [51:12<2:19:39,  6.25s/it]
+ 26%|██▌       | 473/1813 [51:18<2:22:19,  6.37s/it]
+ 26%|██▌       | 474/1813 [51:24<2:20:02,  6.27s/it]
+ 26%|██▌       | 475/1813 [51:30<2:12:29,  5.94s/it]
+ 26%|██▋       | 476/1813 [51:36<2:12:18,  5.94s/it]
+ 26%|██▋       | 477/1813 [51:42<2:16:15,  6.12s/it]
+ 26%|██▋       | 478/1813 [51:49<2:21:54,  6.38s/it]
+ 26%|██▋       | 479/1813 [51:55<2:17:50,  6.20s/it]
+ 26%|██▋       | 480/1813 [52:01<2:19:22,  6.27s/it]
+                                                    
+{'loss': '1.059', 'grad_norm': '0.9186', 'learning_rate': '9.065e-05', 'epoch': '0.2648', 'num_input_tokens_seen': 3193704, 'train_runtime': '3122', 'train_tokens_per_second': '1023'}
+
+ 26%|██▋       | 480/1813 [52:01<2:19:22,  6.27s/it]
+ 27%|██▋       | 481/1813 [52:08<2:19:30,  6.28s/it]
+ 27%|██▋       | 482/1813 [52:14<2:22:30,  6.42s/it]
+ 27%|██▋       | 483/1813 [52:22<2:30:37,  6.80s/it]
+ 27%|██▋       | 484/1813 [52:28<2:24:46,  6.54s/it]
+ 27%|██▋       | 485/1813 [52:36<2:33:15,  6.92s/it]
+ 27%|██▋       | 486/1813 [52:42<2:28:58,  6.74s/it]
+ 27%|██▋       | 487/1813 [52:49<2:30:07,  6.79s/it]
+ 27%|██▋       | 488/1813 [52:55<2:23:35,  6.50s/it]
+ 27%|██▋       | 489/1813 [53:02<2:27:40,  6.69s/it]
+ 27%|██▋       | 490/1813 [53:08<2:25:42,  6.61s/it]
+                                                    
+{'loss': '1.011', 'grad_norm': '0.9458', 'learning_rate': '9.009e-05', 'epoch': '0.2703', 'num_input_tokens_seen': 3268064, 'train_runtime': '3189', 'train_tokens_per_second': '1025'}
+
+ 27%|██▋       | 490/1813 [53:08<2:25:42,  6.61s/it]
+ 27%|██▋       | 491/1813 [53:15<2:27:54,  6.71s/it]
+ 27%|██▋       | 492/1813 [53:23<2:33:49,  6.99s/it]
+ 27%|██▋       | 493/1813 [53:29<2:25:04,  6.59s/it]
+ 27%|██▋       | 494/1813 [53:34<2:20:09,  6.38s/it]
+ 27%|██▋       | 495/1813 [53:42<2:25:48,  6.64s/it]
+ 27%|██▋       | 496/1813 [53:48<2:24:22,  6.58s/it]
+ 27%|██▋       | 497/1813 [53:54<2:17:04,  6.25s/it]
+ 27%|██▋       | 498/1813 [54:01<2:23:07,  6.53s/it]
+ 28%|██▊       | 499/1813 [54:07<2:19:53,  6.39s/it]
+ 28%|██▊       | 500/1813 [54:16<2:37:35,  7.20s/it]
+                                                    
+{'loss': '1.094', 'grad_norm': '0.6796', 'learning_rate': '8.952e-05', 'epoch': '0.2759', 'num_input_tokens_seen': 3341536, 'train_runtime': '3257', 'train_tokens_per_second': '1026'}
+
+ 28%|██▊       | 500/1813 [54:16<2:37:35,  7.20s/it]
+ 28%|██▊       | 501/1813 [54:22<2:29:40,  6.85s/it]
+ 28%|██▊       | 502/1813 [54:28<2:21:17,  6.47s/it]
+ 28%|██▊       | 503/1813 [54:33<2:12:08,  6.05s/it]
+ 28%|██▊       | 504/1813 [54:39<2:13:19,  6.11s/it]
+ 28%|██▊       | 505/1813 [54:46<2:19:34,  6.40s/it]
+ 28%|██▊       | 506/1813 [54:52<2:16:28,  6.27s/it]
+ 28%|██▊       | 507/1813 [54:59<2:23:19,  6.58s/it]
+ 28%|██▊       | 508/1813 [55:05<2:14:37,  6.19s/it]
+ 28%|██▊       | 509/1813 [55:10<2:08:06,  5.89s/it]
+ 28%|██▊       | 510/1813 [55:15<2:05:43,  5.79s/it]
+                                                    
+{'loss': '0.9266', 'grad_norm': '1.227', 'learning_rate': '8.894e-05', 'epoch': '0.2814', 'num_input_tokens_seen': 3402712, 'train_runtime': '3316', 'train_tokens_per_second': '1026'}
+
+ 28%|██▊       | 510/1813 [55:15<2:05:43,  5.79s/it]
+ 28%|██▊       | 511/1813 [55:21<2:04:31,  5.74s/it]
+ 28%|██▊       | 512/1813 [55:28<2:11:30,  6.07s/it]
+ 28%|██▊       | 513/1813 [55:35<2:18:14,  6.38s/it]
+ 28%|██▊       | 514/1813 [55:42<2:20:52,  6.51s/it]
+ 28%|██▊       | 515/1813 [55:47<2:11:26,  6.08s/it]
+ 28%|██▊       | 516/1813 [55:52<2:08:07,  5.93s/it]
+ 29%|██▊       | 517/1813 [55:59<2:12:32,  6.14s/it]
+ 29%|██▊       | 518/1813 [56:05<2:11:57,  6.11s/it]
+ 29%|██▊       | 519/1813 [56:11<2:12:26,  6.14s/it]
+ 29%|██▊       | 520/1813 [56:18<2:17:18,  6.37s/it]
+                                                    
+{'loss': '0.9738', 'grad_norm': '0.9356', 'learning_rate': '8.834e-05', 'epoch': '0.2869', 'num_input_tokens_seen': 3469624, 'train_runtime': '3379', 'train_tokens_per_second': '1027'}
+
+ 29%|██▊       | 520/1813 [56:18<2:17:18,  6.37s/it]
+ 29%|██▊       | 521/1813 [56:22<2:04:08,  5.77s/it]
+ 29%|██▉       | 522/1813 [56:29<2:06:20,  5.87s/it]
+ 29%|██▉       | 523/1813 [56:35<2:07:41,  5.94s/it]
+ 29%|██▉       | 524/1813 [56:41<2:09:52,  6.05s/it]
+ 29%|██▉       | 525/1813 [56:46<2:03:03,  5.73s/it]
+ 29%|██▉       | 526/1813 [56:53<2:09:05,  6.02s/it]
+ 29%|██▉       | 527/1813 [56:59<2:12:40,  6.19s/it]
+ 29%|██▉       | 528/1813 [57:06<2:17:12,  6.41s/it]
+ 29%|██▉       | 529/1813 [57:13<2:21:22,  6.61s/it]
+ 29%|██▉       | 530/1813 [57:19<2:16:08,  6.37s/it]
+                                                    
+{'loss': '0.965', 'grad_norm': '1.142', 'learning_rate': '8.772e-05', 'epoch': '0.2924', 'num_input_tokens_seen': 3534904, 'train_runtime': '3440', 'train_tokens_per_second': '1028'}
+
+ 29%|██▉       | 530/1813 [57:19<2:16:08,  6.37s/it]
+ 29%|██▉       | 531/1813 [57:27<2:27:23,  6.90s/it]
+ 29%|██▉       | 532/1813 [57:33<2:23:11,  6.71s/it]
+ 29%|██▉       | 533/1813 [57:41<2:25:38,  6.83s/it]
+ 29%|██▉       | 534/1813 [57:48<2:28:08,  6.95s/it]
+ 30%|██▉       | 535/1813 [57:52<2:11:29,  6.17s/it]
+ 30%|██▉       | 536/1813 [57:57<2:04:38,  5.86s/it]
+ 30%|██▉       | 537/1813 [58:04<2:09:08,  6.07s/it]
+ 30%|██▉       | 538/1813 [58:09<2:05:50,  5.92s/it]
+ 30%|██▉       | 539/1813 [58:16<2:07:52,  6.02s/it]
+ 30%|██▉       | 540/1813 [58:22<2:09:56,  6.12s/it]
+                                                    
+{'loss': '0.9709', 'grad_norm': '0.7969', 'learning_rate': '8.71e-05', 'epoch': '0.2979', 'num_input_tokens_seen': 3603488, 'train_runtime': '3503', 'train_tokens_per_second': '1029'}
+
+ 30%|██▉       | 540/1813 [58:22<2:09:56,  6.12s/it]
+ 30%|██▉       | 541/1813 [58:27<2:05:31,  5.92s/it]
+ 30%|██▉       | 542/1813 [58:33<2:04:44,  5.89s/it]
+ 30%|██▉       | 543/1813 [58:41<2:14:03,  6.33s/it]
+ 30%|███       | 544/1813 [58:49<2:27:09,  6.96s/it]
+ 30%|███       | 545/1813 [58:56<2:24:30,  6.84s/it]
+ 30%|███       | 546/1813 [59:02<2:22:21,  6.74s/it]
+ 30%|███       | 547/1813 [59:10<2:27:49,  7.01s/it]
+ 30%|███       | 548/1813 [59:17<2:30:28,  7.14s/it]
+ 30%|███       | 549/1813 [59:23<2:22:19,  6.76s/it]
+ 30%|███       | 550/1813 [59:29<2:17:30,  6.53s/it]
+                                                    
+{'loss': '0.9095', 'grad_norm': '0.9145', 'learning_rate': '8.646e-05', 'epoch': '0.3034', 'num_input_tokens_seen': 3676576, 'train_runtime': '3570', 'train_tokens_per_second': '1030'}
+
+ 30%|███       | 550/1813 [59:29<2:17:30,  6.53s/it]
+ 30%|███       | 551/1813 [59:35<2:13:48,  6.36s/it]
+ 30%|███       | 552/1813 [59:40<2:07:27,  6.06s/it]
+ 31%|███       | 553/1813 [59:45<1:59:41,  5.70s/it]
+ 31%|███       | 554/1813 [59:52<2:06:26,  6.03s/it]
+ 31%|███       | 555/1813 [59:57<2:01:14,  5.78s/it]
+ 31%|███       | 556/1813 [1:00:03<2:03:09,  5.88s/it]
+ 31%|███       | 557/1813 [1:00:10<2:09:39,  6.19s/it]
+ 31%|███       | 558/1813 [1:00:16<2:07:13,  6.08s/it]
+ 31%|███       | 559/1813 [1:00:21<2:01:13,  5.80s/it]
+ 31%|███       | 560/1813 [1:00:28<2:05:30,  6.01s/it]
+                                                      
+{'loss': '0.9786', 'grad_norm': '0.6304', 'learning_rate': '8.58e-05', 'epoch': '0.309', 'num_input_tokens_seen': 3738056, 'train_runtime': '3628', 'train_tokens_per_second': '1030'}
+
+ 31%|███       | 560/1813 [1:00:28<2:05:30,  6.01s/it]
+ 31%|███       | 561/1813 [1:00:34<2:04:46,  5.98s/it]
+ 31%|███       | 562/1813 [1:00:40<2:08:41,  6.17s/it]
+ 31%|███       | 563/1813 [1:00:47<2:13:18,  6.40s/it]
+ 31%|███       | 564/1813 [1:00:53<2:11:48,  6.33s/it]
+ 31%|███       | 565/1813 [1:00:59<2:07:35,  6.13s/it]
+ 31%|███       | 566/1813 [1:01:06<2:10:25,  6.28s/it]
+ 31%|███▏      | 567/1813 [1:01:11<2:03:57,  5.97s/it]
+ 31%|███▏      | 568/1813 [1:01:18<2:08:15,  6.18s/it]
+ 31%|███▏      | 569/1813 [1:01:25<2:16:06,  6.56s/it]
+ 31%|███▏      | 570/1813 [1:01:33<2:22:01,  6.86s/it]
+                                                      
+{'loss': '0.9762', 'grad_norm': '0.6829', 'learning_rate': '8.514e-05', 'epoch': '0.3145', 'num_input_tokens_seen': 3808208, 'train_runtime': '3693', 'train_tokens_per_second': '1031'}
+
+ 31%|███▏      | 570/1813 [1:01:33<2:22:01,  6.86s/it]
+ 31%|███▏      | 571/1813 [1:01:39<2:21:19,  6.83s/it]
+ 32%|███▏      | 572/1813 [1:01:45<2:15:52,  6.57s/it]
+ 32%|███▏      | 573/1813 [1:01:51<2:07:03,  6.15s/it]
+ 32%|███▏      | 574/1813 [1:01:56<2:03:53,  6.00s/it]
+ 32%|███▏      | 575/1813 [1:02:03<2:07:42,  6.19s/it]
+ 32%|███▏      | 576/1813 [1:02:08<1:58:57,  5.77s/it]
+ 32%|███▏      | 577/1813 [1:02:14<2:05:01,  6.07s/it]
+ 32%|███▏      | 578/1813 [1:02:21<2:06:16,  6.14s/it]
+ 32%|███▏      | 579/1813 [1:02:27<2:07:20,  6.19s/it]
+ 32%|███▏      | 580/1813 [1:02:33<2:08:36,  6.26s/it]
+                                                      
+{'loss': '0.9771', 'grad_norm': '0.8526', 'learning_rate': '8.446e-05', 'epoch': '0.32', 'num_input_tokens_seen': 3872328, 'train_runtime': '3754', 'train_tokens_per_second': '1032'}
+
+ 32%|███▏      | 580/1813 [1:02:33<2:08:36,  6.26s/it]
+ 32%|███▏      | 581/1813 [1:02:39<2:06:19,  6.15s/it]
+ 32%|███▏      | 582/1813 [1:02:45<2:02:31,  5.97s/it]
+ 32%|███▏      | 583/1813 [1:02:51<2:02:46,  5.99s/it]
+ 32%|███▏      | 584/1813 [1:02:57<2:05:18,  6.12s/it]
+ 32%|███▏      | 585/1813 [1:03:03<2:03:50,  6.05s/it]
+ 32%|███▏      | 586/1813 [1:03:10<2:05:49,  6.15s/it]
+ 32%|███▏      | 587/1813 [1:03:15<2:03:47,  6.06s/it]
+ 32%|███▏      | 588/1813 [1:03:21<2:03:29,  6.05s/it]
+ 32%|███▏      | 589/1813 [1:03:28<2:04:12,  6.09s/it]
+ 33%|███▎      | 590/1813 [1:03:34<2:03:55,  6.08s/it]
+                                                      
+{'loss': '0.8897', 'grad_norm': '0.9513', 'learning_rate': '8.377e-05', 'epoch': '0.3255', 'num_input_tokens_seen': 3936024, 'train_runtime': '3814', 'train_tokens_per_second': '1032'}
+
+ 33%|███▎      | 590/1813 [1:03:34<2:03:55,  6.08s/it]
+ 33%|███▎      | 591/1813 [1:03:39<1:59:42,  5.88s/it]
+ 33%|███▎      | 592/1813 [1:03:45<2:00:52,  5.94s/it]
+ 33%|███▎      | 593/1813 [1:03:52<2:04:03,  6.10s/it]
+ 33%|███▎      | 594/1813 [1:03:58<2:03:00,  6.05s/it]
+ 33%|███▎      | 595/1813 [1:04:03<2:00:12,  5.92s/it]
+ 33%|███▎      | 596/1813 [1:04:09<2:00:46,  5.95s/it]
+ 33%|███▎      | 597/1813 [1:04:15<1:58:44,  5.86s/it]
+ 33%|███▎      | 598/1813 [1:04:21<1:57:52,  5.82s/it]
+ 33%|███▎      | 599/1813 [1:04:27<2:00:33,  5.96s/it]
+ 33%|███▎      | 600/1813 [1:04:34<2:07:05,  6.29s/it]
+                                                      
+{'loss': '0.9384', 'grad_norm': '0.7574', 'learning_rate': '8.307e-05', 'epoch': '0.331', 'num_input_tokens_seen': 3999176, 'train_runtime': '3874', 'train_tokens_per_second': '1032'}
+
+ 33%|███▎      | 600/1813 [1:04:34<2:07:05,  6.29s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.74it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.34it/s]
+
+  1%|          | 9/1000 [00:01<02:37,  6.29it/s]
+
+  1%|          | 11/1000 [00:01<02:05,  7.88it/s]
+
+  1%|▏         | 13/1000 [00:01<02:23,  6.88it/s]
+
+  1%|▏         | 14/1000 [00:01<02:26,  6.74it/s]
+
+  2%|▏         | 15/1000 [00:02<02:46,  5.92it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.69it/s]
+
+  2%|▏         | 18/1000 [00:02<02:43,  6.02it/s]
+
+  2%|▏         | 19/1000 [00:02<03:11,  5.13it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.59it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.58it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.34it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.75it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.94it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.75it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.30it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.11it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.85it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.86it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.19it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.01it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.28it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.40it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.15it/s]
+
+ 11%|█         | 112/1000 [00:14<01:14, 11.98it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.48it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.78it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.23it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:48,  8.09it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.44it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.65it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.89it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.95it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.02it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.07it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.91it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.02it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.38it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.96it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.60it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.77it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.83it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.14it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.14it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.51it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.89it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.37it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.62it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.65it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.78it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.63it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.93it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.68it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.73it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.14it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.81it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.89it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.36it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.19it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.92it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.43it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:25,  8.95it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.00it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.45it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.64it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.24it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:42,  7.27it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:37,  7.62it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:26,  8.59it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:37,  7.62it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:54,  6.47it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:55,  6.39it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.31it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.58it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.51it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:44,  6.99it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.42it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.68it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:30,  8.03it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.01it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.84it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.72it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.99it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.06it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.13it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.77it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.13it/s]
+
+ 31%|███       | 306/1000 [00:41<01:40,  6.93it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.77it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.78it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.19it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.24it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:07, 10.22it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.56it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.40it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.16it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.57it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:36,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.37it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.02it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.86it/s]
+
+ 34%|███▍      | 344/1000 [00:46<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.21it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.40it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.90it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.94it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.63it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.58it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.42it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:58,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.21it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.91it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.76it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.89it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.17it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:07,  9.16it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.13it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.63it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.35it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.77it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.46it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.22it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.23it/s]
+
+ 41%|████      | 406/1000 [00:54<01:10,  8.48it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:56<01:23,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.44it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.92it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.56it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.03it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:48, 11.77it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.50it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.29it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:10,  7.93it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:17,  7.15it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:04,  8.63it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:16,  7.26it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:19,  6.88it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:29,  6.10it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:39,  5.52it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.12it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.44it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.76it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.21it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:50, 10.78it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.61it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.73it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:29,  6.00it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.30it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.84it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.19it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.36it/s]
+
+ 47%|████▋     | 474/1000 [01:04<01:25,  6.17it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.18it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.29it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.09it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.51it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.05it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.29it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.26it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.98it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.53it/s]
+
+ 50%|█████     | 503/1000 [01:08<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.55it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.42it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.95it/s]
+
+ 51%|█████     | 512/1000 [01:09<00:48,  9.96it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.10it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.07it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.38it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<01:00,  7.97it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.65it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.79it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.58it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.39it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.10it/s]
+
+ 53%|█████▎    | 533/1000 [01:12<00:48,  9.66it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.68it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.35it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:22,  5.64it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:20,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:06,  6.92it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.05it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.49it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.91it/s]
+
+ 55%|█████▌    | 551/1000 [01:15<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.86it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.04it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.33it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.81it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.72it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.55it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.57it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.40it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:02,  6.46it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:10,  5.58it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:56,  7.00it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:50,  7.77it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.39it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:40,  9.64it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:36, 10.72it/s]
+
+ 62%|██████▏   | 616/1000 [01:24<00:50,  7.59it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.73it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:42,  9.07it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.00it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.18it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.45it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.63it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.17it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.84it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.42it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.94it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:28<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.54it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.31it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:45,  7.69it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.90it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.08it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  5.99it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.47it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.62it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.25it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.64it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.95it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:36,  8.75it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:32,  9.74it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.81it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.11it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.77it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.80it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.24it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.40it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:48,  6.32it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.93it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:36<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.72it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.87it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.38it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.05it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.22it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▎  | 725/1000 [01:39<00:33,  8.30it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:27, 10.10it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.57it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.31it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.51it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.86it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.51it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.76it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.60it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.44it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.71it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.64it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.88it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.59it/s]
+
+ 78%|███████▊  | 780/1000 [01:46<00:31,  6.94it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.99it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.18it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.63it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.96it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.20it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.21it/s]
+
+ 81%|████████▏ | 813/1000 [01:50<00:20,  9.03it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.25it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.11it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.17it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.03it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.81it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.99it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.00it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 833/1000 [01:53<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.44it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.15it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.54it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.86it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.46it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.03it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.19it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.76it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.63it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.70it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.31it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.27it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.29it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.64it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:59<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.95it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.16it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.76it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.83it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.69it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.76it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.38it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:04<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.93it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+
+ 93%|█████████▎| 927/1000 [02:05<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.64it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:06<00:10,  6.60it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.16it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.13it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.33it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:08<00:05,  8.84it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.40it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.44it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.32it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.00it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.41it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.18it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.78it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.77it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.05it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.43it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.91it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.52it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.8095', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.379', 'eval_steps_per_second': '7.379', 'epoch': '0.331', 'num_input_tokens_seen': 3999176}
+
+ 33%|███▎      | 600/1813 [1:06:49<2:07:05,  6.29s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 33%|███▎      | 601/1813 [1:06:54<15:37:31, 46.41s/it]
+ 33%|███▎      | 602/1813 [1:06:59<11:25:30, 33.96s/it]
+ 33%|███▎      | 603/1813 [1:07:04<8:32:28, 25.41s/it] 
+ 33%|███▎      | 604/1813 [1:07:10<6:35:12, 19.61s/it]
+ 33%|███▎      | 605/1813 [1:07:18<5:24:35, 16.12s/it]
+ 33%|███▎      | 606/1813 [1:07:26<4:31:29, 13.50s/it]
+ 33%|███▎      | 607/1813 [1:07:32<3:46:50, 11.29s/it]
+ 34%|███▎      | 608/1813 [1:07:38<3:16:38,  9.79s/it]
+ 34%|███▎      | 609/1813 [1:07:44<2:54:56,  8.72s/it]
+ 34%|███▎      | 610/1813 [1:07:51<2:40:12,  7.99s/it]
+                                                      
+{'loss': '0.98', 'grad_norm': '0.6249', 'learning_rate': '8.235e-05', 'epoch': '0.3366', 'num_input_tokens_seen': 4066376, 'train_runtime': '4071', 'train_tokens_per_second': '998.8'}
+
+ 34%|███▎      | 610/1813 [1:07:51<2:40:12,  7.99s/it]
+ 34%|███▎      | 611/1813 [1:07:56<2:23:39,  7.17s/it]
+ 34%|███▍      | 612/1813 [1:08:00<2:05:28,  6.27s/it]
+ 34%|███▍      | 613/1813 [1:08:06<2:03:51,  6.19s/it]
+ 34%|███▍      | 614/1813 [1:08:12<2:02:54,  6.15s/it]
+ 34%|███▍      | 615/1813 [1:08:18<1:57:52,  5.90s/it]
+ 34%|███▍      | 616/1813 [1:08:23<1:54:26,  5.74s/it]
+ 34%|███▍      | 617/1813 [1:08:28<1:52:52,  5.66s/it]
+ 34%|███▍      | 618/1813 [1:08:35<2:01:09,  6.08s/it]
+ 34%|███▍      | 619/1813 [1:08:40<1:53:59,  5.73s/it]
+ 34%|███▍      | 620/1813 [1:08:47<1:58:41,  5.97s/it]
+                                                      
+{'loss': '0.8698', 'grad_norm': '0.8595', 'learning_rate': '8.163e-05', 'epoch': '0.3421', 'num_input_tokens_seen': 4123976, 'train_runtime': '4127', 'train_tokens_per_second': '999.2'}
+
+ 34%|███▍      | 620/1813 [1:08:47<1:58:41,  5.97s/it]
+ 34%|███▍      | 621/1813 [1:08:54<2:03:47,  6.23s/it]
+ 34%|███▍      | 622/1813 [1:09:01<2:12:24,  6.67s/it]
+ 34%|███▍      | 623/1813 [1:09:06<2:02:54,  6.20s/it]
+ 34%|███▍      | 624/1813 [1:09:13<2:03:53,  6.25s/it]
+ 34%|███▍      | 625/1813 [1:09:18<1:56:20,  5.88s/it]
+ 35%|███▍      | 626/1813 [1:09:22<1:47:39,  5.44s/it]
+ 35%|███▍      | 627/1813 [1:09:28<1:51:53,  5.66s/it]
+ 35%|███▍      | 628/1813 [1:09:35<1:56:08,  5.88s/it]
+ 35%|███▍      | 629/1813 [1:09:43<2:10:15,  6.60s/it]
+ 35%|███▍      | 630/1813 [1:09:50<2:10:51,  6.64s/it]
+                                                      
+{'loss': '0.9488', 'grad_norm': '0.6753', 'learning_rate': '8.089e-05', 'epoch': '0.3476', 'num_input_tokens_seen': 4191992, 'train_runtime': '4190', 'train_tokens_per_second': '1000'}
+
+ 35%|███▍      | 630/1813 [1:09:50<2:10:51,  6.64s/it]
+ 35%|███▍      | 631/1813 [1:09:56<2:09:24,  6.57s/it]
+ 35%|███▍      | 632/1813 [1:10:02<2:05:53,  6.40s/it]
+ 35%|███▍      | 633/1813 [1:10:08<2:01:34,  6.18s/it]
+ 35%|███▍      | 634/1813 [1:10:15<2:07:48,  6.50s/it]
+ 35%|███▌      | 635/1813 [1:10:22<2:09:53,  6.62s/it]
+ 35%|███▌      | 636/1813 [1:10:27<2:01:54,  6.21s/it]
+ 35%|███▌      | 637/1813 [1:10:34<2:06:17,  6.44s/it]
+ 35%|███▌      | 638/1813 [1:10:40<2:01:43,  6.22s/it]
+ 35%|███▌      | 639/1813 [1:10:45<1:55:40,  5.91s/it]
+ 35%|███▌      | 640/1813 [1:10:51<1:55:59,  5.93s/it]
+                                                      
+{'loss': '0.9611', 'grad_norm': '0.9323', 'learning_rate': '8.014e-05', 'epoch': '0.3531', 'num_input_tokens_seen': 4257760, 'train_runtime': '4252', 'train_tokens_per_second': '1001'}
+
+ 35%|███▌      | 640/1813 [1:10:51<1:55:59,  5.93s/it]
+ 35%|███▌      | 641/1813 [1:10:57<1:53:10,  5.79s/it]
+ 35%|███▌      | 642/1813 [1:11:04<2:01:45,  6.24s/it]
+ 35%|███▌      | 643/1813 [1:11:11<2:03:40,  6.34s/it]
+ 36%|███▌      | 644/1813 [1:11:16<1:58:11,  6.07s/it]
+ 36%|███▌      | 645/1813 [1:11:22<1:57:55,  6.06s/it]
+ 36%|███▌      | 646/1813 [1:11:28<1:56:26,  5.99s/it]
+ 36%|███▌      | 647/1813 [1:11:34<1:57:54,  6.07s/it]
+ 36%|███▌      | 648/1813 [1:11:41<2:03:10,  6.34s/it]
+ 36%|███▌      | 649/1813 [1:11:47<1:58:58,  6.13s/it]
+ 36%|███▌      | 650/1813 [1:11:52<1:52:30,  5.80s/it]
+                                                      
+{'loss': '0.9568', 'grad_norm': '0.9603', 'learning_rate': '7.938e-05', 'epoch': '0.3586', 'num_input_tokens_seen': 4322136, 'train_runtime': '4312', 'train_tokens_per_second': '1002'}
+
+ 36%|███▌      | 650/1813 [1:11:52<1:52:30,  5.80s/it]
+ 36%|███▌      | 651/1813 [1:11:58<1:56:18,  6.01s/it]
+ 36%|███▌      | 652/1813 [1:12:05<2:00:44,  6.24s/it]
+ 36%|███▌      | 653/1813 [1:12:10<1:55:34,  5.98s/it]
+ 36%|███▌      | 654/1813 [1:12:16<1:51:31,  5.77s/it]
+ 36%|███▌      | 655/1813 [1:12:22<1:54:07,  5.91s/it]
+ 36%|███▌      | 656/1813 [1:12:27<1:50:39,  5.74s/it]
+ 36%|███▌      | 657/1813 [1:12:32<1:46:32,  5.53s/it]
+ 36%|███▋      | 658/1813 [1:12:39<1:51:25,  5.79s/it]
+ 36%|███▋      | 659/1813 [1:12:45<1:54:53,  5.97s/it]
+ 36%|███▋      | 660/1813 [1:12:51<1:52:26,  5.85s/it]
+                                                      
+{'loss': '0.8933', 'grad_norm': '0.8715', 'learning_rate': '7.861e-05', 'epoch': '0.3641', 'num_input_tokens_seen': 4385688, 'train_runtime': '4371', 'train_tokens_per_second': '1003'}
+
+ 36%|███▋      | 660/1813 [1:12:51<1:52:26,  5.85s/it]
+ 36%|███▋      | 661/1813 [1:12:57<1:52:57,  5.88s/it]
+ 37%|███▋      | 662/1813 [1:13:03<1:53:58,  5.94s/it]
+ 37%|███▋      | 663/1813 [1:13:09<1:58:34,  6.19s/it]
+ 37%|███▋      | 664/1813 [1:13:16<1:58:55,  6.21s/it]
+ 37%|███▋      | 665/1813 [1:13:21<1:51:19,  5.82s/it]
+ 37%|███▋      | 666/1813 [1:13:26<1:48:10,  5.66s/it]
+ 37%|███▋      | 667/1813 [1:13:33<1:53:59,  5.97s/it]
+ 37%|███▋      | 668/1813 [1:13:39<1:55:49,  6.07s/it]
+ 37%|███▋      | 669/1813 [1:13:45<1:54:34,  6.01s/it]
+ 37%|███▋      | 670/1813 [1:13:50<1:52:05,  5.88s/it]
+                                                      
+{'loss': '0.9825', 'grad_norm': '0.9484', 'learning_rate': '7.783e-05', 'epoch': '0.3697', 'num_input_tokens_seen': 4449456, 'train_runtime': '4431', 'train_tokens_per_second': '1004'}
+
+ 37%|███▋      | 670/1813 [1:13:50<1:52:05,  5.88s/it]
+ 37%|███▋      | 671/1813 [1:13:56<1:53:18,  5.95s/it]
+ 37%|███▋      | 672/1813 [1:14:03<1:57:17,  6.17s/it]
+ 37%|███▋      | 673/1813 [1:14:09<1:57:21,  6.18s/it]
+ 37%|███▋      | 674/1813 [1:14:16<2:00:11,  6.33s/it]
+ 37%|███▋      | 675/1813 [1:14:22<1:57:30,  6.20s/it]
+ 37%|███▋      | 676/1813 [1:14:27<1:53:34,  5.99s/it]
+ 37%|███▋      | 677/1813 [1:14:36<2:06:12,  6.67s/it]
+ 37%|███▋      | 678/1813 [1:14:41<1:57:30,  6.21s/it]
+ 37%|███▋      | 679/1813 [1:14:46<1:51:24,  5.89s/it]
+ 38%|███▊      | 680/1813 [1:14:51<1:49:02,  5.77s/it]
+                                                      
+{'loss': '0.939', 'grad_norm': '0.789', 'learning_rate': '7.704e-05', 'epoch': '0.3752', 'num_input_tokens_seen': 4514984, 'train_runtime': '4492', 'train_tokens_per_second': '1005'}
+
+ 38%|███▊      | 680/1813 [1:14:51<1:49:02,  5.77s/it]
+ 38%|███▊      | 681/1813 [1:14:58<1:52:35,  5.97s/it]
+ 38%|███▊      | 682/1813 [1:15:03<1:47:35,  5.71s/it]
+ 38%|███▊      | 683/1813 [1:15:09<1:49:24,  5.81s/it]
+ 38%|███▊      | 684/1813 [1:15:16<1:53:50,  6.05s/it]
+ 38%|███▊      | 685/1813 [1:15:23<2:02:35,  6.52s/it]
+ 38%|███▊      | 686/1813 [1:15:30<2:01:50,  6.49s/it]
+ 38%|███▊      | 687/1813 [1:15:37<2:08:24,  6.84s/it]
+ 38%|███▊      | 688/1813 [1:15:42<1:55:02,  6.14s/it]
+ 38%|███▊      | 689/1813 [1:15:50<2:09:07,  6.89s/it]
+ 38%|███▊      | 690/1813 [1:15:58<2:10:16,  6.96s/it]
+                                                      
+{'loss': '0.9791', 'grad_norm': '0.9098', 'learning_rate': '7.624e-05', 'epoch': '0.3807', 'num_input_tokens_seen': 4587840, 'train_runtime': '4558', 'train_tokens_per_second': '1007'}
+
+ 38%|███▊      | 690/1813 [1:15:58<2:10:16,  6.96s/it]
+ 38%|███▊      | 691/1813 [1:16:03<2:02:42,  6.56s/it]
+ 38%|███▊      | 692/1813 [1:16:09<2:00:39,  6.46s/it]
+ 38%|███▊      | 693/1813 [1:16:15<1:55:43,  6.20s/it]
+ 38%|███▊      | 694/1813 [1:16:21<1:56:43,  6.26s/it]
+ 38%|███▊      | 695/1813 [1:16:28<1:56:11,  6.24s/it]
+ 38%|███▊      | 696/1813 [1:16:36<2:05:58,  6.77s/it]
+ 38%|███▊      | 697/1813 [1:16:40<1:54:29,  6.16s/it]
+ 38%|███▊      | 698/1813 [1:16:46<1:49:45,  5.91s/it]
+ 39%|███▊      | 699/1813 [1:16:53<1:55:46,  6.24s/it]
+ 39%|███▊      | 700/1813 [1:16:59<1:58:23,  6.38s/it]
+                                                      
+{'loss': '1.001', 'grad_norm': '0.9895', 'learning_rate': '7.543e-05', 'epoch': '0.3862', 'num_input_tokens_seen': 4653312, 'train_runtime': '4620', 'train_tokens_per_second': '1007'}
+
+ 39%|███▊      | 700/1813 [1:16:59<1:58:23,  6.38s/it]
+ 39%|███▊      | 701/1813 [1:17:07<2:03:25,  6.66s/it]
+ 39%|███▊      | 702/1813 [1:17:13<2:00:52,  6.53s/it]
+ 39%|███▉      | 703/1813 [1:17:21<2:08:18,  6.94s/it]
+ 39%|███▉      | 704/1813 [1:17:26<2:01:06,  6.55s/it]
+ 39%|███▉      | 705/1813 [1:17:33<2:02:55,  6.66s/it]
+ 39%|███▉      | 706/1813 [1:17:40<2:00:41,  6.54s/it]
+ 39%|███▉      | 707/1813 [1:17:49<2:17:38,  7.47s/it]
+ 39%|███▉      | 708/1813 [1:17:57<2:17:02,  7.44s/it]
+ 39%|███▉      | 709/1813 [1:18:03<2:12:04,  7.18s/it]
+ 39%|███▉      | 710/1813 [1:18:09<2:03:23,  6.71s/it]
+                                                      
+{'loss': '0.957', 'grad_norm': '0.7076', 'learning_rate': '7.462e-05', 'epoch': '0.3917', 'num_input_tokens_seen': 4732080, 'train_runtime': '4689', 'train_tokens_per_second': '1009'}
+
+ 39%|███▉      | 710/1813 [1:18:09<2:03:23,  6.71s/it]
+ 39%|███▉      | 711/1813 [1:18:16<2:04:32,  6.78s/it]
+ 39%|███▉      | 712/1813 [1:18:23<2:04:10,  6.77s/it]
+ 39%|███▉      | 713/1813 [1:18:28<1:58:06,  6.44s/it]
+ 39%|███▉      | 714/1813 [1:18:34<1:56:27,  6.36s/it]
+ 39%|███▉      | 715/1813 [1:18:41<1:58:20,  6.47s/it]
+ 39%|███▉      | 716/1813 [1:18:48<2:01:15,  6.63s/it]
+ 40%|███▉      | 717/1813 [1:18:54<1:58:54,  6.51s/it]
+ 40%|███▉      | 718/1813 [1:19:01<1:58:53,  6.51s/it]
+ 40%|███▉      | 719/1813 [1:19:06<1:52:15,  6.16s/it]
+ 40%|███▉      | 720/1813 [1:19:13<1:53:10,  6.21s/it]
+                                                      
+{'loss': '0.9892', 'grad_norm': '0.8414', 'learning_rate': '7.379e-05', 'epoch': '0.3972', 'num_input_tokens_seen': 4799552, 'train_runtime': '4753', 'train_tokens_per_second': '1010'}
+
+ 40%|███▉      | 720/1813 [1:19:13<1:53:10,  6.21s/it]
+ 40%|███▉      | 721/1813 [1:19:19<1:55:14,  6.33s/it]
+ 40%|███▉      | 722/1813 [1:19:24<1:48:13,  5.95s/it]
+ 40%|███▉      | 723/1813 [1:19:29<1:43:53,  5.72s/it]
+ 40%|███▉      | 724/1813 [1:19:37<1:55:35,  6.37s/it]
+ 40%|███▉      | 725/1813 [1:19:46<2:06:33,  6.98s/it]
+ 40%|████      | 726/1813 [1:19:52<2:04:37,  6.88s/it]
+ 40%|████      | 727/1813 [1:19:58<2:00:36,  6.66s/it]
+ 40%|████      | 728/1813 [1:20:04<1:52:14,  6.21s/it]
+ 40%|████      | 729/1813 [1:20:09<1:50:06,  6.09s/it]
+ 40%|████      | 730/1813 [1:20:15<1:45:59,  5.87s/it]
+                                                      
+{'loss': '0.9659', 'grad_norm': '1.013', 'learning_rate': '7.296e-05', 'epoch': '0.4028', 'num_input_tokens_seen': 4866864, 'train_runtime': '4815', 'train_tokens_per_second': '1011'}
+
+ 40%|████      | 730/1813 [1:20:15<1:45:59,  5.87s/it]
+ 40%|████      | 731/1813 [1:20:22<1:51:42,  6.19s/it]
+ 40%|████      | 732/1813 [1:20:28<1:51:32,  6.19s/it]
+ 40%|████      | 733/1813 [1:20:35<1:57:44,  6.54s/it]
+ 40%|████      | 734/1813 [1:20:42<1:59:40,  6.66s/it]
+ 41%|████      | 735/1813 [1:20:49<1:59:35,  6.66s/it]
+ 41%|████      | 736/1813 [1:20:55<1:54:36,  6.39s/it]
+ 41%|████      | 737/1813 [1:21:00<1:51:05,  6.19s/it]
+ 41%|████      | 738/1813 [1:21:05<1:44:55,  5.86s/it]
+ 41%|████      | 739/1813 [1:21:11<1:44:38,  5.85s/it]
+ 41%|████      | 740/1813 [1:21:18<1:48:19,  6.06s/it]
+                                                      
+{'loss': '0.8938', 'grad_norm': '0.6765', 'learning_rate': '7.211e-05', 'epoch': '0.4083', 'num_input_tokens_seen': 4936352, 'train_runtime': '4878', 'train_tokens_per_second': '1012'}
+
+ 41%|████      | 740/1813 [1:21:18<1:48:19,  6.06s/it]
+ 41%|████      | 741/1813 [1:21:24<1:50:08,  6.16s/it]
+ 41%|████      | 742/1813 [1:21:32<1:56:50,  6.55s/it]
+ 41%|████      | 743/1813 [1:21:39<2:01:06,  6.79s/it]
+ 41%|████      | 744/1813 [1:21:46<1:59:53,  6.73s/it]
+ 41%|████      | 745/1813 [1:21:53<2:01:11,  6.81s/it]
+ 41%|████      | 746/1813 [1:21:59<1:56:48,  6.57s/it]
+ 41%|████      | 747/1813 [1:22:07<2:03:53,  6.97s/it]
+ 41%|████▏     | 748/1813 [1:22:12<1:55:47,  6.52s/it]
+ 41%|████▏     | 749/1813 [1:22:18<1:54:40,  6.47s/it]
+ 41%|████▏     | 750/1813 [1:22:24<1:50:49,  6.26s/it]
+                                                      
+{'loss': '0.978', 'grad_norm': '0.7119', 'learning_rate': '7.126e-05', 'epoch': '0.4138', 'num_input_tokens_seen': 5011064, 'train_runtime': '4945', 'train_tokens_per_second': '1013'}
+
+ 41%|████▏     | 750/1813 [1:22:24<1:50:49,  6.26s/it]
+ 41%|████▏     | 751/1813 [1:22:30<1:48:53,  6.15s/it]
+ 41%|████▏     | 752/1813 [1:22:36<1:47:00,  6.05s/it]
+ 42%|████▏     | 753/1813 [1:22:42<1:49:25,  6.19s/it]
+ 42%|████▏     | 754/1813 [1:22:51<2:01:25,  6.88s/it]
+ 42%|████▏     | 755/1813 [1:22:56<1:51:51,  6.34s/it]
+ 42%|████▏     | 756/1813 [1:23:02<1:48:16,  6.15s/it]
+ 42%|████▏     | 757/1813 [1:23:07<1:46:45,  6.07s/it]
+ 42%|████▏     | 758/1813 [1:23:14<1:50:35,  6.29s/it]
+ 42%|████▏     | 759/1813 [1:23:21<1:54:32,  6.52s/it]
+ 42%|████▏     | 760/1813 [1:23:28<1:52:46,  6.43s/it]
+                                                      
+{'loss': '0.9966', 'grad_norm': '1.093', 'learning_rate': '7.04e-05', 'epoch': '0.4193', 'num_input_tokens_seen': 5078960, 'train_runtime': '5008', 'train_tokens_per_second': '1014'}
+
+ 42%|████▏     | 760/1813 [1:23:28<1:52:46,  6.43s/it]
+ 42%|████▏     | 761/1813 [1:23:34<1:54:50,  6.55s/it]
+ 42%|████▏     | 762/1813 [1:23:41<1:55:59,  6.62s/it]
+ 42%|████▏     | 763/1813 [1:23:48<1:59:09,  6.81s/it]
+ 42%|████▏     | 764/1813 [1:23:55<1:59:31,  6.84s/it]
+ 42%|████▏     | 765/1813 [1:24:03<2:02:16,  7.00s/it]
+ 42%|████▏     | 766/1813 [1:24:09<1:59:03,  6.82s/it]
+ 42%|████▏     | 767/1813 [1:24:15<1:51:37,  6.40s/it]
+ 42%|████▏     | 768/1813 [1:24:19<1:39:38,  5.72s/it]
+ 42%|████▏     | 769/1813 [1:24:25<1:44:30,  6.01s/it]
+ 42%|████▏     | 770/1813 [1:24:32<1:45:09,  6.05s/it]
+                                                      
+{'loss': '0.9243', 'grad_norm': '0.8774', 'learning_rate': '6.954e-05', 'epoch': '0.4248', 'num_input_tokens_seen': 5148504, 'train_runtime': '5072', 'train_tokens_per_second': '1015'}
+
+ 42%|████▏     | 770/1813 [1:24:32<1:45:09,  6.05s/it]
+ 43%|████▎     | 771/1813 [1:24:38<1:47:11,  6.17s/it]
+ 43%|████▎     | 772/1813 [1:24:45<1:52:43,  6.50s/it]
+ 43%|████▎     | 773/1813 [1:24:51<1:48:56,  6.28s/it]
+ 43%|████▎     | 774/1813 [1:24:56<1:42:06,  5.90s/it]
+ 43%|████▎     | 775/1813 [1:25:02<1:42:16,  5.91s/it]
+ 43%|████▎     | 776/1813 [1:25:09<1:46:28,  6.16s/it]
+ 43%|████▎     | 777/1813 [1:25:16<1:53:59,  6.60s/it]
+ 43%|████▎     | 778/1813 [1:25:22<1:46:49,  6.19s/it]
+ 43%|████▎     | 779/1813 [1:25:27<1:40:57,  5.86s/it]
+ 43%|████▎     | 780/1813 [1:25:33<1:45:57,  6.15s/it]
+                                                      
+{'loss': '0.8832', 'grad_norm': '0.9475', 'learning_rate': '6.866e-05', 'epoch': '0.4303', 'num_input_tokens_seen': 5215784, 'train_runtime': '5134', 'train_tokens_per_second': '1016'}
+
+ 43%|████▎     | 780/1813 [1:25:33<1:45:57,  6.15s/it]
+ 43%|████▎     | 781/1813 [1:25:40<1:49:44,  6.38s/it]
+ 43%|████▎     | 782/1813 [1:25:49<1:59:11,  6.94s/it]
+ 43%|████▎     | 783/1813 [1:25:56<2:00:13,  7.00s/it]
+ 43%|████▎     | 784/1813 [1:26:02<1:54:48,  6.69s/it]
+ 43%|████▎     | 785/1813 [1:26:09<1:56:57,  6.83s/it]
+ 43%|████▎     | 786/1813 [1:26:16<1:58:47,  6.94s/it]
+ 43%|████▎     | 787/1813 [1:26:22<1:51:00,  6.49s/it]
+ 43%|████▎     | 788/1813 [1:26:27<1:46:36,  6.24s/it]
+ 44%|████▎     | 789/1813 [1:26:33<1:42:08,  5.99s/it]
+ 44%|████▎     | 790/1813 [1:26:38<1:41:14,  5.94s/it]
+                                                      
+{'loss': '0.9949', 'grad_norm': '1.252', 'learning_rate': '6.778e-05', 'epoch': '0.4359', 'num_input_tokens_seen': 5285160, 'train_runtime': '5199', 'train_tokens_per_second': '1017'}
+
+ 44%|████▎     | 790/1813 [1:26:38<1:41:14,  5.94s/it]
+ 44%|████▎     | 791/1813 [1:26:45<1:41:56,  5.98s/it]
+ 44%|████▎     | 792/1813 [1:26:52<1:47:44,  6.33s/it]
+ 44%|████▎     | 793/1813 [1:26:59<1:52:06,  6.59s/it]
+ 44%|████▍     | 794/1813 [1:27:05<1:51:17,  6.55s/it]
+ 44%|████▍     | 795/1813 [1:27:11<1:47:30,  6.34s/it]
+ 44%|████▍     | 796/1813 [1:27:18<1:49:39,  6.47s/it]
+ 44%|████▍     | 797/1813 [1:27:26<1:55:10,  6.80s/it]
+ 44%|████▍     | 798/1813 [1:27:32<1:51:27,  6.59s/it]
+ 44%|████▍     | 799/1813 [1:27:38<1:48:22,  6.41s/it]
+ 44%|████▍     | 800/1813 [1:27:43<1:45:16,  6.24s/it]
+                                                      
+{'loss': '0.9382', 'grad_norm': '0.7748', 'learning_rate': '6.69e-05', 'epoch': '0.4414', 'num_input_tokens_seen': 5357720, 'train_runtime': '5264', 'train_tokens_per_second': '1018'}
+
+ 44%|████▍     | 800/1813 [1:27:43<1:45:16,  6.24s/it]
+ 44%|████▍     | 801/1813 [1:27:49<1:40:55,  5.98s/it]
+ 44%|████▍     | 802/1813 [1:27:55<1:41:48,  6.04s/it]
+ 44%|████▍     | 803/1813 [1:28:03<1:51:53,  6.65s/it]
+ 44%|████▍     | 804/1813 [1:28:09<1:50:02,  6.54s/it]
+ 44%|████▍     | 805/1813 [1:28:15<1:46:40,  6.35s/it]
+ 44%|████▍     | 806/1813 [1:28:21<1:44:25,  6.22s/it]
+ 45%|████▍     | 807/1813 [1:28:27<1:40:52,  6.02s/it]
+ 45%|████▍     | 808/1813 [1:28:34<1:49:18,  6.53s/it]
+ 45%|████▍     | 809/1813 [1:28:41<1:49:43,  6.56s/it]
+ 45%|████▍     | 810/1813 [1:28:49<1:55:46,  6.93s/it]
+                                                      
+{'loss': '0.9547', 'grad_norm': '0.6789', 'learning_rate': '6.601e-05', 'epoch': '0.4469', 'num_input_tokens_seen': 5430288, 'train_runtime': '5329', 'train_tokens_per_second': '1019'}
+
+ 45%|████▍     | 810/1813 [1:28:49<1:55:46,  6.93s/it]
+ 45%|████▍     | 811/1813 [1:28:56<1:55:09,  6.90s/it]
+ 45%|████▍     | 812/1813 [1:29:01<1:49:38,  6.57s/it]
+ 45%|████▍     | 813/1813 [1:29:07<1:45:41,  6.34s/it]
+ 45%|████▍     | 814/1813 [1:29:14<1:45:20,  6.33s/it]
+ 45%|████▍     | 815/1813 [1:29:18<1:38:03,  5.90s/it]
+ 45%|████▌     | 816/1813 [1:29:25<1:42:41,  6.18s/it]
+ 45%|████▌     | 817/1813 [1:29:31<1:40:30,  6.05s/it]
+ 45%|████▌     | 818/1813 [1:29:38<1:45:43,  6.38s/it]
+ 45%|████▌     | 819/1813 [1:29:45<1:46:40,  6.44s/it]
+ 45%|████▌     | 820/1813 [1:29:52<1:51:29,  6.74s/it]
+                                                      
+{'loss': '0.8936', 'grad_norm': '0.7883', 'learning_rate': '6.511e-05', 'epoch': '0.4524', 'num_input_tokens_seen': 5497552, 'train_runtime': '5393', 'train_tokens_per_second': '1019'}
+
+ 45%|████▌     | 820/1813 [1:29:52<1:51:29,  6.74s/it]
+ 45%|████▌     | 821/1813 [1:29:59<1:49:50,  6.64s/it]
+ 45%|████▌     | 822/1813 [1:30:03<1:40:48,  6.10s/it]
+ 45%|████▌     | 823/1813 [1:30:08<1:33:52,  5.69s/it]
+ 45%|████▌     | 824/1813 [1:30:15<1:37:06,  5.89s/it]
+ 46%|████▌     | 825/1813 [1:30:21<1:38:57,  6.01s/it]
+ 46%|████▌     | 826/1813 [1:30:28<1:42:58,  6.26s/it]
+ 46%|████▌     | 827/1813 [1:30:33<1:40:26,  6.11s/it]
+ 46%|████▌     | 828/1813 [1:30:39<1:39:17,  6.05s/it]
+ 46%|████▌     | 829/1813 [1:30:45<1:36:50,  5.90s/it]
+ 46%|████▌     | 830/1813 [1:30:52<1:44:01,  6.35s/it]
+                                                      
+{'loss': '0.7945', 'grad_norm': '0.8077', 'learning_rate': '6.421e-05', 'epoch': '0.4579', 'num_input_tokens_seen': 5563144, 'train_runtime': '5453', 'train_tokens_per_second': '1020'}
+
+ 46%|████▌     | 830/1813 [1:30:52<1:44:01,  6.35s/it]
+ 46%|████▌     | 831/1813 [1:30:59<1:43:54,  6.35s/it]
+ 46%|████▌     | 832/1813 [1:31:06<1:49:41,  6.71s/it]
+ 46%|████▌     | 833/1813 [1:31:13<1:48:19,  6.63s/it]
+ 46%|████▌     | 834/1813 [1:31:19<1:48:02,  6.62s/it]
+ 46%|████▌     | 835/1813 [1:31:25<1:44:27,  6.41s/it]
+ 46%|████▌     | 836/1813 [1:31:32<1:45:48,  6.50s/it]
+ 46%|████▌     | 837/1813 [1:31:40<1:51:39,  6.86s/it]
+ 46%|████▌     | 838/1813 [1:31:45<1:45:45,  6.51s/it]
+ 46%|████▋     | 839/1813 [1:31:51<1:42:53,  6.34s/it]
+ 46%|████▋     | 840/1813 [1:31:57<1:39:03,  6.11s/it]
+                                                      
+{'loss': '0.9685', 'grad_norm': '0.9255', 'learning_rate': '6.33e-05', 'epoch': '0.4634', 'num_input_tokens_seen': 5634104, 'train_runtime': '5517', 'train_tokens_per_second': '1021'}
+
+ 46%|████▋     | 840/1813 [1:31:57<1:39:03,  6.11s/it]
+ 46%|████▋     | 841/1813 [1:32:03<1:37:17,  6.01s/it]
+ 46%|████▋     | 842/1813 [1:32:09<1:40:04,  6.18s/it]
+ 46%|████▋     | 843/1813 [1:32:15<1:38:51,  6.12s/it]
+ 47%|████▋     | 844/1813 [1:32:21<1:35:12,  5.90s/it]
+ 47%|████▋     | 845/1813 [1:32:27<1:36:36,  5.99s/it]
+ 47%|████▋     | 846/1813 [1:32:33<1:38:59,  6.14s/it]
+ 47%|████▋     | 847/1813 [1:32:40<1:42:50,  6.39s/it]
+ 47%|████▋     | 848/1813 [1:32:47<1:46:32,  6.62s/it]
+ 47%|████▋     | 849/1813 [1:32:53<1:39:35,  6.20s/it]
+ 47%|████▋     | 850/1813 [1:33:01<1:50:12,  6.87s/it]
+                                                      
+{'loss': '0.9214', 'grad_norm': '0.7279', 'learning_rate': '6.239e-05', 'epoch': '0.469', 'num_input_tokens_seen': 5704744, 'train_runtime': '5581', 'train_tokens_per_second': '1022'}
+
+ 47%|████▋     | 850/1813 [1:33:01<1:50:12,  6.87s/it]
+ 47%|████▋     | 851/1813 [1:33:08<1:49:55,  6.86s/it]
+ 47%|████▋     | 852/1813 [1:33:13<1:40:59,  6.31s/it]
+ 47%|████▋     | 853/1813 [1:33:19<1:40:45,  6.30s/it]
+ 47%|████▋     | 854/1813 [1:33:26<1:42:45,  6.43s/it]
+ 47%|████▋     | 855/1813 [1:33:32<1:41:25,  6.35s/it]
+ 47%|████▋     | 856/1813 [1:33:38<1:40:26,  6.30s/it]
+ 47%|████▋     | 857/1813 [1:33:43<1:35:20,  5.98s/it]
+ 47%|████▋     | 858/1813 [1:33:48<1:26:58,  5.46s/it]
+ 47%|████▋     | 859/1813 [1:33:54<1:31:24,  5.75s/it]
+ 47%|████▋     | 860/1813 [1:34:01<1:35:44,  6.03s/it]
+                                                      
+{'loss': '0.9454', 'grad_norm': '0.9392', 'learning_rate': '6.147e-05', 'epoch': '0.4745', 'num_input_tokens_seen': 5766424, 'train_runtime': '5641', 'train_tokens_per_second': '1022'}
+
+ 47%|████▋     | 860/1813 [1:34:01<1:35:44,  6.03s/it]
+ 47%|████▋     | 861/1813 [1:34:06<1:30:55,  5.73s/it]
+ 48%|████▊     | 862/1813 [1:34:13<1:36:04,  6.06s/it]
+ 48%|████▊     | 863/1813 [1:34:19<1:36:03,  6.07s/it]
+ 48%|████▊     | 864/1813 [1:34:23<1:27:50,  5.55s/it]
+ 48%|████▊     | 865/1813 [1:34:30<1:34:30,  5.98s/it]
+ 48%|████▊     | 866/1813 [1:34:36<1:34:41,  6.00s/it]
+ 48%|████▊     | 867/1813 [1:34:42<1:32:55,  5.89s/it]
+ 48%|████▊     | 868/1813 [1:34:47<1:31:36,  5.82s/it]
+ 48%|████▊     | 869/1813 [1:34:55<1:38:30,  6.26s/it]
+ 48%|████▊     | 870/1813 [1:35:02<1:45:16,  6.70s/it]
+                                                      
+{'loss': '0.9217', 'grad_norm': '0.7004', 'learning_rate': '6.055e-05', 'epoch': '0.48', 'num_input_tokens_seen': 5831352, 'train_runtime': '5703', 'train_tokens_per_second': '1023'}
+
+ 48%|████▊     | 870/1813 [1:35:02<1:45:16,  6.70s/it]
+ 48%|████▊     | 871/1813 [1:35:08<1:39:09,  6.32s/it]
+ 48%|████▊     | 872/1813 [1:35:15<1:43:16,  6.58s/it]
+ 48%|████▊     | 873/1813 [1:35:22<1:44:15,  6.65s/it]
+ 48%|████▊     | 874/1813 [1:35:29<1:48:21,  6.92s/it]
+ 48%|████▊     | 875/1813 [1:35:37<1:52:20,  7.19s/it]
+ 48%|████▊     | 876/1813 [1:35:44<1:50:19,  7.06s/it]
+ 48%|████▊     | 877/1813 [1:35:52<1:54:28,  7.34s/it]
+ 48%|████▊     | 878/1813 [1:35:58<1:49:51,  7.05s/it]
+ 48%|████▊     | 879/1813 [1:36:06<1:51:49,  7.18s/it]
+ 49%|████▊     | 880/1813 [1:36:11<1:39:55,  6.43s/it]
+                                                      
+{'loss': '0.9442', 'grad_norm': '1.086', 'learning_rate': '5.962e-05', 'epoch': '0.4855', 'num_input_tokens_seen': 5907208, 'train_runtime': '5771', 'train_tokens_per_second': '1024'}
+
+ 49%|████▊     | 880/1813 [1:36:11<1:39:55,  6.43s/it]
+ 49%|████▊     | 881/1813 [1:36:16<1:35:17,  6.13s/it]
+ 49%|████▊     | 882/1813 [1:36:22<1:35:52,  6.18s/it]
+ 49%|████▊     | 883/1813 [1:36:29<1:38:55,  6.38s/it]
+ 49%|████▉     | 884/1813 [1:36:35<1:38:17,  6.35s/it]
+ 49%|████▉     | 885/1813 [1:36:41<1:36:45,  6.26s/it]
+ 49%|████▉     | 886/1813 [1:36:48<1:36:31,  6.25s/it]
+ 49%|████▉     | 887/1813 [1:36:55<1:43:53,  6.73s/it]
+ 49%|████▉     | 888/1813 [1:37:02<1:43:27,  6.71s/it]
+ 49%|████▉     | 889/1813 [1:37:09<1:45:26,  6.85s/it]
+ 49%|████▉     | 890/1813 [1:37:16<1:42:33,  6.67s/it]
+                                                      
+{'loss': '0.9144', 'grad_norm': '0.7531', 'learning_rate': '5.869e-05', 'epoch': '0.491', 'num_input_tokens_seen': 5978048, 'train_runtime': '5836', 'train_tokens_per_second': '1024'}
+
+ 49%|████▉     | 890/1813 [1:37:16<1:42:33,  6.67s/it]
+ 49%|████▉     | 891/1813 [1:37:22<1:40:04,  6.51s/it]
+ 49%|████▉     | 892/1813 [1:37:29<1:44:15,  6.79s/it]
+ 49%|████▉     | 893/1813 [1:37:35<1:41:33,  6.62s/it]
+ 49%|████▉     | 894/1813 [1:37:41<1:37:58,  6.40s/it]
+ 49%|████▉     | 895/1813 [1:37:49<1:42:03,  6.67s/it]
+ 49%|████▉     | 896/1813 [1:37:54<1:36:45,  6.33s/it]
+ 49%|████▉     | 897/1813 [1:38:00<1:34:37,  6.20s/it]
+ 50%|████▉     | 898/1813 [1:38:07<1:39:24,  6.52s/it]
+ 50%|████▉     | 899/1813 [1:38:15<1:42:56,  6.76s/it]
+ 50%|████▉     | 900/1813 [1:38:22<1:46:52,  7.02s/it]
+                                                      
+{'loss': '0.9864', 'grad_norm': '2.774', 'learning_rate': '5.776e-05', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856, 'train_runtime': '5903', 'train_tokens_per_second': '1025'}
+
+ 50%|████▉     | 900/1813 [1:38:22<1:46:52,  7.02s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.73it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.22it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.31it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.06it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.99it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.82it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:16,  7.15it/s]
+
+  3%|▎         | 26/1000 [00:04<02:38,  6.16it/s]
+
+  3%|▎         | 28/1000 [00:04<02:43,  5.94it/s]
+
+  3%|▎         | 29/1000 [00:04<02:33,  6.32it/s]
+
+  3%|▎         | 30/1000 [00:04<02:44,  5.90it/s]
+
+  3%|▎         | 31/1000 [00:04<03:02,  5.31it/s]
+
+  3%|▎         | 32/1000 [00:05<02:42,  5.96it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.74it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:47,  8.99it/s]
+
+  4%|▍         | 39/1000 [00:05<02:07,  7.51it/s]
+
+  4%|▍         | 40/1000 [00:06<02:04,  7.71it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.07it/s]
+
+  4%|▍         | 44/1000 [00:06<01:35, 10.05it/s]
+
+  5%|▍         | 46/1000 [00:06<01:47,  8.89it/s]
+
+  5%|▍         | 47/1000 [00:06<02:11,  7.27it/s]
+
+  5%|▍         | 49/1000 [00:07<01:49,  8.67it/s]
+
+  5%|▌         | 51/1000 [00:07<01:36,  9.81it/s]
+
+  5%|▌         | 53/1000 [00:07<01:27, 10.86it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.70it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.28it/s]
+
+  6%|▌         | 59/1000 [00:08<01:55,  8.18it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.74it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.56it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.11it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.66it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.85it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.19it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.94it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.37it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  6.99it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.62it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.31it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.43it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.20it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.03it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.51it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.80it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.23it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.91it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.95it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.07it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.91it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:17, 10.94it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.36it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.63it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:46,  7.94it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.59it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.39it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.66it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.14it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.13it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.21it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.44it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:38,  8.33it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:38,  8.32it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:03,  6.66it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:24,  5.67it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:22,  5.75it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:46,  7.67it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.85it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.37it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.62it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.47it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.65it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.75it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.62it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.91it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.51it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.65it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.72it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.13it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.48it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.54it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.36it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:37,  7.87it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:34,  8.08it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.84it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:21,  9.41it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:25,  8.93it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:31,  8.32it/s]
+
+ 24%|██▍       | 242/1000 [00:32<02:06,  6.00it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.46it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.64it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:08,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.73it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:34<01:36,  7.67it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.73it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.42it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.71it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.75it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.99it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.17it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.45it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.84it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.11it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.75it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.07it/s]
+
+ 30%|███       | 305/1000 [00:41<01:25,  8.13it/s]
+
+ 31%|███       | 306/1000 [00:41<01:40,  6.93it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.78it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.79it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.26it/s]
+
+ 32%|███▏      | 315/1000 [00:42<01:06, 10.23it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.58it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:43<01:38,  6.88it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.40it/s]
+
+ 33%|███▎      | 328/1000 [00:44<01:33,  7.16it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.57it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.37it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.81it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:07,  9.82it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:01, 10.69it/s]
+
+ 34%|███▍      | 344/1000 [00:46<01:26,  7.55it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:25,  7.68it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:31,  7.18it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.37it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.88it/s]
+
+ 35%|███▌      | 350/1000 [00:47<01:49,  5.94it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:25,  7.62it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.57it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.41it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.25it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.90it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:58,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.59it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.54it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.71it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.07it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.21it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:54,  5.47it/s]
+
+ 38%|███▊      | 378/1000 [00:51<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:11,  8.72it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.87it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.18it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.17it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.64it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.36it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.78it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.22it/s]
+
+ 40%|████      | 400/1000 [00:54<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.23it/s]
+
+ 41%|████      | 406/1000 [00:54<01:10,  8.48it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.82it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+
+ 41%|████▏     | 414/1000 [00:56<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:12,  7.85it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.56it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.03it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.81it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.52it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.09it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.32it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.24it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.17it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.17it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.49it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.80it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.25it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.82it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.30it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:04<01:20,  6.50it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.57it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.04it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.98it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.61it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:08<01:13,  6.81it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:55,  8.89it/s]
+
+ 51%|█████     | 512/1000 [01:09<00:49,  9.93it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.27it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.52it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.10it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.05it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.79it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.57it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.39it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:12<00:48,  9.72it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.89it/s]
+
+ 55%|█████▌    | 551/1000 [01:15<00:59,  7.48it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.24it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.16it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.92it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.75it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.01it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.85it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.05it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.36it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.58it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:51,  7.97it/s]
+
+ 59%|█████▉    | 593/1000 [01:21<01:00,  6.70it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.29it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:59,  6.82it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.61it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:02,  6.44it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.38it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.67it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.09it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.86it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:45,  8.48it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.79it/s]
+
+ 62%|██████▏   | 616/1000 [01:24<00:50,  7.60it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.75it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.09it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.02it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.19it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.45it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.63it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.84it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:39,  9.32it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.69it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.04it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.92it/s]
+
+ 64%|██████▍   | 640/1000 [01:27<00:39,  9.15it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:28<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.57it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.33it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:44,  7.72it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.93it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.58it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.65it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.97it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.89it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.55it/s]
+
+ 70%|███████   | 702/1000 [01:36<00:53,  5.59it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.20it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.01it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.61it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.49it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.70it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.36it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.15it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:39<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.14it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.62it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.35it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.60it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.42it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.72it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.66it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.46it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.79it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.91it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.59it/s]
+
+ 78%|███████▊  | 780/1000 [01:46<00:31,  6.94it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.36it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.94it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.19it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:27,  7.70it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:26,  7.94it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.17it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.85it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.21it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.22it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.04it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.30it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.19it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.03it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.99it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.00it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 833/1000 [01:53<00:26,  6.21it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.00it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.52it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.10it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.44it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.15it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.05it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.76it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.61it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.69it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.33it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.28it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.30it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.86it/s]
+
+ 88%|████████▊ | 881/1000 [01:59<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.76it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.75it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.84it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.52it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.68it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.85it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.05it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.78it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.38it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.07it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.59it/s]
+
+ 92%|█████████▏| 919/1000 [02:04<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.16it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.64it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.09it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:06<00:10,  6.60it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.03it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.12it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.57it/s]
+
+ 95%|█████████▌| 950/1000 [02:08<00:05,  8.76it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.38it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.14it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.38it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.39it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.44it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.33it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.41it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.19it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.78it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.80it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.42it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.14it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.97it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.7795', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.379', 'eval_steps_per_second': '7.379', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856}
+
+ 50%|████▉     | 900/1813 [1:40:38<1:46:52,  7.02s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+
+                                                   
+ 50%|████▉     | 901/1813 [1:40:44<12:01:10, 47.45s/it]
+ 50%|████▉     | 902/1813 [1:40:50<8:50:53, 34.97s/it] 
+ 50%|████▉     | 903/1813 [1:40:56<6:38:55, 26.30s/it]
+ 50%|████▉     | 904/1813 [1:41:02<5:06:08, 20.21s/it]
+ 50%|████▉     | 905/1813 [1:41:10<4:10:06, 16.53s/it]
+ 50%|████▉     | 906/1813 [1:41:15<3:19:28, 13.20s/it]
+ 50%|█████     | 907/1813 [1:41:20<2:40:21, 10.62s/it]
+ 50%|█████     | 908/1813 [1:41:26<2:17:46,  9.13s/it]
+ 50%|█████     | 909/1813 [1:41:32<2:06:31,  8.40s/it]
+ 50%|█████     | 910/1813 [1:41:39<1:57:12,  7.79s/it]
+                                                      
+{'loss': '0.9172', 'grad_norm': '1.044', 'learning_rate': '5.683e-05', 'epoch': '0.5021', 'num_input_tokens_seen': 6115160, 'train_runtime': '6099', 'train_tokens_per_second': '1003'}
+
+ 50%|█████     | 910/1813 [1:41:39<1:57:12,  7.79s/it]
+ 50%|█████     | 911/1813 [1:41:44<1:47:27,  7.15s/it]
+ 50%|█████     | 912/1813 [1:41:50<1:40:35,  6.70s/it]
+ 50%|█████     | 913/1813 [1:41:55<1:34:12,  6.28s/it]
+ 50%|█████     | 914/1813 [1:42:00<1:28:26,  5.90s/it]
+ 50%|█████     | 915/1813 [1:42:08<1:36:14,  6.43s/it]
+ 51%|█████     | 916/1813 [1:42:15<1:39:02,  6.63s/it]
+ 51%|█████     | 917/1813 [1:42:22<1:40:17,  6.72s/it]
+ 51%|█████     | 918/1813 [1:42:28<1:35:54,  6.43s/it]
+ 51%|█████     | 919/1813 [1:42:34<1:35:38,  6.42s/it]
+ 51%|█████     | 920/1813 [1:42:40<1:31:17,  6.13s/it]
+                                                      
+{'loss': '0.9485', 'grad_norm': '0.7058', 'learning_rate': '5.589e-05', 'epoch': '0.5076', 'num_input_tokens_seen': 6178904, 'train_runtime': '6160', 'train_tokens_per_second': '1003'}
+
+ 51%|█████     | 920/1813 [1:42:40<1:31:17,  6.13s/it]
+ 51%|█████     | 921/1813 [1:42:45<1:29:00,  5.99s/it]
+ 51%|█████     | 922/1813 [1:42:51<1:27:18,  5.88s/it]
+ 51%|█████     | 923/1813 [1:42:57<1:28:15,  5.95s/it]
+ 51%|█████     | 924/1813 [1:43:04<1:32:52,  6.27s/it]
+ 51%|█████     | 925/1813 [1:43:10<1:32:04,  6.22s/it]
+ 51%|█████     | 926/1813 [1:43:17<1:34:19,  6.38s/it]
+ 51%|█████     | 927/1813 [1:43:23<1:32:24,  6.26s/it]
+ 51%|█████     | 928/1813 [1:43:28<1:29:19,  6.06s/it]
+ 51%|█████     | 929/1813 [1:43:34<1:28:49,  6.03s/it]
+ 51%|█████▏    | 930/1813 [1:43:41<1:32:25,  6.28s/it]
+                                                      
+{'loss': '0.9088', 'grad_norm': '1.081', 'learning_rate': '5.495e-05', 'epoch': '0.5131', 'num_input_tokens_seen': 6244360, 'train_runtime': '6222', 'train_tokens_per_second': '1004'}
+
+ 51%|█████▏    | 930/1813 [1:43:41<1:32:25,  6.28s/it]
+ 51%|█████▏    | 931/1813 [1:43:48<1:34:52,  6.45s/it]
+ 51%|█████▏    | 932/1813 [1:43:53<1:30:16,  6.15s/it]
+ 51%|█████▏    | 933/1813 [1:43:59<1:28:43,  6.05s/it]
+ 52%|█████▏    | 934/1813 [1:44:06<1:30:58,  6.21s/it]
+ 52%|█████▏    | 935/1813 [1:44:15<1:42:25,  7.00s/it]
+ 52%|█████▏    | 936/1813 [1:44:21<1:39:44,  6.82s/it]
+ 52%|█████▏    | 937/1813 [1:44:27<1:36:23,  6.60s/it]
+ 52%|█████▏    | 938/1813 [1:44:34<1:38:16,  6.74s/it]
+ 52%|█████▏    | 939/1813 [1:44:40<1:34:11,  6.47s/it]
+ 52%|█████▏    | 940/1813 [1:44:47<1:36:34,  6.64s/it]
+                                                      
+{'loss': '0.893', 'grad_norm': '0.9855', 'learning_rate': '5.401e-05', 'epoch': '0.5186', 'num_input_tokens_seen': 6318968, 'train_runtime': '6288', 'train_tokens_per_second': '1005'}
+
+ 52%|█████▏    | 940/1813 [1:44:47<1:36:34,  6.64s/it]
+ 52%|█████▏    | 941/1813 [1:44:52<1:29:42,  6.17s/it]
+ 52%|█████▏    | 942/1813 [1:44:58<1:26:41,  5.97s/it]
+ 52%|█████▏    | 943/1813 [1:45:03<1:24:41,  5.84s/it]
+ 52%|█████▏    | 944/1813 [1:45:09<1:24:57,  5.87s/it]
+ 52%|█████▏    | 945/1813 [1:45:15<1:23:15,  5.76s/it]
+ 52%|█████▏    | 946/1813 [1:45:21<1:25:38,  5.93s/it]
+ 52%|█████▏    | 947/1813 [1:45:27<1:26:17,  5.98s/it]
+ 52%|█████▏    | 948/1813 [1:45:33<1:27:24,  6.06s/it]
+ 52%|█████▏    | 949/1813 [1:45:38<1:23:03,  5.77s/it]
+ 52%|█████▏    | 950/1813 [1:45:45<1:25:24,  5.94s/it]
+                                                      
+{'loss': '0.9497', 'grad_norm': '1.029', 'learning_rate': '5.307e-05', 'epoch': '0.5241', 'num_input_tokens_seen': 6379704, 'train_runtime': '6345', 'train_tokens_per_second': '1005'}
+
+ 52%|█████▏    | 950/1813 [1:45:45<1:25:24,  5.94s/it]
+ 52%|█████▏    | 951/1813 [1:45:50<1:22:57,  5.77s/it]
+ 53%|█████▎    | 952/1813 [1:45:56<1:21:48,  5.70s/it]
+ 53%|█████▎    | 953/1813 [1:46:02<1:22:43,  5.77s/it]
+ 53%|█████▎    | 954/1813 [1:46:07<1:22:04,  5.73s/it]
+ 53%|█████▎    | 955/1813 [1:46:13<1:21:48,  5.72s/it]
+ 53%|█████▎    | 956/1813 [1:46:19<1:21:08,  5.68s/it]
+ 53%|█████▎    | 957/1813 [1:46:24<1:20:27,  5.64s/it]
+ 53%|█████▎    | 958/1813 [1:46:29<1:15:36,  5.31s/it]
+ 53%|█████▎    | 959/1813 [1:46:34<1:17:48,  5.47s/it]
+ 53%|█████▎    | 960/1813 [1:46:43<1:29:08,  6.27s/it]
+                                                      
+{'loss': '0.9562', 'grad_norm': '0.655', 'learning_rate': '5.212e-05', 'epoch': '0.5297', 'num_input_tokens_seen': 6441656, 'train_runtime': '6403', 'train_tokens_per_second': '1006'}
+
+ 53%|█████▎    | 960/1813 [1:46:43<1:29:08,  6.27s/it]
+ 53%|█████▎    | 961/1813 [1:46:49<1:30:06,  6.35s/it]
+ 53%|█████▎    | 962/1813 [1:46:57<1:36:05,  6.77s/it]
+ 53%|█████▎    | 963/1813 [1:47:04<1:36:10,  6.79s/it]
+ 53%|█████▎    | 964/1813 [1:47:10<1:31:46,  6.49s/it]
+ 53%|█████▎    | 965/1813 [1:47:15<1:29:02,  6.30s/it]
+ 53%|█████▎    | 966/1813 [1:47:23<1:34:05,  6.67s/it]
+ 53%|█████▎    | 967/1813 [1:47:30<1:37:06,  6.89s/it]
+ 53%|█████▎    | 968/1813 [1:47:36<1:31:07,  6.47s/it]
+ 53%|█████▎    | 969/1813 [1:47:42<1:28:49,  6.31s/it]
+ 54%|█████▎    | 970/1813 [1:47:46<1:21:48,  5.82s/it]
+                                                      
+{'loss': '1.004', 'grad_norm': '0.9707', 'learning_rate': '5.118e-05', 'epoch': '0.5352', 'num_input_tokens_seen': 6512320, 'train_runtime': '6467', 'train_tokens_per_second': '1007'}
+
+ 54%|█████▎    | 970/1813 [1:47:46<1:21:48,  5.82s/it]
+ 54%|█████▎    | 971/1813 [1:47:53<1:25:46,  6.11s/it]
+ 54%|█████▎    | 972/1813 [1:48:00<1:26:52,  6.20s/it]
+ 54%|█████▎    | 973/1813 [1:48:06<1:25:46,  6.13s/it]
+ 54%|█████▎    | 974/1813 [1:48:14<1:35:28,  6.83s/it]
+ 54%|█████▍    | 975/1813 [1:48:20<1:33:16,  6.68s/it]
+ 54%|█████▍    | 976/1813 [1:48:26<1:28:52,  6.37s/it]
+ 54%|█████▍    | 977/1813 [1:48:34<1:34:55,  6.81s/it]
+ 54%|█████▍    | 978/1813 [1:48:40<1:33:13,  6.70s/it]
+ 54%|█████▍    | 979/1813 [1:48:48<1:35:21,  6.86s/it]
+ 54%|█████▍    | 980/1813 [1:48:54<1:34:06,  6.78s/it]
+                                                      
+{'loss': '0.9326', 'grad_norm': '1.548', 'learning_rate': '5.024e-05', 'epoch': '0.5407', 'num_input_tokens_seen': 6586672, 'train_runtime': '6535', 'train_tokens_per_second': '1008'}
+
+ 54%|█████▍    | 980/1813 [1:48:54<1:34:06,  6.78s/it]
+ 54%|█████▍    | 981/1813 [1:49:01<1:32:42,  6.69s/it]
+ 54%|█████▍    | 982/1813 [1:49:06<1:26:01,  6.21s/it]
+ 54%|█████▍    | 983/1813 [1:49:11<1:21:53,  5.92s/it]
+ 54%|█████▍    | 984/1813 [1:49:16<1:19:29,  5.75s/it]
+ 54%|█████▍    | 985/1813 [1:49:23<1:22:56,  6.01s/it]
+ 54%|█████▍    | 986/1813 [1:49:31<1:29:23,  6.49s/it]
+ 54%|█████▍    | 987/1813 [1:49:37<1:28:05,  6.40s/it]
+ 54%|█████▍    | 988/1813 [1:49:42<1:24:27,  6.14s/it]
+ 55%|█████▍    | 989/1813 [1:49:47<1:19:33,  5.79s/it]
+ 55%|█████▍    | 990/1813 [1:49:55<1:28:15,  6.43s/it]
+                                                      
+{'loss': '0.8963', 'grad_norm': '0.9012', 'learning_rate': '4.929e-05', 'epoch': '0.5462', 'num_input_tokens_seen': 6653096, 'train_runtime': '6596', 'train_tokens_per_second': '1009'}
+
+ 55%|█████▍    | 990/1813 [1:49:55<1:28:15,  6.43s/it]
+ 55%|█████▍    | 991/1813 [1:50:00<1:22:47,  6.04s/it]
+ 55%|█████▍    | 992/1813 [1:50:08<1:27:43,  6.41s/it]
+ 55%|█████▍    | 993/1813 [1:50:13<1:25:01,  6.22s/it]
+ 55%|█████▍    | 994/1813 [1:50:22<1:33:28,  6.85s/it]
+ 55%|█████▍    | 995/1813 [1:50:27<1:28:50,  6.52s/it]
+ 55%|█████▍    | 996/1813 [1:50:34<1:27:22,  6.42s/it]
+ 55%|█████▍    | 997/1813 [1:50:39<1:25:08,  6.26s/it]
+ 55%|█████▌    | 998/1813 [1:50:45<1:20:35,  5.93s/it]
+ 55%|█████▌    | 999/1813 [1:50:50<1:19:01,  5.82s/it]
+ 55%|█████▌    | 1000/1813 [1:50:55<1:16:29,  5.65s/it]
+                                                       
+{'loss': '1.002', 'grad_norm': '0.8292', 'learning_rate': '4.835e-05', 'epoch': '0.5517', 'num_input_tokens_seen': 6716784, 'train_runtime': '6656', 'train_tokens_per_second': '1009'}
+
+ 55%|█████▌    | 1000/1813 [1:50:55<1:16:29,  5.65s/it]
+ 55%|█████▌    | 1001/1813 [1:51:02<1:19:47,  5.90s/it]
+ 55%|█████▌    | 1002/1813 [1:51:08<1:21:05,  6.00s/it]
+ 55%|█████▌    | 1003/1813 [1:51:15<1:25:08,  6.31s/it]
+ 55%|█████▌    | 1004/1813 [1:51:21<1:21:06,  6.01s/it]
+ 55%|█████▌    | 1005/1813 [1:51:28<1:25:35,  6.36s/it]
+ 55%|█████▌    | 1006/1813 [1:51:34<1:26:55,  6.46s/it]
+ 56%|█████▌    | 1007/1813 [1:51:40<1:24:38,  6.30s/it]
+ 56%|█████▌    | 1008/1813 [1:51:46<1:23:37,  6.23s/it]
+ 56%|█████▌    | 1009/1813 [1:51:53<1:23:45,  6.25s/it]
+ 56%|█████▌    | 1010/1813 [1:51:58<1:20:59,  6.05s/it]
+                                                       
+{'loss': '0.9426', 'grad_norm': '1.046', 'learning_rate': '4.74e-05', 'epoch': '0.5572', 'num_input_tokens_seen': 6783768, 'train_runtime': '6719', 'train_tokens_per_second': '1010'}
+
+ 56%|█████▌    | 1010/1813 [1:51:58<1:20:59,  6.05s/it]
+ 56%|█████▌    | 1011/1813 [1:52:05<1:23:47,  6.27s/it]
+ 56%|█████▌    | 1012/1813 [1:52:10<1:17:36,  5.81s/it]
+ 56%|█████▌    | 1013/1813 [1:52:14<1:12:31,  5.44s/it]
+ 56%|█████▌    | 1014/1813 [1:52:23<1:24:14,  6.33s/it]
+ 56%|█████▌    | 1015/1813 [1:52:28<1:20:52,  6.08s/it]
+ 56%|█████▌    | 1016/1813 [1:52:34<1:18:27,  5.91s/it]
+ 56%|█████▌    | 1017/1813 [1:52:41<1:23:03,  6.26s/it]
+ 56%|█████▌    | 1018/1813 [1:52:47<1:23:31,  6.30s/it]
+ 56%|█████▌    | 1019/1813 [1:52:52<1:19:11,  5.98s/it]
+ 56%|█████▋    | 1020/1813 [1:52:58<1:16:17,  5.77s/it]
+                                                       
+{'loss': '0.8596', 'grad_norm': '1.019', 'learning_rate': '4.646e-05', 'epoch': '0.5628', 'num_input_tokens_seen': 6844992, 'train_runtime': '6778', 'train_tokens_per_second': '1010'}
+
+ 56%|█████▋    | 1020/1813 [1:52:58<1:16:17,  5.77s/it]
+ 56%|█████▋    | 1021/1813 [1:53:05<1:21:48,  6.20s/it]
+ 56%|█████▋    | 1022/1813 [1:53:11<1:22:17,  6.24s/it]
+ 56%|█████▋    | 1023/1813 [1:53:19<1:27:37,  6.65s/it]
+ 56%|█████▋    | 1024/1813 [1:53:25<1:26:29,  6.58s/it]
+ 57%|█████▋    | 1025/1813 [1:53:35<1:36:39,  7.36s/it]
+ 57%|█████▋    | 1026/1813 [1:53:40<1:28:24,  6.74s/it]
+ 57%|█████▋    | 1027/1813 [1:53:46<1:25:41,  6.54s/it]
+ 57%|█████▋    | 1028/1813 [1:53:53<1:28:47,  6.79s/it]
+ 57%|█████▋    | 1029/1813 [1:54:00<1:27:25,  6.69s/it]
+ 57%|█████▋    | 1030/1813 [1:54:07<1:28:05,  6.75s/it]
+                                                       
+{'loss': '0.9955', 'grad_norm': '0.6896', 'learning_rate': '4.552e-05', 'epoch': '0.5683', 'num_input_tokens_seen': 6920856, 'train_runtime': '6847', 'train_tokens_per_second': '1011'}
+
+ 57%|█████▋    | 1030/1813 [1:54:07<1:28:05,  6.75s/it]
+ 57%|█████▋    | 1031/1813 [1:54:12<1:23:37,  6.42s/it]
+ 57%|█████▋    | 1032/1813 [1:54:19<1:24:50,  6.52s/it]
+ 57%|█████▋    | 1033/1813 [1:54:25<1:23:40,  6.44s/it]
+ 57%|█████▋    | 1034/1813 [1:54:31<1:21:42,  6.29s/it]
+ 57%|█████▋    | 1035/1813 [1:54:38<1:21:53,  6.32s/it]
+ 57%|█████▋    | 1036/1813 [1:54:44<1:22:28,  6.37s/it]
+ 57%|█████▋    | 1037/1813 [1:54:50<1:20:56,  6.26s/it]
+ 57%|█████▋    | 1038/1813 [1:54:56<1:21:03,  6.28s/it]
+ 57%|█████▋    | 1039/1813 [1:55:03<1:23:53,  6.50s/it]
+ 57%|█████▋    | 1040/1813 [1:55:10<1:25:54,  6.67s/it]
+                                                       
+{'loss': '0.9234', 'grad_norm': '0.722', 'learning_rate': '4.458e-05', 'epoch': '0.5738', 'num_input_tokens_seen': 6988568, 'train_runtime': '6911', 'train_tokens_per_second': '1011'}
+
+ 57%|█████▋    | 1040/1813 [1:55:10<1:25:54,  6.67s/it]
+ 57%|█████▋    | 1041/1813 [1:55:17<1:23:41,  6.50s/it]
+ 57%|█████▋    | 1042/1813 [1:55:23<1:23:31,  6.50s/it]
+ 58%|█████▊    | 1043/1813 [1:55:29<1:20:26,  6.27s/it]
+ 58%|█████▊    | 1044/1813 [1:55:34<1:17:07,  6.02s/it]
+ 58%|█████▊    | 1045/1813 [1:55:40<1:16:12,  5.95s/it]
+ 58%|█████▊    | 1046/1813 [1:55:46<1:16:18,  5.97s/it]
+ 58%|█████▊    | 1047/1813 [1:55:53<1:18:38,  6.16s/it]
+ 58%|█████▊    | 1048/1813 [1:56:00<1:22:07,  6.44s/it]
+ 58%|█████▊    | 1049/1813 [1:56:05<1:19:19,  6.23s/it]
+ 58%|█████▊    | 1050/1813 [1:56:12<1:18:30,  6.17s/it]
+                                                       
+{'loss': '0.8991', 'grad_norm': '1.065', 'learning_rate': '4.364e-05', 'epoch': '0.5793', 'num_input_tokens_seen': 7051808, 'train_runtime': '6972', 'train_tokens_per_second': '1011'}
+
+ 58%|█████▊    | 1050/1813 [1:56:12<1:18:30,  6.17s/it]
+ 58%|█████▊    | 1051/1813 [1:56:17<1:14:41,  5.88s/it]
+ 58%|█████▊    | 1052/1813 [1:56:22<1:13:55,  5.83s/it]
+ 58%|█████▊    | 1053/1813 [1:56:28<1:13:32,  5.81s/it]
+ 58%|█████▊    | 1054/1813 [1:56:34<1:12:54,  5.76s/it]
+ 58%|█████▊    | 1055/1813 [1:56:39<1:10:53,  5.61s/it]
+ 58%|█████▊    | 1056/1813 [1:56:45<1:11:23,  5.66s/it]
+ 58%|█████▊    | 1057/1813 [1:56:51<1:14:52,  5.94s/it]
+ 58%|█████▊    | 1058/1813 [1:56:56<1:10:57,  5.64s/it]
+ 58%|█████▊    | 1059/1813 [1:57:01<1:07:49,  5.40s/it]
+ 58%|█████▊    | 1060/1813 [1:57:08<1:13:18,  5.84s/it]
+                                                       
+{'loss': '0.9138', 'grad_norm': '0.6532', 'learning_rate': '4.271e-05', 'epoch': '0.5848', 'num_input_tokens_seen': 7110624, 'train_runtime': '7029', 'train_tokens_per_second': '1012'}
+
+ 58%|█████▊    | 1060/1813 [1:57:08<1:13:18,  5.84s/it]
+ 59%|█████▊    | 1061/1813 [1:57:13<1:09:23,  5.54s/it]
+ 59%|█████▊    | 1062/1813 [1:57:18<1:07:12,  5.37s/it]
+ 59%|█████▊    | 1063/1813 [1:57:25<1:11:56,  5.76s/it]
+ 59%|█████▊    | 1064/1813 [1:57:32<1:18:19,  6.27s/it]
+ 59%|█████▊    | 1065/1813 [1:57:39<1:20:19,  6.44s/it]
+ 59%|█████▉    | 1066/1813 [1:57:46<1:24:26,  6.78s/it]
+ 59%|█████▉    | 1067/1813 [1:57:53<1:23:16,  6.70s/it]
+ 59%|█████▉    | 1068/1813 [1:57:58<1:16:58,  6.20s/it]
+ 59%|█████▉    | 1069/1813 [1:58:06<1:23:20,  6.72s/it]
+ 59%|█████▉    | 1070/1813 [1:58:12<1:19:33,  6.42s/it]
+                                                       
+{'loss': '0.9113', 'grad_norm': '0.9644', 'learning_rate': '4.177e-05', 'epoch': '0.5903', 'num_input_tokens_seen': 7179728, 'train_runtime': '7092', 'train_tokens_per_second': '1012'}
+
+ 59%|█████▉    | 1070/1813 [1:58:12<1:19:33,  6.42s/it]
+ 59%|█████▉    | 1071/1813 [1:58:17<1:15:52,  6.14s/it]
+ 59%|█████▉    | 1072/1813 [1:58:23<1:16:34,  6.20s/it]
+ 59%|█████▉    | 1073/1813 [1:58:29<1:15:18,  6.11s/it]
+ 59%|█████▉    | 1074/1813 [1:58:35<1:13:00,  5.93s/it]
+ 59%|█████▉    | 1075/1813 [1:58:41<1:14:57,  6.09s/it]
+ 59%|█████▉    | 1076/1813 [1:58:48<1:17:20,  6.30s/it]
+ 59%|█████▉    | 1077/1813 [1:58:53<1:11:41,  5.84s/it]
+ 59%|█████▉    | 1078/1813 [1:58:58<1:09:38,  5.68s/it]
+ 60%|█████▉    | 1079/1813 [1:59:05<1:12:54,  5.96s/it]
+ 60%|█████▉    | 1080/1813 [1:59:11<1:14:18,  6.08s/it]
+                                                       
+{'loss': '0.9093', 'grad_norm': '0.7117', 'learning_rate': '4.084e-05', 'epoch': '0.5959', 'num_input_tokens_seen': 7240984, 'train_runtime': '7152', 'train_tokens_per_second': '1012'}
+
+ 60%|█████▉    | 1080/1813 [1:59:11<1:14:18,  6.08s/it]
+ 60%|█████▉    | 1081/1813 [1:59:17<1:14:15,  6.09s/it]
+ 60%|█████▉    | 1082/1813 [1:59:23<1:12:25,  5.95s/it]
+ 60%|█████▉    | 1083/1813 [1:59:29<1:13:21,  6.03s/it]
+ 60%|█████▉    | 1084/1813 [1:59:35<1:13:17,  6.03s/it]
+ 60%|█████▉    | 1085/1813 [1:59:41<1:13:22,  6.05s/it]
+ 60%|█████▉    | 1086/1813 [1:59:46<1:07:05,  5.54s/it]
+ 60%|█████▉    | 1087/1813 [1:59:51<1:08:00,  5.62s/it]
+ 60%|██████    | 1088/1813 [1:59:57<1:08:52,  5.70s/it]
+ 60%|██████    | 1089/1813 [2:00:04<1:12:19,  5.99s/it]
+ 60%|██████    | 1090/1813 [2:00:10<1:12:08,  5.99s/it]
+                                                       
+{'loss': '0.8324', 'grad_norm': '0.8991', 'learning_rate': '3.992e-05', 'epoch': '0.6014', 'num_input_tokens_seen': 7301824, 'train_runtime': '7210', 'train_tokens_per_second': '1013'}
+
+ 60%|██████    | 1090/1813 [2:00:10<1:12:08,  5.99s/it]
+ 60%|██████    | 1091/1813 [2:00:16<1:12:40,  6.04s/it]
+ 60%|██████    | 1092/1813 [2:00:22<1:11:41,  5.97s/it]
+ 60%|██████    | 1093/1813 [2:00:27<1:09:35,  5.80s/it]
+ 60%|██████    | 1094/1813 [2:00:34<1:11:05,  5.93s/it]
+ 60%|██████    | 1095/1813 [2:00:40<1:11:36,  5.98s/it]
+ 60%|██████    | 1096/1813 [2:00:45<1:09:13,  5.79s/it]
+ 61%|██████    | 1097/1813 [2:00:51<1:09:29,  5.82s/it]
+ 61%|██████    | 1098/1813 [2:00:58<1:14:12,  6.23s/it]
+ 61%|██████    | 1099/1813 [2:01:04<1:14:20,  6.25s/it]
+ 61%|██████    | 1100/1813 [2:01:12<1:19:49,  6.72s/it]
+                                                       
+{'loss': '0.8953', 'grad_norm': '0.9429', 'learning_rate': '3.899e-05', 'epoch': '0.6069', 'num_input_tokens_seen': 7369528, 'train_runtime': '7273', 'train_tokens_per_second': '1013'}
+
+ 61%|██████    | 1100/1813 [2:01:12<1:19:49,  6.72s/it]
+ 61%|██████    | 1101/1813 [2:01:19<1:18:28,  6.61s/it]
+ 61%|██████    | 1102/1813 [2:01:26<1:19:40,  6.72s/it]
+ 61%|██████    | 1103/1813 [2:01:31<1:14:38,  6.31s/it]
+ 61%|██████    | 1104/1813 [2:01:38<1:18:36,  6.65s/it]
+ 61%|██████    | 1105/1813 [2:01:43<1:11:25,  6.05s/it]
+ 61%|██████    | 1106/1813 [2:01:49<1:09:38,  5.91s/it]
+ 61%|██████    | 1107/1813 [2:01:57<1:18:35,  6.68s/it]
+ 61%|██████    | 1108/1813 [2:02:03<1:14:47,  6.37s/it]
+ 61%|██████    | 1109/1813 [2:02:09<1:15:23,  6.43s/it]
+ 61%|██████    | 1110/1813 [2:02:17<1:19:52,  6.82s/it]
+                                                       
+{'loss': '0.9365', 'grad_norm': '0.8699', 'learning_rate': '3.807e-05', 'epoch': '0.6124', 'num_input_tokens_seen': 7441040, 'train_runtime': '7337', 'train_tokens_per_second': '1014'}
+
+ 61%|██████    | 1110/1813 [2:02:17<1:19:52,  6.82s/it]
+ 61%|██████▏   | 1111/1813 [2:02:22<1:13:03,  6.24s/it]
+ 61%|██████▏   | 1112/1813 [2:02:28<1:11:57,  6.16s/it]
+ 61%|██████▏   | 1113/1813 [2:02:34<1:12:56,  6.25s/it]
+ 61%|██████▏   | 1114/1813 [2:02:42<1:16:24,  6.56s/it]
+ 62%|██████▏   | 1115/1813 [2:02:48<1:17:14,  6.64s/it]
+ 62%|██████▏   | 1116/1813 [2:02:55<1:16:51,  6.62s/it]
+ 62%|██████▏   | 1117/1813 [2:03:01<1:13:42,  6.35s/it]
+ 62%|██████▏   | 1118/1813 [2:03:06<1:09:14,  5.98s/it]
+ 62%|██████▏   | 1119/1813 [2:03:12<1:10:47,  6.12s/it]
+ 62%|██████▏   | 1120/1813 [2:03:19<1:12:53,  6.31s/it]
+                                                       
+{'loss': '0.8833', 'grad_norm': '1.248', 'learning_rate': '3.716e-05', 'epoch': '0.6179', 'num_input_tokens_seen': 7504976, 'train_runtime': '7400', 'train_tokens_per_second': '1014'}
+
+ 62%|██████▏   | 1120/1813 [2:03:19<1:12:53,  6.31s/it]
+ 62%|██████▏   | 1121/1813 [2:03:27<1:18:28,  6.80s/it]
+ 62%|██████▏   | 1122/1813 [2:03:32<1:11:19,  6.19s/it]
+ 62%|██████▏   | 1123/1813 [2:03:39<1:16:15,  6.63s/it]
+ 62%|██████▏   | 1124/1813 [2:03:45<1:12:46,  6.34s/it]
+ 62%|██████▏   | 1125/1813 [2:03:52<1:14:28,  6.49s/it]
+ 62%|██████▏   | 1126/1813 [2:03:58<1:13:07,  6.39s/it]
+ 62%|██████▏   | 1127/1813 [2:04:04<1:10:06,  6.13s/it]
+ 62%|██████▏   | 1128/1813 [2:04:09<1:07:36,  5.92s/it]
+ 62%|██████▏   | 1129/1813 [2:04:14<1:04:55,  5.69s/it]
+ 62%|██████▏   | 1130/1813 [2:04:19<1:03:12,  5.55s/it]
+                                                       
+{'loss': '0.8771', 'grad_norm': '0.831', 'learning_rate': '3.625e-05', 'epoch': '0.6234', 'num_input_tokens_seen': 7568576, 'train_runtime': '7460', 'train_tokens_per_second': '1015'}
+
+ 62%|██████▏   | 1130/1813 [2:04:19<1:03:12,  5.55s/it]
+ 62%|██████▏   | 1131/1813 [2:04:25<1:04:01,  5.63s/it]
+ 62%|██████▏   | 1132/1813 [2:04:31<1:04:52,  5.72s/it]
+ 62%|██████▏   | 1133/1813 [2:04:38<1:07:10,  5.93s/it]
+ 63%|██████▎   | 1134/1813 [2:04:45<1:13:34,  6.50s/it]
+ 63%|██████▎   | 1135/1813 [2:04:52<1:14:29,  6.59s/it]
+ 63%|██████▎   | 1136/1813 [2:04:59<1:15:40,  6.71s/it]
+ 63%|██████▎   | 1137/1813 [2:05:06<1:14:55,  6.65s/it]
+ 63%|██████▎   | 1138/1813 [2:05:13<1:15:59,  6.75s/it]
+ 63%|██████▎   | 1139/1813 [2:05:18<1:11:18,  6.35s/it]
+ 63%|██████▎   | 1140/1813 [2:05:24<1:08:46,  6.13s/it]
+                                                       
+{'loss': '0.95', 'grad_norm': '0.8641', 'learning_rate': '3.534e-05', 'epoch': '0.629', 'num_input_tokens_seen': 7639168, 'train_runtime': '7524', 'train_tokens_per_second': '1015'}
+
+ 63%|██████▎   | 1140/1813 [2:05:24<1:08:46,  6.13s/it]
+ 63%|██████▎   | 1141/1813 [2:05:30<1:08:29,  6.12s/it]
+ 63%|██████▎   | 1142/1813 [2:05:36<1:07:40,  6.05s/it]
+ 63%|██████▎   | 1143/1813 [2:05:42<1:06:51,  5.99s/it]
+ 63%|██████▎   | 1144/1813 [2:05:48<1:09:36,  6.24s/it]
+ 63%|██████▎   | 1145/1813 [2:05:55<1:09:57,  6.28s/it]
+ 63%|██████▎   | 1146/1813 [2:06:00<1:06:55,  6.02s/it]
+ 63%|██████▎   | 1147/1813 [2:06:08<1:11:47,  6.47s/it]
+ 63%|██████▎   | 1148/1813 [2:06:15<1:13:19,  6.62s/it]
+ 63%|██████▎   | 1149/1813 [2:06:20<1:10:03,  6.33s/it]
+ 63%|██████▎   | 1150/1813 [2:06:28<1:13:57,  6.69s/it]
+                                                       
+{'loss': '0.881', 'grad_norm': '0.7982', 'learning_rate': '3.444e-05', 'epoch': '0.6345', 'num_input_tokens_seen': 7707832, 'train_runtime': '7588', 'train_tokens_per_second': '1016'}
+
+ 63%|██████▎   | 1150/1813 [2:06:28<1:13:57,  6.69s/it]
+ 63%|██████▎   | 1151/1813 [2:06:32<1:04:52,  5.88s/it]
+ 64%|██████▎   | 1152/1813 [2:06:37<1:03:18,  5.75s/it]
+ 64%|██████▎   | 1153/1813 [2:06:44<1:07:38,  6.15s/it]
+ 64%|██████▎   | 1154/1813 [2:06:50<1:07:21,  6.13s/it]
+ 64%|██████▎   | 1155/1813 [2:06:57<1:08:54,  6.28s/it]
+ 64%|██████▍   | 1156/1813 [2:07:03<1:09:13,  6.32s/it]
+ 64%|██████▍   | 1157/1813 [2:07:11<1:13:13,  6.70s/it]
+ 64%|██████▍   | 1158/1813 [2:07:18<1:13:11,  6.70s/it]
+ 64%|██████▍   | 1159/1813 [2:07:25<1:14:37,  6.85s/it]
+ 64%|██████▍   | 1160/1813 [2:07:31<1:12:15,  6.64s/it]
+                                                       
+{'loss': '0.9272', 'grad_norm': '0.6358', 'learning_rate': '3.355e-05', 'epoch': '0.64', 'num_input_tokens_seen': 7775872, 'train_runtime': '7652', 'train_tokens_per_second': '1016'}
+
+ 64%|██████▍   | 1160/1813 [2:07:31<1:12:15,  6.64s/it]
+ 64%|██████▍   | 1161/1813 [2:07:39<1:14:41,  6.87s/it]
+ 64%|██████▍   | 1162/1813 [2:07:45<1:14:20,  6.85s/it]
+ 64%|██████▍   | 1163/1813 [2:07:52<1:12:59,  6.74s/it]
+ 64%|██████▍   | 1164/1813 [2:07:58<1:11:14,  6.59s/it]
+ 64%|██████▍   | 1165/1813 [2:08:04<1:10:33,  6.53s/it]
+ 64%|██████▍   | 1166/1813 [2:08:10<1:08:27,  6.35s/it]
+ 64%|██████▍   | 1167/1813 [2:08:17<1:10:18,  6.53s/it]
+ 64%|██████▍   | 1168/1813 [2:08:24<1:09:51,  6.50s/it]
+ 64%|██████▍   | 1169/1813 [2:08:30<1:08:42,  6.40s/it]
+ 65%|██████▍   | 1170/1813 [2:08:36<1:06:44,  6.23s/it]
+                                                       
+{'loss': '0.908', 'grad_norm': '0.9535', 'learning_rate': '3.266e-05', 'epoch': '0.6455', 'num_input_tokens_seen': 7846688, 'train_runtime': '7716', 'train_tokens_per_second': '1017'}
+
+ 65%|██████▍   | 1170/1813 [2:08:36<1:06:44,  6.23s/it]
+ 65%|██████▍   | 1171/1813 [2:08:43<1:10:15,  6.57s/it]
+ 65%|██████▍   | 1172/1813 [2:08:50<1:11:47,  6.72s/it]
+ 65%|██████▍   | 1173/1813 [2:08:56<1:07:41,  6.35s/it]
+ 65%|██████▍   | 1174/1813 [2:09:01<1:03:55,  6.00s/it]
+ 65%|██████▍   | 1175/1813 [2:09:07<1:03:11,  5.94s/it]
+ 65%|██████▍   | 1176/1813 [2:09:13<1:04:09,  6.04s/it]
+ 65%|██████▍   | 1177/1813 [2:09:19<1:02:48,  5.92s/it]
+ 65%|██████▍   | 1178/1813 [2:09:25<1:03:02,  5.96s/it]
+ 65%|██████▌   | 1179/1813 [2:09:31<1:03:35,  6.02s/it]
+ 65%|██████▌   | 1180/1813 [2:09:38<1:05:55,  6.25s/it]
+                                                       
+{'loss': '0.8907', 'grad_norm': '0.9056', 'learning_rate': '3.177e-05', 'epoch': '0.651', 'num_input_tokens_seen': 7914072, 'train_runtime': '7778', 'train_tokens_per_second': '1017'}
+
+ 65%|██████▌   | 1180/1813 [2:09:38<1:05:55,  6.25s/it]
+ 65%|██████▌   | 1181/1813 [2:09:45<1:11:10,  6.76s/it]
+ 65%|██████▌   | 1182/1813 [2:09:53<1:14:23,  7.07s/it]
+ 65%|██████▌   | 1183/1813 [2:10:00<1:14:28,  7.09s/it]
+ 65%|██████▌   | 1184/1813 [2:10:06<1:11:00,  6.77s/it]
+ 65%|██████▌   | 1185/1813 [2:10:14<1:14:24,  7.11s/it]
+ 65%|██████▌   | 1186/1813 [2:10:22<1:15:24,  7.22s/it]
+ 65%|██████▌   | 1187/1813 [2:10:29<1:14:32,  7.14s/it]
+ 66%|██████▌   | 1188/1813 [2:10:34<1:09:41,  6.69s/it]
+ 66%|██████▌   | 1189/1813 [2:10:40<1:05:47,  6.33s/it]
+ 66%|██████▌   | 1190/1813 [2:10:46<1:06:29,  6.40s/it]
+                                                       
+{'loss': '0.9446', 'grad_norm': '1.067', 'learning_rate': '3.09e-05', 'epoch': '0.6566', 'num_input_tokens_seen': 7989064, 'train_runtime': '7847', 'train_tokens_per_second': '1018'}
+
+ 66%|██████▌   | 1190/1813 [2:10:46<1:06:29,  6.40s/it]
+ 66%|██████▌   | 1191/1813 [2:10:52<1:04:29,  6.22s/it]
+ 66%|██████▌   | 1192/1813 [2:10:59<1:05:53,  6.37s/it]
+ 66%|██████▌   | 1193/1813 [2:11:05<1:05:04,  6.30s/it]
+ 66%|██████▌   | 1194/1813 [2:11:12<1:06:11,  6.42s/it]
+ 66%|██████▌   | 1195/1813 [2:11:17<1:01:46,  6.00s/it]
+ 66%|██████▌   | 1196/1813 [2:11:23<1:02:11,  6.05s/it]
+ 66%|██████▌   | 1197/1813 [2:11:29<1:02:39,  6.10s/it]
+ 66%|██████▌   | 1198/1813 [2:11:35<1:01:29,  6.00s/it]
+ 66%|██████▌   | 1199/1813 [2:11:41<1:00:51,  5.95s/it]
+ 66%|██████▌   | 1200/1813 [2:11:48<1:05:05,  6.37s/it]
+                                                       
+{'loss': '0.8617', 'grad_norm': '0.8818', 'learning_rate': '3.003e-05', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352, 'train_runtime': '7909', 'train_tokens_per_second': '1019'}
+
+ 66%|██████▌   | 1200/1813 [2:11:48<1:05:05,  6.37s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.68it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.22it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.32it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.46it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.07it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.99it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.83it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.59it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:55,  5.58it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.35it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.35it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.98it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.74it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.65it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.09it/s]
+
+  4%|▍         | 39/1000 [00:05<02:07,  7.56it/s]
+
+  4%|▍         | 40/1000 [00:06<02:04,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.35it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.28it/s]
+
+  6%|▌         | 59/1000 [00:08<01:55,  8.18it/s]
+
+  6%|▌         | 61/1000 [00:08<01:42,  9.15it/s]
+
+  6%|▋         | 63/1000 [00:08<01:37,  9.62it/s]
+
+  6%|▋         | 65/1000 [00:08<01:52,  8.31it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.53it/s]
+
+  7%|▋         | 69/1000 [00:09<01:53,  8.22it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.97it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.70it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.11it/s]
+
+  8%|▊         | 75/1000 [00:10<02:15,  6.84it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.66it/s]
+
+  8%|▊         | 78/1000 [00:10<02:26,  6.31it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.82it/s]
+
+  8%|▊         | 82/1000 [00:10<01:40,  9.17it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.00it/s]
+
+  8%|▊         | 85/1000 [00:11<02:15,  6.77it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.94it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.32it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.44it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.21it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.05it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.53it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.81it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.19it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.90it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:48,  8.07it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.71it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.13it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.68it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.12it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.89it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.06it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.91it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.04it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.39it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.60it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.66it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.14it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.73it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.88it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.37it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.10it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.20it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.64it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.76it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.62it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.91it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.51it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.67it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.73it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.79it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:04,  6.28it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:04,  6.28it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.54it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.03it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.08it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.36it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.97it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.47it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.73it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.74it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.52it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.05it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.01it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.74it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.99it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.11it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.78it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.78it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.19it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.25it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.23it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.56it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.22it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.88it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.40it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.16it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.36it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.03it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.81it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.02it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.59it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.42it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.56it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.94it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.56it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.41it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.59it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.53it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.70it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.07it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.20it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.47it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.18it/s]
+
+ 38%|███▊      | 379/1000 [00:51<01:17,  8.03it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:08,  9.08it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.84it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:05,  9.36it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:05,  9.31it/s]
+
+ 39%|███▊      | 387/1000 [00:52<01:24,  7.25it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:27,  7.02it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.70it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.36it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  8.98it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.46it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.82it/s]
+
+ 40%|███▉      | 398/1000 [00:53<01:23,  7.20it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:41,  5.91it/s]
+
+ 40%|████      | 400/1000 [00:53<01:52,  5.34it/s]
+
+ 40%|████      | 402/1000 [00:54<01:44,  5.74it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.59it/s]
+
+ 41%|████      | 407/1000 [00:54<01:23,  7.06it/s]
+
+ 41%|████      | 408/1000 [00:55<01:36,  6.14it/s]
+
+ 41%|████      | 410/1000 [00:55<01:33,  6.29it/s]
+
+ 41%|████      | 411/1000 [00:55<01:44,  5.63it/s]
+
+ 41%|████      | 412/1000 [00:55<01:53,  5.16it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.04it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.79it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.31it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.50it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.51it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.68it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.92it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.92it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:16,  7.52it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:13,  7.82it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.52it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 10.98it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:48, 11.76it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.50it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.24it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.47it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.57it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.17it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.49it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.80it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.25it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.82it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.61it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.30it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.84it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.56it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:21,  6.49it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.38it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.14it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.57it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.05it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.97it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.61it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.19it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.98it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.55it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.42it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.95it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:49,  9.96it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.27it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.52it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.09it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.07it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.38it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.39it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.11it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.71it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.05it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:10,  6.49it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.91it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.50it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.25it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.15it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.93it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:58,  7.53it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:04,  6.88it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:50,  8.72it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:44,  9.93it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.28it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:55,  7.91it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.72it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:06,  6.53it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:07,  6.42it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.41it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:08,  6.29it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.32it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.81it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:14,  5.72it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.24it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.95it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:49,  8.53it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.57it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.89it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.13it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.51it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.74it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.79it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.38it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.86it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:02,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.38it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.68it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.09it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:50,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.46it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.72it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.79it/s]
+
+ 62%|██████▏   | 616/1000 [01:24<00:50,  7.61it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.72it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.07it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.02it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.16it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.41it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.58it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.16it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.20it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.42it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:28<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:44,  7.71it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.09it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  5.99it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.54it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.47it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:03,  5.31it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.70it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.97it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.92it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.89it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.83it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:47,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.93it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.58it/s]
+
+ 70%|███████   | 702/1000 [01:36<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.21it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.70it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.36it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.92it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.15it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.14it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.62it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.30it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.50it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.88it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.29it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.35it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.60it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.44it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.63it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.67it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:33,  7.30it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.69it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.46it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.77it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.37it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.77it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.88it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.60it/s]
+
+ 78%|███████▊  | 780/1000 [01:46<00:31,  6.94it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.36it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.52it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.35it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.98it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.85it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.16it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.96it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.20it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.23it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.30it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.13it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.19it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.81it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.74it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.99it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.00it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:53<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.61it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.13it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.76it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.24it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.54it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.86it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.46it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.03it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.50it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.74it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.60it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.67it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.28it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.25it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.25it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.76it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.62it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.86it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.86it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.34it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.76it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.83it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.69it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.30it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.76it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.13it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.38it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.70it/s]
+
+ 95%|█████████▌| 950/1000 [02:08<00:05,  8.82it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.44it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.41it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.80it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                       
+
+                                                   
+{'eval_loss': '0.7632', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.381', 'eval_steps_per_second': '7.381', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352}
+
+ 66%|██████▌   | 1200/1813 [2:14:04<1:05:05,  6.37s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 66%|██████▌   | 1201/1813 [2:14:09<7:57:02, 46.77s/it]
+ 66%|██████▋   | 1202/1813 [2:14:16<5:53:44, 34.74s/it]
+ 66%|██████▋   | 1203/1813 [2:14:22<4:27:14, 26.29s/it]
+ 66%|██████▋   | 1204/1813 [2:14:29<3:27:01, 20.40s/it]
+ 66%|██████▋   | 1205/1813 [2:14:36<2:44:32, 16.24s/it]
+ 67%|██████▋   | 1206/1813 [2:14:42<2:13:35, 13.21s/it]
+ 67%|██████▋   | 1207/1813 [2:14:48<1:52:07, 11.10s/it]
+ 67%|██████▋   | 1208/1813 [2:14:55<1:41:05, 10.03s/it]
+ 67%|██████▋   | 1209/1813 [2:15:02<1:30:57,  9.04s/it]
+ 67%|██████▋   | 1210/1813 [2:15:08<1:21:05,  8.07s/it]
+                                                       
+{'loss': '0.9283', 'grad_norm': '0.954', 'learning_rate': '2.917e-05', 'epoch': '0.6676', 'num_input_tokens_seen': 8125224, 'train_runtime': '8109', 'train_tokens_per_second': '1002'}
+
+ 67%|██████▋   | 1210/1813 [2:15:08<1:21:05,  8.07s/it]
+ 67%|██████▋   | 1211/1813 [2:15:15<1:17:22,  7.71s/it]
+ 67%|██████▋   | 1212/1813 [2:15:19<1:07:07,  6.70s/it]
+ 67%|██████▋   | 1213/1813 [2:15:26<1:06:21,  6.64s/it]
+ 67%|██████▋   | 1214/1813 [2:15:34<1:11:26,  7.16s/it]
+ 67%|██████▋   | 1215/1813 [2:15:40<1:08:28,  6.87s/it]
+ 67%|██████▋   | 1216/1813 [2:15:47<1:08:56,  6.93s/it]
+ 67%|██████▋   | 1217/1813 [2:15:53<1:05:52,  6.63s/it]
+ 67%|██████▋   | 1218/1813 [2:15:59<1:04:25,  6.50s/it]
+ 67%|██████▋   | 1219/1813 [2:16:05<1:01:02,  6.17s/it]
+ 67%|██████▋   | 1220/1813 [2:16:13<1:06:39,  6.74s/it]
+                                                       
+{'loss': '0.9578', 'grad_norm': '7.803', 'learning_rate': '2.831e-05', 'epoch': '0.6731', 'num_input_tokens_seen': 8197488, 'train_runtime': '8173', 'train_tokens_per_second': '1003'}
+
+ 67%|██████▋   | 1220/1813 [2:16:13<1:06:39,  6.74s/it]
+ 67%|██████▋   | 1221/1813 [2:16:18<1:02:13,  6.31s/it]
+ 67%|██████▋   | 1222/1813 [2:16:23<58:13,  5.91s/it]  
+ 67%|██████▋   | 1223/1813 [2:16:29<56:51,  5.78s/it]
+ 68%|██████▊   | 1224/1813 [2:16:34<54:17,  5.53s/it]
+ 68%|██████▊   | 1225/1813 [2:16:40<55:49,  5.70s/it]
+ 68%|██████▊   | 1226/1813 [2:16:46<56:44,  5.80s/it]
+ 68%|██████▊   | 1227/1813 [2:16:52<58:40,  6.01s/it]
+ 68%|██████▊   | 1228/1813 [2:16:58<56:37,  5.81s/it]
+ 68%|██████▊   | 1229/1813 [2:17:04<58:36,  6.02s/it]
+ 68%|██████▊   | 1230/1813 [2:17:10<58:44,  6.04s/it]
+                                                     
+{'loss': '0.8137', 'grad_norm': '0.7522', 'learning_rate': '2.746e-05', 'epoch': '0.6786', 'num_input_tokens_seen': 8257288, 'train_runtime': '8231', 'train_tokens_per_second': '1003'}
+
+ 68%|██████▊   | 1230/1813 [2:17:10<58:44,  6.04s/it]
+ 68%|██████▊   | 1231/1813 [2:17:18<1:03:19,  6.53s/it]
+ 68%|██████▊   | 1232/1813 [2:17:25<1:05:50,  6.80s/it]
+ 68%|██████▊   | 1233/1813 [2:17:32<1:04:29,  6.67s/it]
+ 68%|██████▊   | 1234/1813 [2:17:38<1:02:47,  6.51s/it]
+ 68%|██████▊   | 1235/1813 [2:17:44<1:01:59,  6.44s/it]
+ 68%|██████▊   | 1236/1813 [2:17:53<1:07:45,  7.05s/it]
+ 68%|██████▊   | 1237/1813 [2:17:59<1:05:36,  6.83s/it]
+ 68%|██████▊   | 1238/1813 [2:18:05<1:02:06,  6.48s/it]
+ 68%|██████▊   | 1239/1813 [2:18:10<58:56,  6.16s/it]  
+ 68%|██████▊   | 1240/1813 [2:18:15<56:33,  5.92s/it]
+                                                     
+{'loss': '0.819', 'grad_norm': '0.8135', 'learning_rate': '2.663e-05', 'epoch': '0.6841', 'num_input_tokens_seen': 8327600, 'train_runtime': '8296', 'train_tokens_per_second': '1004'}
+
+ 68%|██████▊   | 1240/1813 [2:18:15<56:33,  5.92s/it]
+ 68%|██████▊   | 1241/1813 [2:18:21<56:07,  5.89s/it]
+ 69%|██████▊   | 1242/1813 [2:18:27<56:38,  5.95s/it]
+ 69%|██████▊   | 1243/1813 [2:18:33<57:04,  6.01s/it]
+ 69%|██████▊   | 1244/1813 [2:18:40<59:26,  6.27s/it]
+ 69%|██████▊   | 1245/1813 [2:18:48<1:02:25,  6.59s/it]
+ 69%|██████▊   | 1246/1813 [2:18:53<1:00:13,  6.37s/it]
+ 69%|██████▉   | 1247/1813 [2:19:00<1:01:14,  6.49s/it]
+ 69%|██████▉   | 1248/1813 [2:19:07<1:01:52,  6.57s/it]
+ 69%|██████▉   | 1249/1813 [2:19:13<1:01:21,  6.53s/it]
+ 69%|██████▉   | 1250/1813 [2:19:18<56:30,  6.02s/it]  
+                                                     
+{'loss': '0.9586', 'grad_norm': '1.12', 'learning_rate': '2.579e-05', 'epoch': '0.6897', 'num_input_tokens_seen': 8394824, 'train_runtime': '8359', 'train_tokens_per_second': '1004'}
+
+ 69%|██████▉   | 1250/1813 [2:19:18<56:30,  6.02s/it]
+ 69%|██████▉   | 1251/1813 [2:19:24<56:35,  6.04s/it]
+ 69%|██████▉   | 1252/1813 [2:19:30<55:21,  5.92s/it]
+ 69%|██████▉   | 1253/1813 [2:19:36<55:17,  5.92s/it]
+ 69%|██████▉   | 1254/1813 [2:19:43<57:03,  6.12s/it]
+ 69%|██████▉   | 1255/1813 [2:19:49<57:20,  6.17s/it]
+ 69%|██████▉   | 1256/1813 [2:19:55<57:11,  6.16s/it]
+ 69%|██████▉   | 1257/1813 [2:20:01<56:05,  6.05s/it]
+ 69%|██████▉   | 1258/1813 [2:20:08<58:08,  6.29s/it]
+ 69%|██████▉   | 1259/1813 [2:20:14<58:41,  6.36s/it]
+ 69%|██████▉   | 1260/1813 [2:20:21<1:00:24,  6.55s/it]
+                                                       
+{'loss': '0.9177', 'grad_norm': '1.085', 'learning_rate': '2.497e-05', 'epoch': '0.6952', 'num_input_tokens_seen': 8461096, 'train_runtime': '8422', 'train_tokens_per_second': '1005'}
+
+ 69%|██████▉   | 1260/1813 [2:20:21<1:00:24,  6.55s/it]
+ 70%|██████▉   | 1261/1813 [2:20:29<1:04:27,  7.01s/it]
+ 70%|██████▉   | 1262/1813 [2:20:37<1:05:32,  7.14s/it]
+ 70%|██████▉   | 1263/1813 [2:20:44<1:05:41,  7.17s/it]
+ 70%|██████▉   | 1264/1813 [2:20:50<1:01:32,  6.73s/it]
+ 70%|██████▉   | 1265/1813 [2:20:55<56:58,  6.24s/it]  
+ 70%|██████▉   | 1266/1813 [2:21:02<1:00:37,  6.65s/it]
+ 70%|██████▉   | 1267/1813 [2:21:07<55:58,  6.15s/it]  
+ 70%|██████▉   | 1268/1813 [2:21:12<52:07,  5.74s/it]
+ 70%|██████▉   | 1269/1813 [2:21:17<50:28,  5.57s/it]
+ 70%|███████   | 1270/1813 [2:21:23<51:20,  5.67s/it]
+                                                     
+{'loss': '0.9102', 'grad_norm': '0.9747', 'learning_rate': '2.416e-05', 'epoch': '0.7007', 'num_input_tokens_seen': 8529224, 'train_runtime': '8484', 'train_tokens_per_second': '1005'}
+
+ 70%|███████   | 1270/1813 [2:21:23<51:20,  5.67s/it]
+ 70%|███████   | 1271/1813 [2:21:28<48:26,  5.36s/it]
+ 70%|███████   | 1272/1813 [2:21:35<52:24,  5.81s/it]
+ 70%|███████   | 1273/1813 [2:21:41<55:02,  6.11s/it]
+ 70%|███████   | 1274/1813 [2:21:49<57:58,  6.45s/it]
+ 70%|███████   | 1275/1813 [2:21:55<58:31,  6.53s/it]
+ 70%|███████   | 1276/1813 [2:22:01<55:50,  6.24s/it]
+ 70%|███████   | 1277/1813 [2:22:08<57:49,  6.47s/it]
+ 70%|███████   | 1278/1813 [2:22:14<55:20,  6.21s/it]
+ 71%|███████   | 1279/1813 [2:22:19<52:12,  5.87s/it]
+ 71%|███████   | 1280/1813 [2:22:26<55:26,  6.24s/it]
+                                                     
+{'loss': '0.9284', 'grad_norm': '1.056', 'learning_rate': '2.336e-05', 'epoch': '0.7062', 'num_input_tokens_seen': 8596456, 'train_runtime': '8546', 'train_tokens_per_second': '1006'}
+
+ 71%|███████   | 1280/1813 [2:22:26<55:26,  6.24s/it]
+ 71%|███████   | 1281/1813 [2:22:31<53:17,  6.01s/it]
+ 71%|███████   | 1282/1813 [2:22:38<53:59,  6.10s/it]
+ 71%|███████   | 1283/1813 [2:22:44<55:49,  6.32s/it]
+ 71%|███████   | 1284/1813 [2:22:50<54:41,  6.20s/it]
+ 71%|███████   | 1285/1813 [2:22:56<52:38,  5.98s/it]
+ 71%|███████   | 1286/1813 [2:23:03<55:55,  6.37s/it]
+ 71%|███████   | 1287/1813 [2:23:09<55:30,  6.33s/it]
+ 71%|███████   | 1288/1813 [2:23:15<53:04,  6.07s/it]
+ 71%|███████   | 1289/1813 [2:23:22<55:01,  6.30s/it]
+ 71%|███████   | 1290/1813 [2:23:29<57:36,  6.61s/it]
+                                                     
+{'loss': '0.9834', 'grad_norm': '0.6644', 'learning_rate': '2.256e-05', 'epoch': '0.7117', 'num_input_tokens_seen': 8662472, 'train_runtime': '8609', 'train_tokens_per_second': '1006'}
+
+ 71%|███████   | 1290/1813 [2:23:29<57:36,  6.61s/it]
+ 71%|███████   | 1291/1813 [2:23:35<56:48,  6.53s/it]
+ 71%|███████▏  | 1292/1813 [2:23:42<57:43,  6.65s/it]
+ 71%|███████▏  | 1293/1813 [2:23:49<58:27,  6.75s/it]
+ 71%|███████▏  | 1294/1813 [2:23:55<57:25,  6.64s/it]
+ 71%|███████▏  | 1295/1813 [2:24:02<55:54,  6.48s/it]
+ 71%|███████▏  | 1296/1813 [2:24:07<53:29,  6.21s/it]
+ 72%|███████▏  | 1297/1813 [2:24:13<51:16,  5.96s/it]
+ 72%|███████▏  | 1298/1813 [2:24:19<53:04,  6.18s/it]
+ 72%|███████▏  | 1299/1813 [2:24:26<54:51,  6.40s/it]
+ 72%|███████▏  | 1300/1813 [2:24:32<54:12,  6.34s/it]
+                                                     
+{'loss': '0.8826', 'grad_norm': '1.069', 'learning_rate': '2.178e-05', 'epoch': '0.7172', 'num_input_tokens_seen': 8730600, 'train_runtime': '8673', 'train_tokens_per_second': '1007'}
+
+ 72%|███████▏  | 1300/1813 [2:24:32<54:12,  6.34s/it]
+ 72%|███████▏  | 1301/1813 [2:24:38<53:11,  6.23s/it]
+ 72%|███████▏  | 1302/1813 [2:24:45<53:57,  6.34s/it]
+ 72%|███████▏  | 1303/1813 [2:24:53<57:42,  6.79s/it]
+ 72%|███████▏  | 1304/1813 [2:24:58<54:40,  6.44s/it]
+ 72%|███████▏  | 1305/1813 [2:25:06<57:14,  6.76s/it]
+ 72%|███████▏  | 1306/1813 [2:25:12<54:48,  6.49s/it]
+ 72%|███████▏  | 1307/1813 [2:25:18<53:59,  6.40s/it]
+ 72%|███████▏  | 1308/1813 [2:25:24<53:55,  6.41s/it]
+ 72%|███████▏  | 1309/1813 [2:25:31<53:22,  6.35s/it]
+ 72%|███████▏  | 1310/1813 [2:25:38<54:43,  6.53s/it]
+                                                     
+{'loss': '0.9633', 'grad_norm': '0.8692', 'learning_rate': '2.1e-05', 'epoch': '0.7228', 'num_input_tokens_seen': 8802000, 'train_runtime': '8738', 'train_tokens_per_second': '1007'}
+
+ 72%|███████▏  | 1310/1813 [2:25:38<54:43,  6.53s/it]
+ 72%|███████▏  | 1311/1813 [2:25:44<53:56,  6.45s/it]
+ 72%|███████▏  | 1312/1813 [2:25:50<51:59,  6.23s/it]
+ 72%|███████▏  | 1313/1813 [2:25:55<51:14,  6.15s/it]
+ 72%|███████▏  | 1314/1813 [2:26:02<51:26,  6.19s/it]
+ 73%|███████▎  | 1315/1813 [2:26:07<50:11,  6.05s/it]
+ 73%|███████▎  | 1316/1813 [2:26:13<50:01,  6.04s/it]
+ 73%|███████▎  | 1317/1813 [2:26:20<50:28,  6.11s/it]
+ 73%|███████▎  | 1318/1813 [2:26:26<49:31,  6.00s/it]
+ 73%|███████▎  | 1319/1813 [2:26:32<50:57,  6.19s/it]
+ 73%|███████▎  | 1320/1813 [2:26:39<52:33,  6.40s/it]
+                                                     
+{'loss': '0.9467', 'grad_norm': '0.7702', 'learning_rate': '2.024e-05', 'epoch': '0.7283', 'num_input_tokens_seen': 8866824, 'train_runtime': '8800', 'train_tokens_per_second': '1008'}
+
+ 73%|███████▎  | 1320/1813 [2:26:39<52:33,  6.40s/it]
+ 73%|███████▎  | 1321/1813 [2:26:46<52:55,  6.45s/it]
+ 73%|███████▎  | 1322/1813 [2:26:52<51:36,  6.31s/it]
+ 73%|███████▎  | 1323/1813 [2:26:58<52:21,  6.41s/it]
+ 73%|███████▎  | 1324/1813 [2:27:04<50:45,  6.23s/it]
+ 73%|███████▎  | 1325/1813 [2:27:10<49:24,  6.08s/it]
+ 73%|███████▎  | 1326/1813 [2:27:16<49:49,  6.14s/it]
+ 73%|███████▎  | 1327/1813 [2:27:24<53:41,  6.63s/it]
+ 73%|███████▎  | 1328/1813 [2:27:31<54:17,  6.72s/it]
+ 73%|███████▎  | 1329/1813 [2:27:37<52:57,  6.57s/it]
+ 73%|███████▎  | 1330/1813 [2:27:43<50:38,  6.29s/it]
+                                                     
+{'loss': '0.8939', 'grad_norm': '0.9541', 'learning_rate': '1.948e-05', 'epoch': '0.7338', 'num_input_tokens_seen': 8935584, 'train_runtime': '8863', 'train_tokens_per_second': '1008'}
+
+ 73%|███████▎  | 1330/1813 [2:27:43<50:38,  6.29s/it]
+ 73%|███████▎  | 1331/1813 [2:27:48<49:34,  6.17s/it]
+ 73%|███████▎  | 1332/1813 [2:27:54<48:56,  6.11s/it]
+ 74%|███████▎  | 1333/1813 [2:28:00<48:26,  6.06s/it]
+ 74%|███████▎  | 1334/1813 [2:28:06<47:10,  5.91s/it]
+ 74%|███████▎  | 1335/1813 [2:28:12<46:17,  5.81s/it]
+ 74%|███████▎  | 1336/1813 [2:28:18<47:12,  5.94s/it]
+ 74%|███████▎  | 1337/1813 [2:28:25<50:44,  6.40s/it]
+ 74%|███████▍  | 1338/1813 [2:28:33<52:51,  6.68s/it]
+ 74%|███████▍  | 1339/1813 [2:28:38<49:46,  6.30s/it]
+ 74%|███████▍  | 1340/1813 [2:28:45<51:18,  6.51s/it]
+                                                     
+{'loss': '0.9832', 'grad_norm': '0.8213', 'learning_rate': '1.874e-05', 'epoch': '0.7393', 'num_input_tokens_seen': 9000416, 'train_runtime': '8925', 'train_tokens_per_second': '1008'}
+
+ 74%|███████▍  | 1340/1813 [2:28:45<51:18,  6.51s/it]
+ 74%|███████▍  | 1341/1813 [2:28:52<51:26,  6.54s/it]
+ 74%|███████▍  | 1342/1813 [2:28:57<48:55,  6.23s/it]
+ 74%|███████▍  | 1343/1813 [2:29:03<47:31,  6.07s/it]
+ 74%|███████▍  | 1344/1813 [2:29:09<47:06,  6.03s/it]
+ 74%|███████▍  | 1345/1813 [2:29:16<50:42,  6.50s/it]
+ 74%|███████▍  | 1346/1813 [2:29:23<50:22,  6.47s/it]
+ 74%|███████▍  | 1347/1813 [2:29:28<48:28,  6.24s/it]
+ 74%|███████▍  | 1348/1813 [2:29:35<48:20,  6.24s/it]
+ 74%|███████▍  | 1349/1813 [2:29:41<48:59,  6.34s/it]
+ 74%|███████▍  | 1350/1813 [2:29:47<47:17,  6.13s/it]
+                                                     
+{'loss': '0.9344', 'grad_norm': '1.063', 'learning_rate': '1.801e-05', 'epoch': '0.7448', 'num_input_tokens_seen': 9066864, 'train_runtime': '8987', 'train_tokens_per_second': '1009'}
+
+ 74%|███████▍  | 1350/1813 [2:29:47<47:17,  6.13s/it]
+ 75%|███████▍  | 1351/1813 [2:29:55<51:13,  6.65s/it]
+ 75%|███████▍  | 1352/1813 [2:30:01<50:10,  6.53s/it]
+ 75%|███████▍  | 1353/1813 [2:30:08<50:43,  6.62s/it]
+ 75%|███████▍  | 1354/1813 [2:30:15<51:01,  6.67s/it]
+ 75%|███████▍  | 1355/1813 [2:30:23<55:03,  7.21s/it]
+ 75%|███████▍  | 1356/1813 [2:30:29<51:31,  6.76s/it]
+ 75%|███████▍  | 1357/1813 [2:30:35<49:39,  6.53s/it]
+ 75%|███████▍  | 1358/1813 [2:30:41<49:42,  6.56s/it]
+ 75%|███████▍  | 1359/1813 [2:30:47<48:02,  6.35s/it]
+ 75%|███████▌  | 1360/1813 [2:30:53<46:58,  6.22s/it]
+                                                     
+{'loss': '0.8969', 'grad_norm': '1.113', 'learning_rate': '1.729e-05', 'epoch': '0.7503', 'num_input_tokens_seen': 9140520, 'train_runtime': '9054', 'train_tokens_per_second': '1010'}
+
+ 75%|███████▌  | 1360/1813 [2:30:53<46:58,  6.22s/it]
+ 75%|███████▌  | 1361/1813 [2:31:00<47:50,  6.35s/it]
+ 75%|███████▌  | 1362/1813 [2:31:06<47:38,  6.34s/it]
+ 75%|███████▌  | 1363/1813 [2:31:14<49:53,  6.65s/it]
+ 75%|███████▌  | 1364/1813 [2:31:19<48:13,  6.44s/it]
+ 75%|███████▌  | 1365/1813 [2:31:27<50:25,  6.75s/it]
+ 75%|███████▌  | 1366/1813 [2:31:33<48:28,  6.51s/it]
+ 75%|███████▌  | 1367/1813 [2:31:39<48:28,  6.52s/it]
+ 75%|███████▌  | 1368/1813 [2:31:44<45:02,  6.07s/it]
+ 76%|███████▌  | 1369/1813 [2:31:53<49:17,  6.66s/it]
+ 76%|███████▌  | 1370/1813 [2:31:58<47:37,  6.45s/it]
+                                                     
+{'loss': '0.8836', 'grad_norm': '0.8735', 'learning_rate': '1.658e-05', 'epoch': '0.7559', 'num_input_tokens_seen': 9213048, 'train_runtime': '9119', 'train_tokens_per_second': '1010'}
+
+ 76%|███████▌  | 1370/1813 [2:31:58<47:37,  6.45s/it]
+ 76%|███████▌  | 1371/1813 [2:32:06<50:57,  6.92s/it]
+ 76%|███████▌  | 1372/1813 [2:32:12<47:03,  6.40s/it]
+ 76%|███████▌  | 1373/1813 [2:32:18<46:10,  6.30s/it]
+ 76%|███████▌  | 1374/1813 [2:32:24<45:10,  6.18s/it]
+ 76%|███████▌  | 1375/1813 [2:32:30<44:35,  6.11s/it]
+ 76%|███████▌  | 1376/1813 [2:32:35<43:16,  5.94s/it]
+ 76%|███████▌  | 1377/1813 [2:32:41<44:03,  6.06s/it]
+ 76%|███████▌  | 1378/1813 [2:32:47<43:15,  5.97s/it]
+ 76%|███████▌  | 1379/1813 [2:32:53<43:27,  6.01s/it]
+ 76%|███████▌  | 1380/1813 [2:33:00<45:47,  6.35s/it]
+                                                     
+{'loss': '0.8775', 'grad_norm': '1.015', 'learning_rate': '1.588e-05', 'epoch': '0.7614', 'num_input_tokens_seen': 9279088, 'train_runtime': '9181', 'train_tokens_per_second': '1011'}
+
+ 76%|███████▌  | 1380/1813 [2:33:00<45:47,  6.35s/it]
+ 76%|███████▌  | 1381/1813 [2:33:08<47:18,  6.57s/it]
+ 76%|███████▌  | 1382/1813 [2:33:13<45:06,  6.28s/it]
+ 76%|███████▋  | 1383/1813 [2:33:19<44:42,  6.24s/it]
+ 76%|███████▋  | 1384/1813 [2:33:25<43:57,  6.15s/it]
+ 76%|███████▋  | 1385/1813 [2:33:32<45:24,  6.36s/it]
+ 76%|███████▋  | 1386/1813 [2:33:39<45:37,  6.41s/it]
+ 77%|███████▋  | 1387/1813 [2:33:46<47:10,  6.64s/it]
+ 77%|███████▋  | 1388/1813 [2:33:51<44:33,  6.29s/it]
+ 77%|███████▋  | 1389/1813 [2:33:56<40:13,  5.69s/it]
+ 77%|███████▋  | 1390/1813 [2:34:02<42:12,  5.99s/it]
+                                                     
+{'loss': '0.9474', 'grad_norm': '0.9559', 'learning_rate': '1.52e-05', 'epoch': '0.7669', 'num_input_tokens_seen': 9344880, 'train_runtime': '9243', 'train_tokens_per_second': '1011'}
+
+ 77%|███████▋  | 1390/1813 [2:34:02<42:12,  5.99s/it]
+ 77%|███████▋  | 1391/1813 [2:34:07<40:27,  5.75s/it]
+ 77%|███████▋  | 1392/1813 [2:34:14<42:15,  6.02s/it]
+ 77%|███████▋  | 1393/1813 [2:34:20<40:59,  5.86s/it]
+ 77%|███████▋  | 1394/1813 [2:34:26<42:53,  6.14s/it]
+ 77%|███████▋  | 1395/1813 [2:34:33<43:16,  6.21s/it]
+ 77%|███████▋  | 1396/1813 [2:34:39<43:17,  6.23s/it]
+ 77%|███████▋  | 1397/1813 [2:34:46<44:41,  6.44s/it]
+ 77%|███████▋  | 1398/1813 [2:34:51<41:28,  6.00s/it]
+ 77%|███████▋  | 1399/1813 [2:34:56<39:22,  5.71s/it]
+ 77%|███████▋  | 1400/1813 [2:35:03<41:28,  6.03s/it]
+                                                     
+{'loss': '0.8827', 'grad_norm': '0.7981', 'learning_rate': '1.453e-05', 'epoch': '0.7724', 'num_input_tokens_seen': 9408280, 'train_runtime': '9303', 'train_tokens_per_second': '1011'}
+
+ 77%|███████▋  | 1400/1813 [2:35:03<41:28,  6.03s/it]
+ 77%|███████▋  | 1401/1813 [2:35:08<40:18,  5.87s/it]
+ 77%|███████▋  | 1402/1813 [2:35:14<40:45,  5.95s/it]
+ 77%|███████▋  | 1403/1813 [2:35:21<42:13,  6.18s/it]
+ 77%|███████▋  | 1404/1813 [2:35:27<42:35,  6.25s/it]
+ 77%|███████▋  | 1405/1813 [2:35:32<39:52,  5.86s/it]
+ 78%|███████▊  | 1406/1813 [2:35:38<39:34,  5.83s/it]
+ 78%|███████▊  | 1407/1813 [2:35:45<40:50,  6.04s/it]
+ 78%|███████▊  | 1408/1813 [2:35:50<38:52,  5.76s/it]
+ 78%|███████▊  | 1409/1813 [2:35:54<36:21,  5.40s/it]
+ 78%|███████▊  | 1410/1813 [2:36:00<37:00,  5.51s/it]
+                                                     
+{'loss': '0.9374', 'grad_norm': '1.029', 'learning_rate': '1.387e-05', 'epoch': '0.7779', 'num_input_tokens_seen': 9467408, 'train_runtime': '9361', 'train_tokens_per_second': '1011'}
+
+ 78%|███████▊  | 1410/1813 [2:36:00<37:00,  5.51s/it]
+ 78%|███████▊  | 1411/1813 [2:36:05<36:28,  5.44s/it]
+ 78%|███████▊  | 1412/1813 [2:36:12<38:09,  5.71s/it]
+ 78%|███████▊  | 1413/1813 [2:36:17<37:51,  5.68s/it]
+ 78%|███████▊  | 1414/1813 [2:36:24<38:49,  5.84s/it]
+ 78%|███████▊  | 1415/1813 [2:36:31<41:20,  6.23s/it]
+ 78%|███████▊  | 1416/1813 [2:36:37<41:07,  6.21s/it]
+ 78%|███████▊  | 1417/1813 [2:36:42<39:20,  5.96s/it]
+ 78%|███████▊  | 1418/1813 [2:36:48<38:54,  5.91s/it]
+ 78%|███████▊  | 1419/1813 [2:36:57<44:11,  6.73s/it]
+ 78%|███████▊  | 1420/1813 [2:37:01<40:08,  6.13s/it]
+                                                     
+{'loss': '0.8896', 'grad_norm': '0.9979', 'learning_rate': '1.322e-05', 'epoch': '0.7834', 'num_input_tokens_seen': 9532720, 'train_runtime': '9422', 'train_tokens_per_second': '1012'}
+
+ 78%|███████▊  | 1420/1813 [2:37:01<40:08,  6.13s/it]
+ 78%|███████▊  | 1421/1813 [2:37:08<41:37,  6.37s/it]
+ 78%|███████▊  | 1422/1813 [2:37:14<39:58,  6.13s/it]
+ 78%|███████▊  | 1423/1813 [2:37:20<38:48,  5.97s/it]
+ 79%|███████▊  | 1424/1813 [2:37:27<41:40,  6.43s/it]
+ 79%|███████▊  | 1425/1813 [2:37:32<38:32,  5.96s/it]
+ 79%|███████▊  | 1426/1813 [2:37:41<43:33,  6.75s/it]
+ 79%|███████▊  | 1427/1813 [2:37:46<41:01,  6.38s/it]
+ 79%|███████▉  | 1428/1813 [2:37:53<41:36,  6.48s/it]
+ 79%|███████▉  | 1429/1813 [2:37:59<40:57,  6.40s/it]
+ 79%|███████▉  | 1430/1813 [2:38:06<41:40,  6.53s/it]
+                                                     
+{'loss': '0.8718', 'grad_norm': '0.9045', 'learning_rate': '1.259e-05', 'epoch': '0.789', 'num_input_tokens_seen': 9602112, 'train_runtime': '9486', 'train_tokens_per_second': '1012'}
+
+ 79%|███████▉  | 1430/1813 [2:38:06<41:40,  6.53s/it]
+ 79%|███████▉  | 1431/1813 [2:38:12<40:31,  6.36s/it]
+ 79%|███████▉  | 1432/1813 [2:38:18<39:34,  6.23s/it]
+ 79%|███████▉  | 1433/1813 [2:38:24<39:50,  6.29s/it]
+ 79%|███████▉  | 1434/1813 [2:38:31<40:02,  6.34s/it]
+ 79%|███████▉  | 1435/1813 [2:38:36<38:32,  6.12s/it]
+ 79%|███████▉  | 1436/1813 [2:38:43<40:03,  6.38s/it]
+ 79%|███████▉  | 1437/1813 [2:38:49<38:50,  6.20s/it]
+ 79%|███████▉  | 1438/1813 [2:38:56<40:42,  6.51s/it]
+ 79%|███████▉  | 1439/1813 [2:39:02<38:32,  6.18s/it]
+ 79%|███████▉  | 1440/1813 [2:39:07<36:12,  5.82s/it]
+                                                     
+{'loss': '0.8798', 'grad_norm': '0.9689', 'learning_rate': '1.197e-05', 'epoch': '0.7945', 'num_input_tokens_seen': 9667112, 'train_runtime': '9547', 'train_tokens_per_second': '1013'}
+
+ 79%|███████▉  | 1440/1813 [2:39:07<36:12,  5.82s/it]
+ 79%|███████▉  | 1441/1813 [2:39:13<37:06,  5.99s/it]
+ 80%|███████▉  | 1442/1813 [2:39:18<35:46,  5.78s/it]
+ 80%|███████▉  | 1443/1813 [2:39:24<34:39,  5.62s/it]
+ 80%|███████▉  | 1444/1813 [2:39:30<36:37,  5.95s/it]
+ 80%|███████▉  | 1445/1813 [2:39:36<35:52,  5.85s/it]
+ 80%|███████▉  | 1446/1813 [2:39:42<35:28,  5.80s/it]
+ 80%|███████▉  | 1447/1813 [2:39:50<39:45,  6.52s/it]
+ 80%|███████▉  | 1448/1813 [2:39:57<41:11,  6.77s/it]
+ 80%|███████▉  | 1449/1813 [2:40:04<40:52,  6.74s/it]
+ 80%|███████▉  | 1450/1813 [2:40:10<40:33,  6.70s/it]
+                                                     
+{'loss': '0.9669', 'grad_norm': '0.9346', 'learning_rate': '1.136e-05', 'epoch': '0.8', 'num_input_tokens_seen': 9735792, 'train_runtime': '9611', 'train_tokens_per_second': '1013'}
+
+ 80%|███████▉  | 1450/1813 [2:40:10<40:33,  6.70s/it]
+ 80%|████████  | 1451/1813 [2:40:17<40:55,  6.78s/it]
+ 80%|████████  | 1452/1813 [2:40:22<36:42,  6.10s/it]
+ 80%|████████  | 1453/1813 [2:40:30<40:46,  6.80s/it]
+ 80%|████████  | 1454/1813 [2:40:36<38:30,  6.43s/it]
+ 80%|████████  | 1455/1813 [2:40:42<37:58,  6.36s/it]
+ 80%|████████  | 1456/1813 [2:40:49<39:25,  6.63s/it]
+ 80%|████████  | 1457/1813 [2:40:53<34:41,  5.85s/it]
+ 80%|████████  | 1458/1813 [2:40:59<34:28,  5.83s/it]
+ 80%|████████  | 1459/1813 [2:41:06<36:41,  6.22s/it]
+ 81%|████████  | 1460/1813 [2:41:13<38:15,  6.50s/it]
+                                                     
+{'loss': '0.9149', 'grad_norm': '0.947', 'learning_rate': '1.077e-05', 'epoch': '0.8055', 'num_input_tokens_seen': 9802968, 'train_runtime': '9674', 'train_tokens_per_second': '1013'}
+
+ 81%|████████  | 1460/1813 [2:41:13<38:15,  6.50s/it]
+ 81%|████████  | 1461/1813 [2:41:19<37:04,  6.32s/it]
+ 81%|████████  | 1462/1813 [2:41:24<34:54,  5.97s/it]
+ 81%|████████  | 1463/1813 [2:41:32<37:17,  6.39s/it]
+ 81%|████████  | 1464/1813 [2:41:38<37:31,  6.45s/it]
+ 81%|████████  | 1465/1813 [2:41:47<40:29,  6.98s/it]
+ 81%|████████  | 1466/1813 [2:41:52<37:37,  6.51s/it]
+ 81%|████████  | 1467/1813 [2:41:59<38:56,  6.75s/it]
+ 81%|████████  | 1468/1813 [2:42:06<38:29,  6.69s/it]
+ 81%|████████  | 1469/1813 [2:42:15<41:50,  7.30s/it]
+ 81%|████████  | 1470/1813 [2:42:21<40:22,  7.06s/it]
+                                                     
+{'loss': '0.913', 'grad_norm': '0.9012', 'learning_rate': '1.019e-05', 'epoch': '0.811', 'num_input_tokens_seen': 9879232, 'train_runtime': '9742', 'train_tokens_per_second': '1014'}
+
+ 81%|████████  | 1470/1813 [2:42:21<40:22,  7.06s/it]
+ 81%|████████  | 1471/1813 [2:42:27<38:38,  6.78s/it]
+ 81%|████████  | 1472/1813 [2:42:34<38:14,  6.73s/it]
+ 81%|████████  | 1473/1813 [2:42:40<36:22,  6.42s/it]
+ 81%|████████▏ | 1474/1813 [2:42:46<36:31,  6.46s/it]
+ 81%|████████▏ | 1475/1813 [2:42:52<35:55,  6.38s/it]
+ 81%|████████▏ | 1476/1813 [2:42:59<35:34,  6.33s/it]
+ 81%|████████▏ | 1477/1813 [2:43:05<35:22,  6.32s/it]
+ 82%|████████▏ | 1478/1813 [2:43:10<34:04,  6.10s/it]
+ 82%|████████▏ | 1479/1813 [2:43:17<33:56,  6.10s/it]
+ 82%|████████▏ | 1480/1813 [2:43:24<35:26,  6.38s/it]
+                                                     
+{'loss': '0.8914', 'grad_norm': '0.7256', 'learning_rate': '9.627e-06', 'epoch': '0.8166', 'num_input_tokens_seen': 9948032, 'train_runtime': '9804', 'train_tokens_per_second': '1015'}
+
+ 82%|████████▏ | 1480/1813 [2:43:24<35:26,  6.38s/it]
+ 82%|████████▏ | 1481/1813 [2:43:30<35:43,  6.46s/it]
+ 82%|████████▏ | 1482/1813 [2:43:36<34:20,  6.22s/it]
+ 82%|████████▏ | 1483/1813 [2:43:43<35:08,  6.39s/it]
+ 82%|████████▏ | 1484/1813 [2:43:49<35:04,  6.40s/it]
+ 82%|████████▏ | 1485/1813 [2:43:56<35:18,  6.46s/it]
+ 82%|████████▏ | 1486/1813 [2:44:01<33:08,  6.08s/it]
+ 82%|████████▏ | 1487/1813 [2:44:09<36:09,  6.66s/it]
+ 82%|████████▏ | 1488/1813 [2:44:15<35:27,  6.54s/it]
+ 82%|████████▏ | 1489/1813 [2:44:23<36:40,  6.79s/it]
+ 82%|████████▏ | 1490/1813 [2:44:30<36:53,  6.85s/it]
+                                                     
+{'loss': '0.9309', 'grad_norm': '0.9664', 'learning_rate': '9.077e-06', 'epoch': '0.8221', 'num_input_tokens_seen': 10021400, 'train_runtime': '9870', 'train_tokens_per_second': '1015'}
+
+ 82%|████████▏ | 1490/1813 [2:44:30<36:53,  6.85s/it]
+ 82%|████████▏ | 1491/1813 [2:44:36<35:31,  6.62s/it]
+ 82%|████████▏ | 1492/1813 [2:44:44<38:10,  7.13s/it]
+ 82%|████████▏ | 1493/1813 [2:44:50<36:15,  6.80s/it]
+ 82%|████████▏ | 1494/1813 [2:44:56<34:48,  6.55s/it]
+ 82%|████████▏ | 1495/1813 [2:45:04<36:59,  6.98s/it]
+ 83%|████████▎ | 1496/1813 [2:45:09<33:51,  6.41s/it]
+ 83%|████████▎ | 1497/1813 [2:45:14<31:40,  6.01s/it]
+ 83%|████████▎ | 1498/1813 [2:45:20<32:14,  6.14s/it]
+ 83%|████████▎ | 1499/1813 [2:45:25<30:00,  5.73s/it]
+ 83%|████████▎ | 1500/1813 [2:45:33<32:51,  6.30s/it]
+                                                     
+{'loss': '0.8627', 'grad_norm': '0.9089', 'learning_rate': '8.542e-06', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904, 'train_runtime': '9933', 'train_tokens_per_second': '1016'}
+
+ 83%|████████▎ | 1500/1813 [2:45:33<32:51,  6.30s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.77it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.22it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.31it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.06it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.99it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.82it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.04it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.59it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:55,  5.58it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.35it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.98it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:06<02:04,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.09it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.30it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.04it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.34it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.73it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.89it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.91it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.28it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.74it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.85it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.19it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.03it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.94it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:37,  5.77it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.51it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.77it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.35it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.62it/s]
+
+ 10%|█         | 103/1000 [00:14<01:56,  7.73it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.30it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.43it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.19it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.00it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:11, 12.48it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.77it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.21it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.90it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.07it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.14it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.06it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.90it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.02it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.37it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.96it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.59it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:28,  5.64it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.16it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.76it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.80it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.13it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.14it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.20it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.72it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.42it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.56it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.10it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.62it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.20it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.65it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.74it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.62it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.92it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.47it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.66it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.73it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.14it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.79it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.20it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.96it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.37it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.03it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.35it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.68it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.73it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.67it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.74it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.11it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.74it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.07it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.94it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.78it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.79it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.26it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.56it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.40it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.16it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.77it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.37it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.03it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.80it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.02it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.86it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:48,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.65it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.60it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.43it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.75it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.88it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.18it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.18it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.64it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.37it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.77it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.46it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:09,  8.65it/s]
+
+ 40%|███▉      | 398/1000 [00:53<01:23,  7.18it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:41,  5.93it/s]
+
+ 40%|████      | 400/1000 [00:53<01:51,  5.37it/s]
+
+ 40%|████      | 402/1000 [00:54<01:44,  5.73it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.21it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.56it/s]
+
+ 41%|████      | 407/1000 [00:54<01:24,  7.04it/s]
+
+ 41%|████      | 408/1000 [00:55<01:36,  6.14it/s]
+
+ 41%|████      | 410/1000 [00:55<01:33,  6.30it/s]
+
+ 41%|████      | 411/1000 [00:55<01:44,  5.64it/s]
+
+ 41%|████      | 412/1000 [00:55<01:53,  5.17it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:09,  8.33it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.50it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.38it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.91it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:16,  7.50it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:13,  7.82it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.52it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.02it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.79it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.52it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.29it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.71it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.47it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.27it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.84it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.50it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.13it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.54it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.01it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.29it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.98it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.97it/s]
+
+ 50%|████▉     | 496/1000 [01:06<01:00,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.81it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.55it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.97it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.10it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.07it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.01it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.57it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.13it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.72it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.68it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.50it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.25it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.94it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.75it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.85it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.05it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.96it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.55it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.59it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.16it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.79it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.38it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.68it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:50,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.79it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.60it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.75it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.10it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.02it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.20it/s]
+
+ 62%|██████▏   | 624/1000 [01:24<00:44,  8.46it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:53,  6.96it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.49it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:53,  6.91it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<01:02,  5.95it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.72it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:39,  9.36it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.64it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.02it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.91it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.13it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.94it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.57it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.09it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.69it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.91it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.09it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  5.99it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.55it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.47it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.60it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.69it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:56,  5.87it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.90it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.89it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.86it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.85it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.15it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.83it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.25it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.41it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:48,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.59it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.01it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.71it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.36it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.17it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.30it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.12it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.61it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.53it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.88it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.51it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.69it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.69it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.64it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.61it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.98it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.85it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.82it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.16it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.21it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.23it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.04it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.30it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.81it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.00it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.22it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.52it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.54it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.05it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.21it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.76it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.64it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.72it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.28it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.28it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.86it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.34it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.76it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.84it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.69it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.32it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.59it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.78it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.59it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.93it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.16it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.62it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.07it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.64it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.46it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.60it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.13it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.33it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.82it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.39it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.39it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.44it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.33it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.40it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.19it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.79it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.42it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.72it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.14it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.97it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7545', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.383', 'eval_steps_per_second': '7.383', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904}
+
+ 83%|████████▎ | 1500/1813 [2:47:48<32:51,  6.30s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+
+                                                   
+ 83%|████████▎ | 1501/1813 [2:47:55<4:04:05, 46.94s/it]
+ 83%|████████▎ | 1502/1813 [2:47:59<2:57:09, 34.18s/it]
+ 83%|████████▎ | 1503/1813 [2:48:05<2:13:12, 25.78s/it]
+ 83%|████████▎ | 1504/1813 [2:48:10<1:40:41, 19.55s/it]
+ 83%|████████▎ | 1505/1813 [2:48:16<1:18:23, 15.27s/it]
+ 83%|████████▎ | 1506/1813 [2:48:22<1:05:17, 12.76s/it]
+ 83%|████████▎ | 1507/1813 [2:48:27<53:04, 10.41s/it]  
+ 83%|████████▎ | 1508/1813 [2:48:34<47:39,  9.38s/it]
+ 83%|████████▎ | 1509/1813 [2:48:40<42:22,  8.36s/it]
+ 83%|████████▎ | 1510/1813 [2:48:47<38:56,  7.71s/it]
+                                                     
+{'loss': '0.8551', 'grad_norm': '1.092', 'learning_rate': '8.021e-06', 'epoch': '0.8331', 'num_input_tokens_seen': 10151928, 'train_runtime': '1.013e+04', 'train_tokens_per_second': '1002'}
+
+ 83%|████████▎ | 1510/1813 [2:48:47<38:56,  7.71s/it]
+ 83%|████████▎ | 1511/1813 [2:48:53<36:52,  7.33s/it]
+ 83%|████████▎ | 1512/1813 [2:48:59<34:25,  6.86s/it]
+ 83%|████████▎ | 1513/1813 [2:49:04<32:15,  6.45s/it]
+ 84%|████████▎ | 1514/1813 [2:49:11<32:17,  6.48s/it]
+ 84%|████████▎ | 1515/1813 [2:49:18<33:33,  6.76s/it]
+ 84%|████████▎ | 1516/1813 [2:49:25<33:36,  6.79s/it]
+ 84%|████████▎ | 1517/1813 [2:49:32<33:20,  6.76s/it]
+ 84%|████████▎ | 1518/1813 [2:49:39<34:26,  7.01s/it]
+ 84%|████████▍ | 1519/1813 [2:49:46<34:30,  7.04s/it]
+ 84%|████████▍ | 1520/1813 [2:49:53<33:07,  6.78s/it]
+                                                     
+{'loss': '0.8822', 'grad_norm': '1.069', 'learning_rate': '7.515e-06', 'epoch': '0.8386', 'num_input_tokens_seen': 10225024, 'train_runtime': '1.019e+04', 'train_tokens_per_second': '1003'}
+
+ 84%|████████▍ | 1520/1813 [2:49:53<33:07,  6.78s/it]
+ 84%|████████▍ | 1521/1813 [2:49:59<33:06,  6.80s/it]
+ 84%|████████▍ | 1522/1813 [2:50:06<32:08,  6.63s/it]
+ 84%|████████▍ | 1523/1813 [2:50:13<32:58,  6.82s/it]
+ 84%|████████▍ | 1524/1813 [2:50:19<32:06,  6.67s/it]
+ 84%|████████▍ | 1525/1813 [2:50:24<29:15,  6.09s/it]
+ 84%|████████▍ | 1526/1813 [2:50:30<29:31,  6.17s/it]
+ 84%|████████▍ | 1527/1813 [2:50:37<30:03,  6.31s/it]
+ 84%|████████▍ | 1528/1813 [2:50:44<30:18,  6.38s/it]
+ 84%|████████▍ | 1529/1813 [2:50:51<31:10,  6.59s/it]
+ 84%|████████▍ | 1530/1813 [2:50:56<29:58,  6.36s/it]
+                                                     
+{'loss': '0.9278', 'grad_norm': '0.7647', 'learning_rate': '7.025e-06', 'epoch': '0.8441', 'num_input_tokens_seen': 10294368, 'train_runtime': '1.026e+04', 'train_tokens_per_second': '1004'}
+
+ 84%|████████▍ | 1530/1813 [2:50:56<29:58,  6.36s/it]
+ 84%|████████▍ | 1531/1813 [2:51:02<28:11,  6.00s/it]
+ 85%|████████▍ | 1532/1813 [2:51:09<29:23,  6.28s/it]
+ 85%|████████▍ | 1533/1813 [2:51:14<27:30,  5.89s/it]
+ 85%|████████▍ | 1534/1813 [2:51:22<30:26,  6.55s/it]
+ 85%|████████▍ | 1535/1813 [2:51:27<28:43,  6.20s/it]
+ 85%|████████▍ | 1536/1813 [2:51:34<30:26,  6.59s/it]
+ 85%|████████▍ | 1537/1813 [2:51:41<30:29,  6.63s/it]
+ 85%|████████▍ | 1538/1813 [2:51:46<27:39,  6.04s/it]
+ 85%|████████▍ | 1539/1813 [2:51:51<26:53,  5.89s/it]
+ 85%|████████▍ | 1540/1813 [2:51:57<26:13,  5.76s/it]
+                                                     
+{'loss': '0.9113', 'grad_norm': '1.176', 'learning_rate': '6.55e-06', 'epoch': '0.8497', 'num_input_tokens_seen': 10359416, 'train_runtime': '1.032e+04', 'train_tokens_per_second': '1004'}
+
+ 85%|████████▍ | 1540/1813 [2:51:57<26:13,  5.76s/it]
+ 85%|████████▍ | 1541/1813 [2:52:03<27:12,  6.00s/it]
+ 85%|████████▌ | 1542/1813 [2:52:10<27:23,  6.06s/it]
+ 85%|████████▌ | 1543/1813 [2:52:15<25:40,  5.71s/it]
+ 85%|████████▌ | 1544/1813 [2:52:19<24:28,  5.46s/it]
+ 85%|████████▌ | 1545/1813 [2:52:27<27:29,  6.16s/it]
+ 85%|████████▌ | 1546/1813 [2:52:33<27:05,  6.09s/it]
+ 85%|████████▌ | 1547/1813 [2:52:38<25:58,  5.86s/it]
+ 85%|████████▌ | 1548/1813 [2:52:47<29:13,  6.62s/it]
+ 85%|████████▌ | 1549/1813 [2:52:52<27:36,  6.27s/it]
+ 85%|████████▌ | 1550/1813 [2:53:00<29:01,  6.62s/it]
+                                                     
+{'loss': '0.9997', 'grad_norm': '0.8834', 'learning_rate': '6.09e-06', 'epoch': '0.8552', 'num_input_tokens_seen': 10427368, 'train_runtime': '1.038e+04', 'train_tokens_per_second': '1005'}
+
+ 85%|████████▌ | 1550/1813 [2:53:00<29:01,  6.62s/it]
+ 86%|████████▌ | 1551/1813 [2:53:07<30:15,  6.93s/it]
+ 86%|████████▌ | 1552/1813 [2:53:14<29:12,  6.71s/it]
+ 86%|████████▌ | 1553/1813 [2:53:20<28:08,  6.49s/it]
+ 86%|████████▌ | 1554/1813 [2:53:26<28:13,  6.54s/it]
+ 86%|████████▌ | 1555/1813 [2:53:33<28:18,  6.58s/it]
+ 86%|████████▌ | 1556/1813 [2:53:39<27:51,  6.50s/it]
+ 86%|████████▌ | 1557/1813 [2:53:47<29:23,  6.89s/it]
+ 86%|████████▌ | 1558/1813 [2:53:53<28:00,  6.59s/it]
+ 86%|████████▌ | 1559/1813 [2:53:59<27:25,  6.48s/it]
+ 86%|████████▌ | 1560/1813 [2:54:04<25:36,  6.07s/it]
+                                                     
+{'loss': '0.9878', 'grad_norm': '0.7696', 'learning_rate': '5.646e-06', 'epoch': '0.8607', 'num_input_tokens_seen': 10497472, 'train_runtime': '1.044e+04', 'train_tokens_per_second': '1005'}
+
+ 86%|████████▌ | 1560/1813 [2:54:04<25:36,  6.07s/it]
+ 86%|████████▌ | 1561/1813 [2:54:11<26:33,  6.32s/it]
+ 86%|████████▌ | 1562/1813 [2:54:17<26:24,  6.31s/it]
+ 86%|████████▌ | 1563/1813 [2:54:24<26:12,  6.29s/it]
+ 86%|████████▋ | 1564/1813 [2:54:30<26:14,  6.32s/it]
+ 86%|████████▋ | 1565/1813 [2:54:35<24:06,  5.83s/it]
+ 86%|████████▋ | 1566/1813 [2:54:41<24:27,  5.94s/it]
+ 86%|████████▋ | 1567/1813 [2:54:47<24:45,  6.04s/it]
+ 86%|████████▋ | 1568/1813 [2:54:52<23:24,  5.73s/it]
+ 87%|████████▋ | 1569/1813 [2:55:00<25:24,  6.25s/it]
+ 87%|████████▋ | 1570/1813 [2:55:07<26:32,  6.56s/it]
+                                                     
+{'loss': '0.8288', 'grad_norm': '0.941', 'learning_rate': '5.218e-06', 'epoch': '0.8662', 'num_input_tokens_seen': 10564336, 'train_runtime': '1.051e+04', 'train_tokens_per_second': '1005'}
+
+ 87%|████████▋ | 1570/1813 [2:55:07<26:32,  6.56s/it]
+ 87%|████████▋ | 1571/1813 [2:55:12<24:58,  6.19s/it]
+ 87%|████████▋ | 1572/1813 [2:55:18<24:15,  6.04s/it]
+ 87%|████████▋ | 1573/1813 [2:55:24<23:58,  5.99s/it]
+ 87%|████████▋ | 1574/1813 [2:55:30<24:38,  6.18s/it]
+ 87%|████████▋ | 1575/1813 [2:55:36<24:03,  6.06s/it]
+ 87%|████████▋ | 1576/1813 [2:55:44<25:46,  6.52s/it]
+ 87%|████████▋ | 1577/1813 [2:55:51<26:11,  6.66s/it]
+ 87%|████████▋ | 1578/1813 [2:55:57<25:34,  6.53s/it]
+ 87%|████████▋ | 1579/1813 [2:56:03<25:04,  6.43s/it]
+ 87%|████████▋ | 1580/1813 [2:56:08<23:07,  5.96s/it]
+                                                     
+{'loss': '0.9389', 'grad_norm': '1.107', 'learning_rate': '4.806e-06', 'epoch': '0.8717', 'num_input_tokens_seen': 10628400, 'train_runtime': '1.057e+04', 'train_tokens_per_second': '1006'}
+
+ 87%|████████▋ | 1580/1813 [2:56:08<23:07,  5.96s/it]
+ 87%|████████▋ | 1581/1813 [2:56:14<22:46,  5.89s/it]
+ 87%|████████▋ | 1582/1813 [2:56:18<20:42,  5.38s/it]
+ 87%|████████▋ | 1583/1813 [2:56:24<21:34,  5.63s/it]
+ 87%|████████▋ | 1584/1813 [2:56:31<22:24,  5.87s/it]
+ 87%|████████▋ | 1585/1813 [2:56:38<24:03,  6.33s/it]
+ 87%|████████▋ | 1586/1813 [2:56:46<25:15,  6.67s/it]
+ 88%|████████▊ | 1587/1813 [2:56:53<26:11,  6.95s/it]
+ 88%|████████▊ | 1588/1813 [2:56:59<24:52,  6.63s/it]
+ 88%|████████▊ | 1589/1813 [2:57:04<23:11,  6.21s/it]
+ 88%|████████▊ | 1590/1813 [2:57:11<23:40,  6.37s/it]
+                                                     
+{'loss': '0.8877', 'grad_norm': '0.9404', 'learning_rate': '4.41e-06', 'epoch': '0.8772', 'num_input_tokens_seen': 10695288, 'train_runtime': '1.063e+04', 'train_tokens_per_second': '1006'}
+
+ 88%|████████▊ | 1590/1813 [2:57:11<23:40,  6.37s/it]
+ 88%|████████▊ | 1591/1813 [2:57:17<23:31,  6.36s/it]
+ 88%|████████▊ | 1592/1813 [2:57:24<23:39,  6.42s/it]
+ 88%|████████▊ | 1593/1813 [2:57:31<23:58,  6.54s/it]
+ 88%|████████▊ | 1594/1813 [2:57:38<24:11,  6.63s/it]
+ 88%|████████▊ | 1595/1813 [2:57:45<24:55,  6.86s/it]
+ 88%|████████▊ | 1596/1813 [2:57:54<26:55,  7.44s/it]
+ 88%|████████▊ | 1597/1813 [2:58:01<26:10,  7.27s/it]
+ 88%|████████▊ | 1598/1813 [2:58:07<25:09,  7.02s/it]
+ 88%|████████▊ | 1599/1813 [2:58:13<23:22,  6.56s/it]
+ 88%|████████▊ | 1600/1813 [2:58:17<21:11,  5.97s/it]
+                                                     
+{'loss': '0.9118', 'grad_norm': '0.8956', 'learning_rate': '4.031e-06', 'epoch': '0.8828', 'num_input_tokens_seen': 10767512, 'train_runtime': '1.07e+04', 'train_tokens_per_second': '1007'}
+
+ 88%|████████▊ | 1600/1813 [2:58:17<21:11,  5.97s/it]
+ 88%|████████▊ | 1601/1813 [2:58:25<23:08,  6.55s/it]
+ 88%|████████▊ | 1602/1813 [2:58:31<22:23,  6.37s/it]
+ 88%|████████▊ | 1603/1813 [2:58:38<22:44,  6.50s/it]
+ 88%|████████▊ | 1604/1813 [2:58:45<23:02,  6.62s/it]
+ 89%|████████▊ | 1605/1813 [2:58:53<24:43,  7.13s/it]
+ 89%|████████▊ | 1606/1813 [2:58:58<22:28,  6.51s/it]
+ 89%|████████▊ | 1607/1813 [2:59:05<22:31,  6.56s/it]
+ 89%|████████▊ | 1608/1813 [2:59:12<23:08,  6.78s/it]
+ 89%|████████▊ | 1609/1813 [2:59:17<21:26,  6.31s/it]
+ 89%|████████▉ | 1610/1813 [2:59:23<20:53,  6.18s/it]
+                                                     
+{'loss': '0.9112', 'grad_norm': '1.199', 'learning_rate': '3.667e-06', 'epoch': '0.8883', 'num_input_tokens_seen': 10840800, 'train_runtime': '1.076e+04', 'train_tokens_per_second': '1007'}
+
+ 89%|████████▉ | 1610/1813 [2:59:23<20:53,  6.18s/it]
+ 89%|████████▉ | 1611/1813 [2:59:28<19:30,  5.79s/it]
+ 89%|████████▉ | 1612/1813 [2:59:34<19:21,  5.78s/it]
+ 89%|████████▉ | 1613/1813 [2:59:38<17:57,  5.39s/it]
+ 89%|████████▉ | 1614/1813 [2:59:44<18:14,  5.50s/it]
+ 89%|████████▉ | 1615/1813 [2:59:50<19:05,  5.78s/it]
+ 89%|████████▉ | 1616/1813 [2:59:57<19:41,  6.00s/it]
+ 89%|████████▉ | 1617/1813 [3:00:04<20:09,  6.17s/it]
+ 89%|████████▉ | 1618/1813 [3:00:09<18:59,  5.84s/it]
+ 89%|████████▉ | 1619/1813 [3:00:14<18:38,  5.77s/it]
+ 89%|████████▉ | 1620/1813 [3:00:19<17:49,  5.54s/it]
+                                                     
+{'loss': '0.9181', 'grad_norm': '0.9562', 'learning_rate': '3.32e-06', 'epoch': '0.8938', 'num_input_tokens_seen': 10899640, 'train_runtime': '1.082e+04', 'train_tokens_per_second': '1007'}
+
+ 89%|████████▉ | 1620/1813 [3:00:19<17:49,  5.54s/it]
+ 89%|████████▉ | 1621/1813 [3:00:25<17:51,  5.58s/it]
+ 89%|████████▉ | 1622/1813 [3:00:33<19:48,  6.22s/it]
+ 90%|████████▉ | 1623/1813 [3:00:40<20:23,  6.44s/it]
+ 90%|████████▉ | 1624/1813 [3:00:46<19:54,  6.32s/it]
+ 90%|████████▉ | 1625/1813 [3:00:53<20:50,  6.65s/it]
+ 90%|████████▉ | 1626/1813 [3:00:59<20:06,  6.45s/it]
+ 90%|████████▉ | 1627/1813 [3:01:05<19:24,  6.26s/it]
+ 90%|████████▉ | 1628/1813 [3:01:11<18:46,  6.09s/it]
+ 90%|████████▉ | 1629/1813 [3:01:18<19:40,  6.41s/it]
+ 90%|████████▉ | 1630/1813 [3:01:23<18:46,  6.15s/it]
+                                                     
+{'loss': '0.9185', 'grad_norm': '1.372', 'learning_rate': '2.99e-06', 'epoch': '0.8993', 'num_input_tokens_seen': 10968528, 'train_runtime': '1.088e+04', 'train_tokens_per_second': '1008'}
+
+ 90%|████████▉ | 1630/1813 [3:01:23<18:46,  6.15s/it]
+ 90%|████████▉ | 1631/1813 [3:01:29<18:13,  6.01s/it]
+ 90%|█████████ | 1632/1813 [3:01:34<16:59,  5.63s/it]
+ 90%|█████████ | 1633/1813 [3:01:41<18:27,  6.15s/it]
+ 90%|█████████ | 1634/1813 [3:01:46<17:25,  5.84s/it]
+ 90%|█████████ | 1635/1813 [3:01:53<18:20,  6.18s/it]
+ 90%|█████████ | 1636/1813 [3:01:59<17:48,  6.04s/it]
+ 90%|█████████ | 1637/1813 [3:02:05<18:02,  6.15s/it]
+ 90%|█████████ | 1638/1813 [3:02:11<17:21,  5.95s/it]
+ 90%|█████████ | 1639/1813 [3:02:18<18:08,  6.26s/it]
+ 90%|█████████ | 1640/1813 [3:02:23<17:04,  5.92s/it]
+                                                     
+{'loss': '0.9227', 'grad_norm': '0.7844', 'learning_rate': '2.677e-06', 'epoch': '0.9048', 'num_input_tokens_seen': 11030048, 'train_runtime': '1.094e+04', 'train_tokens_per_second': '1008'}
+
+ 90%|█████████ | 1640/1813 [3:02:23<17:04,  5.92s/it]
+ 91%|█████████ | 1641/1813 [3:02:29<16:50,  5.87s/it]
+ 91%|█████████ | 1642/1813 [3:02:35<17:18,  6.07s/it]
+ 91%|█████████ | 1643/1813 [3:02:41<17:10,  6.06s/it]
+ 91%|█████████ | 1644/1813 [3:02:48<17:36,  6.25s/it]
+ 91%|█████████ | 1645/1813 [3:02:57<20:13,  7.22s/it]
+ 91%|█████████ | 1646/1813 [3:03:04<19:32,  7.02s/it]
+ 91%|█████████ | 1647/1813 [3:03:10<18:34,  6.71s/it]
+ 91%|█████████ | 1648/1813 [3:03:17<18:28,  6.72s/it]
+ 91%|█████████ | 1649/1813 [3:03:22<17:08,  6.27s/it]
+ 91%|█████████ | 1650/1813 [3:03:27<16:05,  5.92s/it]
+                                                     
+{'loss': '0.8873', 'grad_norm': '0.8632', 'learning_rate': '2.38e-06', 'epoch': '0.9103', 'num_input_tokens_seen': 11098768, 'train_runtime': '1.101e+04', 'train_tokens_per_second': '1008'}
+
+ 91%|█████████ | 1650/1813 [3:03:27<16:05,  5.92s/it]
+ 91%|█████████ | 1651/1813 [3:03:33<16:22,  6.06s/it]
+ 91%|█████████ | 1652/1813 [3:03:40<16:34,  6.18s/it]
+ 91%|█████████ | 1653/1813 [3:03:47<16:54,  6.34s/it]
+ 91%|█████████ | 1654/1813 [3:03:52<16:21,  6.18s/it]
+ 91%|█████████▏| 1655/1813 [3:03:59<16:25,  6.24s/it]
+ 91%|█████████▏| 1656/1813 [3:04:05<16:02,  6.13s/it]
+ 91%|█████████▏| 1657/1813 [3:04:12<17:09,  6.60s/it]
+ 91%|█████████▏| 1658/1813 [3:04:19<17:04,  6.61s/it]
+ 92%|█████████▏| 1659/1813 [3:04:26<17:16,  6.73s/it]
+ 92%|█████████▏| 1660/1813 [3:04:32<16:49,  6.60s/it]
+                                                     
+{'loss': '0.9432', 'grad_norm': '0.9487', 'learning_rate': '2.101e-06', 'epoch': '0.9159', 'num_input_tokens_seen': 11168848, 'train_runtime': '1.107e+04', 'train_tokens_per_second': '1009'}
+
+ 92%|█████████▏| 1660/1813 [3:04:32<16:49,  6.60s/it]
+ 92%|█████████▏| 1661/1813 [3:04:39<17:08,  6.76s/it]
+ 92%|█████████▏| 1662/1813 [3:04:46<16:54,  6.72s/it]
+ 92%|█████████▏| 1663/1813 [3:04:53<16:57,  6.78s/it]
+ 92%|█████████▏| 1664/1813 [3:05:00<16:53,  6.80s/it]
+ 92%|█████████▏| 1665/1813 [3:05:07<17:22,  7.04s/it]
+ 92%|█████████▏| 1666/1813 [3:05:12<15:19,  6.25s/it]
+ 92%|█████████▏| 1667/1813 [3:05:18<15:27,  6.35s/it]
+ 92%|█████████▏| 1668/1813 [3:05:25<15:16,  6.32s/it]
+ 92%|█████████▏| 1669/1813 [3:05:35<17:46,  7.41s/it]
+ 92%|█████████▏| 1670/1813 [3:05:40<16:17,  6.84s/it]
+                                                     
+{'loss': '0.9028', 'grad_norm': '1.006', 'learning_rate': '1.839e-06', 'epoch': '0.9214', 'num_input_tokens_seen': 11244288, 'train_runtime': '1.114e+04', 'train_tokens_per_second': '1009'}
+
+ 92%|█████████▏| 1670/1813 [3:05:40<16:17,  6.84s/it]
+ 92%|█████████▏| 1671/1813 [3:05:45<15:09,  6.40s/it]
+ 92%|█████████▏| 1672/1813 [3:05:51<14:34,  6.20s/it]
+ 92%|█████████▏| 1673/1813 [3:05:57<14:11,  6.08s/it]
+ 92%|█████████▏| 1674/1813 [3:06:04<14:39,  6.32s/it]
+ 92%|█████████▏| 1675/1813 [3:06:11<15:04,  6.56s/it]
+ 92%|█████████▏| 1676/1813 [3:06:18<15:29,  6.78s/it]
+ 92%|█████████▏| 1677/1813 [3:06:25<15:04,  6.65s/it]
+ 93%|█████████▎| 1678/1813 [3:06:31<14:44,  6.56s/it]
+ 93%|█████████▎| 1679/1813 [3:06:37<14:09,  6.34s/it]
+ 93%|█████████▎| 1680/1813 [3:06:46<15:42,  7.09s/it]
+                                                     
+{'loss': '0.9648', 'grad_norm': '0.8387', 'learning_rate': '1.593e-06', 'epoch': '0.9269', 'num_input_tokens_seen': 11317056, 'train_runtime': '1.121e+04', 'train_tokens_per_second': '1010'}
+
+ 93%|█████████▎| 1680/1813 [3:06:46<15:42,  7.09s/it]
+ 93%|█████████▎| 1681/1813 [3:06:51<14:17,  6.50s/it]
+ 93%|█████████▎| 1682/1813 [3:06:56<13:20,  6.11s/it]
+ 93%|█████████▎| 1683/1813 [3:07:01<12:16,  5.66s/it]
+ 93%|█████████▎| 1684/1813 [3:07:07<12:57,  6.03s/it]
+ 93%|█████████▎| 1685/1813 [3:07:14<13:13,  6.20s/it]
+ 93%|█████████▎| 1686/1813 [3:07:21<13:32,  6.40s/it]
+ 93%|█████████▎| 1687/1813 [3:07:27<13:24,  6.39s/it]
+ 93%|█████████▎| 1688/1813 [3:07:34<13:33,  6.51s/it]
+ 93%|█████████▎| 1689/1813 [3:07:39<12:19,  5.96s/it]
+ 93%|█████████▎| 1690/1813 [3:07:45<12:32,  6.12s/it]
+                                                     
+{'loss': '0.8905', 'grad_norm': '0.7818', 'learning_rate': '1.366e-06', 'epoch': '0.9324', 'num_input_tokens_seen': 11380560, 'train_runtime': '1.127e+04', 'train_tokens_per_second': '1010'}
+
+ 93%|█████████▎| 1690/1813 [3:07:45<12:32,  6.12s/it]
+ 93%|█████████▎| 1691/1813 [3:07:51<12:27,  6.12s/it]
+ 93%|█████████▎| 1692/1813 [3:07:59<13:12,  6.55s/it]
+ 93%|█████████▎| 1693/1813 [3:08:05<12:48,  6.40s/it]
+ 93%|█████████▎| 1694/1813 [3:08:13<13:47,  6.96s/it]
+ 93%|█████████▎| 1695/1813 [3:08:19<13:05,  6.66s/it]
+ 94%|█████████▎| 1696/1813 [3:08:25<12:23,  6.36s/it]
+ 94%|█████████▎| 1697/1813 [3:08:31<12:16,  6.35s/it]
+ 94%|█████████▎| 1698/1813 [3:08:36<11:28,  5.99s/it]
+ 94%|█████████▎| 1699/1813 [3:08:43<11:32,  6.07s/it]
+ 94%|█████████▍| 1700/1813 [3:08:49<11:32,  6.13s/it]
+                                                     
+{'loss': '0.8874', 'grad_norm': '1.025', 'learning_rate': '1.155e-06', 'epoch': '0.9379', 'num_input_tokens_seen': 11450712, 'train_runtime': '1.133e+04', 'train_tokens_per_second': '1011'}
+
+ 94%|█████████▍| 1700/1813 [3:08:49<11:32,  6.13s/it]
+ 94%|█████████▍| 1701/1813 [3:08:54<10:45,  5.77s/it]
+ 94%|█████████▍| 1702/1813 [3:08:59<10:29,  5.67s/it]
+ 94%|█████████▍| 1703/1813 [3:09:05<10:29,  5.72s/it]
+ 94%|█████████▍| 1704/1813 [3:09:13<11:28,  6.32s/it]
+ 94%|█████████▍| 1705/1813 [3:09:19<11:33,  6.42s/it]
+ 94%|█████████▍| 1706/1813 [3:09:26<11:25,  6.41s/it]
+ 94%|█████████▍| 1707/1813 [3:09:31<10:50,  6.14s/it]
+ 94%|█████████▍| 1708/1813 [3:09:36<10:01,  5.73s/it]
+ 94%|█████████▍| 1709/1813 [3:09:42<10:07,  5.84s/it]
+ 94%|█████████▍| 1710/1813 [3:09:48<09:54,  5.77s/it]
+                                                     
+{'loss': '0.9414', 'grad_norm': '1.104', 'learning_rate': '9.619e-07', 'epoch': '0.9434', 'num_input_tokens_seen': 11511984, 'train_runtime': '1.139e+04', 'train_tokens_per_second': '1011'}
+
+ 94%|█████████▍| 1710/1813 [3:09:48<09:54,  5.77s/it]
+ 94%|█████████▍| 1711/1813 [3:09:53<09:31,  5.61s/it]
+ 94%|█████████▍| 1712/1813 [3:09:59<09:27,  5.62s/it]
+ 94%|█████████▍| 1713/1813 [3:10:05<09:30,  5.70s/it]
+ 95%|█████████▍| 1714/1813 [3:10:10<09:28,  5.74s/it]
+ 95%|█████████▍| 1715/1813 [3:10:16<09:14,  5.66s/it]
+ 95%|█████████▍| 1716/1813 [3:10:21<08:50,  5.47s/it]
+ 95%|█████████▍| 1717/1813 [3:10:26<08:37,  5.39s/it]
+ 95%|█████████▍| 1718/1813 [3:10:32<08:53,  5.62s/it]
+ 95%|█████████▍| 1719/1813 [3:10:38<09:06,  5.81s/it]
+ 95%|█████████▍| 1720/1813 [3:10:44<09:04,  5.85s/it]
+                                                     
+{'loss': '0.8947', 'grad_norm': '0.8582', 'learning_rate': '7.863e-07', 'epoch': '0.949', 'num_input_tokens_seen': 11571368, 'train_runtime': '1.144e+04', 'train_tokens_per_second': '1011'}
+
+ 95%|█████████▍| 1720/1813 [3:10:44<09:04,  5.85s/it]
+ 95%|█████████▍| 1721/1813 [3:10:52<09:37,  6.28s/it]
+ 95%|█████████▍| 1722/1813 [3:10:58<09:24,  6.20s/it]
+ 95%|█████████▌| 1723/1813 [3:11:04<09:13,  6.15s/it]
+ 95%|█████████▌| 1724/1813 [3:11:10<09:00,  6.07s/it]
+ 95%|█████████▌| 1725/1813 [3:11:17<09:24,  6.41s/it]
+ 95%|█████████▌| 1726/1813 [3:11:24<09:40,  6.68s/it]
+ 95%|█████████▌| 1727/1813 [3:11:30<09:13,  6.44s/it]
+ 95%|█████████▌| 1728/1813 [3:11:35<08:28,  5.98s/it]
+ 95%|█████████▌| 1729/1813 [3:11:40<08:02,  5.74s/it]
+ 95%|█████████▌| 1730/1813 [3:11:47<08:16,  5.99s/it]
+                                                     
+{'loss': '0.932', 'grad_norm': '0.734', 'learning_rate': '6.282e-07', 'epoch': '0.9545', 'num_input_tokens_seen': 11640392, 'train_runtime': '1.151e+04', 'train_tokens_per_second': '1012'}
+
+ 95%|█████████▌| 1730/1813 [3:11:47<08:16,  5.99s/it]
+ 95%|█████████▌| 1731/1813 [3:11:53<08:14,  6.02s/it]
+ 96%|█████████▌| 1732/1813 [3:12:00<08:29,  6.29s/it]
+ 96%|█████████▌| 1733/1813 [3:12:05<07:54,  5.93s/it]
+ 96%|█████████▌| 1734/1813 [3:12:11<07:45,  5.90s/it]
+ 96%|█████████▌| 1735/1813 [3:12:17<07:58,  6.13s/it]
+ 96%|█████████▌| 1736/1813 [3:12:24<08:09,  6.36s/it]
+ 96%|█████████▌| 1737/1813 [3:12:30<07:59,  6.31s/it]
+ 96%|█████████▌| 1738/1813 [3:12:36<07:28,  5.98s/it]
+ 96%|█████████▌| 1739/1813 [3:12:44<08:07,  6.59s/it]
+ 96%|█████████▌| 1740/1813 [3:12:50<08:00,  6.58s/it]
+                                                     
+{'loss': '0.9223', 'grad_norm': '0.9856', 'learning_rate': '4.878e-07', 'epoch': '0.96', 'num_input_tokens_seen': 11709120, 'train_runtime': '1.157e+04', 'train_tokens_per_second': '1012'}
+
+ 96%|█████████▌| 1740/1813 [3:12:50<08:00,  6.58s/it]
+ 96%|█████████▌| 1741/1813 [3:12:56<07:35,  6.32s/it]
+ 96%|█████████▌| 1742/1813 [3:13:03<07:37,  6.45s/it]
+ 96%|█████████▌| 1743/1813 [3:13:10<07:42,  6.61s/it]
+ 96%|█████████▌| 1744/1813 [3:13:17<07:45,  6.74s/it]
+ 96%|█████████▌| 1745/1813 [3:13:23<07:21,  6.49s/it]
+ 96%|█████████▋| 1746/1813 [3:13:30<07:31,  6.74s/it]
+ 96%|█████████▋| 1747/1813 [3:13:35<06:49,  6.20s/it]
+ 96%|█████████▋| 1748/1813 [3:13:42<07:08,  6.60s/it]
+ 96%|█████████▋| 1749/1813 [3:13:48<06:43,  6.31s/it]
+ 97%|█████████▋| 1750/1813 [3:13:54<06:24,  6.11s/it]
+                                                     
+{'loss': '0.8874', 'grad_norm': '0.7923', 'learning_rate': '3.65e-07', 'epoch': '0.9655', 'num_input_tokens_seen': 11777768, 'train_runtime': '1.163e+04', 'train_tokens_per_second': '1012'}
+
+ 97%|█████████▋| 1750/1813 [3:13:54<06:24,  6.11s/it]
+ 97%|█████████▋| 1751/1813 [3:14:00<06:19,  6.13s/it]
+ 97%|█████████▋| 1752/1813 [3:14:06<06:14,  6.15s/it]
+ 97%|█████████▋| 1753/1813 [3:14:14<06:37,  6.63s/it]
+ 97%|█████████▋| 1754/1813 [3:14:21<06:45,  6.88s/it]
+ 97%|█████████▋| 1755/1813 [3:14:28<06:34,  6.80s/it]
+ 97%|█████████▋| 1756/1813 [3:14:33<06:02,  6.35s/it]
+ 97%|█████████▋| 1757/1813 [3:14:39<05:46,  6.19s/it]
+ 97%|█████████▋| 1758/1813 [3:14:45<05:33,  6.07s/it]
+ 97%|█████████▋| 1759/1813 [3:14:51<05:28,  6.08s/it]
+ 97%|█████████▋| 1760/1813 [3:14:56<05:13,  5.92s/it]
+                                                     
+{'loss': '0.9006', 'grad_norm': '0.9911', 'learning_rate': '2.599e-07', 'epoch': '0.971', 'num_input_tokens_seen': 11843192, 'train_runtime': '1.17e+04', 'train_tokens_per_second': '1013'}
+
+ 97%|█████████▋| 1760/1813 [3:14:56<05:13,  5.92s/it]
+ 97%|█████████▋| 1761/1813 [3:15:02<05:03,  5.83s/it]
+ 97%|█████████▋| 1762/1813 [3:15:07<04:43,  5.57s/it]
+ 97%|█████████▋| 1763/1813 [3:15:15<05:11,  6.23s/it]
+ 97%|█████████▋| 1764/1813 [3:15:20<04:51,  5.94s/it]
+ 97%|█████████▋| 1765/1813 [3:15:25<04:37,  5.78s/it]
+ 97%|█████████▋| 1766/1813 [3:15:31<04:30,  5.75s/it]
+ 97%|█████████▋| 1767/1813 [3:15:37<04:25,  5.76s/it]
+ 98%|█████████▊| 1768/1813 [3:15:43<04:19,  5.77s/it]
+ 98%|█████████▊| 1769/1813 [3:15:49<04:15,  5.81s/it]
+ 98%|█████████▊| 1770/1813 [3:15:55<04:23,  6.13s/it]
+                                                     
+{'loss': '0.9144', 'grad_norm': '0.8835', 'learning_rate': '1.726e-07', 'epoch': '0.9766', 'num_input_tokens_seen': 11903728, 'train_runtime': '1.176e+04', 'train_tokens_per_second': '1013'}
+
+ 98%|█████████▊| 1770/1813 [3:15:55<04:23,  6.13s/it]
+ 98%|█████████▊| 1771/1813 [3:16:02<04:19,  6.18s/it]
+ 98%|█████████▊| 1772/1813 [3:16:08<04:15,  6.22s/it]
+ 98%|█████████▊| 1773/1813 [3:16:15<04:18,  6.45s/it]
+ 98%|█████████▊| 1774/1813 [3:16:20<03:51,  5.94s/it]
+ 98%|█████████▊| 1775/1813 [3:16:27<04:04,  6.42s/it]
+ 98%|█████████▊| 1776/1813 [3:16:35<04:09,  6.74s/it]
+ 98%|█████████▊| 1777/1813 [3:16:42<04:11,  6.99s/it]
+ 98%|█████████▊| 1778/1813 [3:16:48<03:52,  6.63s/it]
+ 98%|█████████▊| 1779/1813 [3:16:54<03:34,  6.31s/it]
+ 98%|█████████▊| 1780/1813 [3:17:00<03:23,  6.16s/it]
+                                                     
+{'loss': '0.9059', 'grad_norm': '0.7636', 'learning_rate': '1.031e-07', 'epoch': '0.9821', 'num_input_tokens_seen': 11972864, 'train_runtime': '1.182e+04', 'train_tokens_per_second': '1013'}
+
+ 98%|█████████▊| 1780/1813 [3:17:00<03:23,  6.16s/it]
+ 98%|█████████▊| 1781/1813 [3:17:06<03:16,  6.15s/it]
+ 98%|█████████▊| 1782/1813 [3:17:12<03:07,  6.06s/it]
+ 98%|█████████▊| 1783/1813 [3:17:17<02:59,  5.99s/it]
+ 98%|█████████▊| 1784/1813 [3:17:24<02:59,  6.18s/it]
+ 98%|█████████▊| 1785/1813 [3:17:30<02:50,  6.11s/it]
+ 99%|█████████▊| 1786/1813 [3:17:36<02:45,  6.14s/it]
+ 99%|█████████▊| 1787/1813 [3:17:41<02:32,  5.88s/it]
+ 99%|█████████▊| 1788/1813 [3:17:48<02:32,  6.09s/it]
+ 99%|█████████▊| 1789/1813 [3:17:56<02:43,  6.81s/it]
+ 99%|█████████▊| 1790/1813 [3:18:04<02:38,  6.89s/it]
+                                                     
+{'loss': '0.9073', 'grad_norm': '0.7609', 'learning_rate': '5.138e-08', 'epoch': '0.9876', 'num_input_tokens_seen': 12043496, 'train_runtime': '1.188e+04', 'train_tokens_per_second': '1013'}
+
+ 99%|█████████▊| 1790/1813 [3:18:04<02:38,  6.89s/it]
+ 99%|█████████▉| 1791/1813 [3:18:09<02:20,  6.40s/it]
+ 99%|█████████▉| 1792/1813 [3:18:15<02:12,  6.31s/it]
+ 99%|█████████▉| 1793/1813 [3:18:22<02:09,  6.46s/it]
+ 99%|█████████▉| 1794/1813 [3:18:29<02:05,  6.62s/it]
+ 99%|█████████▉| 1795/1813 [3:18:35<01:59,  6.65s/it]
+ 99%|█████████▉| 1796/1813 [3:18:42<01:53,  6.67s/it]
+ 99%|█████████▉| 1797/1813 [3:18:48<01:44,  6.55s/it]
+ 99%|█████████▉| 1798/1813 [3:18:58<01:50,  7.36s/it]
+ 99%|█████████▉| 1799/1813 [3:19:04<01:38,  7.07s/it]
+ 99%|█████████▉| 1800/1813 [3:19:09<01:21,  6.28s/it]
+                                                     
+{'loss': '0.8827', 'grad_norm': '0.8935', 'learning_rate': '1.749e-08', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448, 'train_runtime': '1.195e+04', 'train_tokens_per_second': '1014'}
+
+ 99%|█████████▉| 1800/1813 [3:19:09<01:21,  6.28s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.72it/s]
+
+  0%|          | 5/1000 [00:00<02:18,  7.21it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.31it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.46it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.04it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.98it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.81it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.95it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.71it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.03it/s]
+
+  2%|▏         | 19/1000 [00:02<03:11,  5.13it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.59it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.23it/s]
+
+  2%|▏         | 23/1000 [00:03<02:55,  5.58it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.35it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.35it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.98it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:06<02:04,  7.73it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.09it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.30it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.04it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.89it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.91it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.72it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.29it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.20it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.18it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.62it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.29it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.41it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.19it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.01it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.48it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.78it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.21it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.90it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.13it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.27it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.69it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.13it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.08it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.39it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.45it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:26,  9.88it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:17, 11.00it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.39it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.19it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.83it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:57,  7.05it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:16,  6.09it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.18it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:03,  6.70it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:38,  8.40it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.78it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.56it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.48it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.74it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.70it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.78it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.88it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.37it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.09it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.61it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.47it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:18, 10.14it/s]
+
+ 20%|██        | 202/1000 [00:26<01:33,  8.54it/s]
+
+ 20%|██        | 204/1000 [00:26<01:33,  8.50it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.81it/s]
+
+ 21%|██        | 208/1000 [00:27<01:34,  8.43it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.41it/s]
+
+ 21%|██        | 212/1000 [00:27<01:22,  9.60it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.72it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.12it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.49it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.97it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.47it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.36it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.35it/s]
+
+ 26%|██▌       | 256/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.74it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.72it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.74it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.98it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.06it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.15it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.79it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.27it/s]
+
+ 32%|███▏      | 315/1000 [00:42<01:06, 10.26it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.58it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.40it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.16it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.57it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.81it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.02it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.87it/s]
+
+ 34%|███▍      | 344/1000 [00:46<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.90it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.63it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.57it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.43it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:58,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.54it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.78it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.21it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.17it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.66it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.38it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.80it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.76it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.50it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.26it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.49it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:12,  7.83it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.53it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.02it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.79it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.52it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.29it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.24it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.80it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.82it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.20it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.37it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:25,  6.17it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.19it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.29it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.08it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:20,  6.50it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.03it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.29it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:22,  6.26it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.98it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50, 10.00it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.06it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:58,  7.97it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.56it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.65it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.32it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:22,  5.63it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:20,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:06,  6.92it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.49it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.91it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.50it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.25it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.17it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.94it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.75it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.85it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.43it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.72it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.59it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.79it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.38it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.68it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:50,  7.77it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.41it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:40,  9.68it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.75it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.60it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.73it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.08it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.02it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.20it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.44it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.62it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.18it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:39,  9.40it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.95it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.58it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:45,  7.57it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:39,  8.79it/s]
+
+ 66%|██████▌   | 657/1000 [01:29<00:46,  7.35it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:55,  6.17it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.66it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:51,  6.57it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:57,  5.83it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.37it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.64it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.22it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.65it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.57it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.63it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.91it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.90it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.88it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.15it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:47,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.02it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.57it/s]
+
+ 70%|███████   | 702/1000 [01:36<00:53,  5.60it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.20it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.01it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.49it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.67it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.83it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.35it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.05it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.79it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.92it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.15it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.60it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.31it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.51it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.87it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.62it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.63it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.67it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.71it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.13it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.60it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.94it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.35it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.96it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.21it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.18it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.85it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.63it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.82it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.15it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.96it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.21it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.24it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.30it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.13it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:53<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.52it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.10it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.15it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.24it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.54it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.04it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.50it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.64it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.70it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.30it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.25it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.24it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.62it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.53it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.95it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.16it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.84it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.59it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.59it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.16it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.63it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.08it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.64it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.03it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.13it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.40it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.39it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.44it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.78it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.80it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.41it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.04it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.43it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.91it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.34it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.72it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.98it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.13it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.96it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.52it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.51it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.01it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.72it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.63it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7529', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.381', 'eval_steps_per_second': '7.381', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448}
+
+ 99%|█████████▉| 1800/1813 [3:21:24<01:21,  6.28s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.63it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_002_lr_1.000e-04/checkpoint-1800
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+ 99%|█████████▉| 1801/1813 [3:21:33<09:32, 47.67s/it]
+ 99%|█████████▉| 1802/1813 [3:21:38<06:24, 34.94s/it]
+ 99%|█████████▉| 1803/1813 [3:21:44<04:21, 26.13s/it]
+100%|█████████▉| 1804/1813 [3:21:51<03:05, 20.56s/it]
+100%|█████████▉| 1805/1813 [3:21:57<02:09, 16.18s/it]
+100%|█████████▉| 1806/1813 [3:22:04<01:32, 13.26s/it]
+100%|█████████▉| 1807/1813 [3:22:11<01:08, 11.39s/it]
+100%|█████████▉| 1808/1813 [3:22:18<00:51, 10.20s/it]
+100%|█████████▉| 1809/1813 [3:22:24<00:36,  9.05s/it]
+100%|█████████▉| 1810/1813 [3:22:30<00:24,  8.06s/it]
+                                                     
+{'loss': '0.8685', 'grad_norm': '0.8519', 'learning_rate': '1.427e-09', 'epoch': '0.9986', 'num_input_tokens_seen': 12187432, 'train_runtime': '1.215e+04', 'train_tokens_per_second': '1003'}
+
+100%|█████████▉| 1810/1813 [3:22:30<00:24,  8.06s/it]
+100%|█████████▉| 1811/1813 [3:22:36<00:14,  7.33s/it]
+100%|█████████▉| 1812/1813 [3:22:41<00:06,  6.72s/it]
+100%|██████████| 1813/1813 [3:22:44<00:00,  5.64s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.73it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.23it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.08it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  7.00it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.82it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:46,  5.91it/s]
+
+  2%|▏         | 19/1000 [00:02<03:13,  5.07it/s]
+
+  2%|▏         | 20/1000 [00:03<03:35,  4.55it/s]
+
+  2%|▏         | 22/1000 [00:03<02:38,  6.18it/s]
+
+  2%|▏         | 23/1000 [00:03<02:56,  5.55it/s]
+
+  2%|▎         | 25/1000 [00:03<02:13,  7.31it/s]
+
+  3%|▎         | 26/1000 [00:04<02:36,  6.24it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 29/1000 [00:04<02:33,  6.34it/s]
+
+  3%|▎         | 30/1000 [00:04<02:43,  5.95it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:46,  9.08it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.10it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.03it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.90it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.92it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.28it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.32it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.74it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.35it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.98it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.11it/s]
+
+  8%|▊         | 75/1000 [00:10<02:15,  6.85it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.66it/s]
+
+  8%|▊         | 78/1000 [00:10<02:26,  6.30it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.83it/s]
+
+  8%|▊         | 82/1000 [00:10<01:40,  9.18it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.03it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.94it/s]
+
+  9%|▊         | 87/1000 [00:12<02:50,  5.37it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  6.99it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:37,  5.77it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.35it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.61it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.30it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.42it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.18it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.03it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.52it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.80it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.22it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:48,  8.07it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.14it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.93it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.00it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.87it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25, 10.05it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.41it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.46it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:26,  9.88it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:17, 11.00it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.39it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.96it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.39it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.66it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.77it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.83it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.14it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.44it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.49it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.74it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.71it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.78it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.73it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.89it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.67it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.77it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.63it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.92it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.51it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.68it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.73it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.14it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.49it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.54it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.81it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.89it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.36it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.01it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.20it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.91it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.43it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.96it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:31,  8.28it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  5.98it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:19,  5.44it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.63it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.23it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.84it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.73it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.35it/s]
+
+ 26%|██▌       | 256/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:25,  8.73it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:37,  7.63it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.49it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:56,  6.35it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:57,  6.28it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:52,  6.55it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.87it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.50it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:49,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.01it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.42it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.73it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.98it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:57,  6.03it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.15it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.86it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:26,  8.16it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.45it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.83it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.10it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.74it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.06it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.77it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.17it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.24it/s]
+
+ 32%|███▏      | 315/1000 [00:42<01:06, 10.24it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.58it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.22it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.88it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.40it/s]
+
+ 33%|███▎      | 328/1000 [00:44<01:33,  7.16it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.57it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.37it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.02it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.81it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.02it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.87it/s]
+
+ 34%|███▍      | 344/1000 [00:46<01:26,  7.59it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:47<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.62it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.94it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.52it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:08,  9.38it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.17it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.26it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.91it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:58,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.59it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.54it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.71it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.07it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.20it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:54,  5.47it/s]
+
+ 38%|███▊      | 378/1000 [00:51<01:30,  6.91it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.75it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.89it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.17it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.16it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.63it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.36it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.78it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.46it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.74it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:54<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.91it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.23it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+
+ 41%|████▏     | 414/1000 [00:56<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.44it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.53it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.01it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.78it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.50it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.17it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:11,  7.78it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:18,  7.05it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:04,  8.53it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:14,  7.40it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:19,  6.95it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:29,  6.14it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.55it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.15it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.47it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.78it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.23it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.79it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.61it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.73it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.00it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.25it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.80it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.53it/s]
+
+ 47%|████▋     | 474/1000 [01:04<01:21,  6.49it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.38it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.48it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.14it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.57it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.05it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.20it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.25it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.96it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:08<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.55it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.42it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.95it/s]
+
+ 51%|█████     | 512/1000 [01:09<00:48,  9.97it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.10it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.07it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.58it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.52it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.11it/s]
+
+ 53%|█████▎    | 533/1000 [01:12<00:48,  9.68it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.67it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.34it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.91it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:20,  5.74it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:06,  6.92it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.05it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:10,  6.48it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.67it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.90it/s]
+
+ 55%|█████▌    | 551/1000 [01:15<01:00,  7.48it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.23it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.13it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.92it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.73it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  6.99it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:50,  8.81it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.01it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.34it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.95it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.72it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:06,  6.53it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:07,  6.42it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.42it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:08,  6.28it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.32it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:55,  7.59it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:15,  5.62it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:21,  5.19it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.90it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:49,  8.46it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:55,  7.53it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.87it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.14it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.52it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.74it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:50,  8.15it/s]
+
+ 59%|█████▉    | 593/1000 [01:21<00:59,  6.79it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.65it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:02,  6.46it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.38it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.69it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.68it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.08it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:40,  9.54it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:36, 10.64it/s]
+
+ 62%|██████▏   | 616/1000 [01:24<00:50,  7.56it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.71it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:42,  9.05it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.01it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.19it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.45it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.62it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.18it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:26<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.41it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:27<00:39,  9.16it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.94it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.57it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.09it/s]
+
+ 65%|██████▍   | 647/1000 [01:28<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.51it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.31it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:45,  7.69it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.89it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.08it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  5.98it/s]
+
+ 66%|██████▌   | 661/1000 [01:30<00:44,  7.55it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.65it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.91it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.89it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.86it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.85it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.15it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.82it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.25it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.41it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:48,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.59it/s]
+
+ 70%|███████   | 702/1000 [01:36<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.70it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.35it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.05it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.79it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.45it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.11it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.17it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.20it/s]
+
+ 72%|███████▎  | 725/1000 [01:39<00:33,  8.28it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:27, 10.07it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.54it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.30it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.50it/s]
+
+ 74%|███████▎  | 735/1000 [01:40<00:26,  9.87it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:28,  9.22it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:27,  9.57it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.21it/s]
+
+ 74%|███████▍  | 743/1000 [01:41<00:30,  8.29it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.52it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.36it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.60it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.66it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.15it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.21it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.39it/s]
+
+ 76%|███████▌  | 758/1000 [01:43<00:31,  7.59it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:33,  7.26it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.60it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.60it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.57it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.34it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:30,  7.66it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.50it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.04it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.35it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.76it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.88it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.56it/s]
+
+ 78%|███████▊  | 780/1000 [01:46<00:31,  6.92it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.34it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.52it/s]
+
+ 79%|███████▉  | 788/1000 [01:47<00:25,  8.35it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.97it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.20it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.96it/s]
+
+ 80%|███████▉  | 795/1000 [01:48<00:28,  7.18it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.85it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.00it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.70it/s]
+
+ 80%|████████  | 802/1000 [01:49<00:25,  7.63it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.81it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.14it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.94it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.18it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.20it/s]
+
+ 81%|████████▏ | 813/1000 [01:50<00:20,  9.04it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.28it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.11it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.17it/s]
+
+ 82%|████████▏ | 821/1000 [01:51<00:22,  8.02it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.80it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.73it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.98it/s]
+
+ 83%|████████▎ | 826/1000 [01:52<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.72it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.00it/s]
+
+ 83%|████████▎ | 832/1000 [01:53<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:53<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.52it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.09it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.43it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.60it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.13it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.40it/s]
+
+ 84%|████████▍ | 845/1000 [01:55<00:26,  5.76it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.24it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.53it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.86it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.45it/s]
+
+ 86%|████████▌ | 855/1000 [01:56<00:14, 10.03it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.19it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.50it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.70it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:13, 10.52it/s]
+
+ 86%|████████▋ | 865/1000 [01:57<00:11, 11.58it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.27it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.23it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.23it/s]
+
+ 87%|████████▋ | 873/1000 [01:58<00:16,  7.74it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.60it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.85it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.85it/s]
+
+ 88%|████████▊ | 881/1000 [01:59<00:15,  7.58it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.53it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.94it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.15it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.32it/s]
+
+ 89%|████████▉ | 890/1000 [02:00<00:11,  9.75it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.73it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.83it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.69it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.85it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.05it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.30it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.38it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.59it/s]
+
+ 92%|█████████▏| 919/1000 [02:04<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.93it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.16it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.64it/s]
+
+ 93%|█████████▎| 927/1000 [02:05<00:09,  8.08it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.64it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:06<00:10,  6.60it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.16it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.29it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.33it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.07it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:08<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.40it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:09<00:06,  7.28it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.33it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.74it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.77it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.01it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.45it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.42it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.33it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:11<00:03,  9.41it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.00it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.17it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.77it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.80it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.05it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:13<00:01,  8.37it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:14<00:01,  6.14it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:15<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.75it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7529', 'eval_runtime': '135.7', 'eval_samples_per_second': '7.371', 'eval_steps_per_second': '7.371', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:25:00<00:00,  5.64s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_002_lr_1.000e-04/checkpoint-1813
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+
+Training completed. Do not forget to share your model on huggingface.co/models =)
+
+
+
+                                                     
+{'train_runtime': '1.23e+04', 'train_samples_per_second': '2.357', 'train_steps_per_second': '0.147', 'train_loss': '0.9647', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:25:01<00:00,  5.64s/it]
+100%|██████████| 1813/1813 [3:25:01<00:00,  6.79s/it]
+Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_002_lr_1.000e-04
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+***** train metrics *****
+  epoch                    =         1.0
+  num_input_tokens_seen    =    12201816
+  total_flos               = 487685654GF
+  train_loss               =      0.9647
+  train_runtime            =  3:25:01.28
+  train_samples_per_second =       2.357
+  train_steps_per_second   =       0.147
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_002_lr_1.000e-04/training_loss.png
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_002_lr_1.000e-04/training_eval_loss.png
+[WARNING|2026-10-13 06:21:48] llamafactory.extras.ploting:149 >> No metric eval_accuracy to plot.
+
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+  0%|          | 3/1000 [00:00<01:03, 15.82it/s]
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+  1%|          | 11/1000 [00:01<02:02,  8.07it/s]
+  1%|▏         | 13/1000 [00:01<02:20,  7.01it/s]
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+  2%|▏         | 16/1000 [00:02<02:51,  5.72it/s]
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+  2%|▏         | 23/1000 [00:03<02:54,  5.58it/s]
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+  4%|▎         | 37/1000 [00:05<01:45,  9.11it/s]
+  4%|▍         | 39/1000 [00:05<02:06,  7.59it/s]
+  4%|▍         | 40/1000 [00:05<02:03,  7.76it/s]
+  4%|▍         | 42/1000 [00:06<01:45,  9.12it/s]
+  4%|▍         | 44/1000 [00:06<01:32, 10.32it/s]
+  5%|▍         | 46/1000 [00:06<01:45,  9.04it/s]
+  5%|▍         | 47/1000 [00:06<02:09,  7.37it/s]
+  5%|▍         | 49/1000 [00:06<01:48,  8.75it/s]
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+  6%|▌         | 57/1000 [00:07<01:41,  9.29it/s]
+  6%|▌         | 59/1000 [00:08<01:54,  8.21it/s]
+  6%|▌         | 61/1000 [00:08<01:40,  9.35it/s]
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+  6%|▋         | 65/1000 [00:08<01:51,  8.35it/s]
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+  8%|▊         | 80/1000 [00:10<01:57,  7.86it/s]
+  8%|▊         | 82/1000 [00:10<01:39,  9.20it/s]
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+  8%|▊         | 85/1000 [00:11<02:14,  6.78it/s]
+  9%|▊         | 86/1000 [00:11<02:33,  5.94it/s]
+  9%|▊         | 87/1000 [00:11<02:49,  5.38it/s]
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+  9%|▉         | 92/1000 [00:12<02:27,  6.16it/s]
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+ 10%|█         | 101/1000 [00:13<01:33,  9.61it/s]
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+ 11%|█         | 106/1000 [00:14<01:36,  9.31it/s]
+ 11%|█         | 108/1000 [00:14<01:25, 10.43it/s]
+ 11%|█         | 110/1000 [00:14<01:19, 11.20it/s]
+ 11%|█         | 112/1000 [00:14<01:13, 12.03it/s]
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.51it/s]
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.81it/s]
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.23it/s]
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.65it/s]
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.03it/s]
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.06it/s]
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.42it/s]
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.48it/s]
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.90it/s]
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.03it/s]
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.38it/s]
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.96it/s]
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.77it/s]
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.82it/s]
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.14it/s]
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.43it/s]
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.79it/s]
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.78it/s]
+ 18%|█▊        | 185/1000 [00:24<01:47,  7.60it/s]
+ 19%|█▊        | 186/1000 [00:24<01:44,  7.76it/s]
+ 19%|█▊        | 187/1000 [00:24<02:08,  6.31it/s]
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.06it/s]
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.59it/s]
+ 19%|█▉        | 192/1000 [00:25<02:05,  6.45it/s]
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.74it/s]
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.18it/s]
+ 20%|██        | 200/1000 [00:26<01:15, 10.61it/s]
+ 20%|██        | 202/1000 [00:26<01:31,  8.74it/s]
+ 20%|██        | 204/1000 [00:26<01:32,  8.61it/s]
+ 21%|██        | 206/1000 [00:26<01:20,  9.88it/s]
+ 21%|██        | 208/1000 [00:27<01:33,  8.49it/s]
+ 21%|██        | 210/1000 [00:27<01:33,  8.47it/s]
+ 21%|██        | 212/1000 [00:27<01:21,  9.67it/s]
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.72it/s]
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.14it/s]
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.54it/s]
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.81it/s]
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.36it/s]
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.20it/s]
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.91it/s]
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.42it/s]
+ 24%|██▍       | 239/1000 [00:31<01:25,  8.94it/s]
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.34it/s]
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.01it/s]
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.46it/s]
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+ 25%|██▌       | 250/1000 [00:33<02:08,  5.86it/s]
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.35it/s]
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.68it/s]
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.73it/s]
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+ 26%|██▌       | 261/1000 [00:34<01:54,  6.48it/s]
+ 26%|██▌       | 262/1000 [00:34<01:55,  6.39it/s]
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.30it/s]
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.57it/s]
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.88it/s]
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.50it/s]
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.02it/s]
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.68it/s]
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.05it/s]
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.72it/s]
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.99it/s]
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.06it/s]
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.18it/s]
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+ 30%|███       | 300/1000 [00:40<01:42,  6.84it/s]
+ 30%|███       | 301/1000 [00:40<01:38,  7.10it/s]
+ 30%|███       | 303/1000 [00:40<01:19,  8.74it/s]
+ 30%|███       | 304/1000 [00:40<01:26,  8.06it/s]
+ 30%|███       | 305/1000 [00:40<01:25,  8.13it/s]
+ 31%|███       | 306/1000 [00:41<01:39,  6.94it/s]
+ 31%|███       | 307/1000 [00:41<01:42,  6.77it/s]
+ 31%|███       | 309/1000 [00:41<01:18,  8.78it/s]
+ 31%|███       | 311/1000 [00:41<01:07, 10.18it/s]
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.24it/s]
+ 32%|███▏      | 315/1000 [00:41<01:07, 10.22it/s]
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.56it/s]
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.88it/s]
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.73it/s]
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.40it/s]
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.16it/s]
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.57it/s]
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.37it/s]
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.03it/s]
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.81it/s]
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.02it/s]
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.84it/s]
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.59it/s]
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.20it/s]
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.40it/s]
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.90it/s]
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.63it/s]
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.54it/s]
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.34it/s]
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.93it/s]
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.55it/s]
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.40it/s]
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.53it/s]
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.70it/s]
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.06it/s]
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.21it/s]
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.18it/s]
+ 38%|███▊      | 379/1000 [00:51<01:16,  8.14it/s]
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.16it/s]
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.54it/s]
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.06it/s]
+ 39%|███▊      | 387/1000 [00:52<01:21,  7.55it/s]
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.29it/s]
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.82it/s]
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.48it/s]
+ 39%|███▉      | 393/1000 [00:52<01:07,  8.99it/s]
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.51it/s]
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.83it/s]
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+ 40%|████      | 400/1000 [00:53<01:45,  5.67it/s]
+ 40%|████      | 402/1000 [00:54<01:41,  5.91it/s]
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+ 41%|████      | 408/1000 [00:55<01:32,  6.38it/s]
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+ 41%|████      | 411/1000 [00:55<01:41,  5.82it/s]
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.05it/s]
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.24it/s]
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.43it/s]
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.49it/s]
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.68it/s]
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.92it/s]
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.35it/s]
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.94it/s]
+ 43%|████▎     | 428/1000 [00:57<01:16,  7.52it/s]
+ 43%|████▎     | 429/1000 [00:58<01:12,  7.82it/s]
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.53it/s]
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.01it/s]
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.77it/s]
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.49it/s]
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.08it/s]
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.24it/s]
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.71it/s]
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.17it/s]
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.58it/s]
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.50it/s]
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.79it/s]
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.23it/s]
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.79it/s]
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.00it/s]
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.84it/s]
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.56it/s]
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.38it/s]
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.58it/s]
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.04it/s]
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.31it/s]
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.98it/s]
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.21it/s]
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.26it/s]
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.96it/s]
+ 50%|████▉     | 496/1000 [01:06<01:00,  8.39it/s]
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.41it/s]
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+ 50%|█████     | 502/1000 [01:07<01:06,  7.53it/s]
+ 50%|█████     | 503/1000 [01:07<01:13,  6.78it/s]
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+ 51%|█████     | 508/1000 [01:08<01:06,  7.42it/s]
+ 51%|█████     | 510/1000 [01:08<00:54,  8.94it/s]
+ 51%|█████     | 512/1000 [01:08<00:49,  9.95it/s]
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.52it/s]
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.09it/s]
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.06it/s]
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.01it/s]
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.55it/s]
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.57it/s]
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.39it/s]
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.10it/s]
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.66it/s]
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.67it/s]
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.35it/s]
+ 54%|█████▎    | 537/1000 [01:12<01:22,  5.64it/s]
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+ 54%|█████▍    | 540/1000 [01:13<01:20,  5.74it/s]
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.92it/s]
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.49it/s]
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.91it/s]
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.50it/s]
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.25it/s]
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.17it/s]
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.94it/s]
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.75it/s]
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.83it/s]
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.02it/s]
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.35it/s]
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.75it/s]
+ 57%|█████▋    | 568/1000 [01:17<01:06,  6.54it/s]
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.43it/s]
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.43it/s]
+ 57%|█████▋    | 572/1000 [01:18<01:07,  6.30it/s]
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.82it/s]
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.96it/s]
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.54it/s]
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.58it/s]
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.14it/s]
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.52it/s]
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.74it/s]
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.38it/s]
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.86it/s]
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.65it/s]
+ 60%|█████▉    | 599/1000 [01:21<01:02,  6.46it/s]
+ 60%|██████    | 601/1000 [01:22<01:02,  6.38it/s]
+ 60%|██████    | 602/1000 [01:22<00:59,  6.69it/s]
+ 60%|██████    | 604/1000 [01:22<01:09,  5.67it/s]
+ 61%|██████    | 606/1000 [01:22<00:56,  6.99it/s]
+ 61%|██████    | 608/1000 [01:23<00:50,  7.77it/s]
+ 61%|██████    | 610/1000 [01:23<00:46,  8.41it/s]
+ 61%|██████    | 612/1000 [01:23<00:40,  9.67it/s]
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.74it/s]
+ 62%|██████▏   | 616/1000 [01:24<00:50,  7.57it/s]
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.72it/s]
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.07it/s]
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.01it/s]
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.19it/s]
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.44it/s]
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.62it/s]
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.18it/s]
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.20it/s]
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.83it/s]
+ 63%|██████▎   | 633/1000 [01:26<00:39,  9.40it/s]
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.93it/s]
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.16it/s]
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.95it/s]
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.58it/s]
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.10it/s]
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.57it/s]
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.51it/s]
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.54it/s]
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.31it/s]
+ 65%|██████▌   | 654/1000 [01:29<00:44,  7.70it/s]
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.90it/s]
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.10it/s]
+ 66%|██████▌   | 659/1000 [01:29<00:56,  5.99it/s]
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.60it/s]
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.19it/s]
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.25it/s]
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.64it/s]
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.97it/s]
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.91it/s]
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.86it/s]
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.82it/s]
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+ 70%|██████▉   | 696/1000 [01:35<00:47,  6.34it/s]
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+ 70%|███████   | 702/1000 [01:36<00:53,  5.62it/s]
+ 70%|███████   | 704/1000 [01:36<00:41,  7.22it/s]
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+ 71%|███████   | 710/1000 [01:37<00:44,  6.49it/s]
+ 71%|███████   | 712/1000 [01:37<00:37,  7.71it/s]
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.35it/s]
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.79it/s]
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.14it/s]
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.30it/s]
+ 73%|███████▎  | 727/1000 [01:39<00:28,  9.71it/s]
+ 73%|███████▎  | 729/1000 [01:39<00:24, 11.27it/s]
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.22it/s]
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.42it/s]
+ 74%|███████▎  | 735/1000 [01:39<00:27,  9.79it/s]
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.46it/s]
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.73it/s]
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.29it/s]
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.34it/s]
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.59it/s]
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.42it/s]
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.63it/s]
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.63it/s]
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.14it/s]
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.21it/s]
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.20it/s]
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.31it/s]
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.72it/s]
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.46it/s]
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.77it/s]
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.59it/s]
+ 78%|███████▊  | 780/1000 [01:46<00:31,  6.93it/s]
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.36it/s]
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.99it/s]
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.18it/s]
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.85it/s]
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+ 81%|████████  | 806/1000 [01:49<00:21,  9.16it/s]
+ 81%|████████  | 807/1000 [01:49<00:21,  8.96it/s]
+ 81%|████████  | 809/1000 [01:49<00:18, 10.20it/s]
+ 81%|████████  | 811/1000 [01:49<00:16, 11.22it/s]
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.04it/s]
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.31it/s]
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.13it/s]
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.00it/s]
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.80it/s]
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+ 83%|████████▎ | 833/1000 [01:53<00:26,  6.22it/s]
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.61it/s]
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.14it/s]
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.40it/s]
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.76it/s]
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.24it/s]
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.54it/s]
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.85it/s]
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.45it/s]
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.03it/s]
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.17it/s]
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.49it/s]
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.75it/s]
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.60it/s]
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.65it/s]
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.30it/s]
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.25it/s]
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.26it/s]
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.62it/s]
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.86it/s]
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.86it/s]
+ 88%|████████▊ | 881/1000 [01:59<00:15,  7.59it/s]
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.53it/s]
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.94it/s]
+ 89%|████████▊ | 886/1000 [01:59<00:12,  8.94it/s]
+ 89%|████████▉ | 888/1000 [01:59<00:11, 10.16it/s]
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.66it/s]
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.68it/s]
+ 89%|████████▉ | 894/1000 [02:00<00:14,  7.55it/s]
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.79it/s]
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.52it/s]
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.68it/s]
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.83it/s]
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.03it/s]
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.04it/s]
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.30it/s]
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.56it/s]
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.75it/s]
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.38it/s]
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.59it/s]
+ 92%|█████████▏| 919/1000 [02:04<00:14,  5.71it/s]
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.93it/s]
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.16it/s]
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.64it/s]
+ 93%|█████████▎| 927/1000 [02:05<00:09,  8.07it/s]
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.64it/s]
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.46it/s]
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.81it/s]
+ 93%|█████████▎| 933/1000 [02:06<00:10,  6.60it/s]
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.16it/s]
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.29it/s]
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.81it/s]
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.03it/s]
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.12it/s]
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.32it/s]
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.06it/s]
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.37it/s]
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.69it/s]
+ 95%|█████████▌| 950/1000 [02:08<00:05,  8.82it/s]
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.40it/s]
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+ 96%|█████████▌| 956/1000 [02:09<00:05,  7.40it/s]
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.76it/s]
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.43it/s]
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.33it/s]
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.41it/s]
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.19it/s]
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.78it/s]
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.80it/s]
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.41it/s]
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.05it/s]
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.43it/s]
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+100%|█████████▉| 995/1000 [02:14<00:00,  5.52it/s]
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+100%|██████████| 1000/1000 [02:15<00:00,  7.39it/s]
+***** eval metrics *****
+  epoch                   =        1.0
+  eval_loss               =     0.7529
+  eval_runtime            = 0:02:15.53
+  eval_samples_per_second =      7.378
+  eval_steps_per_second   =      7.378
+  num_input_tokens_seen   =   12201816
+Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
diff --git a/optuna_30k_experiments/logs/trial_003_lr_2.000e-04.log b/optuna_30k_experiments/logs/trial_003_lr_2.000e-04.log
new file mode 100644
index 0000000..bd54d1e
--- /dev/null
+++ b/optuna_30k_experiments/logs/trial_003_lr_2.000e-04.log
@@ -0,0 +1,12726 @@
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/torch/cuda/__init__.py:734: UserWarning: Can't initialize NVML
+  warnings.warn("Can't initialize NVML")
+[INFO|2026-10-13 06:24:10] llamafactory.hparams.parser:651 >> Process rank: 0, world size: 1, device: cuda:0, distributed training: False, compute dtype: torch.bfloat16
+Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
+[INFO|configuration_utils.py:765] 2026-10-13 06:24:10,818 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-13 06:24:10,827 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|configuration_utils.py:765] 2026-10-13 06:24:12,510 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-13 06:24:12,514 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|configuration_utils.py:765] 2026-10-13 06:24:12,662 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-13 06:24:12,665 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-13 06:24:13] llamafactory.data.loader:144 >> Loading dataset aya_slovak_hpo_30k.jsonl...
+training example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28828, 2176, 28711, 28863, 29114, 10268, 28709, 6612, 28714, 2892, 493, 363, 6294, 10719, 363, 3811, 28723, 7379, 626, 28889, 307, 14637, 311, 28768, 2218, 22885, 3748, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 284, 8341, 6729, 345, 28738, 28739, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28723, 524, 4634, 28711, 570, 1879, 261, 28863, 532, 613, 14483, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 345, 15962, 28739, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 1879, 15173, 537, 284, 8341, 705, 28723, 367, 280, 28879, 570, 4415, 28716, 28797, 9335, 3175, 29826, 626, 323, 412, 8514, 16986, 29826, 28708, 345, 15962, 28739, 1879, 15173, 537, 1221, 4566, 28718, 325, 28707, 28723, 461, 28723, 724, 494, 8891, 24160, 28725, 8582, 28862, 24160, 28725, 20532, 28764, 271, 28725, 438, 29365, 15745, 1319, 12035, 28862, 8269, 284, 28827, 2762, 28709, 264, 4798, 8359, 1450, 8325, 12121, 1798, 420, 762, 1089, 290, 28906, 18765, 724, 2331, 29114, 1221, 4304, 1780, 363, 13881, 29826, 316, 2085, 28708, 28723, 22627, 28873, 263, 335, 20208, 284, 28827, 2762, 28709, 264, 4798, 5424, 10056, 1286, 290, 28906, 18765, 724, 2331, 29114, 11323, 10719, 12490, 363, 13881, 29826, 316, 284, 8341, 2390, 28723, 22127, 280, 28879, 21022, 1150, 424, 724, 28827, 1144, 28873, 340, 17066, 270, 21022, 485, 284, 8341, 705, 28725, 446, 6909, 28797, 637, 28705, 29114, 28708, 28879, 4798, 6612, 28827, 1100, 28768, 28863, 28723, 320, 412, 28879, 637, 15612, 28716, 28889, 28726, 1150, 424, 19523, 412, 28873, 276, 28797, 2390, 284, 8341, 2390, 363, 23407, 11277, 9538, 2085, 28708, 28723, 9532, 2144, 686, 7865, 28718, 341, 14475, 1687, 28706, 18037, 452, 270, 8269, 284, 8341, 705, 28725, 290, 4827, 486, 2710, 2144, 332, 1800, 16344, 29114, 6588, 269, 28862, 28711, 28863, 686, 1653, 5388, 28718, 710, 261, 1296, 28709, 284, 8341, 705, 28723, 351, 28711, 1371, 28797, 1776, 29826, 8269, 284, 8341, 705, 268, 28863, 1776, 29826, 8269, 1879, 3497, 598, 8269, 8631, 28879, 279, 412, 28725, 12121, 7988, 268, 28863, 24270, 12115, 28797, 1879, 5626, 263, 28862, 8269, 8631, 28879, 279, 412, 28723, 318, 15612, 1473, 276, 15690, 2245, 626, 15690, 307, 14637, 311, 28768, 300, 15469, 28711, 570, 446, 450, 4798, 29826, 333, 28729, 1879, 319, 3976, 294, 405, 28768, 4882, 405, 28725, 534, 28724, 2710, 724, 313, 4827, 2245, 626, 28889, 20532, 28764, 271, 28723, 10650, 300, 12235, 28827, 28873, 424, 268, 27172, 2245, 28723, 1054, 15504, 28879, 424, 724, 313, 8768, 4545, 504, 270, 8269, 2124, 2245, 626, 12689, 4584, 10906, 28718, 710, 18782, 28879, 28715, 28797, 1135, 10356, 28723, 2645, 28764, 12275, 28820, 28873, 29114, 13067, 549, 2892, 5872, 3811, 8631, 25991, 3026, 29114, 408, 28906, 28764, 485, 2082, 1403, 12121, 1798, 1773, 29826, 28729, 22160, 710, 408, 28906, 28764, 485, 1135, 10874, 1776, 4536, 28873, 300, 23407, 11277, 9538, 2085, 28708, 28723, 22127, 3361, 13470, 1150, 317, 28873, 424, 2082, 7330, 511, 2085, 28708, 28723, 401, 492, 13591, 28820, 261, 2220, 28862, 10766, 381, 28827, 486, 29114, 363, 28879, 4689, 3418, 28879, 21775, 29114, 280, 28723, 418, 28820, 11277, 28716, 6294, 26177, 903, 12689, 6612, 20122, 28733, 12868, 291, 2124, 2085, 28708, 1879, 6670, 28862, 9348, 493, 686, 375, 5620, 386, 28862, 28827, 28725, 23374, 4536, 25306, 16388, 957, 450, 15612, 28764, 2126, 29114, 511, 2152, 363, 25769, 14110, 17413, 26177, 903, 28873, 15808, 19396, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 7379, 1879, 277, 7939, 1879, 724, 313, 8768, 2245, 28718, 363, 446, 551, 9538, 12121, 1798, 24473, 21694, 300, 261, 5869, 28723, 384, 28728, 491, 28712, 4566, 15469, 28711, 570, 1879, 2245, 28725, 534, 28724, 2710, 7463, 15612, 13335, 28728, 4371, 1144, 28723, 10650, 300, 15469, 28711, 570, 1879, 1160, 28709, 28879, 3907, 13288, 491, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 28725, 534, 28724, 637, 686, 598, 5237, 5945, 17984, 28718, 2117, 18904, 3146, 515, 13288, 25208, 28723, 367, 280, 28879, 570, 12472, 1006, 5872, 28863, 358, 446, 311, 4845, 1879, 6149, 783, 494, 28718, 2245, 28718, 363, 17984, 28718, 405, 13288, 491, 28723, 13343, 29826, 28729, 385, 29114, 284, 8341, 705, 28747, 399, 3711, 3175, 29826, 626, 12990, 17984, 28718, 2117, 268, 6125, 15690, 345, 28738, 28739, 16986, 29826, 28708, 1773, 29826, 10641, 2124, 345, 28738, 28739, 582, 20524, 12214, 1773, 29826, 28729, 385, 29114, 284, 8341, 705, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 15173, 537, 28718, 325, 849, 3748, 28725, 446, 6909, 28820, 3061, 28720, 300, 28827, 1520, 6612, 749, 1780, 20532, 28764, 271, 586, 28873, 28710, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28731, 1879, 686, 6729, 1773, 29826, 28729, 22160, 284, 8341, 705, 28723, 550, 11245, 412, 28747, 550, 11245, 412, 1683, 28827, 16032, 271, 1342, 4295, 3191, 316, 28729, 6449, 2245, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 345, 28741, 28739, 1879, 22954, 338, 28718, 297, 12689, 345, 28741, 28739, 686, 2129, 28710, 29114, 16986, 28863, 358, 28723, 17563, 971, 28747, 17563, 971, 582, 20524, 12214, 1342, 28764, 1193, 1342, 4295, 5765, 6175, 2959, 15416, 28710, 13288, 491, 3589, 28723, 9532, 484, 28717, 3811, 18904, 17101, 29114, 22399, 971, 28725, 15612, 13335, 28728, 4371, 28711, 570, 13288, 20996, 28725, 446, 6909, 28797, 484, 28717, 3811, 18904, 17101, 29114, 28725, 264, 2513, 300, 8631, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 268, 345, 28741, 28739, 264, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Zhrnúť tento článok v jednej vete. Textový nástroj je ikona, ktorá sa podobá písmenu "T" v paneli nástrojov vľavo. Kliknite na túto ikonu. Použite rozbaľovaciu ponuku vedľa "Character" v hornej časti obrazovky na výber písma. Použite druhé rozbaľovacie menu vedľa "Character" na výber formátu (t. j. pravidelný, tučný, kurzor, atď.). Tradičné písmo ako Times New Roman alebo Garamond môže pridať formálny vzhľad loga. Bezšerifové písmo ako Helvetica môže pridať modernejší vzhľad písmu. Nepoužívajte príliš dekoratívne písma, ktoré sa ťažko čítajú. Tiež sa vyhýbajte zmiešanému písmu v návrhu loga. Ak si z internetu stiahnete bezplatné písma, mali by ste si uvedomiť licenčnú zmluvu pre tieto písma. Mnohé voľné písma sú voľné na osobné použitie, ale nie sú povolené na komerčné použitie. S vybraným textovým nástrojom kliknite kdekoľvek na grafickej doske, aby ste pridali textový kurzor. Potom napíšte svoj text. Zvážte pridanie samostatného textového objektu pre každé slovo. Obzvlášť ak plánujete používať rôzne farby alebo veľkosti pre rôzne slová vo vašom návrhu loga. Nepridávaj ešte farbu do loga. Farebná tlač nemusí byť vždy možnosťou. Návrh jednoduchého čierno-bieleho loga na začiatok zabezpečí, že vaše logo bude vyzerať dobre v jeho najjednoduchšej forme. Môžete tiež použiť nástroj Text na ceste na pridanie textu v kruhu alebo zakrivenom tvare. Dvakrát kliknite na text, aby ste ho vyzdvihli. Potom kliknite na položku znak v hornej časti obrazovky, aby sa zobrazila ponuka nastavenia znakov. Použite nasledujúce kroky na úpravu textu v ponuke znak. Veľkosť písma: Rozbaľovacia ponuka s malým "T" vedľa veľkého "T" upravuje veľkosť písma. Môžete tiež použiť nástroj výberu (ikona, ktorá pripomína čierny kurzor myši v paneli nástrojov vľavo) na zmenu veľkosti písma. Vedenie: Vedenie mení priestor medzi riadkami textu. Použite rozbaľovaciu ponuku vedľa ikony, ktorá sa podobá "A" na vrchu iného "A" zmeniť vedúce. Kerning: Kerning upravuje medzery medzi konkrétnymi znakmi. Ak chcete nastaviť kerning, vyzdvihnite znaky, ktoré chcete nastaviť, a potom použite rozbaľovaciu ponuku vedľa ikony s "A" aVyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+labels:
+Vyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+eval example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28965, 28709, 637, 8631, 25991, 3026, 28725, 13067, 10531, 28729, 350, 2591, 686, 3023, 28862, 28827, 284, 28873, 269, 13096, 28711, 28863, 6149, 3399, 28804, 550, 13881, 29826, 316, 300, 1879, 7871, 338, 28820, 6936, 1150, 28863, 7340, 6957, 8905, 3907, 12235, 28827, 28873, 424, 5765, 772, 28725, 446, 6909, 28889, 14848, 912, 12214, 4321, 28720, 751, 29365, 28723, 351, 28906, 18765, 298, 486, 29114, 28705, 28740, 387, 28705, 28750, 28734, 22116, 28827, 28723, 8294, 772, 28747, 13, 13, 27332, 12107, 28747, 13, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Čo sa používa, ak škodca zničí pšeničnú úrodu? Vzhľadom na predchádzajúcu otázku napíšte kontext, ktorý obsahuje odpoveď. Môže to byť 1 - 20 vetí. Kontext:
+
+### Response:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+labels:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+[INFO|configuration_utils.py:765] 2026-10-13 06:24:14,178 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-13 06:24:14,181 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-13 06:24:14] llamafactory.model.model_utils.kv_cache:144 >> KV cache is disabled during training.
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/unsloth/__init__.py:165: UserWarning: WARNING: Unsloth should be imported before [trl, transformers, peft] to ensure all optimizations are applied. Your code may run slower or encounter memory issues without these optimizations.
+
+Please restructure your imports with 'import unsloth' at the top of your file.
+  from ._gpu_init import *
+🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/torch/cuda/__init__.py:734: UserWarning: Can't initialize NVML
+  warnings.warn("Can't initialize NVML")
+🦥 Unsloth Zoo will now patch everything to make training faster!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: WARNING `trust_remote_code` is True.
+Are you certain you want to do remote code execution?
+==((====))==  Unsloth 2026.6.1: Fast Mistral patching. Transformers: 5.5.0.
+   \\   /|    NVIDIA RTX A4500. Num GPUs = 1. Max memory: 19.698 GB. Platform: Linux.
+O^O/ \_/ \    Torch: 2.6.0+cu124. CUDA: 8.6. CUDA Toolkit: 12.4. Triton: 3.2.0
+\        /    Bfloat16 = TRUE. FA [Xformers = 0.0.29.post3. FA2 = False]
+ "-____-"     Free license: http://github.com/unslothai/unsloth
+Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading weights file model.safetensors from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/model.safetensors.index.json
+Will use dtype=torch.bfloat16 as defined in model's config object
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 2,
+  "use_cache": true
+}
+
+
+Loading weights:   0%|          | 0/291 [00:00<?, ?it/s]
+Loading weights:   2%|▏         | 5/291 [00:00<00:06, 46.54it/s]
+Loading weights:   5%|▌         | 15/291 [00:00<00:03, 71.34it/s]
+Loading weights:  11%|█         | 31/291 [00:00<00:02, 101.07it/s]
+Loading weights:  14%|█▍        | 42/291 [00:00<00:02, 100.71it/s]
+Loading weights:  19%|█▉        | 55/291 [00:00<00:02, 108.28it/s]
+Loading weights:  23%|██▎       | 67/291 [00:00<00:02, 108.44it/s]
+Loading weights:  27%|██▋       | 78/291 [00:00<00:02, 99.66it/s] 
+Loading weights:  32%|███▏      | 92/291 [00:00<00:01, 110.97it/s]
+Loading weights:  36%|███▌      | 104/291 [00:01<00:01, 101.44it/s]
+Loading weights:  40%|███▉      | 115/291 [00:01<00:01, 97.15it/s] 
+Loading weights:  44%|████▎     | 127/291 [00:01<00:01, 102.07it/s]
+Loading weights:  48%|████▊     | 139/291 [00:01<00:01, 105.72it/s]
+Loading weights:  53%|█████▎    | 154/291 [00:01<00:01, 117.18it/s]
+Loading weights:  57%|█████▋    | 167/291 [00:01<00:01, 120.72it/s]
+Loading weights:  62%|██████▏   | 180/291 [00:01<00:00, 118.21it/s]
+Loading weights:  66%|██████▋   | 193/291 [00:01<00:00, 119.89it/s]
+Loading weights:  71%|███████   | 206/291 [00:01<00:00, 110.42it/s]
+Loading weights:  76%|███████▌  | 220/291 [00:02<00:00, 114.00it/s]
+Loading weights:  80%|███████▉  | 232/291 [00:02<00:00, 103.49it/s]
+Loading weights:  85%|████████▍ | 247/291 [00:02<00:00, 111.87it/s]
+Loading weights:  89%|████████▉ | 259/291 [00:02<00:00, 102.47it/s]
+Loading weights:  93%|█████████▎| 272/291 [00:02<00:00, 108.87it/s]
+Loading weights:  98%|█████████▊| 284/291 [00:02<00:00, 104.33it/s]
+Loading weights: 100%|██████████| 291/291 [00:02<00:00, 107.20it/s]
+loading configuration file generation_config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/generation_config.json
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2
+}
+
+Could not locate the custom_generate/generate.py inside slovak-nlp/mistral-sk-7b.
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: slovak-nlp/mistral-sk-7b had a bad pad_token (</s>). Using pad_token = <unk>.
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:68: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_fwd
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:85: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_bwd
+[INFO|2026-10-13 06:24:31] llamafactory.model.model_utils.checkpointing:144 >> Gradient checkpointing enabled.
+[INFO|2026-10-13 06:24:31] llamafactory.model.adapter:144 >> Upcasting trainable params to float32.
+[INFO|2026-10-13 06:24:31] llamafactory.model.adapter:144 >> Fine-tuning method: LoRA
+Unsloth 2026.6.1 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers.
+[INFO|2026-10-13 06:24:34] llamafactory.model.loader:144 >> trainable params: 41,943,040 || all params: 7,283,675,136 || trainable%: 0.5758
+The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'pad_token_id': 2}.
+==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 1
+   \\   /|    Num examples = 29,000 | Num Epochs = 1 | Total steps = 1,813
+O^O/ \_/ \    Batch size per device = 1 | Gradient accumulation steps = 16
+\        /    Data Parallel GPUs = 1 | Total batch size (1 x 16 x 1) = 16
+ "-____-"     Trainable parameters = 41,943,040 of 7,283,675,136 (0.58% trained)
+
+  0%|          | 0/1813 [00:00<?, ?it/s]`use_return_dict` is deprecated! Use `return_dict` instead!
+Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
+Unsloth: Will smartly offload gradients to save VRAM!
+
+  0%|          | 1/1813 [00:06<3:24:19,  6.77s/it]
+  0%|          | 2/1813 [00:13<3:19:41,  6.62s/it]
+  0%|          | 3/1813 [00:18<3:01:54,  6.03s/it]
+  0%|          | 4/1813 [00:22<2:38:10,  5.25s/it]
+  0%|          | 5/1813 [00:29<2:52:04,  5.71s/it]
+  0%|          | 6/1813 [00:36<3:09:01,  6.28s/it]
+  0%|          | 7/1813 [00:42<3:10:29,  6.33s/it]
+  0%|          | 8/1813 [00:48<2:58:58,  5.95s/it]
+  0%|          | 9/1813 [00:55<3:13:30,  6.44s/it]
+  1%|          | 10/1813 [01:01<3:04:31,  6.14s/it]
+                                                   
+{'loss': '1.777', 'grad_norm': '3.533', 'learning_rate': '1.2e-05', 'epoch': '0.005517', 'num_input_tokens_seen': 65928, 'train_runtime': '61.12', 'train_tokens_per_second': '1079'}
+
+  1%|          | 10/1813 [01:01<3:04:31,  6.14s/it]
+  1%|          | 11/1813 [01:07<3:05:24,  6.17s/it]
+  1%|          | 12/1813 [01:15<3:20:21,  6.68s/it]
+  1%|          | 13/1813 [01:19<2:58:25,  5.95s/it]
+  1%|          | 14/1813 [01:25<2:59:23,  5.98s/it]
+  1%|          | 15/1813 [01:30<2:50:34,  5.69s/it]
+  1%|          | 16/1813 [01:36<2:53:36,  5.80s/it]
+  1%|          | 17/1813 [01:42<2:58:47,  5.97s/it]
+  1%|          | 18/1813 [01:49<3:00:06,  6.02s/it]
+  1%|          | 19/1813 [01:55<3:06:43,  6.24s/it]
+  1%|          | 20/1813 [02:01<3:04:37,  6.18s/it]
+                                                   
+{'loss': '1.454', 'grad_norm': '1.221', 'learning_rate': '2.533e-05', 'epoch': '0.01103', 'num_input_tokens_seen': 131352, 'train_runtime': '121.9', 'train_tokens_per_second': '1078'}
+
+  1%|          | 20/1813 [02:01<3:04:37,  6.18s/it]
+  1%|          | 21/1813 [02:07<3:01:16,  6.07s/it]
+  1%|          | 22/1813 [02:12<2:49:57,  5.69s/it]
+  1%|▏         | 23/1813 [02:17<2:45:09,  5.54s/it]
+  1%|▏         | 24/1813 [02:23<2:46:22,  5.58s/it]
+  1%|▏         | 25/1813 [02:30<2:58:42,  6.00s/it]
+  1%|▏         | 26/1813 [02:37<3:09:39,  6.37s/it]
+  1%|▏         | 27/1813 [02:43<3:04:24,  6.20s/it]
+  2%|▏         | 28/1813 [02:50<3:09:03,  6.35s/it]
+  2%|▏         | 29/1813 [02:57<3:22:30,  6.81s/it]
+  2%|▏         | 30/1813 [03:02<3:03:04,  6.16s/it]
+                                                   
+{'loss': '1.323', 'grad_norm': '1.325', 'learning_rate': '3.867e-05', 'epoch': '0.01655', 'num_input_tokens_seen': 197376, 'train_runtime': '182.6', 'train_tokens_per_second': '1081'}
+
+  2%|▏         | 30/1813 [03:02<3:03:04,  6.16s/it]
+  2%|▏         | 31/1813 [03:08<3:03:23,  6.18s/it]
+  2%|▏         | 32/1813 [03:14<2:54:57,  5.89s/it]
+  2%|▏         | 33/1813 [03:20<2:55:21,  5.91s/it]
+  2%|▏         | 34/1813 [03:27<3:09:39,  6.40s/it]
+  2%|▏         | 35/1813 [03:33<3:09:58,  6.41s/it]
+  2%|▏         | 36/1813 [03:40<3:09:49,  6.41s/it]
+  2%|▏         | 37/1813 [03:45<2:58:17,  6.02s/it]
+  2%|▏         | 38/1813 [03:50<2:48:35,  5.70s/it]
+  2%|▏         | 39/1813 [03:56<2:47:53,  5.68s/it]
+  2%|▏         | 40/1813 [04:01<2:47:51,  5.68s/it]
+                                                   
+{'loss': '1.207', 'grad_norm': '1.188', 'learning_rate': '5.2e-05', 'epoch': '0.02207', 'num_input_tokens_seen': 261168, 'train_runtime': '241.8', 'train_tokens_per_second': '1080'}
+
+  2%|▏         | 40/1813 [04:01<2:47:51,  5.68s/it]
+  2%|▏         | 41/1813 [04:06<2:43:09,  5.52s/it]
+  2%|▏         | 42/1813 [04:12<2:46:04,  5.63s/it]
+  2%|▏         | 43/1813 [04:20<3:03:14,  6.21s/it]
+  2%|▏         | 44/1813 [04:26<3:05:31,  6.29s/it]
+  2%|▏         | 45/1813 [04:34<3:15:13,  6.63s/it]
+  3%|▎         | 46/1813 [04:40<3:09:27,  6.43s/it]
+  3%|▎         | 47/1813 [04:45<3:01:55,  6.18s/it]
+  3%|▎         | 48/1813 [04:52<3:06:17,  6.33s/it]
+  3%|▎         | 49/1813 [04:58<3:05:55,  6.32s/it]
+  3%|▎         | 50/1813 [05:05<3:04:47,  6.29s/it]
+                                                   
+{'loss': '1.276', 'grad_norm': '1.357', 'learning_rate': '6.533e-05', 'epoch': '0.02759', 'num_input_tokens_seen': 330104, 'train_runtime': '305', 'train_tokens_per_second': '1082'}
+
+  3%|▎         | 50/1813 [05:05<3:04:47,  6.29s/it]
+  3%|▎         | 51/1813 [05:12<3:15:02,  6.64s/it]
+  3%|▎         | 52/1813 [05:21<3:38:19,  7.44s/it]
+  3%|▎         | 53/1813 [05:27<3:25:37,  7.01s/it]
+  3%|▎         | 54/1813 [05:35<3:29:34,  7.15s/it]
+  3%|▎         | 55/1813 [05:41<3:21:37,  6.88s/it]
+  3%|▎         | 56/1813 [05:48<3:19:21,  6.81s/it]
+  3%|▎         | 57/1813 [05:53<3:03:09,  6.26s/it]
+  3%|▎         | 58/1813 [06:01<3:21:13,  6.88s/it]
+  3%|▎         | 59/1813 [06:08<3:22:00,  6.91s/it]
+  3%|▎         | 60/1813 [06:13<3:07:15,  6.41s/it]
+                                                   
+{'loss': '1.17', 'grad_norm': '1.461', 'learning_rate': '7.867e-05', 'epoch': '0.0331', 'num_input_tokens_seen': 407792, 'train_runtime': '373.7', 'train_tokens_per_second': '1091'}
+
+  3%|▎         | 60/1813 [06:13<3:07:15,  6.41s/it]
+  3%|▎         | 61/1813 [06:18<2:55:19,  6.00s/it]
+  3%|▎         | 62/1813 [06:24<2:51:00,  5.86s/it]
+  3%|▎         | 63/1813 [06:29<2:49:22,  5.81s/it]
+  4%|▎         | 64/1813 [06:35<2:48:26,  5.78s/it]
+  4%|▎         | 65/1813 [06:42<2:59:59,  6.18s/it]
+  4%|▎         | 66/1813 [06:49<3:00:49,  6.21s/it]
+  4%|▎         | 67/1813 [06:54<2:54:15,  5.99s/it]
+  4%|▍         | 68/1813 [07:00<2:55:12,  6.02s/it]
+  4%|▍         | 69/1813 [07:07<3:04:13,  6.34s/it]
+  4%|▍         | 70/1813 [07:15<3:13:45,  6.67s/it]
+                                                   
+{'loss': '1.066', 'grad_norm': '1.061', 'learning_rate': '9.2e-05', 'epoch': '0.03862', 'num_input_tokens_seen': 474368, 'train_runtime': '435.2', 'train_tokens_per_second': '1090'}
+
+  4%|▍         | 70/1813 [07:15<3:13:45,  6.67s/it]
+  4%|▍         | 71/1813 [07:20<3:03:12,  6.31s/it]
+  4%|▍         | 72/1813 [07:29<3:24:35,  7.05s/it]
+  4%|▍         | 73/1813 [07:35<3:12:27,  6.64s/it]
+  4%|▍         | 74/1813 [07:41<3:08:54,  6.52s/it]
+  4%|▍         | 75/1813 [07:47<3:09:11,  6.53s/it]
+  4%|▍         | 76/1813 [07:54<3:05:24,  6.40s/it]
+  4%|▍         | 77/1813 [08:00<3:09:02,  6.53s/it]
+  4%|▍         | 78/1813 [08:06<3:00:35,  6.24s/it]
+  4%|▍         | 79/1813 [08:11<2:54:43,  6.05s/it]
+  4%|▍         | 80/1813 [08:18<2:56:17,  6.10s/it]
+                                                   
+{'loss': '1.089', 'grad_norm': '1.242', 'learning_rate': '0.0001053', 'epoch': '0.04414', 'num_input_tokens_seen': 541616, 'train_runtime': '498.2', 'train_tokens_per_second': '1087'}
+
+  4%|▍         | 80/1813 [08:18<2:56:17,  6.10s/it]
+  4%|▍         | 81/1813 [08:24<2:54:09,  6.03s/it]
+  5%|▍         | 82/1813 [08:31<3:01:39,  6.30s/it]
+  5%|▍         | 83/1813 [08:38<3:08:06,  6.52s/it]
+  5%|▍         | 84/1813 [08:44<3:11:34,  6.65s/it]
+  5%|▍         | 85/1813 [08:51<3:08:31,  6.55s/it]
+  5%|▍         | 86/1813 [08:57<3:07:37,  6.52s/it]
+  5%|▍         | 87/1813 [09:02<2:54:51,  6.08s/it]
+  5%|▍         | 88/1813 [09:09<2:56:07,  6.13s/it]
+  5%|▍         | 89/1813 [09:14<2:54:04,  6.06s/it]
+  5%|▍         | 90/1813 [09:21<2:56:52,  6.16s/it]
+                                                   
+{'loss': '1.025', 'grad_norm': '0.8814', 'learning_rate': '0.0001187', 'epoch': '0.04966', 'num_input_tokens_seen': 608792, 'train_runtime': '561.3', 'train_tokens_per_second': '1085'}
+
+  5%|▍         | 90/1813 [09:21<2:56:52,  6.16s/it]
+  5%|▌         | 91/1813 [09:26<2:52:03,  6.00s/it]
+  5%|▌         | 92/1813 [09:31<2:41:53,  5.64s/it]
+  5%|▌         | 93/1813 [09:37<2:43:49,  5.71s/it]
+  5%|▌         | 94/1813 [09:43<2:48:43,  5.89s/it]
+  5%|▌         | 95/1813 [09:50<2:52:42,  6.03s/it]
+  5%|▌         | 96/1813 [09:56<2:51:24,  5.99s/it]
+  5%|▌         | 97/1813 [10:02<2:55:40,  6.14s/it]
+  5%|▌         | 98/1813 [10:07<2:46:30,  5.83s/it]
+  5%|▌         | 99/1813 [10:13<2:43:48,  5.73s/it]
+  6%|▌         | 100/1813 [10:19<2:48:25,  5.90s/it]
+                                                    
+{'loss': '1.065', 'grad_norm': '0.8532', 'learning_rate': '0.000132', 'epoch': '0.05517', 'num_input_tokens_seen': 670936, 'train_runtime': '619.6', 'train_tokens_per_second': '1083'}
+
+  6%|▌         | 100/1813 [10:19<2:48:25,  5.90s/it]
+  6%|▌         | 101/1813 [10:26<2:54:33,  6.12s/it]
+  6%|▌         | 102/1813 [10:32<2:59:05,  6.28s/it]
+  6%|▌         | 103/1813 [10:38<2:55:52,  6.17s/it]
+  6%|▌         | 104/1813 [10:45<2:56:22,  6.19s/it]
+  6%|▌         | 105/1813 [10:50<2:49:38,  5.96s/it]
+  6%|▌         | 106/1813 [10:56<2:49:41,  5.96s/it]
+  6%|▌         | 107/1813 [11:02<2:52:46,  6.08s/it]
+  6%|▌         | 108/1813 [11:09<2:55:05,  6.16s/it]
+  6%|▌         | 109/1813 [11:15<2:54:23,  6.14s/it]
+  6%|▌         | 110/1813 [11:22<2:59:41,  6.33s/it]
+                                                    
+{'loss': '1.096', 'grad_norm': '0.7748', 'learning_rate': '0.0001453', 'epoch': '0.06069', 'num_input_tokens_seen': 738632, 'train_runtime': '682', 'train_tokens_per_second': '1083'}
+
+  6%|▌         | 110/1813 [11:22<2:59:41,  6.33s/it]
+  6%|▌         | 111/1813 [11:28<3:03:54,  6.48s/it]
+  6%|▌         | 112/1813 [11:35<3:05:50,  6.56s/it]
+  6%|▌         | 113/1813 [11:41<3:02:36,  6.45s/it]
+  6%|▋         | 114/1813 [11:49<3:11:43,  6.77s/it]
+  6%|▋         | 115/1813 [11:55<3:03:21,  6.48s/it]
+  6%|▋         | 116/1813 [12:01<3:06:24,  6.59s/it]
+  6%|▋         | 117/1813 [12:07<2:56:20,  6.24s/it]
+  7%|▋         | 118/1813 [12:13<2:52:33,  6.11s/it]
+  7%|▋         | 119/1813 [12:19<2:53:52,  6.16s/it]
+  7%|▋         | 120/1813 [12:25<2:54:26,  6.18s/it]
+                                                    
+{'loss': '1.1', 'grad_norm': '1.095', 'learning_rate': '0.0001587', 'epoch': '0.06621', 'num_input_tokens_seen': 808824, 'train_runtime': '745.7', 'train_tokens_per_second': '1085'}
+
+  7%|▋         | 120/1813 [12:25<2:54:26,  6.18s/it]
+  7%|▋         | 121/1813 [12:29<2:36:52,  5.56s/it]
+  7%|▋         | 122/1813 [12:34<2:31:32,  5.38s/it]
+  7%|▋         | 123/1813 [12:41<2:40:04,  5.68s/it]
+  7%|▋         | 124/1813 [12:46<2:36:01,  5.54s/it]
+  7%|▋         | 125/1813 [12:52<2:44:48,  5.86s/it]
+  7%|▋         | 126/1813 [12:59<2:47:08,  5.94s/it]
+  7%|▋         | 127/1813 [13:03<2:35:57,  5.55s/it]
+  7%|▋         | 128/1813 [13:08<2:29:06,  5.31s/it]
+  7%|▋         | 129/1813 [13:16<2:52:42,  6.15s/it]
+  7%|▋         | 130/1813 [13:23<2:56:34,  6.30s/it]
+                                                    
+{'loss': '1.042', 'grad_norm': '0.7589', 'learning_rate': '0.000172', 'epoch': '0.07172', 'num_input_tokens_seen': 869720, 'train_runtime': '803.2', 'train_tokens_per_second': '1083'}
+
+  7%|▋         | 130/1813 [13:23<2:56:34,  6.30s/it]
+  7%|▋         | 131/1813 [13:29<2:52:35,  6.16s/it]
+  7%|▋         | 132/1813 [13:34<2:44:15,  5.86s/it]
+  7%|▋         | 133/1813 [13:39<2:40:35,  5.74s/it]
+  7%|▋         | 134/1813 [13:44<2:34:10,  5.51s/it]
+  7%|▋         | 135/1813 [13:49<2:30:36,  5.39s/it]
+  8%|▊         | 136/1813 [13:55<2:34:05,  5.51s/it]
+  8%|▊         | 137/1813 [14:01<2:38:52,  5.69s/it]
+  8%|▊         | 138/1813 [14:08<2:50:46,  6.12s/it]
+  8%|▊         | 139/1813 [14:15<2:55:02,  6.27s/it]
+  8%|▊         | 140/1813 [14:22<3:00:09,  6.46s/it]
+                                                    
+{'loss': '1.042', 'grad_norm': '1.091', 'learning_rate': '0.0001853', 'epoch': '0.07724', 'num_input_tokens_seen': 932000, 'train_runtime': '862.3', 'train_tokens_per_second': '1081'}
+
+  8%|▊         | 140/1813 [14:22<3:00:09,  6.46s/it]
+  8%|▊         | 141/1813 [14:27<2:52:10,  6.18s/it]
+  8%|▊         | 142/1813 [14:35<3:06:21,  6.69s/it]
+  8%|▊         | 143/1813 [14:41<2:54:55,  6.28s/it]
+  8%|▊         | 144/1813 [14:46<2:44:58,  5.93s/it]
+  8%|▊         | 145/1813 [14:53<2:54:24,  6.27s/it]
+  8%|▊         | 146/1813 [14:59<2:54:55,  6.30s/it]
+  8%|▊         | 147/1813 [15:05<2:49:37,  6.11s/it]
+  8%|▊         | 148/1813 [15:11<2:49:00,  6.09s/it]
+  8%|▊         | 149/1813 [15:17<2:46:53,  6.02s/it]
+  8%|▊         | 150/1813 [15:23<2:48:32,  6.08s/it]
+                                                    
+{'loss': '1.1', 'grad_norm': '0.8783', 'learning_rate': '0.0001987', 'epoch': '0.08276', 'num_input_tokens_seen': 997296, 'train_runtime': '923.4', 'train_tokens_per_second': '1080'}
+
+  8%|▊         | 150/1813 [15:23<2:48:32,  6.08s/it]
+  8%|▊         | 151/1813 [15:28<2:43:25,  5.90s/it]
+  8%|▊         | 152/1813 [15:34<2:41:23,  5.83s/it]
+  8%|▊         | 153/1813 [15:40<2:42:59,  5.89s/it]
+  8%|▊         | 154/1813 [15:47<2:47:37,  6.06s/it]
+  9%|▊         | 155/1813 [15:54<2:56:26,  6.38s/it]
+  9%|▊         | 156/1813 [16:00<2:53:18,  6.28s/it]
+  9%|▊         | 157/1813 [16:06<2:52:44,  6.26s/it]
+  9%|▊         | 158/1813 [16:11<2:42:17,  5.88s/it]
+  9%|▉         | 159/1813 [16:17<2:40:47,  5.83s/it]
+  9%|▉         | 160/1813 [16:22<2:38:40,  5.76s/it]
+                                                    
+{'loss': '1.026', 'grad_norm': '1.363', 'learning_rate': '0.0002', 'epoch': '0.08828', 'num_input_tokens_seen': 1060096, 'train_runtime': '982.7', 'train_tokens_per_second': '1079'}
+
+  9%|▉         | 160/1813 [16:22<2:38:40,  5.76s/it]
+  9%|▉         | 161/1813 [16:28<2:38:21,  5.75s/it]
+  9%|▉         | 162/1813 [16:32<2:25:33,  5.29s/it]
+  9%|▉         | 163/1813 [16:39<2:36:38,  5.70s/it]
+  9%|▉         | 164/1813 [16:46<2:49:29,  6.17s/it]
+  9%|▉         | 165/1813 [16:52<2:49:24,  6.17s/it]
+  9%|▉         | 166/1813 [17:00<3:02:11,  6.64s/it]
+  9%|▉         | 167/1813 [17:07<3:02:24,  6.65s/it]
+  9%|▉         | 168/1813 [17:14<3:06:19,  6.80s/it]
+  9%|▉         | 169/1813 [17:19<2:56:50,  6.45s/it]
+  9%|▉         | 170/1813 [17:26<2:58:35,  6.52s/it]
+                                                    
+{'loss': '1.053', 'grad_norm': '0.6226', 'learning_rate': '0.0001999', 'epoch': '0.09379', 'num_input_tokens_seen': 1130368, 'train_runtime': '1047', 'train_tokens_per_second': '1080'}
+
+  9%|▉         | 170/1813 [17:26<2:58:35,  6.52s/it]
+  9%|▉         | 171/1813 [17:31<2:48:43,  6.17s/it]
+  9%|▉         | 172/1813 [17:37<2:41:58,  5.92s/it]
+ 10%|▉         | 173/1813 [17:43<2:45:46,  6.07s/it]
+ 10%|▉         | 174/1813 [17:49<2:43:05,  5.97s/it]
+ 10%|▉         | 175/1813 [17:55<2:45:07,  6.05s/it]
+ 10%|▉         | 176/1813 [18:00<2:37:22,  5.77s/it]
+ 10%|▉         | 177/1813 [18:08<2:51:03,  6.27s/it]
+ 10%|▉         | 178/1813 [18:13<2:39:23,  5.85s/it]
+ 10%|▉         | 179/1813 [18:19<2:39:48,  5.87s/it]
+ 10%|▉         | 180/1813 [18:25<2:41:31,  5.93s/it]
+                                                    
+{'loss': '1.037', 'grad_norm': '0.9835', 'learning_rate': '0.0001998', 'epoch': '0.09931', 'num_input_tokens_seen': 1193192, 'train_runtime': '1105', 'train_tokens_per_second': '1080'}
+
+ 10%|▉         | 180/1813 [18:25<2:41:31,  5.93s/it]
+ 10%|▉         | 181/1813 [18:30<2:37:55,  5.81s/it]
+ 10%|█         | 182/1813 [18:35<2:34:14,  5.67s/it]
+ 10%|█         | 183/1813 [18:41<2:33:22,  5.65s/it]
+ 10%|█         | 184/1813 [18:46<2:31:26,  5.58s/it]
+ 10%|█         | 185/1813 [18:55<2:51:19,  6.31s/it]
+ 10%|█         | 186/1813 [19:01<2:51:12,  6.31s/it]
+ 10%|█         | 187/1813 [19:07<2:49:10,  6.24s/it]
+ 10%|█         | 188/1813 [19:12<2:40:39,  5.93s/it]
+ 10%|█         | 189/1813 [19:19<2:46:11,  6.14s/it]
+ 10%|█         | 190/1813 [19:25<2:50:30,  6.30s/it]
+                                                    
+{'loss': '0.9777', 'grad_norm': '0.9615', 'learning_rate': '0.0001997', 'epoch': '0.1048', 'num_input_tokens_seen': 1259336, 'train_runtime': '1166', 'train_tokens_per_second': '1080'}
+
+ 10%|█         | 190/1813 [19:25<2:50:30,  6.30s/it]
+ 11%|█         | 191/1813 [19:31<2:44:58,  6.10s/it]
+ 11%|█         | 192/1813 [19:38<2:49:12,  6.26s/it]
+ 11%|█         | 193/1813 [19:44<2:52:52,  6.40s/it]
+ 11%|█         | 194/1813 [19:51<2:50:27,  6.32s/it]
+ 11%|█         | 195/1813 [19:58<3:00:52,  6.71s/it]
+ 11%|█         | 196/1813 [20:04<2:50:15,  6.32s/it]
+ 11%|█         | 197/1813 [20:11<2:57:06,  6.58s/it]
+ 11%|█         | 198/1813 [20:18<3:03:34,  6.82s/it]
+ 11%|█         | 199/1813 [20:25<3:03:06,  6.81s/it]
+ 11%|█         | 200/1813 [20:32<3:01:52,  6.77s/it]
+                                                    
+{'loss': '1.086', 'grad_norm': '1.075', 'learning_rate': '0.0001996', 'epoch': '0.1103', 'num_input_tokens_seen': 1331224, 'train_runtime': '1232', 'train_tokens_per_second': '1080'}
+
+ 11%|█         | 200/1813 [20:32<3:01:52,  6.77s/it]
+ 11%|█         | 201/1813 [20:37<2:52:39,  6.43s/it]
+ 11%|█         | 202/1813 [20:42<2:43:23,  6.09s/it]
+ 11%|█         | 203/1813 [20:48<2:35:35,  5.80s/it]
+ 11%|█▏        | 204/1813 [20:55<2:45:05,  6.16s/it]
+ 11%|█▏        | 205/1813 [21:00<2:38:56,  5.93s/it]
+ 11%|█▏        | 206/1813 [21:06<2:38:21,  5.91s/it]
+ 11%|█▏        | 207/1813 [21:12<2:38:34,  5.92s/it]
+ 11%|█▏        | 208/1813 [21:20<2:53:52,  6.50s/it]
+ 12%|█▏        | 209/1813 [21:25<2:42:19,  6.07s/it]
+ 12%|█▏        | 210/1813 [21:31<2:44:02,  6.14s/it]
+                                                    
+{'loss': '0.9652', 'grad_norm': '1.053', 'learning_rate': '0.0001994', 'epoch': '0.1159', 'num_input_tokens_seen': 1392472, 'train_runtime': '1292', 'train_tokens_per_second': '1078'}
+
+ 12%|█▏        | 210/1813 [21:31<2:44:02,  6.14s/it]
+ 12%|█▏        | 211/1813 [21:40<3:04:57,  6.93s/it]
+ 12%|█▏        | 212/1813 [21:46<2:55:59,  6.60s/it]
+ 12%|█▏        | 213/1813 [21:51<2:47:05,  6.27s/it]
+ 12%|█▏        | 214/1813 [21:57<2:40:25,  6.02s/it]
+ 12%|█▏        | 215/1813 [22:02<2:35:52,  5.85s/it]
+ 12%|█▏        | 216/1813 [22:07<2:25:11,  5.45s/it]
+ 12%|█▏        | 217/1813 [22:12<2:22:36,  5.36s/it]
+ 12%|█▏        | 218/1813 [22:18<2:30:06,  5.65s/it]
+ 12%|█▏        | 219/1813 [22:24<2:34:15,  5.81s/it]
+ 12%|█▏        | 220/1813 [22:30<2:36:59,  5.91s/it]
+                                                    
+{'loss': '1.042', 'grad_norm': '0.8483', 'learning_rate': '0.0001992', 'epoch': '0.1214', 'num_input_tokens_seen': 1454040, 'train_runtime': '1351', 'train_tokens_per_second': '1076'}
+
+ 12%|█▏        | 220/1813 [22:30<2:36:59,  5.91s/it]
+ 12%|█▏        | 221/1813 [22:37<2:40:39,  6.06s/it]
+ 12%|█▏        | 222/1813 [22:42<2:34:00,  5.81s/it]
+ 12%|█▏        | 223/1813 [22:50<2:47:45,  6.33s/it]
+ 12%|█▏        | 224/1813 [22:55<2:37:16,  5.94s/it]
+ 12%|█▏        | 225/1813 [23:02<2:45:49,  6.27s/it]
+ 12%|█▏        | 226/1813 [23:08<2:50:08,  6.43s/it]
+ 13%|█▎        | 227/1813 [23:14<2:45:24,  6.26s/it]
+ 13%|█▎        | 228/1813 [23:21<2:47:35,  6.34s/it]
+ 13%|█▎        | 229/1813 [23:28<2:53:59,  6.59s/it]
+ 13%|█▎        | 230/1813 [23:33<2:41:23,  6.12s/it]
+                                                    
+{'loss': '1.007', 'grad_norm': '1.187', 'learning_rate': '0.0001989', 'epoch': '0.1269', 'num_input_tokens_seen': 1522056, 'train_runtime': '1413', 'train_tokens_per_second': '1077'}
+
+ 13%|█▎        | 230/1813 [23:33<2:41:23,  6.12s/it]
+ 13%|█▎        | 231/1813 [23:38<2:33:03,  5.81s/it]
+ 13%|█▎        | 232/1813 [23:45<2:45:38,  6.29s/it]
+ 13%|█▎        | 233/1813 [23:51<2:36:27,  5.94s/it]
+ 13%|█▎        | 234/1813 [23:56<2:31:53,  5.77s/it]
+ 13%|█▎        | 235/1813 [24:03<2:41:09,  6.13s/it]
+ 13%|█▎        | 236/1813 [24:11<2:52:48,  6.58s/it]
+ 13%|█▎        | 237/1813 [24:18<2:55:41,  6.69s/it]
+ 13%|█▎        | 238/1813 [24:23<2:47:48,  6.39s/it]
+ 13%|█▎        | 239/1813 [24:31<2:57:39,  6.77s/it]
+ 13%|█▎        | 240/1813 [24:37<2:54:02,  6.64s/it]
+                                                    
+{'loss': '1.002', 'grad_norm': '0.8349', 'learning_rate': '0.0001986', 'epoch': '0.1324', 'num_input_tokens_seen': 1590200, 'train_runtime': '1478', 'train_tokens_per_second': '1076'}
+
+ 13%|█▎        | 240/1813 [24:37<2:54:02,  6.64s/it]
+ 13%|█▎        | 241/1813 [24:44<2:52:13,  6.57s/it]
+ 13%|█▎        | 242/1813 [24:50<2:50:00,  6.49s/it]
+ 13%|█▎        | 243/1813 [24:56<2:45:02,  6.31s/it]
+ 13%|█▎        | 244/1813 [25:03<2:51:57,  6.58s/it]
+ 14%|█▎        | 245/1813 [25:10<2:56:34,  6.76s/it]
+ 14%|█▎        | 246/1813 [25:16<2:47:33,  6.42s/it]
+ 14%|█▎        | 247/1813 [25:23<2:51:54,  6.59s/it]
+ 14%|█▎        | 248/1813 [25:31<3:01:50,  6.97s/it]
+ 14%|█▎        | 249/1813 [25:36<2:51:16,  6.57s/it]
+ 14%|█▍        | 250/1813 [25:42<2:43:59,  6.30s/it]
+                                                    
+{'loss': '1.08', 'grad_norm': '0.9302', 'learning_rate': '0.0001983', 'epoch': '0.1379', 'num_input_tokens_seen': 1658648, 'train_runtime': '1542', 'train_tokens_per_second': '1075'}
+
+ 14%|█▍        | 250/1813 [25:42<2:43:59,  6.30s/it]
+ 14%|█▍        | 251/1813 [25:47<2:30:19,  5.77s/it]
+ 14%|█▍        | 252/1813 [25:52<2:26:41,  5.64s/it]
+ 14%|█▍        | 253/1813 [25:58<2:30:51,  5.80s/it]
+ 14%|█▍        | 254/1813 [26:05<2:42:21,  6.25s/it]
+ 14%|█▍        | 255/1813 [26:12<2:45:12,  6.36s/it]
+ 14%|█▍        | 256/1813 [26:17<2:37:18,  6.06s/it]
+ 14%|█▍        | 257/1813 [26:24<2:42:46,  6.28s/it]
+ 14%|█▍        | 258/1813 [26:31<2:48:22,  6.50s/it]
+ 14%|█▍        | 259/1813 [26:36<2:34:18,  5.96s/it]
+ 14%|█▍        | 260/1813 [26:42<2:33:53,  5.95s/it]
+                                                    
+{'loss': '0.8734', 'grad_norm': '1.161', 'learning_rate': '0.0001979', 'epoch': '0.1434', 'num_input_tokens_seen': 1722792, 'train_runtime': '1602', 'train_tokens_per_second': '1075'}
+
+ 14%|█▍        | 260/1813 [26:42<2:33:53,  5.95s/it]
+ 14%|█▍        | 261/1813 [26:49<2:43:17,  6.31s/it]
+ 14%|█▍        | 262/1813 [26:55<2:45:31,  6.40s/it]
+ 15%|█▍        | 263/1813 [27:02<2:46:19,  6.44s/it]
+ 15%|█▍        | 264/1813 [27:08<2:45:02,  6.39s/it]
+ 15%|█▍        | 265/1813 [27:14<2:43:28,  6.34s/it]
+ 15%|█▍        | 266/1813 [27:19<2:32:34,  5.92s/it]
+ 15%|█▍        | 267/1813 [27:25<2:28:52,  5.78s/it]
+ 15%|█▍        | 268/1813 [27:30<2:23:21,  5.57s/it]
+ 15%|█▍        | 269/1813 [27:36<2:27:55,  5.75s/it]
+ 15%|█▍        | 270/1813 [27:42<2:28:23,  5.77s/it]
+                                                    
+{'loss': '0.9334', 'grad_norm': '1.075', 'learning_rate': '0.0001975', 'epoch': '0.149', 'num_input_tokens_seen': 1786808, 'train_runtime': '1662', 'train_tokens_per_second': '1075'}
+
+ 15%|█▍        | 270/1813 [27:42<2:28:23,  5.77s/it]
+ 15%|█▍        | 271/1813 [27:50<2:44:56,  6.42s/it]
+ 15%|█▌        | 272/1813 [27:55<2:37:04,  6.12s/it]
+ 15%|█▌        | 273/1813 [28:04<2:54:23,  6.79s/it]
+ 15%|█▌        | 274/1813 [28:10<2:53:35,  6.77s/it]
+ 15%|█▌        | 275/1813 [28:16<2:41:43,  6.31s/it]
+ 15%|█▌        | 276/1813 [28:21<2:32:41,  5.96s/it]
+ 15%|█▌        | 277/1813 [28:27<2:34:18,  6.03s/it]
+ 15%|█▌        | 278/1813 [28:33<2:34:18,  6.03s/it]
+ 15%|█▌        | 279/1813 [28:41<2:46:22,  6.51s/it]
+ 15%|█▌        | 280/1813 [28:47<2:45:37,  6.48s/it]
+                                                    
+{'loss': '1.018', 'grad_norm': '0.9419', 'learning_rate': '0.000197', 'epoch': '0.1545', 'num_input_tokens_seen': 1858112, 'train_runtime': '1728', 'train_tokens_per_second': '1076'}
+
+ 15%|█▌        | 280/1813 [28:47<2:45:37,  6.48s/it]
+ 15%|█▌        | 281/1813 [28:53<2:39:07,  6.23s/it]
+ 16%|█▌        | 282/1813 [28:58<2:34:19,  6.05s/it]
+ 16%|█▌        | 283/1813 [29:04<2:29:00,  5.84s/it]
+ 16%|█▌        | 284/1813 [29:09<2:26:49,  5.76s/it]
+ 16%|█▌        | 285/1813 [29:16<2:34:39,  6.07s/it]
+ 16%|█▌        | 286/1813 [29:22<2:34:30,  6.07s/it]
+ 16%|█▌        | 287/1813 [29:28<2:35:57,  6.13s/it]
+ 16%|█▌        | 288/1813 [29:34<2:31:47,  5.97s/it]
+ 16%|█▌        | 289/1813 [29:41<2:36:35,  6.16s/it]
+ 16%|█▌        | 290/1813 [29:47<2:40:42,  6.33s/it]
+                                                    
+{'loss': '0.9688', 'grad_norm': '0.933', 'learning_rate': '0.0001966', 'epoch': '0.16', 'num_input_tokens_seen': 1920600, 'train_runtime': '1788', 'train_tokens_per_second': '1074'}
+
+ 16%|█▌        | 290/1813 [29:47<2:40:42,  6.33s/it]
+ 16%|█▌        | 291/1813 [29:54<2:46:58,  6.58s/it]
+ 16%|█▌        | 292/1813 [30:02<2:53:00,  6.82s/it]
+ 16%|█▌        | 293/1813 [30:08<2:45:55,  6.55s/it]
+ 16%|█▌        | 294/1813 [30:15<2:50:16,  6.73s/it]
+ 16%|█▋        | 295/1813 [30:22<2:55:11,  6.92s/it]
+ 16%|█▋        | 296/1813 [30:28<2:45:51,  6.56s/it]
+ 16%|█▋        | 297/1813 [30:34<2:41:42,  6.40s/it]
+ 16%|█▋        | 298/1813 [30:39<2:30:32,  5.96s/it]
+ 16%|█▋        | 299/1813 [30:45<2:31:47,  6.02s/it]
+ 17%|█▋        | 300/1813 [30:51<2:31:21,  6.00s/it]
+                                                    
+{'loss': '1.039', 'grad_norm': '0.9525', 'learning_rate': '0.0001961', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312, 'train_runtime': '1852', 'train_tokens_per_second': '1074'}
+
+ 17%|█▋        | 300/1813 [30:51<2:31:21,  6.00s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.81it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.09it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.01it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 31/1000 [00:04<03:00,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:05<02:04,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.10it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.32it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.37it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.76it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.92it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.94it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.75it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.30it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.27it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.13it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.85it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.20it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:56,  7.73it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.34it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.47it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.22it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.05it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.54it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.81it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.24it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.16it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.95it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.06it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.92it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.05it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.43it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.98it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.77it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.15it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.52it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.12it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:01,  6.64it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 193/1000 [00:25<01:54,  7.02it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:14,  6.00it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:44,  7.72it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:25,  9.37it/s]
+
+ 20%|██        | 200/1000 [00:26<01:12, 10.96it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.79it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.65it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.99it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.53it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.69it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.73it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.49it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.81it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.96it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:25,  8.88it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:31,  8.30it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.00it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:19,  5.44it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.63it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.22it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.84it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.74it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.05it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.42it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.73it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.75it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.07it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.94it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.78it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.80it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.27it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.57it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.05it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.81it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.01it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.87it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.20it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.40it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.54it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:08,  9.39it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.17it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.21it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.78it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.21it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.18it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.15it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.65it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.37it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.78it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.22it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.23it/s]
+
+ 41%|████      | 406/1000 [00:54<01:10,  8.48it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.82it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.43it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.94it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:16,  7.52it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.83it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.54it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.00it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:48, 11.76it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.48it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.07it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.27it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.84it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.20it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.36it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:25,  6.17it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.18it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.29it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:15,  6.86it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:21,  6.37it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:05,  7.92it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:11,  7.25it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:22,  6.25it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:22,  6.24it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.96it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.60it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.19it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.97it/s]
+
+ 50%|████▉     | 496/1000 [01:06<01:00,  8.39it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.42it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.55it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:55,  8.79it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:49,  9.83it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:07,  7.23it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.49it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.06it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.04it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.36it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.58it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.13it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:47,  9.75it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.49it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.91it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.76it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.85it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.04it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.36it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.58it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.90it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.38it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:50,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.46it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.72it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.79it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.61it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.74it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.09it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.02it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.19it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.45it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.57it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:52,  7.15it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.19it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.82it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:39,  9.40it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.94it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.16it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.94it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.58it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.09it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.57it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.50it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.91it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.57it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.49it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.65it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.99it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.92it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.89it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.49it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.57it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.17it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.38it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  6.99it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.60it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.49it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.68it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.83it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.34it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.04it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.79it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:24, 11.04it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:33,  8.14it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.37it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:27,  9.80it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.43it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.69it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.27it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.33it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.58it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.41it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.62it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.66it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.69it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.64it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.59it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.46it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.78it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.79it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.94it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.99it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.84it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.18it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.19it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.21it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.06it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.32it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.13it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  7.87it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.73it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.67it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.93it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.73it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:33,  5.21it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.71it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  6.99it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 833/1000 [01:53<00:26,  6.22it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.00it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.52it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.08it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.43it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.61it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.14it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.46it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.03it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.51it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.75it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.61it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.68it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.30it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.26it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.26it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.56it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:14,  8.77it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.84it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.57it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.52it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.93it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.15it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.34it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.78it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.77it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.84it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.71it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.93it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.67it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.07it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.38it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:08<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.03it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.44it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.04it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.22it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.81it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.44it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.07it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.37it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+                                                    
+
+                                                   
+{'eval_loss': '0.8436', 'eval_runtime': '135.6', 'eval_samples_per_second': '7.377', 'eval_steps_per_second': '7.377', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312}
+
+ 17%|█▋        | 300/1813 [33:07<2:31:21,  6.00s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+
+                                                   
+ 17%|█▋        | 301/1813 [33:12<19:31:38, 46.49s/it]
+ 17%|█▋        | 302/1813 [33:17<14:16:47, 34.02s/it]
+ 17%|█▋        | 303/1813 [33:24<10:51:23, 25.88s/it]
+ 17%|█▋        | 304/1813 [33:32<8:33:20, 20.41s/it] 
+ 17%|█▋        | 305/1813 [33:38<6:47:00, 16.19s/it]
+ 17%|█▋        | 306/1813 [33:44<5:30:29, 13.16s/it]
+ 17%|█▋        | 307/1813 [33:50<4:34:20, 10.93s/it]
+ 17%|█▋        | 308/1813 [33:55<3:54:17,  9.34s/it]
+ 17%|█▋        | 309/1813 [34:02<3:32:45,  8.49s/it]
+ 17%|█▋        | 310/1813 [34:07<3:07:45,  7.50s/it]
+                                                    
+{'loss': '0.9655', 'grad_norm': '1.183', 'learning_rate': '0.0001955', 'epoch': '0.171', 'num_input_tokens_seen': 2050672, 'train_runtime': '2047', 'train_tokens_per_second': '1002'}
+
+ 17%|█▋        | 310/1813 [34:07<3:07:45,  7.50s/it]
+ 17%|█▋        | 311/1813 [34:14<3:05:34,  7.41s/it]
+ 17%|█▋        | 312/1813 [34:22<3:06:09,  7.44s/it]
+ 17%|█▋        | 313/1813 [34:29<3:01:22,  7.26s/it]
+ 17%|█▋        | 314/1813 [34:34<2:48:01,  6.73s/it]
+ 17%|█▋        | 315/1813 [34:41<2:48:27,  6.75s/it]
+ 17%|█▋        | 316/1813 [34:47<2:44:45,  6.60s/it]
+ 17%|█▋        | 317/1813 [34:53<2:38:04,  6.34s/it]
+ 18%|█▊        | 318/1813 [34:58<2:30:22,  6.04s/it]
+ 18%|█▊        | 319/1813 [35:03<2:20:27,  5.64s/it]
+ 18%|█▊        | 320/1813 [35:09<2:21:23,  5.68s/it]
+                                                    
+{'loss': '1.047', 'grad_norm': '0.6493', 'learning_rate': '0.0001949', 'epoch': '0.1766', 'num_input_tokens_seen': 2118000, 'train_runtime': '2109', 'train_tokens_per_second': '1004'}
+
+ 18%|█▊        | 320/1813 [35:09<2:21:23,  5.68s/it]
+ 18%|█▊        | 321/1813 [35:14<2:22:20,  5.72s/it]
+ 18%|█▊        | 322/1813 [35:20<2:22:29,  5.73s/it]
+ 18%|█▊        | 323/1813 [35:26<2:26:05,  5.88s/it]
+ 18%|█▊        | 324/1813 [35:34<2:35:18,  6.26s/it]
+ 18%|█▊        | 325/1813 [35:41<2:41:16,  6.50s/it]
+ 18%|█▊        | 326/1813 [35:47<2:38:49,  6.41s/it]
+ 18%|█▊        | 327/1813 [35:53<2:34:59,  6.26s/it]
+ 18%|█▊        | 328/1813 [35:59<2:38:12,  6.39s/it]
+ 18%|█▊        | 329/1813 [36:05<2:35:09,  6.27s/it]
+ 18%|█▊        | 330/1813 [36:11<2:31:47,  6.14s/it]
+                                                    
+{'loss': '0.9619', 'grad_norm': '0.995', 'learning_rate': '0.0001943', 'epoch': '0.1821', 'num_input_tokens_seen': 2184640, 'train_runtime': '2172', 'train_tokens_per_second': '1006'}
+
+ 18%|█▊        | 330/1813 [36:11<2:31:47,  6.14s/it]
+ 18%|█▊        | 331/1813 [36:18<2:34:05,  6.24s/it]
+ 18%|█▊        | 332/1813 [36:23<2:27:01,  5.96s/it]
+ 18%|█▊        | 333/1813 [36:29<2:27:53,  6.00s/it]
+ 18%|█▊        | 334/1813 [36:35<2:26:11,  5.93s/it]
+ 18%|█▊        | 335/1813 [36:43<2:39:57,  6.49s/it]
+ 19%|█▊        | 336/1813 [36:50<2:44:21,  6.68s/it]
+ 19%|█▊        | 337/1813 [36:55<2:36:05,  6.35s/it]
+ 19%|█▊        | 338/1813 [37:01<2:34:10,  6.27s/it]
+ 19%|█▊        | 339/1813 [37:07<2:28:00,  6.02s/it]
+ 19%|█▉        | 340/1813 [37:14<2:34:53,  6.31s/it]
+                                                    
+{'loss': '0.9429', 'grad_norm': '1.139', 'learning_rate': '0.0001937', 'epoch': '0.1876', 'num_input_tokens_seen': 2251392, 'train_runtime': '2234', 'train_tokens_per_second': '1008'}
+
+ 19%|█▉        | 340/1813 [37:14<2:34:53,  6.31s/it]
+ 19%|█▉        | 341/1813 [37:20<2:34:15,  6.29s/it]
+ 19%|█▉        | 342/1813 [37:28<2:42:05,  6.61s/it]
+ 19%|█▉        | 343/1813 [37:33<2:34:45,  6.32s/it]
+ 19%|█▉        | 344/1813 [37:39<2:29:06,  6.09s/it]
+ 19%|█▉        | 345/1813 [37:46<2:38:00,  6.46s/it]
+ 19%|█▉        | 346/1813 [37:53<2:38:06,  6.47s/it]
+ 19%|█▉        | 347/1813 [37:58<2:28:01,  6.06s/it]
+ 19%|█▉        | 348/1813 [38:04<2:32:11,  6.23s/it]
+ 19%|█▉        | 349/1813 [38:13<2:48:21,  6.90s/it]
+ 19%|█▉        | 350/1813 [38:21<2:54:58,  7.18s/it]
+                                                    
+{'loss': '1.059', 'grad_norm': '0.858', 'learning_rate': '0.000193', 'epoch': '0.1931', 'num_input_tokens_seen': 2324712, 'train_runtime': '2301', 'train_tokens_per_second': '1010'}
+
+ 19%|█▉        | 350/1813 [38:21<2:54:58,  7.18s/it]
+ 19%|█▉        | 351/1813 [38:28<2:53:56,  7.14s/it]
+ 19%|█▉        | 352/1813 [38:34<2:45:44,  6.81s/it]
+ 19%|█▉        | 353/1813 [38:39<2:37:19,  6.47s/it]
+ 20%|█▉        | 354/1813 [38:45<2:32:29,  6.27s/it]
+ 20%|█▉        | 355/1813 [38:51<2:31:04,  6.22s/it]
+ 20%|█▉        | 356/1813 [38:58<2:36:25,  6.44s/it]
+ 20%|█▉        | 357/1813 [39:04<2:32:48,  6.30s/it]
+ 20%|█▉        | 358/1813 [39:11<2:33:58,  6.35s/it]
+ 20%|█▉        | 359/1813 [39:17<2:30:50,  6.22s/it]
+ 20%|█▉        | 360/1813 [39:24<2:36:36,  6.47s/it]
+                                                    
+{'loss': '1.002', 'grad_norm': '0.8958', 'learning_rate': '0.0001923', 'epoch': '0.1986', 'num_input_tokens_seen': 2390776, 'train_runtime': '2364', 'train_tokens_per_second': '1011'}
+
+ 20%|█▉        | 360/1813 [39:24<2:36:36,  6.47s/it]
+ 20%|█▉        | 361/1813 [39:29<2:28:32,  6.14s/it]
+ 20%|█▉        | 362/1813 [39:35<2:25:38,  6.02s/it]
+ 20%|██        | 363/1813 [39:42<2:33:52,  6.37s/it]
+ 20%|██        | 364/1813 [39:49<2:36:43,  6.49s/it]
+ 20%|██        | 365/1813 [39:55<2:33:46,  6.37s/it]
+ 20%|██        | 366/1813 [40:02<2:38:04,  6.55s/it]
+ 20%|██        | 367/1813 [40:08<2:34:03,  6.39s/it]
+ 20%|██        | 368/1813 [40:13<2:27:38,  6.13s/it]
+ 20%|██        | 369/1813 [40:18<2:18:34,  5.76s/it]
+ 20%|██        | 370/1813 [40:24<2:21:15,  5.87s/it]
+                                                    
+{'loss': '0.9964', 'grad_norm': '1.142', 'learning_rate': '0.0001916', 'epoch': '0.2041', 'num_input_tokens_seen': 2454808, 'train_runtime': '2425', 'train_tokens_per_second': '1012'}
+
+ 20%|██        | 370/1813 [40:24<2:21:15,  5.87s/it]
+ 20%|██        | 371/1813 [40:30<2:16:49,  5.69s/it]
+ 21%|██        | 372/1813 [40:38<2:35:06,  6.46s/it]
+ 21%|██        | 373/1813 [40:44<2:34:36,  6.44s/it]
+ 21%|██        | 374/1813 [40:52<2:41:23,  6.73s/it]
+ 21%|██        | 375/1813 [40:59<2:42:49,  6.79s/it]
+ 21%|██        | 376/1813 [41:04<2:32:41,  6.38s/it]
+ 21%|██        | 377/1813 [41:10<2:30:17,  6.28s/it]
+ 21%|██        | 378/1813 [41:16<2:29:30,  6.25s/it]
+ 21%|██        | 379/1813 [41:22<2:23:38,  6.01s/it]
+ 21%|██        | 380/1813 [41:27<2:16:44,  5.73s/it]
+                                                    
+{'loss': '0.9651', 'grad_norm': '1.171', 'learning_rate': '0.0001908', 'epoch': '0.2097', 'num_input_tokens_seen': 2520472, 'train_runtime': '2487', 'train_tokens_per_second': '1013'}
+
+ 21%|██        | 380/1813 [41:27<2:16:44,  5.73s/it]
+ 21%|██        | 381/1813 [41:34<2:26:00,  6.12s/it]
+ 21%|██        | 382/1813 [41:41<2:31:58,  6.37s/it]
+ 21%|██        | 383/1813 [41:47<2:30:47,  6.33s/it]
+ 21%|██        | 384/1813 [41:53<2:29:58,  6.30s/it]
+ 21%|██        | 385/1813 [41:59<2:29:47,  6.29s/it]
+ 21%|██▏       | 386/1813 [42:04<2:18:44,  5.83s/it]
+ 21%|██▏       | 387/1813 [42:10<2:21:18,  5.95s/it]
+ 21%|██▏       | 388/1813 [42:18<2:30:08,  6.32s/it]
+ 21%|██▏       | 389/1813 [42:25<2:41:02,  6.79s/it]
+ 22%|██▏       | 390/1813 [42:32<2:40:49,  6.78s/it]
+                                                    
+{'loss': '1.001', 'grad_norm': '0.851', 'learning_rate': '0.00019', 'epoch': '0.2152', 'num_input_tokens_seen': 2591512, 'train_runtime': '2553', 'train_tokens_per_second': '1015'}
+
+ 22%|██▏       | 390/1813 [42:32<2:40:49,  6.78s/it]
+ 22%|██▏       | 391/1813 [42:38<2:36:06,  6.59s/it]
+ 22%|██▏       | 392/1813 [42:45<2:34:45,  6.53s/it]
+ 22%|██▏       | 393/1813 [42:51<2:35:12,  6.56s/it]
+ 22%|██▏       | 394/1813 [42:56<2:23:56,  6.09s/it]
+ 22%|██▏       | 395/1813 [43:02<2:20:33,  5.95s/it]
+ 22%|██▏       | 396/1813 [43:08<2:24:22,  6.11s/it]
+ 22%|██▏       | 397/1813 [43:13<2:16:01,  5.76s/it]
+ 22%|██▏       | 398/1813 [43:20<2:19:08,  5.90s/it]
+ 22%|██▏       | 399/1813 [43:27<2:31:14,  6.42s/it]
+ 22%|██▏       | 400/1813 [43:36<2:43:51,  6.96s/it]
+                                                    
+{'loss': '0.9838', 'grad_norm': '0.9736', 'learning_rate': '0.0001891', 'epoch': '0.2207', 'num_input_tokens_seen': 2660336, 'train_runtime': '2616', 'train_tokens_per_second': '1017'}
+
+ 22%|██▏       | 400/1813 [43:36<2:43:51,  6.96s/it]
+ 22%|██▏       | 401/1813 [43:43<2:46:08,  7.06s/it]
+ 22%|██▏       | 402/1813 [43:48<2:30:23,  6.39s/it]
+ 22%|██▏       | 403/1813 [43:54<2:27:37,  6.28s/it]
+ 22%|██▏       | 404/1813 [44:01<2:31:44,  6.46s/it]
+ 22%|██▏       | 405/1813 [44:08<2:35:59,  6.65s/it]
+ 22%|██▏       | 406/1813 [44:12<2:22:51,  6.09s/it]
+ 22%|██▏       | 407/1813 [44:17<2:10:16,  5.56s/it]
+ 23%|██▎       | 408/1813 [44:22<2:11:35,  5.62s/it]
+ 23%|██▎       | 409/1813 [44:29<2:15:48,  5.80s/it]
+ 23%|██▎       | 410/1813 [44:34<2:14:09,  5.74s/it]
+                                                    
+{'loss': '1.012', 'grad_norm': '1.155', 'learning_rate': '0.0001883', 'epoch': '0.2262', 'num_input_tokens_seen': 2722056, 'train_runtime': '2675', 'train_tokens_per_second': '1018'}
+
+ 23%|██▎       | 410/1813 [44:34<2:14:09,  5.74s/it]
+ 23%|██▎       | 411/1813 [44:41<2:18:13,  5.92s/it]
+ 23%|██▎       | 412/1813 [44:47<2:22:28,  6.10s/it]
+ 23%|██▎       | 413/1813 [44:55<2:30:58,  6.47s/it]
+ 23%|██▎       | 414/1813 [45:00<2:23:59,  6.18s/it]
+ 23%|██▎       | 415/1813 [45:06<2:25:34,  6.25s/it]
+ 23%|██▎       | 416/1813 [45:13<2:27:57,  6.35s/it]
+ 23%|██▎       | 417/1813 [45:21<2:35:58,  6.70s/it]
+ 23%|██▎       | 418/1813 [45:26<2:29:04,  6.41s/it]
+ 23%|██▎       | 419/1813 [45:33<2:29:32,  6.44s/it]
+ 23%|██▎       | 420/1813 [45:39<2:25:09,  6.25s/it]
+                                                    
+{'loss': '1.003', 'grad_norm': '0.8105', 'learning_rate': '0.0001874', 'epoch': '0.2317', 'num_input_tokens_seen': 2790960, 'train_runtime': '2739', 'train_tokens_per_second': '1019'}
+
+ 23%|██▎       | 420/1813 [45:39<2:25:09,  6.25s/it]
+ 23%|██▎       | 421/1813 [45:45<2:25:54,  6.29s/it]
+ 23%|██▎       | 422/1813 [45:52<2:33:01,  6.60s/it]
+ 23%|██▎       | 423/1813 [45:58<2:29:28,  6.45s/it]
+ 23%|██▎       | 424/1813 [46:05<2:27:12,  6.36s/it]
+ 23%|██▎       | 425/1813 [46:10<2:20:56,  6.09s/it]
+ 23%|██▎       | 426/1813 [46:17<2:28:39,  6.43s/it]
+ 24%|██▎       | 427/1813 [46:23<2:24:45,  6.27s/it]
+ 24%|██▎       | 428/1813 [46:29<2:24:02,  6.24s/it]
+ 24%|██▎       | 429/1813 [46:35<2:17:15,  5.95s/it]
+ 24%|██▎       | 430/1813 [46:42<2:27:37,  6.40s/it]
+                                                    
+{'loss': '0.9886', 'grad_norm': '0.8128', 'learning_rate': '0.0001864', 'epoch': '0.2372', 'num_input_tokens_seen': 2860096, 'train_runtime': '2803', 'train_tokens_per_second': '1021'}
+
+ 24%|██▎       | 430/1813 [46:42<2:27:37,  6.40s/it]
+ 24%|██▍       | 431/1813 [46:48<2:24:43,  6.28s/it]
+ 24%|██▍       | 432/1813 [46:54<2:21:05,  6.13s/it]
+ 24%|██▍       | 433/1813 [46:59<2:15:31,  5.89s/it]
+ 24%|██▍       | 434/1813 [47:06<2:19:29,  6.07s/it]
+ 24%|██▍       | 435/1813 [47:12<2:18:06,  6.01s/it]
+ 24%|██▍       | 436/1813 [47:18<2:20:30,  6.12s/it]
+ 24%|██▍       | 437/1813 [47:24<2:17:34,  6.00s/it]
+ 24%|██▍       | 438/1813 [47:30<2:19:46,  6.10s/it]
+ 24%|██▍       | 439/1813 [47:36<2:20:00,  6.11s/it]
+ 24%|██▍       | 440/1813 [47:43<2:22:07,  6.21s/it]
+                                                    
+{'loss': '0.9073', 'grad_norm': '1.271', 'learning_rate': '0.0001855', 'epoch': '0.2428', 'num_input_tokens_seen': 2924856, 'train_runtime': '2863', 'train_tokens_per_second': '1022'}
+
+ 24%|██▍       | 440/1813 [47:43<2:22:07,  6.21s/it]
+ 24%|██▍       | 441/1813 [47:49<2:20:35,  6.15s/it]
+ 24%|██▍       | 442/1813 [47:55<2:20:55,  6.17s/it]
+ 24%|██▍       | 443/1813 [48:02<2:25:15,  6.36s/it]
+ 24%|██▍       | 444/1813 [48:07<2:20:00,  6.14s/it]
+ 25%|██▍       | 445/1813 [48:14<2:27:59,  6.49s/it]
+ 25%|██▍       | 446/1813 [48:21<2:28:15,  6.51s/it]
+ 25%|██▍       | 447/1813 [48:27<2:26:33,  6.44s/it]
+ 25%|██▍       | 448/1813 [48:34<2:26:22,  6.43s/it]
+ 25%|██▍       | 449/1813 [48:39<2:19:35,  6.14s/it]
+ 25%|██▍       | 450/1813 [48:45<2:16:04,  5.99s/it]
+                                                    
+{'loss': '1.023', 'grad_norm': '1.042', 'learning_rate': '0.0001845', 'epoch': '0.2483', 'num_input_tokens_seen': 2990040, 'train_runtime': '2925', 'train_tokens_per_second': '1022'}
+
+ 25%|██▍       | 450/1813 [48:45<2:16:04,  5.99s/it]
+ 25%|██▍       | 451/1813 [48:52<2:26:21,  6.45s/it]
+ 25%|██▍       | 452/1813 [48:58<2:21:12,  6.22s/it]
+ 25%|██▍       | 453/1813 [49:03<2:12:53,  5.86s/it]
+ 25%|██▌       | 454/1813 [49:10<2:20:04,  6.18s/it]
+ 25%|██▌       | 455/1813 [49:17<2:22:40,  6.30s/it]
+ 25%|██▌       | 456/1813 [49:21<2:11:48,  5.83s/it]
+ 25%|██▌       | 457/1813 [49:27<2:14:03,  5.93s/it]
+ 25%|██▌       | 458/1813 [49:35<2:27:33,  6.53s/it]
+ 25%|██▌       | 459/1813 [49:42<2:29:39,  6.63s/it]
+ 25%|██▌       | 460/1813 [49:49<2:27:56,  6.56s/it]
+                                                    
+{'loss': '1.088', 'grad_norm': '0.9134', 'learning_rate': '0.0001834', 'epoch': '0.2538', 'num_input_tokens_seen': 3058848, 'train_runtime': '2989', 'train_tokens_per_second': '1023'}
+
+ 25%|██▌       | 460/1813 [49:49<2:27:56,  6.56s/it]
+ 25%|██▌       | 461/1813 [49:55<2:23:45,  6.38s/it]
+ 25%|██▌       | 462/1813 [50:02<2:30:37,  6.69s/it]
+ 26%|██▌       | 463/1813 [50:08<2:25:04,  6.45s/it]
+ 26%|██▌       | 464/1813 [50:13<2:15:20,  6.02s/it]
+ 26%|██▌       | 465/1813 [50:19<2:12:19,  5.89s/it]
+ 26%|██▌       | 466/1813 [50:25<2:18:12,  6.16s/it]
+ 26%|██▌       | 467/1813 [50:31<2:17:04,  6.11s/it]
+ 26%|██▌       | 468/1813 [50:38<2:20:24,  6.26s/it]
+ 26%|██▌       | 469/1813 [50:44<2:21:49,  6.33s/it]
+ 26%|██▌       | 470/1813 [50:51<2:23:09,  6.40s/it]
+                                                    
+{'loss': '0.9592', 'grad_norm': '0.7299', 'learning_rate': '0.0001824', 'epoch': '0.2593', 'num_input_tokens_seen': 3126680, 'train_runtime': '3051', 'train_tokens_per_second': '1025'}
+
+ 26%|██▌       | 470/1813 [50:51<2:23:09,  6.40s/it]
+ 26%|██▌       | 471/1813 [50:57<2:18:18,  6.18s/it]
+ 26%|██▌       | 472/1813 [51:03<2:20:38,  6.29s/it]
+ 26%|██▌       | 473/1813 [51:10<2:24:29,  6.47s/it]
+ 26%|██▌       | 474/1813 [51:16<2:22:42,  6.40s/it]
+ 26%|██▌       | 475/1813 [51:22<2:16:00,  6.10s/it]
+ 26%|██▋       | 476/1813 [51:28<2:16:31,  6.13s/it]
+ 26%|██▋       | 477/1813 [51:35<2:20:15,  6.30s/it]
+ 26%|██▋       | 478/1813 [51:42<2:25:58,  6.56s/it]
+ 26%|██▋       | 479/1813 [51:48<2:20:47,  6.33s/it]
+ 26%|██▋       | 480/1813 [51:54<2:23:36,  6.46s/it]
+                                                    
+{'loss': '1.067', 'grad_norm': '1.134', 'learning_rate': '0.0001813', 'epoch': '0.2648', 'num_input_tokens_seen': 3193704, 'train_runtime': '3115', 'train_tokens_per_second': '1025'}
+
+ 26%|██▋       | 480/1813 [51:54<2:23:36,  6.46s/it]
+ 27%|██▋       | 481/1813 [52:01<2:24:13,  6.50s/it]
+ 27%|██▋       | 482/1813 [52:08<2:27:29,  6.65s/it]
+ 27%|██▋       | 483/1813 [52:16<2:35:05,  7.00s/it]
+ 27%|██▋       | 484/1813 [52:22<2:27:08,  6.64s/it]
+ 27%|██▋       | 485/1813 [52:29<2:34:09,  6.96s/it]
+ 27%|██▋       | 486/1813 [52:36<2:29:32,  6.76s/it]
+ 27%|██▋       | 487/1813 [52:43<2:31:32,  6.86s/it]
+ 27%|██▋       | 488/1813 [52:49<2:25:17,  6.58s/it]
+ 27%|██▋       | 489/1813 [52:56<2:28:46,  6.74s/it]
+ 27%|██▋       | 490/1813 [53:02<2:26:02,  6.62s/it]
+                                                    
+{'loss': '1.012', 'grad_norm': '1.279', 'learning_rate': '0.0001802', 'epoch': '0.2703', 'num_input_tokens_seen': 3268064, 'train_runtime': '3183', 'train_tokens_per_second': '1027'}
+
+ 27%|██▋       | 490/1813 [53:02<2:26:02,  6.62s/it]
+ 27%|██▋       | 491/1813 [53:09<2:28:00,  6.72s/it]
+ 27%|██▋       | 492/1813 [53:17<2:34:06,  7.00s/it]
+ 27%|██▋       | 493/1813 [53:22<2:24:47,  6.58s/it]
+ 27%|██▋       | 494/1813 [53:28<2:19:37,  6.35s/it]
+ 27%|██▋       | 495/1813 [53:35<2:25:27,  6.62s/it]
+ 27%|██▋       | 496/1813 [53:42<2:23:56,  6.56s/it]
+ 27%|██▋       | 497/1813 [53:47<2:17:21,  6.26s/it]
+ 27%|██▋       | 498/1813 [53:54<2:23:03,  6.53s/it]
+ 28%|██▊       | 499/1813 [54:01<2:20:30,  6.42s/it]
+ 28%|██▊       | 500/1813 [54:10<2:38:18,  7.23s/it]
+                                                    
+{'loss': '1.085', 'grad_norm': '0.8142', 'learning_rate': '0.000179', 'epoch': '0.2759', 'num_input_tokens_seen': 3341536, 'train_runtime': '3250', 'train_tokens_per_second': '1028'}
+
+ 28%|██▊       | 500/1813 [54:10<2:38:18,  7.23s/it]
+ 28%|██▊       | 501/1813 [54:16<2:32:23,  6.97s/it]
+ 28%|██▊       | 502/1813 [54:22<2:22:48,  6.54s/it]
+ 28%|██▊       | 503/1813 [54:27<2:12:20,  6.06s/it]
+ 28%|██▊       | 504/1813 [54:33<2:11:58,  6.05s/it]
+ 28%|██▊       | 505/1813 [54:40<2:18:55,  6.37s/it]
+ 28%|██▊       | 506/1813 [54:46<2:15:58,  6.24s/it]
+ 28%|██▊       | 507/1813 [54:53<2:22:28,  6.55s/it]
+ 28%|██▊       | 508/1813 [54:58<2:13:00,  6.12s/it]
+ 28%|██▊       | 509/1813 [55:03<2:07:03,  5.85s/it]
+ 28%|██▊       | 510/1813 [55:09<2:04:07,  5.72s/it]
+                                                    
+{'loss': '0.9232', 'grad_norm': '1.323', 'learning_rate': '0.0001779', 'epoch': '0.2814', 'num_input_tokens_seen': 3402712, 'train_runtime': '3309', 'train_tokens_per_second': '1028'}
+
+ 28%|██▊       | 510/1813 [55:09<2:04:07,  5.72s/it]
+ 28%|██▊       | 511/1813 [55:14<2:01:53,  5.62s/it]
+ 28%|██▊       | 512/1813 [55:21<2:09:49,  5.99s/it]
+ 28%|██▊       | 513/1813 [55:28<2:18:21,  6.39s/it]
+ 28%|██▊       | 514/1813 [55:35<2:21:45,  6.55s/it]
+ 28%|██▊       | 515/1813 [55:40<2:13:59,  6.19s/it]
+ 28%|██▊       | 516/1813 [55:46<2:11:41,  6.09s/it]
+ 29%|██▊       | 517/1813 [55:53<2:15:12,  6.26s/it]
+ 29%|██▊       | 518/1813 [55:59<2:14:41,  6.24s/it]
+ 29%|██▊       | 519/1813 [56:05<2:13:39,  6.20s/it]
+ 29%|██▊       | 520/1813 [56:12<2:16:59,  6.36s/it]
+                                                    
+{'loss': '0.9762', 'grad_norm': '1.003', 'learning_rate': '0.0001767', 'epoch': '0.2869', 'num_input_tokens_seen': 3469624, 'train_runtime': '3372', 'train_tokens_per_second': '1029'}
+
+ 29%|██▊       | 520/1813 [56:12<2:16:59,  6.36s/it]
+ 29%|██▊       | 521/1813 [56:16<2:01:45,  5.65s/it]
+ 29%|██▉       | 522/1813 [56:22<2:03:40,  5.75s/it]
+ 29%|██▉       | 523/1813 [56:28<2:07:39,  5.94s/it]
+ 29%|██▉       | 524/1813 [56:35<2:10:23,  6.07s/it]
+ 29%|██▉       | 525/1813 [56:40<2:03:27,  5.75s/it]
+ 29%|██▉       | 526/1813 [56:46<2:09:40,  6.05s/it]
+ 29%|██▉       | 527/1813 [56:53<2:14:30,  6.28s/it]
+ 29%|██▉       | 528/1813 [57:01<2:20:28,  6.56s/it]
+ 29%|██▉       | 529/1813 [57:08<2:24:18,  6.74s/it]
+ 29%|██▉       | 530/1813 [57:14<2:19:58,  6.55s/it]
+                                                    
+{'loss': '0.9689', 'grad_norm': '1.2', 'learning_rate': '0.0001754', 'epoch': '0.2924', 'num_input_tokens_seen': 3534904, 'train_runtime': '3434', 'train_tokens_per_second': '1029'}
+
+ 29%|██▉       | 530/1813 [57:14<2:19:58,  6.55s/it]
+ 29%|██▉       | 531/1813 [57:22<2:30:46,  7.06s/it]
+ 29%|██▉       | 532/1813 [57:28<2:25:40,  6.82s/it]
+ 29%|██▉       | 533/1813 [57:36<2:28:04,  6.94s/it]
+ 29%|██▉       | 534/1813 [57:43<2:29:46,  7.03s/it]
+ 30%|██▉       | 535/1813 [57:47<2:12:55,  6.24s/it]
+ 30%|██▉       | 536/1813 [57:52<2:05:40,  5.91s/it]
+ 30%|██▉       | 537/1813 [57:59<2:09:51,  6.11s/it]
+ 30%|██▉       | 538/1813 [58:04<2:05:13,  5.89s/it]
+ 30%|██▉       | 539/1813 [58:10<2:06:45,  5.97s/it]
+ 30%|██▉       | 540/1813 [58:16<2:06:53,  5.98s/it]
+                                                    
+{'loss': '0.9774', 'grad_norm': '0.8625', 'learning_rate': '0.0001742', 'epoch': '0.2979', 'num_input_tokens_seen': 3603488, 'train_runtime': '3497', 'train_tokens_per_second': '1030'}
+
+ 30%|██▉       | 540/1813 [58:16<2:06:53,  5.98s/it]
+ 30%|██▉       | 541/1813 [58:21<2:01:07,  5.71s/it]
+ 30%|██▉       | 542/1813 [58:27<1:59:31,  5.64s/it]
+ 30%|██▉       | 543/1813 [58:34<2:09:27,  6.12s/it]
+ 30%|███       | 544/1813 [58:43<2:23:39,  6.79s/it]
+ 30%|███       | 545/1813 [58:49<2:22:40,  6.75s/it]
+ 30%|███       | 546/1813 [58:56<2:21:51,  6.72s/it]
+ 30%|███       | 547/1813 [59:04<2:28:40,  7.05s/it]
+ 30%|███       | 548/1813 [59:11<2:32:13,  7.22s/it]
+ 30%|███       | 549/1813 [59:17<2:23:27,  6.81s/it]
+ 30%|███       | 550/1813 [59:23<2:17:01,  6.51s/it]
+                                                    
+{'loss': '0.9111', 'grad_norm': '1.151', 'learning_rate': '0.0001729', 'epoch': '0.3034', 'num_input_tokens_seen': 3676576, 'train_runtime': '3563', 'train_tokens_per_second': '1032'}
+
+ 30%|███       | 550/1813 [59:23<2:17:01,  6.51s/it]
+ 30%|███       | 551/1813 [59:29<2:14:09,  6.38s/it]
+ 30%|███       | 552/1813 [59:34<2:08:07,  6.10s/it]
+ 31%|███       | 553/1813 [59:39<2:00:07,  5.72s/it]
+ 31%|███       | 554/1813 [59:46<2:06:22,  6.02s/it]
+ 31%|███       | 555/1813 [59:51<1:59:22,  5.69s/it]
+ 31%|███       | 556/1813 [59:57<2:01:11,  5.78s/it]
+ 31%|███       | 557/1813 [1:00:04<2:07:19,  6.08s/it]
+ 31%|███       | 558/1813 [1:00:09<2:04:39,  5.96s/it]
+ 31%|███       | 559/1813 [1:00:15<1:59:25,  5.71s/it]
+ 31%|███       | 560/1813 [1:00:21<2:03:51,  5.93s/it]
+                                                      
+{'loss': '0.979', 'grad_norm': '0.739', 'learning_rate': '0.0001716', 'epoch': '0.309', 'num_input_tokens_seen': 3738056, 'train_runtime': '3621', 'train_tokens_per_second': '1032'}
+
+ 31%|███       | 560/1813 [1:00:21<2:03:51,  5.93s/it]
+ 31%|███       | 561/1813 [1:00:27<2:03:35,  5.92s/it]
+ 31%|███       | 562/1813 [1:00:34<2:07:52,  6.13s/it]
+ 31%|███       | 563/1813 [1:00:40<2:13:00,  6.38s/it]
+ 31%|███       | 564/1813 [1:00:46<2:10:06,  6.25s/it]
+ 31%|███       | 565/1813 [1:00:52<2:06:35,  6.09s/it]
+ 31%|███       | 566/1813 [1:00:59<2:09:51,  6.25s/it]
+ 31%|███▏      | 567/1813 [1:01:04<2:05:02,  6.02s/it]
+ 31%|███▏      | 568/1813 [1:01:11<2:10:55,  6.31s/it]
+ 31%|███▏      | 569/1813 [1:01:19<2:18:39,  6.69s/it]
+ 31%|███▏      | 570/1813 [1:01:26<2:22:43,  6.89s/it]
+                                                      
+{'loss': '0.9748', 'grad_norm': '0.7982', 'learning_rate': '0.0001703', 'epoch': '0.3145', 'num_input_tokens_seen': 3808208, 'train_runtime': '3687', 'train_tokens_per_second': '1033'}
+
+ 31%|███▏      | 570/1813 [1:01:26<2:22:43,  6.89s/it]
+ 31%|███▏      | 571/1813 [1:01:33<2:22:01,  6.86s/it]
+ 32%|███▏      | 572/1813 [1:01:39<2:17:55,  6.67s/it]
+ 32%|███▏      | 573/1813 [1:01:45<2:09:47,  6.28s/it]
+ 32%|███▏      | 574/1813 [1:01:50<2:07:29,  6.17s/it]
+ 32%|███▏      | 575/1813 [1:01:57<2:10:14,  6.31s/it]
+ 32%|███▏      | 576/1813 [1:02:02<2:00:51,  5.86s/it]
+ 32%|███▏      | 577/1813 [1:02:09<2:06:17,  6.13s/it]
+ 32%|███▏      | 578/1813 [1:02:15<2:06:44,  6.16s/it]
+ 32%|███▏      | 579/1813 [1:02:21<2:07:53,  6.22s/it]
+ 32%|███▏      | 580/1813 [1:02:28<2:08:25,  6.25s/it]
+                                                      
+{'loss': '0.9811', 'grad_norm': '0.9324', 'learning_rate': '0.0001689', 'epoch': '0.32', 'num_input_tokens_seen': 3872328, 'train_runtime': '3748', 'train_tokens_per_second': '1033'}
+
+ 32%|███▏      | 580/1813 [1:02:28<2:08:25,  6.25s/it]
+ 32%|███▏      | 581/1813 [1:02:33<2:05:45,  6.12s/it]
+ 32%|███▏      | 582/1813 [1:02:39<2:01:43,  5.93s/it]
+ 32%|███▏      | 583/1813 [1:02:45<2:00:15,  5.87s/it]
+ 32%|███▏      | 584/1813 [1:02:51<2:03:23,  6.02s/it]
+ 32%|███▏      | 585/1813 [1:02:57<2:02:48,  6.00s/it]
+ 32%|███▏      | 586/1813 [1:03:03<2:04:30,  6.09s/it]
+ 32%|███▏      | 587/1813 [1:03:09<2:02:57,  6.02s/it]
+ 32%|███▏      | 588/1813 [1:03:15<2:05:08,  6.13s/it]
+ 32%|███▏      | 589/1813 [1:03:22<2:07:10,  6.23s/it]
+ 33%|███▎      | 590/1813 [1:03:28<2:06:08,  6.19s/it]
+                                                      
+{'loss': '0.898', 'grad_norm': '1.095', 'learning_rate': '0.0001675', 'epoch': '0.3255', 'num_input_tokens_seen': 3936024, 'train_runtime': '3809', 'train_tokens_per_second': '1033'}
+
+ 33%|███▎      | 590/1813 [1:03:28<2:06:08,  6.19s/it]
+ 33%|███▎      | 591/1813 [1:03:33<2:01:05,  5.95s/it]
+ 33%|███▎      | 592/1813 [1:03:39<2:00:35,  5.93s/it]
+ 33%|███▎      | 593/1813 [1:03:46<2:04:03,  6.10s/it]
+ 33%|███▎      | 594/1813 [1:03:52<2:03:30,  6.08s/it]
+ 33%|███▎      | 595/1813 [1:03:57<1:59:58,  5.91s/it]
+ 33%|███▎      | 596/1813 [1:04:03<1:58:27,  5.84s/it]
+ 33%|███▎      | 597/1813 [1:04:08<1:55:17,  5.69s/it]
+ 33%|███▎      | 598/1813 [1:04:14<1:54:21,  5.65s/it]
+ 33%|███▎      | 599/1813 [1:04:20<1:56:29,  5.76s/it]
+ 33%|███▎      | 600/1813 [1:04:27<2:03:21,  6.10s/it]
+                                                      
+{'loss': '0.9298', 'grad_norm': '0.9432', 'learning_rate': '0.0001661', 'epoch': '0.331', 'num_input_tokens_seen': 3999176, 'train_runtime': '3867', 'train_tokens_per_second': '1034'}
+
+ 33%|███▎      | 600/1813 [1:04:27<2:03:21,  6.10s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.71it/s]
+
+  0%|          | 5/1000 [00:00<02:18,  7.21it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.31it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.08it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  7.00it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.83it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.35it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.98it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.07it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.30it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.03it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.35it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.73it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.89it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.75it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.31it/s]
+
+  6%|▌         | 59/1000 [00:08<01:55,  8.18it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.11it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.65it/s]
+
+  8%|▊         | 78/1000 [00:10<02:26,  6.31it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.85it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.18it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.01it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.94it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.62it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.83it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.33it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.44it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.20it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.03it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.51it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.77it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.19it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.90it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.07it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.71it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.63it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.87it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.95it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.88it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:28,  9.67it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:43,  8.25it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:41,  8.36it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:26,  9.80it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:18, 10.81it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:22, 10.25it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.60it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:46,  7.92it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:38,  8.55it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.77it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.47it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.39it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.66it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.17it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.77it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.83it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.14it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.14it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.21it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.44it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.89it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.74it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.70it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.74it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.60it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.89it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.51it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.47it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.68it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.72it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.13it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.36it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.03it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.92it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.44it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:25,  8.94it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:31,  8.33it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.00it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.46it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.04it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.75it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.99it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.84it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.78it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.79it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.19it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.24it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:07, 10.21it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.56it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.88it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.84it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.05it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.91it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.23it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.42it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:25,  7.62it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.54it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:08,  9.39it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.17it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.26it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.91it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:58,  5.35it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.59it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.54it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.07it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.20it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.47it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.77it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.21it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.17it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.65it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.37it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.79it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.45it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:09,  8.73it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.24it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:13,  7.91it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:10,  8.22it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:18,  7.36it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:27,  6.62it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:38,  5.86it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:31,  6.30it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.88it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:16,  7.49it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:13,  7.80it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.53it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.00it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:48, 11.76it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.47it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.31it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.08it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.24it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.82it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.61it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:29,  6.00it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.81it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.30it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.56it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.50it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.31it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.29it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:08<01:13,  6.79it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.54it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.42it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.95it/s]
+
+ 51%|█████     | 512/1000 [01:09<00:48,  9.97it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.38it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:47,  9.74it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:20,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:15<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.86it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.33it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.24it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.96it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.55it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.59it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.40it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.46it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.71it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.77it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.60it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.73it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:42,  9.07it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.02it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.20it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.44it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.62it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.45it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.95it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.57it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.51it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.10it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  5.99it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.65it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.96it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.88it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.41it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:48,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.21it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.49it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.69it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.84it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.35it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.05it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.79it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.92it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.14it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.14it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.62it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.88it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.51it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.76it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.24it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.71it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.66it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.62it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.46it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.78it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.14it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.77it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.88it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.59it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.93it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.36it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.35it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.98it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.21it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.18it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.85it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.63it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.20it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.22it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.29it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.12it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.17it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.81it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.74it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.99it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  6.93it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.71it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.20it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  5.99it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.50it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.08it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.44it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.15it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.46it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.03it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.17it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.50it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.76it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.61it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.67it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.30it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.25it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.28it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:14,  8.76it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.82it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.56it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.51it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.93it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.15it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.34it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.75it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.83it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.69it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.84it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.05it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.30it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.64it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.81it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.02it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.11it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.33it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.70it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.81it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.40it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.39it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.44it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.33it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.40it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.80it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.41it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.05it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.43it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.8095', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.382', 'eval_steps_per_second': '7.382', 'epoch': '0.331', 'num_input_tokens_seen': 3999176}
+
+ 33%|███▎      | 600/1813 [1:06:42<2:03:21,  6.10s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 33%|███▎      | 601/1813 [1:06:47<15:36:27, 46.36s/it]
+ 33%|███▎      | 602/1813 [1:06:52<11:25:39, 33.97s/it]
+ 33%|███▎      | 603/1813 [1:06:58<8:33:13, 25.45s/it] 
+ 33%|███▎      | 604/1813 [1:07:04<6:35:46, 19.64s/it]
+ 33%|███▎      | 605/1813 [1:07:12<5:25:22, 16.16s/it]
+ 33%|███▎      | 606/1813 [1:07:19<4:32:50, 13.56s/it]
+ 33%|███▎      | 607/1813 [1:07:26<3:49:16, 11.41s/it]
+ 34%|███▎      | 608/1813 [1:07:32<3:19:17,  9.92s/it]
+ 34%|███▎      | 609/1813 [1:07:39<2:58:21,  8.89s/it]
+ 34%|███▎      | 610/1813 [1:07:45<2:44:10,  8.19s/it]
+                                                      
+{'loss': '0.976', 'grad_norm': '0.7217', 'learning_rate': '0.0001647', 'epoch': '0.3366', 'num_input_tokens_seen': 4066376, 'train_runtime': '4066', 'train_tokens_per_second': '1000'}
+
+ 34%|███▎      | 610/1813 [1:07:45<2:44:10,  8.19s/it]
+ 34%|███▎      | 611/1813 [1:07:51<2:26:46,  7.33s/it]
+ 34%|███▍      | 612/1813 [1:07:55<2:07:37,  6.38s/it]
+ 34%|███▍      | 613/1813 [1:08:00<2:03:49,  6.19s/it]
+ 34%|███▍      | 614/1813 [1:08:06<2:02:22,  6.12s/it]
+ 34%|███▍      | 615/1813 [1:08:12<1:57:13,  5.87s/it]
+ 34%|███▍      | 616/1813 [1:08:17<1:51:44,  5.60s/it]
+ 34%|███▍      | 617/1813 [1:08:22<1:51:03,  5.57s/it]
+ 34%|███▍      | 618/1813 [1:08:29<2:00:13,  6.04s/it]
+ 34%|███▍      | 619/1813 [1:08:34<1:53:22,  5.70s/it]
+ 34%|███▍      | 620/1813 [1:08:41<1:58:22,  5.95s/it]
+                                                      
+{'loss': '0.8778', 'grad_norm': '1.008', 'learning_rate': '0.0001633', 'epoch': '0.3421', 'num_input_tokens_seen': 4123976, 'train_runtime': '4121', 'train_tokens_per_second': '1001'}
+
+ 34%|███▍      | 620/1813 [1:08:41<1:58:22,  5.95s/it]
+ 34%|███▍      | 621/1813 [1:08:48<2:03:35,  6.22s/it]
+ 34%|███▍      | 622/1813 [1:08:55<2:11:36,  6.63s/it]
+ 34%|███▍      | 623/1813 [1:09:00<2:01:16,  6.11s/it]
+ 34%|███▍      | 624/1813 [1:09:06<2:02:17,  6.17s/it]
+ 34%|███▍      | 625/1813 [1:09:11<1:54:42,  5.79s/it]
+ 35%|███▍      | 626/1813 [1:09:16<1:45:04,  5.31s/it]
+ 35%|███▍      | 627/1813 [1:09:22<1:49:33,  5.54s/it]
+ 35%|███▍      | 628/1813 [1:09:28<1:52:32,  5.70s/it]
+ 35%|███▍      | 629/1813 [1:09:36<2:07:45,  6.47s/it]
+ 35%|███▍      | 630/1813 [1:09:43<2:09:00,  6.54s/it]
+                                                      
+{'loss': '0.9491', 'grad_norm': '0.8515', 'learning_rate': '0.0001618', 'epoch': '0.3476', 'num_input_tokens_seen': 4191992, 'train_runtime': '4183', 'train_tokens_per_second': '1002'}
+
+ 35%|███▍      | 630/1813 [1:09:43<2:09:00,  6.54s/it]
+ 35%|███▍      | 631/1813 [1:09:49<2:08:05,  6.50s/it]
+ 35%|███▍      | 632/1813 [1:09:55<2:05:00,  6.35s/it]
+ 35%|███▍      | 633/1813 [1:10:01<2:00:59,  6.15s/it]
+ 35%|███▍      | 634/1813 [1:10:08<2:07:25,  6.48s/it]
+ 35%|███▌      | 635/1813 [1:10:15<2:08:10,  6.53s/it]
+ 35%|███▌      | 636/1813 [1:10:20<1:59:38,  6.10s/it]
+ 35%|███▌      | 637/1813 [1:10:27<2:04:32,  6.35s/it]
+ 35%|███▌      | 638/1813 [1:10:32<1:59:40,  6.11s/it]
+ 35%|███▌      | 639/1813 [1:10:37<1:54:13,  5.84s/it]
+ 35%|███▌      | 640/1813 [1:10:43<1:54:55,  5.88s/it]
+                                                      
+{'loss': '0.9522', 'grad_norm': '1.237', 'learning_rate': '0.0001603', 'epoch': '0.3531', 'num_input_tokens_seen': 4257760, 'train_runtime': '4244', 'train_tokens_per_second': '1003'}
+
+ 35%|███▌      | 640/1813 [1:10:43<1:54:55,  5.88s/it]
+ 35%|███▌      | 641/1813 [1:10:49<1:50:17,  5.65s/it]
+ 35%|███▌      | 642/1813 [1:10:56<1:59:38,  6.13s/it]
+ 35%|███▌      | 643/1813 [1:11:02<2:02:40,  6.29s/it]
+ 36%|███▌      | 644/1813 [1:11:08<1:57:25,  6.03s/it]
+ 36%|███▌      | 645/1813 [1:11:14<1:57:11,  6.02s/it]
+ 36%|███▌      | 646/1813 [1:11:20<1:55:50,  5.96s/it]
+ 36%|███▌      | 647/1813 [1:11:26<1:57:24,  6.04s/it]
+ 36%|███▌      | 648/1813 [1:11:33<2:02:48,  6.32s/it]
+ 36%|███▌      | 649/1813 [1:11:39<1:58:43,  6.12s/it]
+ 36%|███▌      | 650/1813 [1:11:44<1:53:33,  5.86s/it]
+                                                      
+{'loss': '0.9441', 'grad_norm': '1.01', 'learning_rate': '0.0001588', 'epoch': '0.3586', 'num_input_tokens_seen': 4322136, 'train_runtime': '4304', 'train_tokens_per_second': '1004'}
+
+ 36%|███▌      | 650/1813 [1:11:44<1:53:33,  5.86s/it]
+ 36%|███▌      | 651/1813 [1:11:50<1:58:13,  6.10s/it]
+ 36%|███▌      | 652/1813 [1:11:57<2:02:44,  6.34s/it]
+ 36%|███▌      | 653/1813 [1:12:03<1:57:57,  6.10s/it]
+ 36%|███▌      | 654/1813 [1:12:08<1:54:27,  5.92s/it]
+ 36%|███▌      | 655/1813 [1:12:15<1:57:52,  6.11s/it]
+ 36%|███▌      | 656/1813 [1:12:21<1:54:45,  5.95s/it]
+ 36%|███▌      | 657/1813 [1:12:26<1:51:47,  5.80s/it]
+ 36%|███▋      | 658/1813 [1:12:33<1:56:55,  6.07s/it]
+ 36%|███▋      | 659/1813 [1:12:39<1:59:48,  6.23s/it]
+ 36%|███▋      | 660/1813 [1:12:45<1:55:52,  6.03s/it]
+                                                      
+{'loss': '0.9016', 'grad_norm': '0.9785', 'learning_rate': '0.0001572', 'epoch': '0.3641', 'num_input_tokens_seen': 4385688, 'train_runtime': '4365', 'train_tokens_per_second': '1005'}
+
+ 36%|███▋      | 660/1813 [1:12:45<1:55:52,  6.03s/it]
+ 36%|███▋      | 661/1813 [1:12:51<1:55:29,  6.02s/it]
+ 37%|███▋      | 662/1813 [1:12:57<1:55:42,  6.03s/it]
+ 37%|███▋      | 663/1813 [1:13:04<2:00:04,  6.27s/it]
+ 37%|███▋      | 664/1813 [1:13:10<2:00:18,  6.28s/it]
+ 37%|███▋      | 665/1813 [1:13:15<1:52:13,  5.87s/it]
+ 37%|███▋      | 666/1813 [1:13:20<1:48:38,  5.68s/it]
+ 37%|███▋      | 667/1813 [1:13:27<1:54:13,  5.98s/it]
+ 37%|███▋      | 668/1813 [1:13:33<1:55:57,  6.08s/it]
+ 37%|███▋      | 669/1813 [1:13:39<1:54:54,  6.03s/it]
+ 37%|███▋      | 670/1813 [1:13:45<1:52:39,  5.91s/it]
+                                                      
+{'loss': '0.9796', 'grad_norm': '1.205', 'learning_rate': '0.0001557', 'epoch': '0.3697', 'num_input_tokens_seen': 4449456, 'train_runtime': '4425', 'train_tokens_per_second': '1005'}
+
+ 37%|███▋      | 670/1813 [1:13:45<1:52:39,  5.91s/it]
+ 37%|███▋      | 671/1813 [1:13:51<1:55:15,  6.06s/it]
+ 37%|███▋      | 672/1813 [1:13:58<1:59:34,  6.29s/it]
+ 37%|███▋      | 673/1813 [1:14:04<2:00:18,  6.33s/it]
+ 37%|███▋      | 674/1813 [1:14:11<2:02:30,  6.45s/it]
+ 37%|███▋      | 675/1813 [1:14:17<1:59:07,  6.28s/it]
+ 37%|███▋      | 676/1813 [1:14:22<1:54:43,  6.05s/it]
+ 37%|███▋      | 677/1813 [1:14:31<2:06:44,  6.69s/it]
+ 37%|███▋      | 678/1813 [1:14:36<1:58:23,  6.26s/it]
+ 37%|███▋      | 679/1813 [1:14:41<1:52:37,  5.96s/it]
+ 38%|███▊      | 680/1813 [1:14:47<1:51:41,  5.92s/it]
+                                                      
+{'loss': '0.9305', 'grad_norm': '0.9702', 'learning_rate': '0.0001541', 'epoch': '0.3752', 'num_input_tokens_seen': 4514984, 'train_runtime': '4488', 'train_tokens_per_second': '1006'}
+
+ 38%|███▊      | 680/1813 [1:14:47<1:51:41,  5.92s/it]
+ 38%|███▊      | 681/1813 [1:14:54<1:54:59,  6.10s/it]
+ 38%|███▊      | 682/1813 [1:14:59<1:50:04,  5.84s/it]
+ 38%|███▊      | 683/1813 [1:15:05<1:51:35,  5.93s/it]
+ 38%|███▊      | 684/1813 [1:15:12<1:55:29,  6.14s/it]
+ 38%|███▊      | 685/1813 [1:15:19<2:03:59,  6.60s/it]
+ 38%|███▊      | 686/1813 [1:15:25<2:02:14,  6.51s/it]
+ 38%|███▊      | 687/1813 [1:15:33<2:09:18,  6.89s/it]
+ 38%|███▊      | 688/1813 [1:15:38<1:57:28,  6.27s/it]
+ 38%|███▊      | 689/1813 [1:15:47<2:11:21,  7.01s/it]
+ 38%|███▊      | 690/1813 [1:15:54<2:12:59,  7.11s/it]
+                                                      
+{'loss': '0.9827', 'grad_norm': '1.091', 'learning_rate': '0.0001525', 'epoch': '0.3807', 'num_input_tokens_seen': 4587840, 'train_runtime': '4555', 'train_tokens_per_second': '1007'}
+
+ 38%|███▊      | 690/1813 [1:15:54<2:12:59,  7.11s/it]
+ 38%|███▊      | 691/1813 [1:16:00<2:04:35,  6.66s/it]
+ 38%|███▊      | 692/1813 [1:16:06<2:01:44,  6.52s/it]
+ 38%|███▊      | 693/1813 [1:16:11<1:55:44,  6.20s/it]
+ 38%|███▊      | 694/1813 [1:16:18<1:56:27,  6.24s/it]
+ 38%|███▊      | 695/1813 [1:16:24<1:56:03,  6.23s/it]
+ 38%|███▊      | 696/1813 [1:16:32<2:05:26,  6.74s/it]
+ 38%|███▊      | 697/1813 [1:16:37<1:54:59,  6.18s/it]
+ 38%|███▊      | 698/1813 [1:16:42<1:51:17,  5.99s/it]
+ 39%|███▊      | 699/1813 [1:16:49<1:56:34,  6.28s/it]
+ 39%|███▊      | 700/1813 [1:16:56<1:58:06,  6.37s/it]
+                                                      
+{'loss': '1.005', 'grad_norm': '1.161', 'learning_rate': '0.0001509', 'epoch': '0.3862', 'num_input_tokens_seen': 4653312, 'train_runtime': '4616', 'train_tokens_per_second': '1008'}
+
+ 39%|███▊      | 700/1813 [1:16:56<1:58:06,  6.37s/it]
+ 39%|███▊      | 701/1813 [1:17:03<2:03:10,  6.65s/it]
+ 39%|███▊      | 702/1813 [1:17:10<2:02:09,  6.60s/it]
+ 39%|███▉      | 703/1813 [1:17:18<2:09:53,  7.02s/it]
+ 39%|███▉      | 704/1813 [1:17:23<2:03:17,  6.67s/it]
+ 39%|███▉      | 705/1813 [1:17:31<2:05:32,  6.80s/it]
+ 39%|███▉      | 706/1813 [1:17:37<2:02:29,  6.64s/it]
+ 39%|███▉      | 707/1813 [1:17:46<2:18:27,  7.51s/it]
+ 39%|███▉      | 708/1813 [1:17:54<2:17:23,  7.46s/it]
+ 39%|███▉      | 709/1813 [1:18:00<2:13:24,  7.25s/it]
+ 39%|███▉      | 710/1813 [1:18:06<2:06:04,  6.86s/it]
+                                                      
+{'loss': '0.9653', 'grad_norm': '0.8149', 'learning_rate': '0.0001492', 'epoch': '0.3917', 'num_input_tokens_seen': 4732080, 'train_runtime': '4687', 'train_tokens_per_second': '1010'}
+
+ 39%|███▉      | 710/1813 [1:18:06<2:06:04,  6.86s/it]
+ 39%|███▉      | 711/1813 [1:18:13<2:05:32,  6.83s/it]
+ 39%|███▉      | 712/1813 [1:18:20<2:04:29,  6.78s/it]
+ 39%|███▉      | 713/1813 [1:18:26<1:59:38,  6.53s/it]
+ 39%|███▉      | 714/1813 [1:18:32<1:57:03,  6.39s/it]
+ 39%|███▉      | 715/1813 [1:18:38<1:58:01,  6.45s/it]
+ 39%|███▉      | 716/1813 [1:18:45<2:00:40,  6.60s/it]
+ 40%|███▉      | 717/1813 [1:18:52<1:58:36,  6.49s/it]
+ 40%|███▉      | 718/1813 [1:18:58<1:58:45,  6.51s/it]
+ 40%|███▉      | 719/1813 [1:19:04<1:52:28,  6.17s/it]
+ 40%|███▉      | 720/1813 [1:19:10<1:53:18,  6.22s/it]
+                                                      
+{'loss': '0.9908', 'grad_norm': '1.018', 'learning_rate': '0.0001476', 'epoch': '0.3972', 'num_input_tokens_seen': 4799552, 'train_runtime': '4750', 'train_tokens_per_second': '1010'}
+
+ 40%|███▉      | 720/1813 [1:19:10<1:53:18,  6.22s/it]
+ 40%|███▉      | 721/1813 [1:19:17<1:55:57,  6.37s/it]
+ 40%|███▉      | 722/1813 [1:19:22<1:50:34,  6.08s/it]
+ 40%|███▉      | 723/1813 [1:19:28<1:47:09,  5.90s/it]
+ 40%|███▉      | 724/1813 [1:19:35<1:58:05,  6.51s/it]
+ 40%|███▉      | 725/1813 [1:19:44<2:08:17,  7.07s/it]
+ 40%|████      | 726/1813 [1:19:50<2:04:51,  6.89s/it]
+ 40%|████      | 727/1813 [1:19:56<1:59:41,  6.61s/it]
+ 40%|████      | 728/1813 [1:20:02<1:52:47,  6.24s/it]
+ 40%|████      | 729/1813 [1:20:07<1:49:51,  6.08s/it]
+ 40%|████      | 730/1813 [1:20:13<1:46:09,  5.88s/it]
+                                                      
+{'loss': '0.9616', 'grad_norm': '1.052', 'learning_rate': '0.0001459', 'epoch': '0.4028', 'num_input_tokens_seen': 4866864, 'train_runtime': '4813', 'train_tokens_per_second': '1011'}
+
+ 40%|████      | 730/1813 [1:20:13<1:46:09,  5.88s/it]
+ 40%|████      | 731/1813 [1:20:20<1:52:27,  6.24s/it]
+ 40%|████      | 732/1813 [1:20:26<1:53:57,  6.33s/it]
+ 40%|████      | 733/1813 [1:20:34<2:00:14,  6.68s/it]
+ 40%|████      | 734/1813 [1:20:41<2:01:48,  6.77s/it]
+ 41%|████      | 735/1813 [1:20:48<2:01:39,  6.77s/it]
+ 41%|████      | 736/1813 [1:20:54<1:56:48,  6.51s/it]
+ 41%|████      | 737/1813 [1:21:00<1:54:00,  6.36s/it]
+ 41%|████      | 738/1813 [1:21:05<1:48:41,  6.07s/it]
+ 41%|████      | 739/1813 [1:21:11<1:47:26,  6.00s/it]
+ 41%|████      | 740/1813 [1:21:17<1:51:13,  6.22s/it]
+                                                      
+{'loss': '0.8881', 'grad_norm': '0.8816', 'learning_rate': '0.0001442', 'epoch': '0.4083', 'num_input_tokens_seen': 4936352, 'train_runtime': '4878', 'train_tokens_per_second': '1012'}
+
+ 41%|████      | 740/1813 [1:21:17<1:51:13,  6.22s/it]
+ 41%|████      | 741/1813 [1:21:24<1:53:28,  6.35s/it]
+ 41%|████      | 742/1813 [1:21:32<1:59:42,  6.71s/it]
+ 41%|████      | 743/1813 [1:21:39<2:03:48,  6.94s/it]
+ 41%|████      | 744/1813 [1:21:46<2:02:49,  6.89s/it]
+ 41%|████      | 745/1813 [1:21:53<2:04:06,  6.97s/it]
+ 41%|████      | 746/1813 [1:21:59<2:00:14,  6.76s/it]
+ 41%|████      | 747/1813 [1:22:07<2:06:29,  7.12s/it]
+ 41%|████▏     | 748/1813 [1:22:13<1:57:34,  6.62s/it]
+ 41%|████▏     | 749/1813 [1:22:19<1:55:56,  6.54s/it]
+ 41%|████▏     | 750/1813 [1:22:25<1:51:44,  6.31s/it]
+                                                      
+{'loss': '0.9725', 'grad_norm': '0.8841', 'learning_rate': '0.0001425', 'epoch': '0.4138', 'num_input_tokens_seen': 5011064, 'train_runtime': '4945', 'train_tokens_per_second': '1013'}
+
+ 41%|████▏     | 750/1813 [1:22:25<1:51:44,  6.31s/it]
+ 41%|████▏     | 751/1813 [1:22:31<1:50:20,  6.23s/it]
+ 41%|████▏     | 752/1813 [1:22:37<1:48:41,  6.15s/it]
+ 42%|████▏     | 753/1813 [1:22:43<1:50:47,  6.27s/it]
+ 42%|████▏     | 754/1813 [1:22:52<2:02:28,  6.94s/it]
+ 42%|████▏     | 755/1813 [1:22:57<1:51:24,  6.32s/it]
+ 42%|████▏     | 756/1813 [1:23:02<1:46:46,  6.06s/it]
+ 42%|████▏     | 757/1813 [1:23:08<1:45:04,  5.97s/it]
+ 42%|████▏     | 758/1813 [1:23:15<1:48:12,  6.15s/it]
+ 42%|████▏     | 759/1813 [1:23:22<1:52:31,  6.41s/it]
+ 42%|████▏     | 760/1813 [1:23:28<1:49:59,  6.27s/it]
+                                                      
+{'loss': '1.003', 'grad_norm': '1.215', 'learning_rate': '0.0001408', 'epoch': '0.4193', 'num_input_tokens_seen': 5078960, 'train_runtime': '5008', 'train_tokens_per_second': '1014'}
+
+ 42%|████▏     | 760/1813 [1:23:28<1:49:59,  6.27s/it]
+ 42%|████▏     | 761/1813 [1:23:34<1:51:58,  6.39s/it]
+ 42%|████▏     | 762/1813 [1:23:41<1:52:59,  6.45s/it]
+ 42%|████▏     | 763/1813 [1:23:48<1:55:42,  6.61s/it]
+ 42%|████▏     | 764/1813 [1:23:55<1:57:09,  6.70s/it]
+ 42%|████▏     | 765/1813 [1:24:02<2:00:54,  6.92s/it]
+ 42%|████▏     | 766/1813 [1:24:09<1:58:48,  6.81s/it]
+ 42%|████▏     | 767/1813 [1:24:14<1:51:27,  6.39s/it]
+ 42%|████▏     | 768/1813 [1:24:19<1:41:20,  5.82s/it]
+ 42%|████▏     | 769/1813 [1:24:25<1:46:15,  6.11s/it]
+ 42%|████▏     | 770/1813 [1:24:32<1:47:15,  6.17s/it]
+                                                      
+{'loss': '0.9154', 'grad_norm': '1.08', 'learning_rate': '0.0001391', 'epoch': '0.4248', 'num_input_tokens_seen': 5148504, 'train_runtime': '5072', 'train_tokens_per_second': '1015'}
+
+ 42%|████▏     | 770/1813 [1:24:32<1:47:15,  6.17s/it]
+ 43%|████▎     | 771/1813 [1:24:38<1:49:28,  6.30s/it]
+ 43%|████▎     | 772/1813 [1:24:46<1:54:17,  6.59s/it]
+ 43%|████▎     | 773/1813 [1:24:52<1:51:12,  6.42s/it]
+ 43%|████▎     | 774/1813 [1:24:57<1:45:12,  6.08s/it]
+ 43%|████▎     | 775/1813 [1:25:03<1:45:15,  6.08s/it]
+ 43%|████▎     | 776/1813 [1:25:10<1:48:35,  6.28s/it]
+ 43%|████▎     | 777/1813 [1:25:17<1:55:47,  6.71s/it]
+ 43%|████▎     | 778/1813 [1:25:23<1:47:53,  6.25s/it]
+ 43%|████▎     | 779/1813 [1:25:28<1:43:11,  5.99s/it]
+ 43%|████▎     | 780/1813 [1:25:35<1:47:11,  6.23s/it]
+                                                      
+{'loss': '0.8878', 'grad_norm': '1.062', 'learning_rate': '0.0001373', 'epoch': '0.4303', 'num_input_tokens_seen': 5215784, 'train_runtime': '5135', 'train_tokens_per_second': '1016'}
+
+ 43%|████▎     | 780/1813 [1:25:35<1:47:11,  6.23s/it]
+ 43%|████▎     | 781/1813 [1:25:42<1:49:57,  6.39s/it]
+ 43%|████▎     | 782/1813 [1:25:50<1:58:52,  6.92s/it]
+ 43%|████▎     | 783/1813 [1:25:57<1:59:02,  6.93s/it]
+ 43%|████▎     | 784/1813 [1:26:03<1:53:35,  6.62s/it]
+ 43%|████▎     | 785/1813 [1:26:10<1:56:05,  6.78s/it]
+ 43%|████▎     | 786/1813 [1:26:17<1:58:01,  6.89s/it]
+ 43%|████▎     | 787/1813 [1:26:22<1:48:58,  6.37s/it]
+ 43%|████▎     | 788/1813 [1:26:27<1:43:53,  6.08s/it]
+ 44%|████▎     | 789/1813 [1:26:32<1:38:20,  5.76s/it]
+ 44%|████▎     | 790/1813 [1:26:38<1:37:43,  5.73s/it]
+                                                      
+{'loss': '0.9875', 'grad_norm': '0.94', 'learning_rate': '0.0001356', 'epoch': '0.4359', 'num_input_tokens_seen': 5285160, 'train_runtime': '5199', 'train_tokens_per_second': '1017'}
+
+ 44%|████▎     | 790/1813 [1:26:38<1:37:43,  5.73s/it]
+ 44%|████▎     | 791/1813 [1:26:44<1:39:35,  5.85s/it]
+ 44%|████▎     | 792/1813 [1:26:51<1:46:09,  6.24s/it]
+ 44%|████▎     | 793/1813 [1:26:59<1:51:09,  6.54s/it]
+ 44%|████▍     | 794/1813 [1:27:05<1:50:43,  6.52s/it]
+ 44%|████▍     | 795/1813 [1:27:11<1:47:15,  6.32s/it]
+ 44%|████▍     | 796/1813 [1:27:18<1:49:33,  6.46s/it]
+ 44%|████▍     | 797/1813 [1:27:25<1:55:08,  6.80s/it]
+ 44%|████▍     | 798/1813 [1:27:31<1:51:28,  6.59s/it]
+ 44%|████▍     | 799/1813 [1:27:37<1:48:17,  6.41s/it]
+ 44%|████▍     | 800/1813 [1:27:43<1:46:29,  6.31s/it]
+                                                      
+{'loss': '0.9421', 'grad_norm': '0.9114', 'learning_rate': '0.0001338', 'epoch': '0.4414', 'num_input_tokens_seen': 5357720, 'train_runtime': '5264', 'train_tokens_per_second': '1018'}
+
+ 44%|████▍     | 800/1813 [1:27:43<1:46:29,  6.31s/it]
+ 44%|████▍     | 801/1813 [1:27:49<1:42:10,  6.06s/it]
+ 44%|████▍     | 802/1813 [1:27:55<1:42:49,  6.10s/it]
+ 44%|████▍     | 803/1813 [1:28:03<1:52:26,  6.68s/it]
+ 44%|████▍     | 804/1813 [1:28:10<1:50:42,  6.58s/it]
+ 44%|████▍     | 805/1813 [1:28:16<1:47:39,  6.41s/it]
+ 44%|████▍     | 806/1813 [1:28:22<1:46:15,  6.33s/it]
+ 45%|████▍     | 807/1813 [1:28:27<1:42:40,  6.12s/it]
+ 45%|████▍     | 808/1813 [1:28:35<1:51:00,  6.63s/it]
+ 45%|████▍     | 809/1813 [1:28:42<1:52:03,  6.70s/it]
+ 45%|████▍     | 810/1813 [1:28:50<1:57:15,  7.01s/it]
+                                                      
+{'loss': '0.9589', 'grad_norm': '0.8725', 'learning_rate': '0.000132', 'epoch': '0.4469', 'num_input_tokens_seen': 5430288, 'train_runtime': '5330', 'train_tokens_per_second': '1019'}
+
+ 45%|████▍     | 810/1813 [1:28:50<1:57:15,  7.01s/it]
+ 45%|████▍     | 811/1813 [1:28:57<1:55:47,  6.93s/it]
+ 45%|████▍     | 812/1813 [1:29:02<1:49:40,  6.57s/it]
+ 45%|████▍     | 813/1813 [1:29:08<1:43:53,  6.23s/it]
+ 45%|████▍     | 814/1813 [1:29:14<1:43:25,  6.21s/it]
+ 45%|████▍     | 815/1813 [1:29:19<1:35:45,  5.76s/it]
+ 45%|████▌     | 816/1813 [1:29:25<1:40:30,  6.05s/it]
+ 45%|████▌     | 817/1813 [1:29:31<1:38:46,  5.95s/it]
+ 45%|████▌     | 818/1813 [1:29:38<1:45:24,  6.36s/it]
+ 45%|████▌     | 819/1813 [1:29:45<1:46:08,  6.41s/it]
+ 45%|████▌     | 820/1813 [1:29:52<1:50:58,  6.71s/it]
+                                                      
+{'loss': '0.8875', 'grad_norm': '0.9094', 'learning_rate': '0.0001302', 'epoch': '0.4524', 'num_input_tokens_seen': 5497552, 'train_runtime': '5393', 'train_tokens_per_second': '1019'}
+
+ 45%|████▌     | 820/1813 [1:29:52<1:50:58,  6.71s/it]
+ 45%|████▌     | 821/1813 [1:29:59<1:49:50,  6.64s/it]
+ 45%|████▌     | 822/1813 [1:30:04<1:40:50,  6.11s/it]
+ 45%|████▌     | 823/1813 [1:30:08<1:34:15,  5.71s/it]
+ 45%|████▌     | 824/1813 [1:30:15<1:37:36,  5.92s/it]
+ 46%|████▌     | 825/1813 [1:30:21<1:39:34,  6.05s/it]
+ 46%|████▌     | 826/1813 [1:30:28<1:43:31,  6.29s/it]
+ 46%|████▌     | 827/1813 [1:30:34<1:40:52,  6.14s/it]
+ 46%|████▌     | 828/1813 [1:30:40<1:39:39,  6.07s/it]
+ 46%|████▌     | 829/1813 [1:30:45<1:37:51,  5.97s/it]
+ 46%|████▌     | 830/1813 [1:30:53<1:44:55,  6.40s/it]
+                                                      
+{'loss': '0.7936', 'grad_norm': '0.8778', 'learning_rate': '0.0001284', 'epoch': '0.4579', 'num_input_tokens_seen': 5563144, 'train_runtime': '5453', 'train_tokens_per_second': '1020'}
+
+ 46%|████▌     | 830/1813 [1:30:53<1:44:55,  6.40s/it]
+ 46%|████▌     | 831/1813 [1:30:59<1:43:57,  6.35s/it]
+ 46%|████▌     | 832/1813 [1:31:06<1:49:02,  6.67s/it]
+ 46%|████▌     | 833/1813 [1:31:13<1:47:24,  6.58s/it]
+ 46%|████▌     | 834/1813 [1:31:19<1:47:21,  6.58s/it]
+ 46%|████▌     | 835/1813 [1:31:25<1:44:19,  6.40s/it]
+ 46%|████▌     | 836/1813 [1:31:32<1:46:46,  6.56s/it]
+ 46%|████▌     | 837/1813 [1:31:40<1:52:58,  6.95s/it]
+ 46%|████▌     | 838/1813 [1:31:46<1:48:00,  6.65s/it]
+ 46%|████▋     | 839/1813 [1:31:52<1:45:41,  6.51s/it]
+ 46%|████▋     | 840/1813 [1:31:58<1:42:24,  6.31s/it]
+                                                      
+{'loss': '0.9675', 'grad_norm': '1.183', 'learning_rate': '0.0001266', 'epoch': '0.4634', 'num_input_tokens_seen': 5634104, 'train_runtime': '5519', 'train_tokens_per_second': '1021'}
+
+ 46%|████▋     | 840/1813 [1:31:58<1:42:24,  6.31s/it]
+ 46%|████▋     | 841/1813 [1:32:04<1:40:32,  6.21s/it]
+ 46%|████▋     | 842/1813 [1:32:11<1:43:36,  6.40s/it]
+ 46%|████▋     | 843/1813 [1:32:17<1:42:14,  6.32s/it]
+ 47%|████▋     | 844/1813 [1:32:23<1:39:16,  6.15s/it]
+ 47%|████▋     | 845/1813 [1:32:29<1:40:18,  6.22s/it]
+ 47%|████▋     | 846/1813 [1:32:36<1:42:48,  6.38s/it]
+ 47%|████▋     | 847/1813 [1:32:43<1:46:06,  6.59s/it]
+ 47%|████▋     | 848/1813 [1:32:50<1:49:50,  6.83s/it]
+ 47%|████▋     | 849/1813 [1:32:56<1:43:00,  6.41s/it]
+ 47%|████▋     | 850/1813 [1:33:04<1:52:25,  7.00s/it]
+                                                      
+{'loss': '0.922', 'grad_norm': '0.8316', 'learning_rate': '0.0001248', 'epoch': '0.469', 'num_input_tokens_seen': 5704744, 'train_runtime': '5585', 'train_tokens_per_second': '1021'}
+
+ 47%|████▋     | 850/1813 [1:33:04<1:52:25,  7.00s/it]
+ 47%|████▋     | 851/1813 [1:33:11<1:51:24,  6.95s/it]
+ 47%|████▋     | 852/1813 [1:33:16<1:41:57,  6.37s/it]
+ 47%|████▋     | 853/1813 [1:33:22<1:41:18,  6.33s/it]
+ 47%|████▋     | 854/1813 [1:33:29<1:44:23,  6.53s/it]
+ 47%|████▋     | 855/1813 [1:33:36<1:42:30,  6.42s/it]
+ 47%|████▋     | 856/1813 [1:33:42<1:41:07,  6.34s/it]
+ 47%|████▋     | 857/1813 [1:33:47<1:35:34,  6.00s/it]
+ 47%|████▋     | 858/1813 [1:33:51<1:25:05,  5.35s/it]
+ 47%|████▋     | 859/1813 [1:33:57<1:28:40,  5.58s/it]
+ 47%|████▋     | 860/1813 [1:34:03<1:32:48,  5.84s/it]
+                                                      
+{'loss': '0.9451', 'grad_norm': '1.165', 'learning_rate': '0.0001229', 'epoch': '0.4745', 'num_input_tokens_seen': 5766424, 'train_runtime': '5644', 'train_tokens_per_second': '1022'}
+
+ 47%|████▋     | 860/1813 [1:34:03<1:32:48,  5.84s/it]
+ 47%|████▋     | 861/1813 [1:34:09<1:30:01,  5.67s/it]
+ 48%|████▊     | 862/1813 [1:34:15<1:35:44,  6.04s/it]
+ 48%|████▊     | 863/1813 [1:34:22<1:36:57,  6.12s/it]
+ 48%|████▊     | 864/1813 [1:34:26<1:29:07,  5.63s/it]
+ 48%|████▊     | 865/1813 [1:34:33<1:35:33,  6.05s/it]
+ 48%|████▊     | 866/1813 [1:34:39<1:35:32,  6.05s/it]
+ 48%|████▊     | 867/1813 [1:34:45<1:33:29,  5.93s/it]
+ 48%|████▊     | 868/1813 [1:34:51<1:31:38,  5.82s/it]
+ 48%|████▊     | 869/1813 [1:34:58<1:38:43,  6.28s/it]
+ 48%|████▊     | 870/1813 [1:35:06<1:45:51,  6.73s/it]
+                                                      
+{'loss': '0.9045', 'grad_norm': '0.8641', 'learning_rate': '0.0001211', 'epoch': '0.48', 'num_input_tokens_seen': 5831352, 'train_runtime': '5706', 'train_tokens_per_second': '1022'}
+
+ 48%|████▊     | 870/1813 [1:35:06<1:45:51,  6.73s/it]
+ 48%|████▊     | 871/1813 [1:35:11<1:39:57,  6.37s/it]
+ 48%|████▊     | 872/1813 [1:35:19<1:44:40,  6.67s/it]
+ 48%|████▊     | 873/1813 [1:35:25<1:44:30,  6.67s/it]
+ 48%|████▊     | 874/1813 [1:35:33<1:48:36,  6.94s/it]
+ 48%|████▊     | 875/1813 [1:35:41<1:52:54,  7.22s/it]
+ 48%|████▊     | 876/1813 [1:35:48<1:51:38,  7.15s/it]
+ 48%|████▊     | 877/1813 [1:35:56<1:55:35,  7.41s/it]
+ 48%|████▊     | 878/1813 [1:36:02<1:51:23,  7.15s/it]
+ 48%|████▊     | 879/1813 [1:36:10<1:52:59,  7.26s/it]
+ 49%|████▊     | 880/1813 [1:36:14<1:41:02,  6.50s/it]
+                                                      
+{'loss': '0.9384', 'grad_norm': '1.411', 'learning_rate': '0.0001192', 'epoch': '0.4855', 'num_input_tokens_seen': 5907208, 'train_runtime': '5775', 'train_tokens_per_second': '1023'}
+
+ 49%|████▊     | 880/1813 [1:36:14<1:41:02,  6.50s/it]
+ 49%|████▊     | 881/1813 [1:36:20<1:36:25,  6.21s/it]
+ 49%|████▊     | 882/1813 [1:36:26<1:36:32,  6.22s/it]
+ 49%|████▊     | 883/1813 [1:36:33<1:39:20,  6.41s/it]
+ 49%|████▉     | 884/1813 [1:36:39<1:38:21,  6.35s/it]
+ 49%|████▉     | 885/1813 [1:36:45<1:37:20,  6.29s/it]
+ 49%|████▉     | 886/1813 [1:36:52<1:38:06,  6.35s/it]
+ 49%|████▉     | 887/1813 [1:37:00<1:45:43,  6.85s/it]
+ 49%|████▉     | 888/1813 [1:37:07<1:45:11,  6.82s/it]
+ 49%|████▉     | 889/1813 [1:37:14<1:47:27,  6.98s/it]
+ 49%|████▉     | 890/1813 [1:37:20<1:44:21,  6.78s/it]
+                                                      
+{'loss': '0.9001', 'grad_norm': '0.9493', 'learning_rate': '0.0001174', 'epoch': '0.491', 'num_input_tokens_seen': 5978048, 'train_runtime': '5841', 'train_tokens_per_second': '1023'}
+
+ 49%|████▉     | 890/1813 [1:37:20<1:44:21,  6.78s/it]
+ 49%|████▉     | 891/1813 [1:37:27<1:41:02,  6.58s/it]
+ 49%|████▉     | 892/1813 [1:37:34<1:45:15,  6.86s/it]
+ 49%|████▉     | 893/1813 [1:37:40<1:42:14,  6.67s/it]
+ 49%|████▉     | 894/1813 [1:37:46<1:37:46,  6.38s/it]
+ 49%|████▉     | 895/1813 [1:37:53<1:42:45,  6.72s/it]
+ 49%|████▉     | 896/1813 [1:37:59<1:36:36,  6.32s/it]
+ 49%|████▉     | 897/1813 [1:38:05<1:34:23,  6.18s/it]
+ 50%|████▉     | 898/1813 [1:38:12<1:38:26,  6.46s/it]
+ 50%|████▉     | 899/1813 [1:38:19<1:42:05,  6.70s/it]
+ 50%|████▉     | 900/1813 [1:38:27<1:47:00,  7.03s/it]
+                                                      
+{'loss': '0.9884', 'grad_norm': '1.043', 'learning_rate': '0.0001155', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856, 'train_runtime': '5907', 'train_tokens_per_second': '1024'}
+
+ 50%|████▉     | 900/1813 [1:38:27<1:47:00,  7.03s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.68it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.21it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.30it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.45it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.05it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.99it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.82it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.71it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.04it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.59it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.23it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.58it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.35it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.35it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.98it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.74it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.65it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:06<02:04,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.09it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.30it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.03it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.73it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.89it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.91it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.72it/s]
+
+  6%|▌         | 57/1000 [00:07<01:43,  9.15it/s]
+
+  6%|▌         | 59/1000 [00:08<01:55,  8.12it/s]
+
+  6%|▌         | 61/1000 [00:08<01:41,  9.26it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.68it/s]
+
+  6%|▋         | 65/1000 [00:08<01:52,  8.32it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.55it/s]
+
+  7%|▋         | 69/1000 [00:09<01:53,  8.23it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.97it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.70it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.10it/s]
+
+  8%|▊         | 75/1000 [00:10<02:15,  6.85it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.65it/s]
+
+  8%|▊         | 78/1000 [00:10<02:26,  6.31it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.84it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.19it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.00it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.94it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  6.99it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.23it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.16it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.77it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.34it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.62it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.80it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.30it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.44it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.19it/s]
+
+ 11%|█         | 112/1000 [00:14<01:14, 11.91it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:11, 12.41it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.73it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.18it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.90it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:48,  8.07it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.71it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.13it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.78it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.27it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.08it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.11it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.62it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.93it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.01it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.88it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.07it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.47it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.89it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:17, 11.01it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.36it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.63it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:46,  7.94it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.58it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.48it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.39it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.66it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.17it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.77it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.83it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.13it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:15,  6.13it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.21it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:36,  8.55it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.47it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.74it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.71it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.88it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.37it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.10it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.74it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.18it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.67it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.77it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.63it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.88it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.46it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.66it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.71it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.12it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.49it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:29,  8.77it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:04,  6.28it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.40it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.79it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.89it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.35it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:37,  7.91it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:34,  8.10it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:18,  9.79it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:21,  9.35it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:25,  8.90it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:31,  8.31it/s]
+
+ 24%|██▍       | 242/1000 [00:32<02:06,  5.99it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.45it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.63it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.22it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:50,  6.83it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:08,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.01it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.73it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.35it/s]
+
+ 26%|██▌       | 256/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:25,  8.72it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.67it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:55,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.32it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.58it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.52it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.01it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.42it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.70it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.74it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.98it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:26,  8.11it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:49,  6.43it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.82it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.10it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.07it/s]
+
+ 30%|███       | 305/1000 [00:41<01:25,  8.11it/s]
+
+ 31%|███       | 306/1000 [00:41<01:40,  6.93it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.78it/s]
+
+ 31%|███       | 309/1000 [00:41<01:20,  8.57it/s]
+
+ 31%|███       | 311/1000 [00:41<01:08, 10.04it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.14it/s]
+
+ 32%|███▏      | 315/1000 [00:42<01:07, 10.13it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.50it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.21it/s]
+
+ 32%|███▏      | 321/1000 [00:43<01:38,  6.86it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.41it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.72it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:31,  7.39it/s]
+
+ 33%|███▎      | 328/1000 [00:44<01:33,  7.15it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.77it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.57it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:36,  6.94it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.37it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.03it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.80it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.00it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.85it/s]
+
+ 34%|███▍      | 344/1000 [00:46<01:26,  7.59it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:25,  7.69it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.20it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.40it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:47<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.63it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.54it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.94it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.56it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.40it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.17it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:58,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.59it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.51it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.69it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.06it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.21it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:54,  5.47it/s]
+
+ 38%|███▊      | 378/1000 [00:51<01:29,  6.91it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.75it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.89it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.19it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.17it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.13it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.64it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.37it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.79it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.46it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:09,  8.73it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:54<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.91it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.23it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:56<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.80it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.49it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.67it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.92it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.94it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.53it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:12,  7.83it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.53it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.00it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.77it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.48it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.06it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.27it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.24it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.47it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.80it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.22it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.79it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.00it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.30it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.56it/s]
+
+ 47%|████▋     | 474/1000 [01:04<01:21,  6.49it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.38it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.14it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.57it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.04it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.31it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.20it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.97it/s]
+
+ 50%|████▉     | 496/1000 [01:06<01:00,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.42it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:08<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.55it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:55,  8.84it/s]
+
+ 51%|█████     | 512/1000 [01:09<00:49,  9.87it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:07,  7.25it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.50it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.08it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.04it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.39it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:12<00:48,  9.68it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.91it/s]
+
+ 55%|█████▌    | 551/1000 [01:15<00:59,  7.50it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.17it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.75it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.83it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.04it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.96it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.14it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.51it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:50,  8.16it/s]
+
+ 59%|█████▉    | 593/1000 [01:21<01:00,  6.78it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.38it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.86it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:03,  6.36it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:03,  6.33it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.65it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:10,  5.65it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.06it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:50,  7.83it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.46it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.72it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.78it/s]
+
+ 62%|██████▏   | 616/1000 [01:24<00:50,  7.60it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.75it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.10it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.20it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.45it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:44,  8.42it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:52,  7.09it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<01:00,  6.16it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.80it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:39,  9.37it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.68it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.04it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.93it/s]
+
+ 64%|██████▍   | 640/1000 [01:27<00:39,  9.13it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.94it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.57it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.09it/s]
+
+ 65%|██████▍   | 647/1000 [01:28<00:48,  7.35it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:55,  6.39it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:54,  6.46it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:56,  6.20it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:45,  7.59it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:39,  8.81it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.05it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:57,  5.96it/s]
+
+ 66%|██████▌   | 661/1000 [01:30<00:45,  7.52it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.46it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.75it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:03,  5.31it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.60it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:04,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.69it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.62it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.25it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.64it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.67it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.94it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.90it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.86it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.85it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.15it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.83it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.22it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.37it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:48,  6.31it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.92it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.03it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:40,  7.45it/s]
+
+ 70%|███████   | 702/1000 [01:36<00:53,  5.56it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.16it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.39it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:42,  6.92it/s]
+
+ 71%|███████   | 709/1000 [01:37<00:38,  7.55it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.45it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.62it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.78it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.28it/s]
+
+ 72%|███████▏  | 717/1000 [01:38<00:35,  7.93it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.73it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.89it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.43it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.11it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.18it/s]
+
+ 72%|███████▏  | 724/1000 [01:39<00:33,  8.19it/s]
+
+ 72%|███████▎  | 725/1000 [01:39<00:33,  8.27it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:27, 10.11it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.58it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.30it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.51it/s]
+
+ 74%|███████▎  | 735/1000 [01:40<00:26,  9.85it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.50it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.75it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.29it/s]
+
+ 74%|███████▍  | 743/1000 [01:41<00:30,  8.35it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.59it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.41it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.62it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+
+ 76%|███████▌  | 758/1000 [01:43<00:32,  7.34it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:33,  7.11it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:28,  8.53it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.59it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:36,  6.55it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.42it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:30,  7.60it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.50it/s]
+
+ 77%|███████▋  | 772/1000 [01:45<00:28,  8.06it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.35it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.73it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.85it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.55it/s]
+
+ 78%|███████▊  | 780/1000 [01:46<00:31,  6.91it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.34it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.13it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.51it/s]
+
+ 79%|███████▉  | 788/1000 [01:47<00:25,  8.35it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.96it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.19it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:27,  7.70it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:26,  7.95it/s]
+
+ 80%|███████▉  | 795/1000 [01:48<00:28,  7.18it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.84it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.00it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:49<00:25,  7.63it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.82it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.15it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.94it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.18it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.20it/s]
+
+ 81%|████████▏ | 813/1000 [01:50<00:20,  8.92it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:20,  9.22it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.08it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.12it/s]
+
+ 82%|████████▏ | 821/1000 [01:51<00:22,  8.01it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.80it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.73it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.97it/s]
+
+ 83%|████████▎ | 826/1000 [01:52<00:30,  5.77it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:33,  5.14it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:30,  5.65it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  6.92it/s]
+
+ 83%|████████▎ | 832/1000 [01:53<00:29,  5.69it/s]
+
+ 83%|████████▎ | 833/1000 [01:53<00:27,  6.17it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  5.98it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.50it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.10it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.43it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.60it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:20,  7.82it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.31it/s]
+
+ 84%|████████▍ | 845/1000 [01:55<00:27,  5.70it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.13it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.43it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.44it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.83it/s]
+
+ 85%|████████▌ | 853/1000 [01:56<00:17,  8.41it/s]
+
+ 86%|████████▌ | 855/1000 [01:56<00:14,  9.99it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.13it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.47it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.74it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.61it/s]
+
+ 86%|████████▋ | 865/1000 [01:57<00:11, 11.62it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.25it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.20it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.20it/s]
+
+ 87%|████████▋ | 873/1000 [01:58<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.61it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.86it/s]
+
+ 88%|████████▊ | 879/1000 [01:59<00:17,  6.83it/s]
+
+ 88%|████████▊ | 881/1000 [01:59<00:15,  7.57it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.52it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.89it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.10it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.29it/s]
+
+ 89%|████████▉ | 890/1000 [02:00<00:11,  9.73it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.73it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.58it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.81it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.69it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.83it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.04it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.57it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.57it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.29it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.03it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.32it/s]
+
+ 92%|█████████▏| 918/1000 [02:04<00:14,  5.57it/s]
+
+ 92%|█████████▏| 919/1000 [02:04<00:14,  5.69it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.57it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.92it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.15it/s]
+
+ 93%|█████████▎| 926/1000 [02:05<00:08,  8.64it/s]
+
+ 93%|█████████▎| 927/1000 [02:05<00:09,  8.08it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.62it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.46it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.81it/s]
+
+ 93%|█████████▎| 933/1000 [02:06<00:10,  6.60it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.16it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.81it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.03it/s]
+
+ 94%|█████████▍| 940/1000 [02:07<00:07,  8.12it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.33it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.07it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.29it/s]
+
+ 95%|█████████▍| 948/1000 [02:08<00:05,  9.62it/s]
+
+ 95%|█████████▌| 950/1000 [02:08<00:05,  8.79it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.39it/s]
+
+ 95%|█████████▌| 954/1000 [02:09<00:07,  6.15it/s]
+
+ 96%|█████████▌| 956/1000 [02:09<00:05,  7.39it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:10<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.41it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.40it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.31it/s]
+
+ 97%|█████████▋| 968/1000 [02:11<00:04,  7.98it/s]
+
+ 97%|█████████▋| 970/1000 [02:11<00:03,  9.19it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  7.91it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.09it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.65it/s]
+
+ 98%|█████████▊| 978/1000 [02:12<00:02, 10.69it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.23it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  6.95it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.38it/s]
+
+ 98%|█████████▊| 984/1000 [02:13<00:02,  6.86it/s]
+
+ 99%|█████████▊| 986/1000 [02:13<00:01,  8.30it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.72it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.97it/s]
+
+ 99%|█████████▉| 991/1000 [02:14<00:01,  6.13it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.97it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:15<00:00,  5.52it/s]
+
+100%|█████████▉| 997/1000 [02:15<00:00,  7.04it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.7774', 'eval_runtime': '135.8', 'eval_samples_per_second': '7.363', 'eval_steps_per_second': '7.363', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856}
+
+ 50%|████▉     | 900/1813 [1:40:43<1:47:00,  7.03s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+
+                                                   
+ 50%|████▉     | 901/1813 [1:40:49<12:02:31, 47.53s/it]
+ 50%|████▉     | 902/1813 [1:40:55<8:51:55, 35.03s/it] 
+ 50%|████▉     | 903/1813 [1:41:01<6:39:14, 26.32s/it]
+ 50%|████▉     | 904/1813 [1:41:07<5:06:25, 20.23s/it]
+ 50%|████▉     | 905/1813 [1:41:15<4:10:10, 16.53s/it]
+ 50%|████▉     | 906/1813 [1:41:20<3:19:00, 13.17s/it]
+ 50%|█████     | 907/1813 [1:41:24<2:38:43, 10.51s/it]
+ 50%|█████     | 908/1813 [1:41:30<2:15:30,  8.98s/it]
+ 50%|█████     | 909/1813 [1:41:36<2:04:16,  8.25s/it]
+ 50%|█████     | 910/1813 [1:41:42<1:54:22,  7.60s/it]
+                                                      
+{'loss': '0.9143', 'grad_norm': '1.114', 'learning_rate': '0.0001137', 'epoch': '0.5021', 'num_input_tokens_seen': 6115160, 'train_runtime': '6103', 'train_tokens_per_second': '1002'}
+
+ 50%|█████     | 910/1813 [1:41:42<1:54:22,  7.60s/it]
+ 50%|█████     | 911/1813 [1:41:48<1:44:12,  6.93s/it]
+ 50%|█████     | 912/1813 [1:41:53<1:37:25,  6.49s/it]
+ 50%|█████     | 913/1813 [1:41:58<1:31:32,  6.10s/it]
+ 50%|█████     | 914/1813 [1:42:03<1:24:52,  5.66s/it]
+ 50%|█████     | 915/1813 [1:42:11<1:33:51,  6.27s/it]
+ 51%|█████     | 916/1813 [1:42:18<1:37:30,  6.52s/it]
+ 51%|█████     | 917/1813 [1:42:25<1:38:16,  6.58s/it]
+ 51%|█████     | 918/1813 [1:42:30<1:33:37,  6.28s/it]
+ 51%|█████     | 919/1813 [1:42:36<1:33:37,  6.28s/it]
+ 51%|█████     | 920/1813 [1:42:42<1:30:26,  6.08s/it]
+                                                      
+{'loss': '0.9424', 'grad_norm': '0.8236', 'learning_rate': '0.0001118', 'epoch': '0.5076', 'num_input_tokens_seen': 6178904, 'train_runtime': '6163', 'train_tokens_per_second': '1003'}
+
+ 51%|█████     | 920/1813 [1:42:42<1:30:26,  6.08s/it]
+ 51%|█████     | 921/1813 [1:42:48<1:29:04,  5.99s/it]
+ 51%|█████     | 922/1813 [1:42:54<1:28:23,  5.95s/it]
+ 51%|█████     | 923/1813 [1:43:00<1:29:22,  6.03s/it]
+ 51%|█████     | 924/1813 [1:43:07<1:33:40,  6.32s/it]
+ 51%|█████     | 925/1813 [1:43:13<1:31:07,  6.16s/it]
+ 51%|█████     | 926/1813 [1:43:19<1:33:00,  6.29s/it]
+ 51%|█████     | 927/1813 [1:43:25<1:31:28,  6.19s/it]
+ 51%|█████     | 928/1813 [1:43:31<1:28:32,  6.00s/it]
+ 51%|█████     | 929/1813 [1:43:37<1:28:20,  6.00s/it]
+ 51%|█████▏    | 930/1813 [1:43:44<1:32:08,  6.26s/it]
+                                                      
+{'loss': '0.9029', 'grad_norm': '0.9303', 'learning_rate': '0.0001099', 'epoch': '0.5131', 'num_input_tokens_seen': 6244360, 'train_runtime': '6224', 'train_tokens_per_second': '1003'}
+
+ 51%|█████▏    | 930/1813 [1:43:44<1:32:08,  6.26s/it]
+ 51%|█████▏    | 931/1813 [1:43:51<1:35:28,  6.50s/it]
+ 51%|█████▏    | 932/1813 [1:43:56<1:31:45,  6.25s/it]
+ 51%|█████▏    | 933/1813 [1:44:02<1:29:53,  6.13s/it]
+ 52%|█████▏    | 934/1813 [1:44:09<1:31:59,  6.28s/it]
+ 52%|█████▏    | 935/1813 [1:44:18<1:43:11,  7.05s/it]
+ 52%|█████▏    | 936/1813 [1:44:24<1:40:17,  6.86s/it]
+ 52%|█████▏    | 937/1813 [1:44:30<1:37:25,  6.67s/it]
+ 52%|█████▏    | 938/1813 [1:44:37<1:39:10,  6.80s/it]
+ 52%|█████▏    | 939/1813 [1:44:43<1:35:33,  6.56s/it]
+ 52%|█████▏    | 940/1813 [1:44:51<1:37:50,  6.72s/it]
+                                                      
+{'loss': '0.8899', 'grad_norm': '1.102', 'learning_rate': '0.000108', 'epoch': '0.5186', 'num_input_tokens_seen': 6318968, 'train_runtime': '6291', 'train_tokens_per_second': '1004'}
+
+ 52%|█████▏    | 940/1813 [1:44:51<1:37:50,  6.72s/it]
+ 52%|█████▏    | 941/1813 [1:44:56<1:31:41,  6.31s/it]
+ 52%|█████▏    | 942/1813 [1:45:02<1:28:43,  6.11s/it]
+ 52%|█████▏    | 943/1813 [1:45:07<1:26:29,  5.97s/it]
+ 52%|█████▏    | 944/1813 [1:45:13<1:26:25,  5.97s/it]
+ 52%|█████▏    | 945/1813 [1:45:18<1:23:11,  5.75s/it]
+ 52%|█████▏    | 946/1813 [1:45:25<1:24:53,  5.87s/it]
+ 52%|█████▏    | 947/1813 [1:45:31<1:26:48,  6.01s/it]
+ 52%|█████▏    | 948/1813 [1:45:37<1:27:52,  6.10s/it]
+ 52%|█████▏    | 949/1813 [1:45:42<1:23:34,  5.80s/it]
+ 52%|█████▏    | 950/1813 [1:45:49<1:26:06,  5.99s/it]
+                                                      
+{'loss': '0.9461', 'grad_norm': '1.297', 'learning_rate': '0.0001061', 'epoch': '0.5241', 'num_input_tokens_seen': 6379704, 'train_runtime': '6349', 'train_tokens_per_second': '1005'}
+
+ 52%|█████▏    | 950/1813 [1:45:49<1:26:06,  5.99s/it]
+ 52%|█████▏    | 951/1813 [1:45:54<1:23:41,  5.83s/it]
+ 53%|█████▎    | 952/1813 [1:46:00<1:22:17,  5.73s/it]
+ 53%|█████▎    | 953/1813 [1:46:06<1:23:00,  5.79s/it]
+ 53%|█████▎    | 954/1813 [1:46:11<1:22:13,  5.74s/it]
+ 53%|█████▎    | 955/1813 [1:46:17<1:21:50,  5.72s/it]
+ 53%|█████▎    | 956/1813 [1:46:22<1:21:03,  5.67s/it]
+ 53%|█████▎    | 957/1813 [1:46:28<1:20:23,  5.63s/it]
+ 53%|█████▎    | 958/1813 [1:46:33<1:15:27,  5.30s/it]
+ 53%|█████▎    | 959/1813 [1:46:38<1:17:40,  5.46s/it]
+ 53%|█████▎    | 960/1813 [1:46:46<1:29:03,  6.26s/it]
+                                                      
+{'loss': '0.9544', 'grad_norm': '0.8503', 'learning_rate': '0.0001042', 'epoch': '0.5297', 'num_input_tokens_seen': 6441656, 'train_runtime': '6407', 'train_tokens_per_second': '1005'}
+
+ 53%|█████▎    | 960/1813 [1:46:46<1:29:03,  6.26s/it]
+ 53%|█████▎    | 961/1813 [1:46:53<1:30:00,  6.34s/it]
+ 53%|█████▎    | 962/1813 [1:47:01<1:35:58,  6.77s/it]
+ 53%|█████▎    | 963/1813 [1:47:08<1:36:12,  6.79s/it]
+ 53%|█████▎    | 964/1813 [1:47:14<1:32:17,  6.52s/it]
+ 53%|█████▎    | 965/1813 [1:47:19<1:29:22,  6.32s/it]
+ 53%|█████▎    | 966/1813 [1:47:27<1:34:17,  6.68s/it]
+ 53%|█████▎    | 967/1813 [1:47:34<1:37:14,  6.90s/it]
+ 53%|█████▎    | 968/1813 [1:47:40<1:31:13,  6.48s/it]
+ 53%|█████▎    | 969/1813 [1:47:46<1:28:49,  6.31s/it]
+ 54%|█████▎    | 970/1813 [1:47:50<1:21:45,  5.82s/it]
+                                                      
+{'loss': '0.9867', 'grad_norm': '0.9779', 'learning_rate': '0.0001024', 'epoch': '0.5352', 'num_input_tokens_seen': 6512320, 'train_runtime': '6471', 'train_tokens_per_second': '1006'}
+
+ 54%|█████▎    | 970/1813 [1:47:50<1:21:45,  5.82s/it]
+ 54%|█████▎    | 971/1813 [1:47:57<1:25:17,  6.08s/it]
+ 54%|█████▎    | 972/1813 [1:48:03<1:25:56,  6.13s/it]
+ 54%|█████▎    | 973/1813 [1:48:09<1:24:49,  6.06s/it]
+ 54%|█████▎    | 974/1813 [1:48:18<1:34:29,  6.76s/it]
+ 54%|█████▍    | 975/1813 [1:48:24<1:33:28,  6.69s/it]
+ 54%|█████▍    | 976/1813 [1:48:30<1:28:58,  6.38s/it]
+ 54%|█████▍    | 977/1813 [1:48:38<1:35:00,  6.82s/it]
+ 54%|█████▍    | 978/1813 [1:48:44<1:32:41,  6.66s/it]
+ 54%|█████▍    | 979/1813 [1:48:51<1:34:10,  6.78s/it]
+ 54%|█████▍    | 980/1813 [1:48:57<1:32:34,  6.67s/it]
+                                                      
+{'loss': '0.9339', 'grad_norm': '1.53', 'learning_rate': '0.0001005', 'epoch': '0.5407', 'num_input_tokens_seen': 6586672, 'train_runtime': '6538', 'train_tokens_per_second': '1007'}
+
+ 54%|█████▍    | 980/1813 [1:48:57<1:32:34,  6.67s/it]
+ 54%|█████▍    | 981/1813 [1:49:04<1:32:19,  6.66s/it]
+ 54%|█████▍    | 982/1813 [1:49:09<1:25:45,  6.19s/it]
+ 54%|█████▍    | 983/1813 [1:49:14<1:21:51,  5.92s/it]
+ 54%|█████▍    | 984/1813 [1:49:20<1:20:00,  5.79s/it]
+ 54%|█████▍    | 985/1813 [1:49:27<1:23:48,  6.07s/it]
+ 54%|█████▍    | 986/1813 [1:49:34<1:29:53,  6.52s/it]
+ 54%|█████▍    | 987/1813 [1:49:40<1:28:24,  6.42s/it]
+ 54%|█████▍    | 988/1813 [1:49:46<1:24:29,  6.14s/it]
+ 55%|█████▍    | 989/1813 [1:49:51<1:19:31,  5.79s/it]
+ 55%|█████▍    | 990/1813 [1:49:59<1:28:09,  6.43s/it]
+                                                      
+{'loss': '0.8833', 'grad_norm': '1.01', 'learning_rate': '9.858e-05', 'epoch': '0.5462', 'num_input_tokens_seen': 6653096, 'train_runtime': '6599', 'train_tokens_per_second': '1008'}
+
+ 55%|█████▍    | 990/1813 [1:49:59<1:28:09,  6.43s/it]
+ 55%|█████▍    | 991/1813 [1:50:04<1:23:27,  6.09s/it]
+ 55%|█████▍    | 992/1813 [1:50:11<1:28:32,  6.47s/it]
+ 55%|█████▍    | 993/1813 [1:50:17<1:26:00,  6.29s/it]
+ 55%|█████▍    | 994/1813 [1:50:26<1:34:36,  6.93s/it]
+ 55%|█████▍    | 995/1813 [1:50:32<1:30:13,  6.62s/it]
+ 55%|█████▍    | 996/1813 [1:50:38<1:28:22,  6.49s/it]
+ 55%|█████▍    | 997/1813 [1:50:44<1:25:06,  6.26s/it]
+ 55%|█████▌    | 998/1813 [1:50:48<1:19:33,  5.86s/it]
+ 55%|█████▌    | 999/1813 [1:50:54<1:18:45,  5.80s/it]
+ 55%|█████▌    | 1000/1813 [1:50:59<1:15:25,  5.57s/it]
+                                                       
+{'loss': '1', 'grad_norm': '1.051', 'learning_rate': '9.669e-05', 'epoch': '0.5517', 'num_input_tokens_seen': 6716784, 'train_runtime': '6660', 'train_tokens_per_second': '1009'}
+
+ 55%|█████▌    | 1000/1813 [1:50:59<1:15:25,  5.57s/it]
+ 55%|█████▌    | 1001/1813 [1:51:06<1:19:02,  5.84s/it]
+ 55%|█████▌    | 1002/1813 [1:51:12<1:20:18,  5.94s/it]
+ 55%|█████▌    | 1003/1813 [1:51:19<1:23:45,  6.20s/it]
+ 55%|█████▌    | 1004/1813 [1:51:24<1:19:40,  5.91s/it]
+ 55%|█████▌    | 1005/1813 [1:51:31<1:24:24,  6.27s/it]
+ 55%|█████▌    | 1006/1813 [1:51:38<1:25:59,  6.39s/it]
+ 56%|█████▌    | 1007/1813 [1:51:43<1:23:02,  6.18s/it]
+ 56%|█████▌    | 1008/1813 [1:51:49<1:21:08,  6.05s/it]
+ 56%|█████▌    | 1009/1813 [1:51:55<1:20:53,  6.04s/it]
+ 56%|█████▌    | 1010/1813 [1:52:01<1:18:57,  5.90s/it]
+                                                       
+{'loss': '0.9323', 'grad_norm': '1.135', 'learning_rate': '9.481e-05', 'epoch': '0.5572', 'num_input_tokens_seen': 6783768, 'train_runtime': '6721', 'train_tokens_per_second': '1009'}
+
+ 56%|█████▌    | 1010/1813 [1:52:01<1:18:57,  5.90s/it]
+ 56%|█████▌    | 1011/1813 [1:52:07<1:22:18,  6.16s/it]
+ 56%|█████▌    | 1012/1813 [1:52:12<1:16:32,  5.73s/it]
+ 56%|█████▌    | 1013/1813 [1:52:16<1:10:47,  5.31s/it]
+ 56%|█████▌    | 1014/1813 [1:52:25<1:22:42,  6.21s/it]
+ 56%|█████▌    | 1015/1813 [1:52:30<1:18:15,  5.88s/it]
+ 56%|█████▌    | 1016/1813 [1:52:35<1:15:26,  5.68s/it]
+ 56%|█████▌    | 1017/1813 [1:52:42<1:20:00,  6.03s/it]
+ 56%|█████▌    | 1018/1813 [1:52:48<1:21:00,  6.11s/it]
+ 56%|█████▌    | 1019/1813 [1:52:53<1:16:20,  5.77s/it]
+ 56%|█████▋    | 1020/1813 [1:52:58<1:13:12,  5.54s/it]
+                                                       
+{'loss': '0.8484', 'grad_norm': '1.222', 'learning_rate': '9.292e-05', 'epoch': '0.5628', 'num_input_tokens_seen': 6844992, 'train_runtime': '6779', 'train_tokens_per_second': '1010'}
+
+ 56%|█████▋    | 1020/1813 [1:52:58<1:13:12,  5.54s/it]
+ 56%|█████▋    | 1021/1813 [1:53:05<1:18:28,  5.95s/it]
+ 56%|█████▋    | 1022/1813 [1:53:11<1:19:02,  6.00s/it]
+ 56%|█████▋    | 1023/1813 [1:53:19<1:24:59,  6.46s/it]
+ 56%|█████▋    | 1024/1813 [1:53:25<1:24:38,  6.44s/it]
+ 57%|█████▋    | 1025/1813 [1:53:34<1:35:23,  7.26s/it]
+ 57%|█████▋    | 1026/1813 [1:53:40<1:28:24,  6.74s/it]
+ 57%|█████▋    | 1027/1813 [1:53:46<1:25:42,  6.54s/it]
+ 57%|█████▋    | 1028/1813 [1:53:53<1:29:00,  6.80s/it]
+ 57%|█████▋    | 1029/1813 [1:54:00<1:27:57,  6.73s/it]
+ 57%|█████▋    | 1030/1813 [1:54:07<1:28:25,  6.78s/it]
+                                                       
+{'loss': '0.9876', 'grad_norm': '0.8492', 'learning_rate': '9.104e-05', 'epoch': '0.5683', 'num_input_tokens_seen': 6920856, 'train_runtime': '6847', 'train_tokens_per_second': '1011'}
+
+ 57%|█████▋    | 1030/1813 [1:54:07<1:28:25,  6.78s/it]
+ 57%|█████▋    | 1031/1813 [1:54:13<1:24:26,  6.48s/it]
+ 57%|█████▋    | 1032/1813 [1:54:19<1:25:43,  6.59s/it]
+ 57%|█████▋    | 1033/1813 [1:54:26<1:24:14,  6.48s/it]
+ 57%|█████▋    | 1034/1813 [1:54:31<1:21:35,  6.28s/it]
+ 57%|█████▋    | 1035/1813 [1:54:38<1:21:05,  6.25s/it]
+ 57%|█████▋    | 1036/1813 [1:54:44<1:21:49,  6.32s/it]
+ 57%|█████▋    | 1037/1813 [1:54:50<1:19:28,  6.14s/it]
+ 57%|█████▋    | 1038/1813 [1:54:56<1:19:44,  6.17s/it]
+ 57%|█████▋    | 1039/1813 [1:55:03<1:22:59,  6.43s/it]
+ 57%|█████▋    | 1040/1813 [1:55:10<1:25:34,  6.64s/it]
+                                                       
+{'loss': '0.912', 'grad_norm': '0.8267', 'learning_rate': '8.916e-05', 'epoch': '0.5738', 'num_input_tokens_seen': 6988568, 'train_runtime': '6911', 'train_tokens_per_second': '1011'}
+
+ 57%|█████▋    | 1040/1813 [1:55:10<1:25:34,  6.64s/it]
+ 57%|█████▋    | 1041/1813 [1:55:16<1:22:35,  6.42s/it]
+ 57%|█████▋    | 1042/1813 [1:55:23<1:22:32,  6.42s/it]
+ 58%|█████▊    | 1043/1813 [1:55:28<1:19:00,  6.16s/it]
+ 58%|█████▊    | 1044/1813 [1:55:34<1:15:58,  5.93s/it]
+ 58%|█████▊    | 1045/1813 [1:55:39<1:15:02,  5.86s/it]
+ 58%|█████▊    | 1046/1813 [1:55:45<1:14:13,  5.81s/it]
+ 58%|█████▊    | 1047/1813 [1:55:52<1:17:11,  6.05s/it]
+ 58%|█████▊    | 1048/1813 [1:55:58<1:20:03,  6.28s/it]
+ 58%|█████▊    | 1049/1813 [1:56:04<1:17:32,  6.09s/it]
+ 58%|█████▊    | 1050/1813 [1:56:10<1:17:00,  6.06s/it]
+                                                       
+{'loss': '0.9023', 'grad_norm': '1.239', 'learning_rate': '8.728e-05', 'epoch': '0.5793', 'num_input_tokens_seen': 7051808, 'train_runtime': '6970', 'train_tokens_per_second': '1012'}
+
+ 58%|█████▊    | 1050/1813 [1:56:10<1:17:00,  6.06s/it]
+ 58%|█████▊    | 1051/1813 [1:56:15<1:13:47,  5.81s/it]
+ 58%|█████▊    | 1052/1813 [1:56:21<1:14:06,  5.84s/it]
+ 58%|█████▊    | 1053/1813 [1:56:27<1:14:44,  5.90s/it]
+ 58%|█████▊    | 1054/1813 [1:56:33<1:14:08,  5.86s/it]
+ 58%|█████▊    | 1055/1813 [1:56:38<1:11:43,  5.68s/it]
+ 58%|█████▊    | 1056/1813 [1:56:44<1:11:55,  5.70s/it]
+ 58%|█████▊    | 1057/1813 [1:56:51<1:15:12,  5.97s/it]
+ 58%|█████▊    | 1058/1813 [1:56:55<1:11:10,  5.66s/it]
+ 58%|█████▊    | 1059/1813 [1:57:00<1:07:06,  5.34s/it]
+ 58%|█████▊    | 1060/1813 [1:57:07<1:13:33,  5.86s/it]
+                                                       
+{'loss': '0.9028', 'grad_norm': '0.7496', 'learning_rate': '8.541e-05', 'epoch': '0.5848', 'num_input_tokens_seen': 7110624, 'train_runtime': '7028', 'train_tokens_per_second': '1012'}
+
+ 58%|█████▊    | 1060/1813 [1:57:07<1:13:33,  5.86s/it]
+ 59%|█████▊    | 1061/1813 [1:57:12<1:11:04,  5.67s/it]
+ 59%|█████▊    | 1062/1813 [1:57:17<1:08:28,  5.47s/it]
+ 59%|█████▊    | 1063/1813 [1:57:24<1:13:41,  5.90s/it]
+ 59%|█████▊    | 1064/1813 [1:57:32<1:19:39,  6.38s/it]
+ 59%|█████▊    | 1065/1813 [1:57:38<1:20:28,  6.46s/it]
+ 59%|█████▉    | 1066/1813 [1:57:46<1:24:02,  6.75s/it]
+ 59%|█████▉    | 1067/1813 [1:57:52<1:22:58,  6.67s/it]
+ 59%|█████▉    | 1068/1813 [1:57:58<1:18:07,  6.29s/it]
+ 59%|█████▉    | 1069/1813 [1:58:06<1:24:44,  6.83s/it]
+ 59%|█████▉    | 1070/1813 [1:58:12<1:20:50,  6.53s/it]
+                                                       
+{'loss': '0.9053', 'grad_norm': '1.243', 'learning_rate': '8.355e-05', 'epoch': '0.5903', 'num_input_tokens_seen': 7179728, 'train_runtime': '7092', 'train_tokens_per_second': '1012'}
+
+ 59%|█████▉    | 1070/1813 [1:58:12<1:20:50,  6.53s/it]
+ 59%|█████▉    | 1071/1813 [1:58:17<1:16:05,  6.15s/it]
+ 59%|█████▉    | 1072/1813 [1:58:23<1:16:50,  6.22s/it]
+ 59%|█████▉    | 1073/1813 [1:58:29<1:15:25,  6.12s/it]
+ 59%|█████▉    | 1074/1813 [1:58:35<1:12:54,  5.92s/it]
+ 59%|█████▉    | 1075/1813 [1:58:41<1:14:29,  6.06s/it]
+ 59%|█████▉    | 1076/1813 [1:58:48<1:15:57,  6.18s/it]
+ 59%|█████▉    | 1077/1813 [1:58:52<1:09:40,  5.68s/it]
+ 59%|█████▉    | 1078/1813 [1:58:57<1:07:16,  5.49s/it]
+ 60%|█████▉    | 1079/1813 [1:59:03<1:10:33,  5.77s/it]
+ 60%|█████▉    | 1080/1813 [1:59:10<1:12:53,  5.97s/it]
+                                                       
+{'loss': '0.9028', 'grad_norm': '0.7934', 'learning_rate': '8.169e-05', 'epoch': '0.5959', 'num_input_tokens_seen': 7240984, 'train_runtime': '7150', 'train_tokens_per_second': '1013'}
+
+ 60%|█████▉    | 1080/1813 [1:59:10<1:12:53,  5.97s/it]
+ 60%|█████▉    | 1081/1813 [1:59:16<1:14:01,  6.07s/it]
+ 60%|█████▉    | 1082/1813 [1:59:22<1:12:47,  5.97s/it]
+ 60%|█████▉    | 1083/1813 [1:59:28<1:12:59,  6.00s/it]
+ 60%|█████▉    | 1084/1813 [1:59:34<1:12:10,  5.94s/it]
+ 60%|█████▉    | 1085/1813 [1:59:40<1:11:53,  5.93s/it]
+ 60%|█████▉    | 1086/1813 [1:59:44<1:04:27,  5.32s/it]
+ 60%|█████▉    | 1087/1813 [1:59:49<1:05:34,  5.42s/it]
+ 60%|██████    | 1088/1813 [1:59:55<1:06:24,  5.50s/it]
+ 60%|██████    | 1089/1813 [2:00:02<1:10:16,  5.82s/it]
+ 60%|██████    | 1090/1813 [2:00:08<1:11:22,  5.92s/it]
+                                                       
+{'loss': '0.8074', 'grad_norm': '1.013', 'learning_rate': '7.983e-05', 'epoch': '0.6014', 'num_input_tokens_seen': 7301824, 'train_runtime': '7208', 'train_tokens_per_second': '1013'}
+
+ 60%|██████    | 1090/1813 [2:00:08<1:11:22,  5.92s/it]
+ 60%|██████    | 1091/1813 [2:00:14<1:12:13,  6.00s/it]
+ 60%|██████    | 1092/1813 [2:00:20<1:12:04,  6.00s/it]
+ 60%|██████    | 1093/1813 [2:00:26<1:10:42,  5.89s/it]
+ 60%|██████    | 1094/1813 [2:00:32<1:12:26,  6.05s/it]
+ 60%|██████    | 1095/1813 [2:00:38<1:13:17,  6.12s/it]
+ 60%|██████    | 1096/1813 [2:00:44<1:10:22,  5.89s/it]
+ 61%|██████    | 1097/1813 [2:00:49<1:10:04,  5.87s/it]
+ 61%|██████    | 1098/1813 [2:00:56<1:14:03,  6.22s/it]
+ 61%|██████    | 1099/1813 [2:01:03<1:13:43,  6.20s/it]
+ 61%|██████    | 1100/1813 [2:01:10<1:19:25,  6.68s/it]
+                                                       
+{'loss': '0.8957', 'grad_norm': '0.9555', 'learning_rate': '7.798e-05', 'epoch': '0.6069', 'num_input_tokens_seen': 7369528, 'train_runtime': '7271', 'train_tokens_per_second': '1014'}
+
+ 61%|██████    | 1100/1813 [2:01:10<1:19:25,  6.68s/it]
+ 61%|██████    | 1101/1813 [2:01:17<1:18:54,  6.65s/it]
+ 61%|██████    | 1102/1813 [2:01:24<1:20:49,  6.82s/it]
+ 61%|██████    | 1103/1813 [2:01:30<1:16:30,  6.46s/it]
+ 61%|██████    | 1104/1813 [2:01:37<1:20:24,  6.81s/it]
+ 61%|██████    | 1105/1813 [2:01:43<1:14:14,  6.29s/it]
+ 61%|██████    | 1106/1813 [2:01:48<1:12:33,  6.16s/it]
+ 61%|██████    | 1107/1813 [2:01:57<1:20:31,  6.84s/it]
+ 61%|██████    | 1108/1813 [2:02:03<1:16:35,  6.52s/it]
+ 61%|██████    | 1109/1813 [2:02:09<1:16:49,  6.55s/it]
+ 61%|██████    | 1110/1813 [2:02:17<1:20:18,  6.85s/it]
+                                                       
+{'loss': '0.9344', 'grad_norm': '1.018', 'learning_rate': '7.615e-05', 'epoch': '0.6124', 'num_input_tokens_seen': 7441040, 'train_runtime': '7337', 'train_tokens_per_second': '1014'}
+
+ 61%|██████    | 1110/1813 [2:02:17<1:20:18,  6.85s/it]
+ 61%|██████▏   | 1111/1813 [2:02:21<1:12:06,  6.16s/it]
+ 61%|██████▏   | 1112/1813 [2:02:27<1:10:55,  6.07s/it]
+ 61%|██████▏   | 1113/1813 [2:02:34<1:11:52,  6.16s/it]
+ 61%|██████▏   | 1114/1813 [2:02:41<1:15:00,  6.44s/it]
+ 62%|██████▏   | 1115/1813 [2:02:47<1:15:43,  6.51s/it]
+ 62%|██████▏   | 1116/1813 [2:02:53<1:14:31,  6.42s/it]
+ 62%|██████▏   | 1117/1813 [2:02:59<1:10:49,  6.11s/it]
+ 62%|██████▏   | 1118/1813 [2:03:04<1:06:08,  5.71s/it]
+ 62%|██████▏   | 1119/1813 [2:03:10<1:07:53,  5.87s/it]
+ 62%|██████▏   | 1120/1813 [2:03:16<1:10:04,  6.07s/it]
+                                                       
+{'loss': '0.872', 'grad_norm': '2.986', 'learning_rate': '7.432e-05', 'epoch': '0.6179', 'num_input_tokens_seen': 7504976, 'train_runtime': '7397', 'train_tokens_per_second': '1015'}
+
+ 62%|██████▏   | 1120/1813 [2:03:16<1:10:04,  6.07s/it]
+ 62%|██████▏   | 1121/1813 [2:03:24<1:16:01,  6.59s/it]
+ 62%|██████▏   | 1122/1813 [2:03:29<1:09:31,  6.04s/it]
+ 62%|██████▏   | 1123/1813 [2:03:36<1:14:34,  6.48s/it]
+ 62%|██████▏   | 1124/1813 [2:03:42<1:10:39,  6.15s/it]
+ 62%|██████▏   | 1125/1813 [2:03:49<1:12:15,  6.30s/it]
+ 62%|██████▏   | 1126/1813 [2:03:54<1:10:51,  6.19s/it]
+ 62%|██████▏   | 1127/1813 [2:04:00<1:07:36,  5.91s/it]
+ 62%|██████▏   | 1128/1813 [2:04:05<1:05:15,  5.72s/it]
+ 62%|██████▏   | 1129/1813 [2:04:10<1:03:17,  5.55s/it]
+ 62%|██████▏   | 1130/1813 [2:04:15<1:01:31,  5.40s/it]
+                                                       
+{'loss': '0.8494', 'grad_norm': '0.9379', 'learning_rate': '7.249e-05', 'epoch': '0.6234', 'num_input_tokens_seen': 7568576, 'train_runtime': '7456', 'train_tokens_per_second': '1015'}
+
+ 62%|██████▏   | 1130/1813 [2:04:15<1:01:31,  5.40s/it]
+ 62%|██████▏   | 1131/1813 [2:04:21<1:02:12,  5.47s/it]
+ 62%|██████▏   | 1132/1813 [2:04:27<1:03:37,  5.61s/it]
+ 62%|██████▏   | 1133/1813 [2:04:33<1:06:19,  5.85s/it]
+ 63%|██████▎   | 1134/1813 [2:04:41<1:12:59,  6.45s/it]
+ 63%|██████▎   | 1135/1813 [2:04:48<1:13:56,  6.54s/it]
+ 63%|██████▎   | 1136/1813 [2:04:55<1:14:56,  6.64s/it]
+ 63%|██████▎   | 1137/1813 [2:05:01<1:14:23,  6.60s/it]
+ 63%|██████▎   | 1138/1813 [2:05:08<1:15:36,  6.72s/it]
+ 63%|██████▎   | 1139/1813 [2:05:14<1:11:13,  6.34s/it]
+ 63%|██████▎   | 1140/1813 [2:05:19<1:08:35,  6.12s/it]
+                                                       
+{'loss': '0.9318', 'grad_norm': '1.038', 'learning_rate': '7.068e-05', 'epoch': '0.629', 'num_input_tokens_seen': 7639168, 'train_runtime': '7520', 'train_tokens_per_second': '1016'}
+
+ 63%|██████▎   | 1140/1813 [2:05:19<1:08:35,  6.12s/it]
+ 63%|██████▎   | 1141/1813 [2:05:26<1:09:07,  6.17s/it]
+ 63%|██████▎   | 1142/1813 [2:05:32<1:08:32,  6.13s/it]
+ 63%|██████▎   | 1143/1813 [2:05:37<1:07:47,  6.07s/it]
+ 63%|██████▎   | 1144/1813 [2:05:44<1:10:26,  6.32s/it]
+ 63%|██████▎   | 1145/1813 [2:05:51<1:11:13,  6.40s/it]
+ 63%|██████▎   | 1146/1813 [2:05:56<1:07:54,  6.11s/it]
+ 63%|██████▎   | 1147/1813 [2:06:04<1:12:43,  6.55s/it]
+ 63%|██████▎   | 1148/1813 [2:06:11<1:14:43,  6.74s/it]
+ 63%|██████▎   | 1149/1813 [2:06:17<1:11:54,  6.50s/it]
+ 63%|██████▎   | 1150/1813 [2:06:25<1:15:16,  6.81s/it]
+                                                       
+{'loss': '0.8448', 'grad_norm': '0.9389', 'learning_rate': '6.888e-05', 'epoch': '0.6345', 'num_input_tokens_seen': 7707832, 'train_runtime': '7585', 'train_tokens_per_second': '1016'}
+
+ 63%|██████▎   | 1150/1813 [2:06:25<1:15:16,  6.81s/it]
+ 63%|██████▎   | 1151/1813 [2:06:29<1:05:43,  5.96s/it]
+ 64%|██████▎   | 1152/1813 [2:06:34<1:03:41,  5.78s/it]
+ 64%|██████▎   | 1153/1813 [2:06:41<1:07:23,  6.13s/it]
+ 64%|██████▎   | 1154/1813 [2:06:47<1:06:21,  6.04s/it]
+ 64%|██████▎   | 1155/1813 [2:06:53<1:07:30,  6.16s/it]
+ 64%|██████▍   | 1156/1813 [2:06:59<1:07:39,  6.18s/it]
+ 64%|██████▍   | 1157/1813 [2:07:07<1:12:05,  6.59s/it]
+ 64%|██████▍   | 1158/1813 [2:07:14<1:12:14,  6.62s/it]
+ 64%|██████▍   | 1159/1813 [2:07:21<1:14:50,  6.87s/it]
+ 64%|██████▍   | 1160/1813 [2:07:27<1:12:56,  6.70s/it]
+                                                       
+{'loss': '0.9194', 'grad_norm': '0.7847', 'learning_rate': '6.709e-05', 'epoch': '0.64', 'num_input_tokens_seen': 7775872, 'train_runtime': '7648', 'train_tokens_per_second': '1017'}
+
+ 64%|██████▍   | 1160/1813 [2:07:27<1:12:56,  6.70s/it]
+ 64%|██████▍   | 1161/1813 [2:07:35<1:15:13,  6.92s/it]
+ 64%|██████▍   | 1162/1813 [2:07:42<1:14:45,  6.89s/it]
+ 64%|██████▍   | 1163/1813 [2:07:48<1:13:01,  6.74s/it]
+ 64%|██████▍   | 1164/1813 [2:07:55<1:12:14,  6.68s/it]
+ 64%|██████▍   | 1165/1813 [2:08:01<1:11:42,  6.64s/it]
+ 64%|██████▍   | 1166/1813 [2:08:07<1:09:34,  6.45s/it]
+ 64%|██████▍   | 1167/1813 [2:08:14<1:11:04,  6.60s/it]
+ 64%|██████▍   | 1168/1813 [2:08:21<1:10:23,  6.55s/it]
+ 64%|██████▍   | 1169/1813 [2:08:27<1:09:31,  6.48s/it]
+ 65%|██████▍   | 1170/1813 [2:08:33<1:07:43,  6.32s/it]
+                                                       
+{'loss': '0.8965', 'grad_norm': '1.08', 'learning_rate': '6.532e-05', 'epoch': '0.6455', 'num_input_tokens_seen': 7846688, 'train_runtime': '7713', 'train_tokens_per_second': '1017'}
+
+ 65%|██████▍   | 1170/1813 [2:08:33<1:07:43,  6.32s/it]
+ 65%|██████▍   | 1171/1813 [2:08:40<1:11:26,  6.68s/it]
+ 65%|██████▍   | 1172/1813 [2:08:48<1:13:27,  6.88s/it]
+ 65%|██████▍   | 1173/1813 [2:08:53<1:09:30,  6.52s/it]
+ 65%|██████▍   | 1174/1813 [2:08:59<1:06:28,  6.24s/it]
+ 65%|██████▍   | 1175/1813 [2:09:05<1:05:40,  6.18s/it]
+ 65%|██████▍   | 1176/1813 [2:09:11<1:06:43,  6.28s/it]
+ 65%|██████▍   | 1177/1813 [2:09:17<1:05:29,  6.18s/it]
+ 65%|██████▍   | 1178/1813 [2:09:23<1:04:49,  6.13s/it]
+ 65%|██████▌   | 1179/1813 [2:09:30<1:04:49,  6.14s/it]
+ 65%|██████▌   | 1180/1813 [2:09:36<1:06:38,  6.32s/it]
+                                                       
+{'loss': '0.8674', 'grad_norm': '1.108', 'learning_rate': '6.355e-05', 'epoch': '0.651', 'num_input_tokens_seen': 7914072, 'train_runtime': '7777', 'train_tokens_per_second': '1018'}
+
+ 65%|██████▌   | 1180/1813 [2:09:36<1:06:38,  6.32s/it]
+ 65%|██████▌   | 1181/1813 [2:09:44<1:11:42,  6.81s/it]
+ 65%|██████▌   | 1182/1813 [2:09:52<1:14:46,  7.11s/it]
+ 65%|██████▌   | 1183/1813 [2:09:59<1:14:40,  7.11s/it]
+ 65%|██████▌   | 1184/1813 [2:10:05<1:10:40,  6.74s/it]
+ 65%|██████▌   | 1185/1813 [2:10:13<1:13:41,  7.04s/it]
+ 65%|██████▌   | 1186/1813 [2:10:20<1:14:30,  7.13s/it]
+ 65%|██████▌   | 1187/1813 [2:10:27<1:13:47,  7.07s/it]
+ 66%|██████▌   | 1188/1813 [2:10:32<1:08:12,  6.55s/it]
+ 66%|██████▌   | 1189/1813 [2:10:38<1:03:44,  6.13s/it]
+ 66%|██████▌   | 1190/1813 [2:10:44<1:04:24,  6.20s/it]
+                                                       
+{'loss': '0.9325', 'grad_norm': '1.167', 'learning_rate': '6.18e-05', 'epoch': '0.6566', 'num_input_tokens_seen': 7989064, 'train_runtime': '7844', 'train_tokens_per_second': '1018'}
+
+ 66%|██████▌   | 1190/1813 [2:10:44<1:04:24,  6.20s/it]
+ 66%|██████▌   | 1191/1813 [2:10:50<1:03:17,  6.11s/it]
+ 66%|██████▌   | 1192/1813 [2:10:57<1:05:02,  6.28s/it]
+ 66%|██████▌   | 1193/1813 [2:11:02<1:03:58,  6.19s/it]
+ 66%|██████▌   | 1194/1813 [2:11:09<1:05:01,  6.30s/it]
+ 66%|██████▌   | 1195/1813 [2:11:14<1:00:29,  5.87s/it]
+ 66%|██████▌   | 1196/1813 [2:11:20<1:01:23,  5.97s/it]
+ 66%|██████▌   | 1197/1813 [2:11:27<1:02:42,  6.11s/it]
+ 66%|██████▌   | 1198/1813 [2:11:33<1:02:32,  6.10s/it]
+ 66%|██████▌   | 1199/1813 [2:11:39<1:02:26,  6.10s/it]
+ 66%|██████▌   | 1200/1813 [2:11:46<1:06:29,  6.51s/it]
+                                                       
+{'loss': '0.8622', 'grad_norm': '0.9519', 'learning_rate': '6.006e-05', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352, 'train_runtime': '7907', 'train_tokens_per_second': '1019'}
+
+ 66%|██████▌   | 1200/1813 [2:11:46<1:06:29,  6.51s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.78it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+
+  1%|          | 7/1000 [00:00<02:00,  8.26it/s]
+
+  1%|          | 9/1000 [00:01<02:35,  6.37it/s]
+
+  1%|          | 11/1000 [00:01<02:04,  7.95it/s]
+
+  1%|▏         | 13/1000 [00:01<02:22,  6.94it/s]
+
+  1%|▏         | 14/1000 [00:01<02:25,  6.78it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.94it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.71it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.04it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  6.00it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  6.00it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.68it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.12it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.35it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.37it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.90it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+
+  6%|▌         | 55/1000 [00:07<01:29, 10.59it/s]
+
+  6%|▌         | 57/1000 [00:07<01:42,  9.22it/s]
+
+  6%|▌         | 59/1000 [00:08<01:55,  8.16it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.30it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.73it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.13it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.21it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.83it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.33it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.46it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.22it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.06it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.53it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.81it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.24it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.09it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:05,  6.93it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:41,  8.54it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:26, 10.01it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:30,  9.59it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.86it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.90it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.00it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.87it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25, 10.06it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.47it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.90it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.04it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.42it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.98it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.62it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.15it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:22,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.20it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.67it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.78it/s]
+
+ 20%|██        | 204/1000 [00:26<01:34,  8.45it/s]
+
+ 21%|██        | 206/1000 [00:26<01:21,  9.79it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.45it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.43it/s]
+
+ 21%|██        | 212/1000 [00:27<01:22,  9.61it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.72it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:27,  8.94it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:46,  7.38it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:31,  8.56it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:05,  6.21it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:05,  6.22it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.51it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.02it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.08it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.40it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.94it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.47it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.37it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 244/1000 [00:32<02:03,  6.10it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:54,  6.58it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:27,  8.58it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.87it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:07,  5.88it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:09,  5.77it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  5.99it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.70it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:34<01:36,  7.72it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.55it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.04it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.71it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.71it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.08it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.71it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.75it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.99it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:37,  7.14it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.77it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:41<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:41,  6.80it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.82it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.22it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.25it/s]
+
+ 32%|███▏      | 315/1000 [00:42<01:06, 10.24it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.57it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:43<01:39,  6.85it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.41it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.73it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:31,  7.39it/s]
+
+ 33%|███▎      | 328/1000 [00:44<01:33,  7.15it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.05it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+
+ 34%|███▍      | 344/1000 [00:46<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.73it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.37it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.89it/s]
+
+ 35%|███▌      | 350/1000 [00:47<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.63it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.54it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.58it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.43it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:58,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:51<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.77it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.21it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.19it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.65it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.37it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.79it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.76it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:54<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:56<01:23,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.27it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.46it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.94it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.38it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.97it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:12,  7.83it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.52it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.02it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.78it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.52it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.31it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.71it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.47it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.28it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.85it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.05it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.98it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.45it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.57it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.99it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.30it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.55it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.55it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.13it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:47,  9.73it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.77it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.51it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.52it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.28it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.86it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.05it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.26it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.99it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.57it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.93it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.17it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.40it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.68it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:45,  8.48it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.74it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.83it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.11it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.22it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.65it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.44it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.97it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.58it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.49it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.64it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.95it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.91it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.88it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.27it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.23it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.70it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.35it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.15it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.22it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.61it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.55it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.79it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.35it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.09it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:40,  6.15it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.34it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:32,  7.52it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:33,  7.23it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.64it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.56it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:36,  6.55it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.42it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:30,  7.73it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.37it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.77it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:25,  8.39it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  9.00it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.96it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.20it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.23it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.31it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.81it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.81it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.12it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.46it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.43it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.56it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.50it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.88it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.48it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:13, 10.22it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.53it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.63it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.72it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.33it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.29it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.29it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.59it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.62it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.85it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.87it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.07it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.32it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.60it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.79it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.70it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.57it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.93it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.09it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.64it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.62it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.85it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.06it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.16it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.84it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.42it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.41it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.51it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.03it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.21it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.80it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.45it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.07it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.45it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.52it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                       
+
+                                                   
+{'eval_loss': '0.7551', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.385', 'eval_steps_per_second': '7.385', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352}
+
+ 66%|██████▌   | 1200/1813 [2:14:02<1:06:29,  6.51s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 66%|██████▌   | 1201/1813 [2:14:07<7:57:45, 46.84s/it]
+ 66%|██████▋   | 1202/1813 [2:14:14<5:54:42, 34.83s/it]
+ 66%|██████▋   | 1203/1813 [2:14:20<4:27:22, 26.30s/it]
+ 66%|██████▋   | 1204/1813 [2:14:27<3:26:51, 20.38s/it]
+ 66%|██████▋   | 1205/1813 [2:14:34<2:45:04, 16.29s/it]
+ 67%|██████▋   | 1206/1813 [2:14:40<2:14:25, 13.29s/it]
+ 67%|██████▋   | 1207/1813 [2:14:46<1:53:19, 11.22s/it]
+ 67%|██████▋   | 1208/1813 [2:14:54<1:42:26, 10.16s/it]
+ 67%|██████▋   | 1209/1813 [2:15:01<1:32:15,  9.16s/it]
+ 67%|██████▋   | 1210/1813 [2:15:07<1:22:47,  8.24s/it]
+                                                       
+{'loss': '0.921', 'grad_norm': '1.073', 'learning_rate': '5.833e-05', 'epoch': '0.6676', 'num_input_tokens_seen': 8125224, 'train_runtime': '8107', 'train_tokens_per_second': '1002'}
+
+ 67%|██████▋   | 1210/1813 [2:15:07<1:22:47,  8.24s/it]
+ 67%|██████▋   | 1211/1813 [2:15:14<1:18:48,  7.86s/it]
+ 67%|██████▋   | 1212/1813 [2:15:18<1:08:07,  6.80s/it]
+ 67%|██████▋   | 1213/1813 [2:15:25<1:07:15,  6.73s/it]
+ 67%|██████▋   | 1214/1813 [2:15:33<1:12:26,  7.26s/it]
+ 67%|██████▋   | 1215/1813 [2:15:40<1:10:04,  7.03s/it]
+ 67%|██████▋   | 1216/1813 [2:15:47<1:10:32,  7.09s/it]
+ 67%|██████▋   | 1217/1813 [2:15:53<1:07:31,  6.80s/it]
+ 67%|██████▋   | 1218/1813 [2:16:00<1:06:14,  6.68s/it]
+ 67%|██████▋   | 1219/1813 [2:16:05<1:02:35,  6.32s/it]
+ 67%|██████▋   | 1220/1813 [2:16:13<1:07:58,  6.88s/it]
+                                                       
+{'loss': '0.9406', 'grad_norm': '14.71', 'learning_rate': '5.662e-05', 'epoch': '0.6731', 'num_input_tokens_seen': 8197488, 'train_runtime': '8174', 'train_tokens_per_second': '1003'}
+
+ 67%|██████▋   | 1220/1813 [2:16:13<1:07:58,  6.88s/it]
+ 67%|██████▋   | 1221/1813 [2:16:18<1:03:08,  6.40s/it]
+ 67%|██████▋   | 1222/1813 [2:16:23<58:47,  5.97s/it]  
+ 67%|██████▋   | 1223/1813 [2:16:29<57:21,  5.83s/it]
+ 68%|██████▊   | 1224/1813 [2:16:34<55:27,  5.65s/it]
+ 68%|██████▊   | 1225/1813 [2:16:40<57:01,  5.82s/it]
+ 68%|██████▊   | 1226/1813 [2:16:47<58:00,  5.93s/it]
+ 68%|██████▊   | 1227/1813 [2:16:53<1:00:00,  6.14s/it]
+ 68%|██████▊   | 1228/1813 [2:16:59<57:24,  5.89s/it]  
+ 68%|██████▊   | 1229/1813 [2:17:05<59:12,  6.08s/it]
+ 68%|██████▊   | 1230/1813 [2:17:11<58:48,  6.05s/it]
+                                                     
+{'loss': '0.7933', 'grad_norm': '0.8689', 'learning_rate': '5.493e-05', 'epoch': '0.6786', 'num_input_tokens_seen': 8257288, 'train_runtime': '8232', 'train_tokens_per_second': '1003'}
+
+ 68%|██████▊   | 1230/1813 [2:17:11<58:48,  6.05s/it]
+ 68%|██████▊   | 1231/1813 [2:17:18<1:02:41,  6.46s/it]
+ 68%|██████▊   | 1232/1813 [2:17:26<1:04:46,  6.69s/it]
+ 68%|██████▊   | 1233/1813 [2:17:32<1:02:59,  6.52s/it]
+ 68%|██████▊   | 1234/1813 [2:17:38<1:01:02,  6.33s/it]
+ 68%|██████▊   | 1235/1813 [2:17:44<1:00:16,  6.26s/it]
+ 68%|██████▊   | 1236/1813 [2:17:52<1:06:31,  6.92s/it]
+ 68%|██████▊   | 1237/1813 [2:17:58<1:04:33,  6.72s/it]
+ 68%|██████▊   | 1238/1813 [2:18:04<1:00:33,  6.32s/it]
+ 68%|██████▊   | 1239/1813 [2:18:09<57:00,  5.96s/it]  
+ 68%|██████▊   | 1240/1813 [2:18:14<54:24,  5.70s/it]
+                                                     
+{'loss': '0.7911', 'grad_norm': '0.9318', 'learning_rate': '5.325e-05', 'epoch': '0.6841', 'num_input_tokens_seen': 8327600, 'train_runtime': '8295', 'train_tokens_per_second': '1004'}
+
+ 68%|██████▊   | 1240/1813 [2:18:14<54:24,  5.70s/it]
+ 68%|██████▊   | 1241/1813 [2:18:20<54:14,  5.69s/it]
+ 69%|██████▊   | 1242/1813 [2:18:26<54:50,  5.76s/it]
+ 69%|██████▊   | 1243/1813 [2:18:32<55:48,  5.87s/it]
+ 69%|██████▊   | 1244/1813 [2:18:39<58:54,  6.21s/it]
+ 69%|██████▊   | 1245/1813 [2:18:46<1:02:03,  6.55s/it]
+ 69%|██████▊   | 1246/1813 [2:18:52<1:00:30,  6.40s/it]
+ 69%|██████▉   | 1247/1813 [2:18:59<1:02:08,  6.59s/it]
+ 69%|██████▉   | 1248/1813 [2:19:06<1:02:30,  6.64s/it]
+ 69%|██████▉   | 1249/1813 [2:19:12<1:01:08,  6.50s/it]
+ 69%|██████▉   | 1250/1813 [2:19:17<55:36,  5.93s/it]  
+                                                     
+{'loss': '0.9418', 'grad_norm': '1.292', 'learning_rate': '5.159e-05', 'epoch': '0.6897', 'num_input_tokens_seen': 8394824, 'train_runtime': '8357', 'train_tokens_per_second': '1004'}
+
+ 69%|██████▉   | 1250/1813 [2:19:17<55:36,  5.93s/it]
+ 69%|██████▉   | 1251/1813 [2:19:23<56:12,  6.00s/it]
+ 69%|██████▉   | 1252/1813 [2:19:29<55:00,  5.88s/it]
+ 69%|██████▉   | 1253/1813 [2:19:34<54:48,  5.87s/it]
+ 69%|██████▉   | 1254/1813 [2:19:41<56:24,  6.06s/it]
+ 69%|██████▉   | 1255/1813 [2:19:47<57:00,  6.13s/it]
+ 69%|██████▉   | 1256/1813 [2:19:53<56:53,  6.13s/it]
+ 69%|██████▉   | 1257/1813 [2:19:59<55:45,  6.02s/it]
+ 69%|██████▉   | 1258/1813 [2:20:06<57:35,  6.23s/it]
+ 69%|██████▉   | 1259/1813 [2:20:13<58:56,  6.38s/it]
+ 69%|██████▉   | 1260/1813 [2:20:20<1:00:41,  6.59s/it]
+                                                       
+{'loss': '0.9137', 'grad_norm': '1.113', 'learning_rate': '4.995e-05', 'epoch': '0.6952', 'num_input_tokens_seen': 8461096, 'train_runtime': '8420', 'train_tokens_per_second': '1005'}
+
+ 69%|██████▉   | 1260/1813 [2:20:20<1:00:41,  6.59s/it]
+ 70%|██████▉   | 1261/1813 [2:20:28<1:05:01,  7.07s/it]
+ 70%|██████▉   | 1262/1813 [2:20:35<1:06:14,  7.21s/it]
+ 70%|██████▉   | 1263/1813 [2:20:43<1:06:09,  7.22s/it]
+ 70%|██████▉   | 1264/1813 [2:20:48<1:01:55,  6.77s/it]
+ 70%|██████▉   | 1265/1813 [2:20:53<57:34,  6.30s/it]  
+ 70%|██████▉   | 1266/1813 [2:21:01<1:01:29,  6.74s/it]
+ 70%|██████▉   | 1267/1813 [2:21:07<57:39,  6.34s/it]  
+ 70%|██████▉   | 1268/1813 [2:21:12<54:08,  5.96s/it]
+ 70%|██████▉   | 1269/1813 [2:21:17<52:14,  5.76s/it]
+ 70%|███████   | 1270/1813 [2:21:23<52:57,  5.85s/it]
+                                                     
+{'loss': '0.8917', 'grad_norm': '1.074', 'learning_rate': '4.832e-05', 'epoch': '0.7007', 'num_input_tokens_seen': 8529224, 'train_runtime': '8484', 'train_tokens_per_second': '1005'}
+
+ 70%|███████   | 1270/1813 [2:21:23<52:57,  5.85s/it]
+ 70%|███████   | 1271/1813 [2:21:28<49:42,  5.50s/it]
+ 70%|███████   | 1272/1813 [2:21:35<53:28,  5.93s/it]
+ 70%|███████   | 1273/1813 [2:21:42<56:27,  6.27s/it]
+ 70%|███████   | 1274/1813 [2:21:49<59:39,  6.64s/it]
+ 70%|███████   | 1275/1813 [2:21:56<59:54,  6.68s/it]
+ 70%|███████   | 1276/1813 [2:22:02<56:47,  6.35s/it]
+ 70%|███████   | 1277/1813 [2:22:09<58:28,  6.55s/it]
+ 70%|███████   | 1278/1813 [2:22:14<55:21,  6.21s/it]
+ 71%|███████   | 1279/1813 [2:22:19<51:07,  5.74s/it]
+ 71%|███████   | 1280/1813 [2:22:26<54:10,  6.10s/it]
+                                                     
+{'loss': '0.9203', 'grad_norm': '1.26', 'learning_rate': '4.671e-05', 'epoch': '0.7062', 'num_input_tokens_seen': 8596456, 'train_runtime': '8546', 'train_tokens_per_second': '1006'}
+
+ 71%|███████   | 1280/1813 [2:22:26<54:10,  6.10s/it]
+ 71%|███████   | 1281/1813 [2:22:31<51:29,  5.81s/it]
+ 71%|███████   | 1282/1813 [2:22:37<52:36,  5.94s/it]
+ 71%|███████   | 1283/1813 [2:22:44<54:23,  6.16s/it]
+ 71%|███████   | 1284/1813 [2:22:49<53:18,  6.05s/it]
+ 71%|███████   | 1285/1813 [2:22:55<51:44,  5.88s/it]
+ 71%|███████   | 1286/1813 [2:23:02<55:18,  6.30s/it]
+ 71%|███████   | 1287/1813 [2:23:08<54:54,  6.26s/it]
+ 71%|███████   | 1288/1813 [2:23:14<52:18,  5.98s/it]
+ 71%|███████   | 1289/1813 [2:23:21<54:28,  6.24s/it]
+ 71%|███████   | 1290/1813 [2:23:28<57:01,  6.54s/it]
+                                                     
+{'loss': '0.9674', 'grad_norm': '0.7892', 'learning_rate': '4.512e-05', 'epoch': '0.7117', 'num_input_tokens_seen': 8662472, 'train_runtime': '8608', 'train_tokens_per_second': '1006'}
+
+ 71%|███████   | 1290/1813 [2:23:28<57:01,  6.54s/it]
+ 71%|███████   | 1291/1813 [2:23:34<56:02,  6.44s/it]
+ 71%|███████▏  | 1292/1813 [2:23:41<57:21,  6.61s/it]
+ 71%|███████▏  | 1293/1813 [2:23:48<58:43,  6.78s/it]
+ 71%|███████▏  | 1294/1813 [2:23:55<57:31,  6.65s/it]
+ 71%|███████▏  | 1295/1813 [2:24:00<55:29,  6.43s/it]
+ 71%|███████▏  | 1296/1813 [2:24:06<52:29,  6.09s/it]
+ 72%|███████▏  | 1297/1813 [2:24:11<50:49,  5.91s/it]
+ 72%|███████▏  | 1298/1813 [2:24:18<52:52,  6.16s/it]
+ 72%|███████▏  | 1299/1813 [2:24:25<54:58,  6.42s/it]
+ 72%|███████▏  | 1300/1813 [2:24:31<54:15,  6.35s/it]
+                                                     
+{'loss': '0.8595', 'grad_norm': '1.249', 'learning_rate': '4.355e-05', 'epoch': '0.7172', 'num_input_tokens_seen': 8730600, 'train_runtime': '8672', 'train_tokens_per_second': '1007'}
+
+ 72%|███████▏  | 1300/1813 [2:24:31<54:15,  6.35s/it]
+ 72%|███████▏  | 1301/1813 [2:24:37<52:33,  6.16s/it]
+ 72%|███████▏  | 1302/1813 [2:24:43<53:30,  6.28s/it]
+ 72%|███████▏  | 1303/1813 [2:24:51<57:00,  6.71s/it]
+ 72%|███████▏  | 1304/1813 [2:24:57<53:49,  6.35s/it]
+ 72%|███████▏  | 1305/1813 [2:25:04<56:25,  6.66s/it]
+ 72%|███████▏  | 1306/1813 [2:25:10<53:53,  6.38s/it]
+ 72%|███████▏  | 1307/1813 [2:25:16<54:14,  6.43s/it]
+ 72%|███████▏  | 1308/1813 [2:25:23<54:29,  6.47s/it]
+ 72%|███████▏  | 1309/1813 [2:25:29<54:09,  6.45s/it]
+ 72%|███████▏  | 1310/1813 [2:25:36<55:45,  6.65s/it]
+                                                     
+{'loss': '0.9525', 'grad_norm': '1.013', 'learning_rate': '4.2e-05', 'epoch': '0.7228', 'num_input_tokens_seen': 8802000, 'train_runtime': '8737', 'train_tokens_per_second': '1007'}
+
+ 72%|███████▏  | 1310/1813 [2:25:36<55:45,  6.65s/it]
+ 72%|███████▏  | 1311/1813 [2:25:43<54:51,  6.56s/it]
+ 72%|███████▏  | 1312/1813 [2:25:49<52:43,  6.31s/it]
+ 72%|███████▏  | 1313/1813 [2:25:55<52:14,  6.27s/it]
+ 72%|███████▏  | 1314/1813 [2:26:01<52:13,  6.28s/it]
+ 73%|███████▎  | 1315/1813 [2:26:07<50:25,  6.08s/it]
+ 73%|███████▎  | 1316/1813 [2:26:12<49:31,  5.98s/it]
+ 73%|███████▎  | 1317/1813 [2:26:18<49:23,  5.98s/it]
+ 73%|███████▎  | 1318/1813 [2:26:24<47:58,  5.82s/it]
+ 73%|███████▎  | 1319/1813 [2:26:30<49:30,  6.01s/it]
+ 73%|███████▎  | 1320/1813 [2:26:37<50:57,  6.20s/it]
+                                                     
+{'loss': '0.9255', 'grad_norm': '0.8742', 'learning_rate': '4.048e-05', 'epoch': '0.7283', 'num_input_tokens_seen': 8866824, 'train_runtime': '8797', 'train_tokens_per_second': '1008'}
+
+ 73%|███████▎  | 1320/1813 [2:26:37<50:57,  6.20s/it]
+ 73%|███████▎  | 1321/1813 [2:26:44<51:53,  6.33s/it]
+ 73%|███████▎  | 1322/1813 [2:26:50<51:25,  6.29s/it]
+ 73%|███████▎  | 1323/1813 [2:26:56<52:19,  6.41s/it]
+ 73%|███████▎  | 1324/1813 [2:27:03<51:32,  6.33s/it]
+ 73%|███████▎  | 1325/1813 [2:27:09<50:38,  6.23s/it]
+ 73%|███████▎  | 1326/1813 [2:27:15<50:53,  6.27s/it]
+ 73%|███████▎  | 1327/1813 [2:27:23<54:27,  6.72s/it]
+ 73%|███████▎  | 1328/1813 [2:27:30<54:44,  6.77s/it]
+ 73%|███████▎  | 1329/1813 [2:27:36<53:12,  6.60s/it]
+ 73%|███████▎  | 1330/1813 [2:27:41<50:42,  6.30s/it]
+                                                     
+{'loss': '0.8798', 'grad_norm': '1.073', 'learning_rate': '3.897e-05', 'epoch': '0.7338', 'num_input_tokens_seen': 8935584, 'train_runtime': '8862', 'train_tokens_per_second': '1008'}
+
+ 73%|███████▎  | 1330/1813 [2:27:41<50:42,  6.30s/it]
+ 73%|███████▎  | 1331/1813 [2:27:47<49:01,  6.10s/it]
+ 73%|███████▎  | 1332/1813 [2:27:53<48:02,  5.99s/it]
+ 74%|███████▎  | 1333/1813 [2:27:59<47:43,  5.97s/it]
+ 74%|███████▎  | 1334/1813 [2:28:04<46:54,  5.88s/it]
+ 74%|███████▎  | 1335/1813 [2:28:10<46:03,  5.78s/it]
+ 74%|███████▎  | 1336/1813 [2:28:16<47:12,  5.94s/it]
+ 74%|███████▎  | 1337/1813 [2:28:24<50:43,  6.39s/it]
+ 74%|███████▍  | 1338/1813 [2:28:31<52:53,  6.68s/it]
+ 74%|███████▍  | 1339/1813 [2:28:36<49:40,  6.29s/it]
+ 74%|███████▍  | 1340/1813 [2:28:43<50:46,  6.44s/it]
+                                                     
+{'loss': '0.9731', 'grad_norm': '1.005', 'learning_rate': '3.748e-05', 'epoch': '0.7393', 'num_input_tokens_seen': 9000416, 'train_runtime': '8924', 'train_tokens_per_second': '1009'}
+
+ 74%|███████▍  | 1340/1813 [2:28:43<50:46,  6.44s/it]
+ 74%|███████▍  | 1341/1813 [2:28:49<50:25,  6.41s/it]
+ 74%|███████▍  | 1342/1813 [2:28:55<47:29,  6.05s/it]
+ 74%|███████▍  | 1343/1813 [2:29:00<46:04,  5.88s/it]
+ 74%|███████▍  | 1344/1813 [2:29:06<46:10,  5.91s/it]
+ 74%|███████▍  | 1345/1813 [2:29:14<50:12,  6.44s/it]
+ 74%|███████▍  | 1346/1813 [2:29:20<49:59,  6.42s/it]
+ 74%|███████▍  | 1347/1813 [2:29:26<48:11,  6.20s/it]
+ 74%|███████▍  | 1348/1813 [2:29:32<48:23,  6.24s/it]
+ 74%|███████▍  | 1349/1813 [2:29:39<49:32,  6.41s/it]
+ 74%|███████▍  | 1350/1813 [2:29:45<48:07,  6.24s/it]
+                                                     
+{'loss': '0.9161', 'grad_norm': '1.297', 'learning_rate': '3.602e-05', 'epoch': '0.7448', 'num_input_tokens_seen': 9066864, 'train_runtime': '8985', 'train_tokens_per_second': '1009'}
+
+ 74%|███████▍  | 1350/1813 [2:29:45<48:07,  6.24s/it]
+ 75%|███████▍  | 1351/1813 [2:29:53<52:00,  6.75s/it]
+ 75%|███████▍  | 1352/1813 [2:29:59<50:45,  6.61s/it]
+ 75%|███████▍  | 1353/1813 [2:30:06<51:06,  6.67s/it]
+ 75%|███████▍  | 1354/1813 [2:30:13<51:17,  6.70s/it]
+ 75%|███████▍  | 1355/1813 [2:30:21<55:14,  7.24s/it]
+ 75%|███████▍  | 1356/1813 [2:30:27<51:39,  6.78s/it]
+ 75%|███████▍  | 1357/1813 [2:30:33<49:46,  6.55s/it]
+ 75%|███████▍  | 1358/1813 [2:30:39<49:44,  6.56s/it]
+ 75%|███████▍  | 1359/1813 [2:30:45<47:39,  6.30s/it]
+ 75%|███████▌  | 1360/1813 [2:30:51<46:41,  6.19s/it]
+                                                     
+{'loss': '0.868', 'grad_norm': '1.254', 'learning_rate': '3.458e-05', 'epoch': '0.7503', 'num_input_tokens_seen': 9140520, 'train_runtime': '9052', 'train_tokens_per_second': '1010'}
+
+ 75%|███████▌  | 1360/1813 [2:30:51<46:41,  6.19s/it]
+ 75%|███████▌  | 1361/1813 [2:30:58<48:15,  6.41s/it]
+ 75%|███████▌  | 1362/1813 [2:31:05<48:32,  6.46s/it]
+ 75%|███████▌  | 1363/1813 [2:31:12<50:46,  6.77s/it]
+ 75%|███████▌  | 1364/1813 [2:31:18<48:49,  6.52s/it]
+ 75%|███████▌  | 1365/1813 [2:31:26<50:49,  6.81s/it]
+ 75%|███████▌  | 1366/1813 [2:31:31<48:45,  6.55s/it]
+ 75%|███████▌  | 1367/1813 [2:31:38<48:43,  6.55s/it]
+ 75%|███████▌  | 1368/1813 [2:31:43<45:51,  6.18s/it]
+ 76%|███████▌  | 1369/1813 [2:31:51<50:07,  6.77s/it]
+ 76%|███████▌  | 1370/1813 [2:31:57<48:12,  6.53s/it]
+                                                     
+{'loss': '0.8542', 'grad_norm': '1.008', 'learning_rate': '3.316e-05', 'epoch': '0.7559', 'num_input_tokens_seen': 9213048, 'train_runtime': '9118', 'train_tokens_per_second': '1010'}
+
+ 76%|███████▌  | 1370/1813 [2:31:57<48:12,  6.53s/it]
+ 76%|███████▌  | 1371/1813 [2:32:05<51:21,  6.97s/it]
+ 76%|███████▌  | 1372/1813 [2:32:11<47:20,  6.44s/it]
+ 76%|███████▌  | 1373/1813 [2:32:17<46:21,  6.32s/it]
+ 76%|███████▌  | 1374/1813 [2:32:23<45:18,  6.19s/it]
+ 76%|███████▌  | 1375/1813 [2:32:28<44:02,  6.03s/it]
+ 76%|███████▌  | 1376/1813 [2:32:34<42:22,  5.82s/it]
+ 76%|███████▌  | 1377/1813 [2:32:40<42:55,  5.91s/it]
+ 76%|███████▌  | 1378/1813 [2:32:45<41:55,  5.78s/it]
+ 76%|███████▌  | 1379/1813 [2:32:51<42:12,  5.84s/it]
+ 76%|███████▌  | 1380/1813 [2:32:58<44:32,  6.17s/it]
+                                                     
+{'loss': '0.8455', 'grad_norm': '1.114', 'learning_rate': '3.177e-05', 'epoch': '0.7614', 'num_input_tokens_seen': 9279088, 'train_runtime': '9179', 'train_tokens_per_second': '1011'}
+
+ 76%|███████▌  | 1380/1813 [2:32:58<44:32,  6.17s/it]
+ 76%|███████▌  | 1381/1813 [2:33:05<46:03,  6.40s/it]
+ 76%|███████▌  | 1382/1813 [2:33:10<43:40,  6.08s/it]
+ 76%|███████▋  | 1383/1813 [2:33:16<43:05,  6.01s/it]
+ 76%|███████▋  | 1384/1813 [2:33:22<42:58,  6.01s/it]
+ 76%|███████▋  | 1385/1813 [2:33:29<44:47,  6.28s/it]
+ 76%|███████▋  | 1386/1813 [2:33:36<45:24,  6.38s/it]
+ 77%|███████▋  | 1387/1813 [2:33:43<47:04,  6.63s/it]
+ 77%|███████▋  | 1388/1813 [2:33:49<44:49,  6.33s/it]
+ 77%|███████▋  | 1389/1813 [2:33:53<41:09,  5.83s/it]
+ 77%|███████▋  | 1390/1813 [2:34:00<43:15,  6.14s/it]
+                                                     
+{'loss': '0.9327', 'grad_norm': '1.097', 'learning_rate': '3.04e-05', 'epoch': '0.7669', 'num_input_tokens_seen': 9344880, 'train_runtime': '9241', 'train_tokens_per_second': '1011'}
+
+ 77%|███████▋  | 1390/1813 [2:34:00<43:15,  6.14s/it]
+ 77%|███████▋  | 1391/1813 [2:34:05<41:10,  5.85s/it]
+ 77%|███████▋  | 1392/1813 [2:34:12<42:35,  6.07s/it]
+ 77%|███████▋  | 1393/1813 [2:34:17<40:54,  5.84s/it]
+ 77%|███████▋  | 1394/1813 [2:34:24<42:52,  6.14s/it]
+ 77%|███████▋  | 1395/1813 [2:34:30<43:25,  6.23s/it]
+ 77%|███████▋  | 1396/1813 [2:34:37<43:31,  6.26s/it]
+ 77%|███████▋  | 1397/1813 [2:34:44<44:47,  6.46s/it]
+ 77%|███████▋  | 1398/1813 [2:34:48<41:02,  5.93s/it]
+ 77%|███████▋  | 1399/1813 [2:34:53<38:37,  5.60s/it]
+ 77%|███████▋  | 1400/1813 [2:35:00<40:45,  5.92s/it]
+                                                     
+{'loss': '0.8799', 'grad_norm': '0.9047', 'learning_rate': '2.906e-05', 'epoch': '0.7724', 'num_input_tokens_seen': 9408280, 'train_runtime': '9300', 'train_tokens_per_second': '1012'}
+
+ 77%|███████▋  | 1400/1813 [2:35:00<40:45,  5.92s/it]
+ 77%|███████▋  | 1401/1813 [2:35:05<40:01,  5.83s/it]
+ 77%|███████▋  | 1402/1813 [2:35:12<40:46,  5.95s/it]
+ 77%|███████▋  | 1403/1813 [2:35:18<42:09,  6.17s/it]
+ 77%|███████▋  | 1404/1813 [2:35:25<42:22,  6.22s/it]
+ 77%|███████▋  | 1405/1813 [2:35:30<39:37,  5.83s/it]
+ 78%|███████▊  | 1406/1813 [2:35:35<39:27,  5.82s/it]
+ 78%|███████▊  | 1407/1813 [2:35:42<41:05,  6.07s/it]
+ 78%|███████▊  | 1408/1813 [2:35:47<39:01,  5.78s/it]
+ 78%|███████▊  | 1409/1813 [2:35:52<36:28,  5.42s/it]
+ 78%|███████▊  | 1410/1813 [2:35:57<36:57,  5.50s/it]
+                                                     
+{'loss': '0.9233', 'grad_norm': '1.028', 'learning_rate': '2.774e-05', 'epoch': '0.7779', 'num_input_tokens_seen': 9467408, 'train_runtime': '9358', 'train_tokens_per_second': '1012'}
+
+ 78%|███████▊  | 1410/1813 [2:35:57<36:57,  5.50s/it]
+ 78%|███████▊  | 1411/1813 [2:36:03<37:04,  5.53s/it]
+ 78%|███████▊  | 1412/1813 [2:36:10<38:46,  5.80s/it]
+ 78%|███████▊  | 1413/1813 [2:36:15<38:27,  5.77s/it]
+ 78%|███████▊  | 1414/1813 [2:36:22<39:23,  5.92s/it]
+ 78%|███████▊  | 1415/1813 [2:36:29<41:57,  6.33s/it]
+ 78%|███████▊  | 1416/1813 [2:36:35<41:42,  6.30s/it]
+ 78%|███████▊  | 1417/1813 [2:36:41<40:07,  6.08s/it]
+ 78%|███████▊  | 1418/1813 [2:36:47<39:45,  6.04s/it]
+ 78%|███████▊  | 1419/1813 [2:36:55<44:54,  6.84s/it]
+ 78%|███████▊  | 1420/1813 [2:37:00<40:39,  6.21s/it]
+                                                     
+{'loss': '0.8733', 'grad_norm': '1.122', 'learning_rate': '2.644e-05', 'epoch': '0.7834', 'num_input_tokens_seen': 9532720, 'train_runtime': '9420', 'train_tokens_per_second': '1012'}
+
+ 78%|███████▊  | 1420/1813 [2:37:00<40:39,  6.21s/it]
+ 78%|███████▊  | 1421/1813 [2:37:07<42:09,  6.45s/it]
+ 78%|███████▊  | 1422/1813 [2:37:13<40:43,  6.25s/it]
+ 78%|███████▊  | 1423/1813 [2:37:19<39:48,  6.12s/it]
+ 79%|███████▊  | 1424/1813 [2:37:26<42:04,  6.49s/it]
+ 79%|███████▊  | 1425/1813 [2:37:30<38:09,  5.90s/it]
+ 79%|███████▊  | 1426/1813 [2:37:39<43:11,  6.70s/it]
+ 79%|███████▊  | 1427/1813 [2:37:44<40:29,  6.29s/it]
+ 79%|███████▉  | 1428/1813 [2:37:51<40:50,  6.37s/it]
+ 79%|███████▉  | 1429/1813 [2:37:57<40:17,  6.30s/it]
+ 79%|███████▉  | 1430/1813 [2:38:04<41:12,  6.46s/it]
+                                                     
+{'loss': '0.8383', 'grad_norm': '0.9834', 'learning_rate': '2.518e-05', 'epoch': '0.789', 'num_input_tokens_seen': 9602112, 'train_runtime': '9484', 'train_tokens_per_second': '1012'}
+
+ 79%|███████▉  | 1430/1813 [2:38:04<41:12,  6.46s/it]
+ 79%|███████▉  | 1431/1813 [2:38:10<40:25,  6.35s/it]
+ 79%|███████▉  | 1432/1813 [2:38:16<39:23,  6.20s/it]
+ 79%|███████▉  | 1433/1813 [2:38:22<39:41,  6.27s/it]
+ 79%|███████▉  | 1434/1813 [2:38:29<39:55,  6.32s/it]
+ 79%|███████▉  | 1435/1813 [2:38:34<38:38,  6.13s/it]
+ 79%|███████▉  | 1436/1813 [2:38:41<40:06,  6.38s/it]
+ 79%|███████▉  | 1437/1813 [2:38:47<39:07,  6.24s/it]
+ 79%|███████▉  | 1438/1813 [2:38:55<40:54,  6.55s/it]
+ 79%|███████▉  | 1439/1813 [2:39:00<38:40,  6.20s/it]
+ 79%|███████▉  | 1440/1813 [2:39:05<36:17,  5.84s/it]
+                                                     
+{'loss': '0.8573', 'grad_norm': '0.998', 'learning_rate': '2.394e-05', 'epoch': '0.7945', 'num_input_tokens_seen': 9667112, 'train_runtime': '9545', 'train_tokens_per_second': '1013'}
+
+ 79%|███████▉  | 1440/1813 [2:39:05<36:17,  5.84s/it]
+ 79%|███████▉  | 1441/1813 [2:39:11<37:10,  6.00s/it]
+ 80%|███████▉  | 1442/1813 [2:39:16<35:35,  5.76s/it]
+ 80%|███████▉  | 1443/1813 [2:39:21<34:03,  5.52s/it]
+ 80%|███████▉  | 1444/1813 [2:39:28<36:11,  5.89s/it]
+ 80%|███████▉  | 1445/1813 [2:39:34<35:26,  5.78s/it]
+ 80%|███████▉  | 1446/1813 [2:39:39<34:40,  5.67s/it]
+ 80%|███████▉  | 1447/1813 [2:39:47<39:11,  6.42s/it]
+ 80%|███████▉  | 1448/1813 [2:39:55<40:29,  6.66s/it]
+ 80%|███████▉  | 1449/1813 [2:40:01<40:06,  6.61s/it]
+ 80%|███████▉  | 1450/1813 [2:40:07<39:39,  6.56s/it]
+                                                     
+{'loss': '0.9389', 'grad_norm': '1.036', 'learning_rate': '2.273e-05', 'epoch': '0.8', 'num_input_tokens_seen': 9735792, 'train_runtime': '9608', 'train_tokens_per_second': '1013'}
+
+ 80%|███████▉  | 1450/1813 [2:40:07<39:39,  6.56s/it]
+ 80%|████████  | 1451/1813 [2:40:14<40:22,  6.69s/it]
+ 80%|████████  | 1452/1813 [2:40:19<36:31,  6.07s/it]
+ 80%|████████  | 1453/1813 [2:40:27<40:31,  6.75s/it]
+ 80%|████████  | 1454/1813 [2:40:33<37:50,  6.32s/it]
+ 80%|████████  | 1455/1813 [2:40:39<37:26,  6.28s/it]
+ 80%|████████  | 1456/1813 [2:40:46<38:45,  6.51s/it]
+ 80%|████████  | 1457/1813 [2:40:50<33:27,  5.64s/it]
+ 80%|████████  | 1458/1813 [2:40:55<33:07,  5.60s/it]
+ 80%|████████  | 1459/1813 [2:41:02<35:30,  6.02s/it]
+ 81%|████████  | 1460/1813 [2:41:09<37:26,  6.36s/it]
+                                                     
+{'loss': '0.8932', 'grad_norm': '1.063', 'learning_rate': '2.154e-05', 'epoch': '0.8055', 'num_input_tokens_seen': 9802968, 'train_runtime': '9670', 'train_tokens_per_second': '1014'}
+
+ 81%|████████  | 1460/1813 [2:41:09<37:26,  6.36s/it]
+ 81%|████████  | 1461/1813 [2:41:15<36:26,  6.21s/it]
+ 81%|████████  | 1462/1813 [2:41:20<34:41,  5.93s/it]
+ 81%|████████  | 1463/1813 [2:41:28<37:15,  6.39s/it]
+ 81%|████████  | 1464/1813 [2:41:34<37:32,  6.46s/it]
+ 81%|████████  | 1465/1813 [2:41:43<40:31,  6.99s/it]
+ 81%|████████  | 1466/1813 [2:41:48<37:39,  6.51s/it]
+ 81%|████████  | 1467/1813 [2:41:55<38:58,  6.76s/it]
+ 81%|████████  | 1468/1813 [2:42:02<38:29,  6.69s/it]
+ 81%|████████  | 1469/1813 [2:42:11<41:56,  7.31s/it]
+ 81%|████████  | 1470/1813 [2:42:17<40:33,  7.09s/it]
+                                                     
+{'loss': '0.9012', 'grad_norm': '0.9799', 'learning_rate': '2.038e-05', 'epoch': '0.811', 'num_input_tokens_seen': 9879232, 'train_runtime': '9738', 'train_tokens_per_second': '1015'}
+
+ 81%|████████  | 1470/1813 [2:42:17<40:33,  7.09s/it]
+ 81%|████████  | 1471/1813 [2:42:23<38:45,  6.80s/it]
+ 81%|████████  | 1472/1813 [2:42:30<38:19,  6.74s/it]
+ 81%|████████  | 1473/1813 [2:42:36<36:37,  6.46s/it]
+ 81%|████████▏ | 1474/1813 [2:42:42<36:41,  6.49s/it]
+ 81%|████████▏ | 1475/1813 [2:42:49<36:02,  6.40s/it]
+ 81%|████████▏ | 1476/1813 [2:42:55<35:39,  6.35s/it]
+ 81%|████████▏ | 1477/1813 [2:43:01<35:49,  6.40s/it]
+ 82%|████████▏ | 1478/1813 [2:43:07<34:31,  6.18s/it]
+ 82%|████████▏ | 1479/1813 [2:43:13<34:25,  6.18s/it]
+ 82%|████████▏ | 1480/1813 [2:43:20<35:53,  6.47s/it]
+                                                     
+{'loss': '0.8726', 'grad_norm': '0.8065', 'learning_rate': '1.925e-05', 'epoch': '0.8166', 'num_input_tokens_seen': 9948032, 'train_runtime': '9801', 'train_tokens_per_second': '1015'}
+
+ 82%|████████▏ | 1480/1813 [2:43:20<35:53,  6.47s/it]
+ 82%|████████▏ | 1481/1813 [2:43:27<36:16,  6.55s/it]
+ 82%|████████▏ | 1482/1813 [2:43:33<34:42,  6.29s/it]
+ 82%|████████▏ | 1483/1813 [2:43:40<35:37,  6.48s/it]
+ 82%|████████▏ | 1484/1813 [2:43:46<35:34,  6.49s/it]
+ 82%|████████▏ | 1485/1813 [2:43:53<35:39,  6.52s/it]
+ 82%|████████▏ | 1486/1813 [2:43:58<33:25,  6.13s/it]
+ 82%|████████▏ | 1487/1813 [2:44:06<36:07,  6.65s/it]
+ 82%|████████▏ | 1488/1813 [2:44:12<35:24,  6.54s/it]
+ 82%|████████▏ | 1489/1813 [2:44:20<36:39,  6.79s/it]
+ 82%|████████▏ | 1490/1813 [2:44:26<36:43,  6.82s/it]
+                                                     
+{'loss': '0.887', 'grad_norm': '1.024', 'learning_rate': '1.815e-05', 'epoch': '0.8221', 'num_input_tokens_seen': 10021400, 'train_runtime': '9867', 'train_tokens_per_second': '1016'}
+
+ 82%|████████▏ | 1490/1813 [2:44:26<36:43,  6.82s/it]
+ 82%|████████▏ | 1491/1813 [2:44:32<34:52,  6.50s/it]
+ 82%|████████▏ | 1492/1813 [2:44:40<37:32,  7.02s/it]
+ 82%|████████▏ | 1493/1813 [2:44:47<36:39,  6.87s/it]
+ 82%|████████▏ | 1494/1813 [2:44:54<36:26,  6.85s/it]
+ 82%|████████▏ | 1495/1813 [2:45:02<39:10,  7.39s/it]
+ 83%|████████▎ | 1496/1813 [2:45:08<35:53,  6.79s/it]
+ 83%|████████▎ | 1497/1813 [2:45:13<33:32,  6.37s/it]
+ 83%|████████▎ | 1498/1813 [2:45:20<34:30,  6.57s/it]
+ 83%|████████▎ | 1499/1813 [2:45:25<32:23,  6.19s/it]
+ 83%|████████▎ | 1500/1813 [2:45:34<36:02,  6.91s/it]
+                                                     
+{'loss': '0.8516', 'grad_norm': '0.9367', 'learning_rate': '1.708e-05', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904, 'train_runtime': '9935', 'train_tokens_per_second': '1016'}
+
+ 83%|████████▎ | 1500/1813 [2:45:34<36:02,  6.91s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:12, 13.83it/s]
+
+  0%|          | 5/1000 [00:00<02:38,  6.29it/s]
+
+  1%|          | 6/1000 [00:00<02:24,  6.90it/s]
+
+  1%|          | 7/1000 [00:00<02:20,  7.09it/s]
+
+  1%|          | 9/1000 [00:01<03:05,  5.34it/s]
+
+  1%|          | 11/1000 [00:01<02:25,  6.81it/s]
+
+  1%|          | 12/1000 [00:01<02:23,  6.89it/s]
+
+  1%|▏         | 13/1000 [00:02<02:54,  5.66it/s]
+
+  1%|▏         | 14/1000 [00:02<02:56,  5.60it/s]
+
+  2%|▏         | 15/1000 [00:02<03:28,  4.73it/s]
+
+  2%|▏         | 16/1000 [00:02<03:32,  4.64it/s]
+
+  2%|▏         | 18/1000 [00:03<03:13,  5.09it/s]
+
+  2%|▏         | 19/1000 [00:03<03:51,  4.24it/s]
+
+  2%|▏         | 20/1000 [00:03<04:17,  3.81it/s]
+
+  2%|▏         | 22/1000 [00:03<03:04,  5.29it/s]
+
+  2%|▏         | 23/1000 [00:04<03:29,  4.67it/s]
+
+  2%|▎         | 25/1000 [00:04<02:37,  6.20it/s]
+
+  3%|▎         | 26/1000 [00:04<03:02,  5.34it/s]
+
+  3%|▎         | 27/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 28/1000 [00:05<03:21,  4.82it/s]
+
+  3%|▎         | 29/1000 [00:05<03:03,  5.29it/s]
+
+  3%|▎         | 30/1000 [00:05<03:14,  4.98it/s]
+
+  3%|▎         | 31/1000 [00:05<03:40,  4.39it/s]
+
+  3%|▎         | 32/1000 [00:05<03:14,  4.99it/s]
+
+  3%|▎         | 33/1000 [00:06<03:18,  4.87it/s]
+
+  4%|▎         | 35/1000 [00:06<02:25,  6.63it/s]
+
+  4%|▎         | 37/1000 [00:06<02:01,  7.94it/s]
+
+  4%|▍         | 39/1000 [00:06<02:26,  6.56it/s]
+
+  4%|▍         | 40/1000 [00:07<02:22,  6.73it/s]
+
+  4%|▍         | 42/1000 [00:07<02:00,  7.97it/s]
+
+  4%|▍         | 44/1000 [00:07<01:49,  8.77it/s]
+
+  4%|▍         | 45/1000 [00:07<02:10,  7.30it/s]
+
+  5%|▍         | 47/1000 [00:07<02:25,  6.54it/s]
+
+  5%|▍         | 49/1000 [00:08<02:03,  7.72it/s]
+
+  5%|▌         | 51/1000 [00:08<01:48,  8.72it/s]
+
+  5%|▌         | 53/1000 [00:08<01:38,  9.58it/s]
+
+  6%|▌         | 55/1000 [00:08<01:43,  9.16it/s]
+
+  6%|▌         | 56/1000 [00:08<02:06,  7.49it/s]
+
+  6%|▌         | 58/1000 [00:09<01:57,  8.03it/s]
+
+  6%|▌         | 59/1000 [00:09<02:15,  6.93it/s]
+
+  6%|▌         | 61/1000 [00:09<01:55,  8.16it/s]
+
+  6%|▋         | 63/1000 [00:09<01:49,  8.59it/s]
+
+  6%|▋         | 64/1000 [00:10<02:21,  6.60it/s]
+
+  7%|▋         | 66/1000 [00:10<01:59,  7.82it/s]
+
+  7%|▋         | 67/1000 [00:10<02:28,  6.30it/s]
+
+  7%|▋         | 69/1000 [00:10<02:10,  7.11it/s]
+
+  7%|▋         | 70/1000 [00:11<02:42,  5.74it/s]
+
+  7%|▋         | 72/1000 [00:11<02:45,  5.62it/s]
+
+  7%|▋         | 74/1000 [00:11<02:12,  6.97it/s]
+
+  8%|▊         | 75/1000 [00:11<02:37,  5.88it/s]
+
+  8%|▊         | 76/1000 [00:12<02:45,  5.59it/s]
+
+  8%|▊         | 77/1000 [00:12<02:29,  6.16it/s]
+
+  8%|▊         | 78/1000 [00:12<02:57,  5.19it/s]
+
+  8%|▊         | 80/1000 [00:12<02:13,  6.89it/s]
+
+  8%|▊         | 82/1000 [00:12<01:54,  8.03it/s]
+
+  8%|▊         | 83/1000 [00:13<02:24,  6.35it/s]
+
+  8%|▊         | 85/1000 [00:13<02:35,  5.89it/s]
+
+  9%|▊         | 86/1000 [00:13<02:57,  5.16it/s]
+
+  9%|▊         | 87/1000 [00:14<03:14,  4.68it/s]
+
+  9%|▉         | 89/1000 [00:14<02:28,  6.12it/s]
+
+  9%|▉         | 90/1000 [00:14<02:23,  6.34it/s]
+
+  9%|▉         | 91/1000 [00:14<02:51,  5.31it/s]
+
+  9%|▉         | 92/1000 [00:14<02:48,  5.38it/s]
+
+  9%|▉         | 93/1000 [00:15<03:00,  5.02it/s]
+
+  9%|▉         | 94/1000 [00:15<02:38,  5.71it/s]
+
+ 10%|▉         | 95/1000 [00:15<03:23,  4.45it/s]
+
+ 10%|▉         | 97/1000 [00:15<02:34,  5.83it/s]
+
+ 10%|▉         | 99/1000 [00:15<02:01,  7.41it/s]
+
+ 10%|█         | 101/1000 [00:16<01:44,  8.60it/s]
+
+ 10%|█         | 102/1000 [00:16<02:14,  6.66it/s]
+
+ 10%|█         | 103/1000 [00:16<02:16,  6.59it/s]
+
+ 10%|█         | 104/1000 [00:16<02:13,  6.72it/s]
+
+ 11%|█         | 106/1000 [00:16<01:46,  8.39it/s]
+
+ 11%|█         | 108/1000 [00:16<01:34,  9.47it/s]
+
+ 11%|█         | 110/1000 [00:17<01:27, 10.12it/s]
+
+ 11%|█         | 112/1000 [00:17<01:21, 10.86it/s]
+
+ 11%|█▏        | 114/1000 [00:17<01:21, 10.81it/s]
+
+ 12%|█▏        | 116/1000 [00:17<01:20, 11.03it/s]
+
+ 12%|█▏        | 118/1000 [00:17<01:16, 11.48it/s]
+
+ 12%|█▏        | 120/1000 [00:18<02:29,  5.89it/s]
+
+ 12%|█▏        | 122/1000 [00:18<02:06,  6.92it/s]
+
+ 12%|█▏        | 124/1000 [00:19<02:18,  6.33it/s]
+
+ 12%|█▎        | 125/1000 [00:19<02:59,  4.88it/s]
+
+ 13%|█▎        | 126/1000 [00:19<02:46,  5.26it/s]
+
+ 13%|█▎        | 127/1000 [00:19<02:55,  4.97it/s]
+
+ 13%|█▎        | 128/1000 [00:20<03:11,  4.56it/s]
+
+ 13%|█▎        | 130/1000 [00:20<02:22,  6.13it/s]
+
+ 13%|█▎        | 132/1000 [00:20<01:54,  7.55it/s]
+
+ 13%|█▎        | 134/1000 [00:20<01:40,  8.57it/s]
+
+ 14%|█▎        | 135/1000 [00:20<01:51,  7.74it/s]
+
+ 14%|█▎        | 137/1000 [00:20<01:44,  8.25it/s]
+
+ 14%|█▍        | 139/1000 [00:21<01:41,  8.50it/s]
+
+ 14%|█▍        | 140/1000 [00:21<01:39,  8.66it/s]
+
+ 14%|█▍        | 141/1000 [00:21<01:42,  8.40it/s]
+
+ 14%|█▍        | 142/1000 [00:21<01:52,  7.62it/s]
+
+ 14%|█▍        | 143/1000 [00:21<01:58,  7.21it/s]
+
+ 14%|█▍        | 145/1000 [00:21<01:38,  8.68it/s]
+
+ 15%|█▍        | 147/1000 [00:22<01:59,  7.13it/s]
+
+ 15%|█▍        | 148/1000 [00:22<01:57,  7.25it/s]
+
+ 15%|█▌        | 150/1000 [00:22<01:38,  8.60it/s]
+
+ 15%|█▌        | 152/1000 [00:22<01:27,  9.66it/s]
+
+ 15%|█▌        | 154/1000 [00:23<01:32,  9.16it/s]
+
+ 16%|█▌        | 155/1000 [00:23<02:04,  6.81it/s]
+
+ 16%|█▌        | 156/1000 [00:23<02:38,  5.31it/s]
+
+ 16%|█▌        | 158/1000 [00:23<02:06,  6.65it/s]
+
+ 16%|█▌        | 160/1000 [00:24<01:53,  7.41it/s]
+
+ 16%|█▌        | 162/1000 [00:24<02:05,  6.69it/s]
+
+ 16%|█▋        | 163/1000 [00:24<02:10,  6.43it/s]
+
+ 16%|█▋        | 164/1000 [00:24<02:32,  5.49it/s]
+
+ 16%|█▋        | 165/1000 [00:25<02:55,  4.76it/s]
+
+ 17%|█▋        | 166/1000 [00:25<03:09,  4.39it/s]
+
+ 17%|█▋        | 167/1000 [00:25<02:48,  4.95it/s]
+
+ 17%|█▋        | 169/1000 [00:25<02:02,  6.77it/s]
+
+ 17%|█▋        | 171/1000 [00:26<02:13,  6.22it/s]
+
+ 17%|█▋        | 172/1000 [00:26<02:35,  5.33it/s]
+
+ 17%|█▋        | 173/1000 [00:26<02:33,  5.39it/s]
+
+ 17%|█▋        | 174/1000 [00:26<02:21,  5.86it/s]
+
+ 18%|█▊        | 176/1000 [00:26<01:54,  7.17it/s]
+
+ 18%|█▊        | 177/1000 [00:27<02:20,  5.85it/s]
+
+ 18%|█▊        | 179/1000 [00:27<01:50,  7.44it/s]
+
+ 18%|█▊        | 180/1000 [00:27<01:50,  7.41it/s]
+
+ 18%|█▊        | 181/1000 [00:27<02:19,  5.85it/s]
+
+ 18%|█▊        | 182/1000 [00:28<02:46,  4.91it/s]
+
+ 18%|█▊        | 183/1000 [00:28<02:46,  4.91it/s]
+
+ 18%|█▊        | 185/1000 [00:28<02:02,  6.63it/s]
+
+ 19%|█▊        | 186/1000 [00:28<01:59,  6.81it/s]
+
+ 19%|█▊        | 187/1000 [00:28<02:26,  5.54it/s]
+
+ 19%|█▉        | 189/1000 [00:28<01:54,  7.07it/s]
+
+ 19%|█▉        | 190/1000 [00:29<02:19,  5.79it/s]
+
+ 19%|█▉        | 192/1000 [00:29<02:22,  5.68it/s]
+
+ 19%|█▉        | 194/1000 [00:29<02:23,  5.61it/s]
+
+ 20%|█▉        | 196/1000 [00:30<01:57,  6.82it/s]
+
+ 20%|█▉        | 198/1000 [00:30<01:39,  8.08it/s]
+
+ 20%|██        | 200/1000 [00:30<01:24,  9.42it/s]
+
+ 20%|██        | 202/1000 [00:30<01:43,  7.71it/s]
+
+ 20%|██        | 204/1000 [00:31<01:44,  7.60it/s]
+
+ 21%|██        | 206/1000 [00:31<01:30,  8.80it/s]
+
+ 21%|██        | 208/1000 [00:31<01:45,  7.50it/s]
+
+ 21%|██        | 210/1000 [00:31<01:46,  7.45it/s]
+
+ 21%|██        | 212/1000 [00:32<01:32,  8.50it/s]
+
+ 21%|██▏       | 213/1000 [00:32<01:42,  7.69it/s]
+
+ 21%|██▏       | 214/1000 [00:32<01:42,  7.70it/s]
+
+ 22%|██▏       | 216/1000 [00:32<01:36,  8.15it/s]
+
+ 22%|██▏       | 217/1000 [00:32<02:01,  6.44it/s]
+
+ 22%|██▏       | 219/1000 [00:32<01:40,  7.74it/s]
+
+ 22%|██▏       | 220/1000 [00:33<02:24,  5.39it/s]
+
+ 22%|██▏       | 221/1000 [00:33<02:23,  5.42it/s]
+
+ 22%|██▏       | 223/1000 [00:34<02:42,  4.78it/s]
+
+ 22%|██▏       | 224/1000 [00:34<02:28,  5.23it/s]
+
+ 22%|██▎       | 225/1000 [00:34<02:26,  5.28it/s]
+
+ 23%|██▎       | 226/1000 [00:34<02:44,  4.70it/s]
+
+ 23%|██▎       | 228/1000 [00:34<02:09,  5.96it/s]
+
+ 23%|██▎       | 229/1000 [00:35<02:29,  5.16it/s]
+
+ 23%|██▎       | 230/1000 [00:35<02:18,  5.58it/s]
+
+ 23%|██▎       | 232/1000 [00:35<01:48,  7.05it/s]
+
+ 23%|██▎       | 233/1000 [00:35<01:46,  7.20it/s]
+
+ 24%|██▎       | 235/1000 [00:35<01:29,  8.54it/s]
+
+ 24%|██▎       | 236/1000 [00:35<01:30,  8.42it/s]
+
+ 24%|██▎       | 237/1000 [00:36<01:34,  8.08it/s]
+
+ 24%|██▍       | 239/1000 [00:36<01:39,  7.69it/s]
+
+ 24%|██▍       | 241/1000 [00:36<01:46,  7.15it/s]
+
+ 24%|██▍       | 242/1000 [00:37<02:34,  4.90it/s]
+
+ 24%|██▍       | 243/1000 [00:37<02:47,  4.52it/s]
+
+ 24%|██▍       | 245/1000 [00:37<02:13,  5.64it/s]
+
+ 25%|██▍       | 247/1000 [00:37<01:48,  6.91it/s]
+
+ 25%|██▍       | 248/1000 [00:38<02:09,  5.81it/s]
+
+ 25%|██▍       | 249/1000 [00:38<02:27,  5.10it/s]
+
+ 25%|██▌       | 250/1000 [00:38<02:28,  5.05it/s]
+
+ 25%|██▌       | 252/1000 [00:38<02:23,  5.23it/s]
+
+ 25%|██▌       | 253/1000 [00:39<02:30,  4.98it/s]
+
+ 26%|██▌       | 255/1000 [00:39<01:55,  6.42it/s]
+
+ 26%|██▌       | 256/1000 [00:39<01:54,  6.51it/s]
+
+ 26%|██▌       | 258/1000 [00:39<01:39,  7.49it/s]
+
+ 26%|██▌       | 260/1000 [00:39<01:51,  6.66it/s]
+
+ 26%|██▌       | 261/1000 [00:40<02:10,  5.68it/s]
+
+ 26%|██▌       | 262/1000 [00:40<02:15,  5.45it/s]
+
+ 26%|██▋       | 264/1000 [00:40<02:15,  5.44it/s]
+
+ 26%|██▋       | 265/1000 [00:40<02:09,  5.69it/s]
+
+ 27%|██▋       | 266/1000 [00:41<02:02,  5.99it/s]
+
+ 27%|██▋       | 268/1000 [00:41<01:51,  6.55it/s]
+
+ 27%|██▋       | 270/1000 [00:41<01:59,  6.13it/s]
+
+ 27%|██▋       | 271/1000 [00:42<02:37,  4.62it/s]
+
+ 27%|██▋       | 273/1000 [00:42<02:28,  4.89it/s]
+
+ 28%|██▊       | 275/1000 [00:42<02:08,  5.64it/s]
+
+ 28%|██▊       | 277/1000 [00:42<01:45,  6.82it/s]
+
+ 28%|██▊       | 278/1000 [00:43<02:03,  5.83it/s]
+
+ 28%|██▊       | 280/1000 [00:43<02:05,  5.73it/s]
+
+ 28%|██▊       | 282/1000 [00:43<01:43,  6.96it/s]
+
+ 28%|██▊       | 283/1000 [00:44<02:24,  4.96it/s]
+
+ 28%|██▊       | 284/1000 [00:44<02:27,  4.85it/s]
+
+ 29%|██▊       | 286/1000 [00:44<01:52,  6.35it/s]
+
+ 29%|██▉       | 288/1000 [00:44<01:37,  7.31it/s]
+
+ 29%|██▉       | 290/1000 [00:45<01:47,  6.62it/s]
+
+ 29%|██▉       | 291/1000 [00:45<01:58,  5.98it/s]
+
+ 29%|██▉       | 292/1000 [00:45<02:19,  5.07it/s]
+
+ 29%|██▉       | 294/1000 [00:45<01:55,  6.12it/s]
+
+ 30%|██▉       | 296/1000 [00:46<01:58,  5.94it/s]
+
+ 30%|██▉       | 298/1000 [00:46<01:39,  7.08it/s]
+
+ 30%|██▉       | 299/1000 [00:46<02:05,  5.60it/s]
+
+ 30%|███       | 300/1000 [00:46<01:58,  5.90it/s]
+
+ 30%|███       | 301/1000 [00:47<01:52,  6.19it/s]
+
+ 30%|███       | 303/1000 [00:47<01:30,  7.67it/s]
+
+ 30%|███       | 304/1000 [00:47<01:38,  7.08it/s]
+
+ 30%|███       | 305/1000 [00:47<01:37,  7.13it/s]
+
+ 31%|███       | 306/1000 [00:47<01:54,  6.04it/s]
+
+ 31%|███       | 307/1000 [00:47<01:57,  5.91it/s]
+
+ 31%|███       | 309/1000 [00:48<01:30,  7.64it/s]
+
+ 31%|███       | 310/1000 [00:48<01:28,  7.80it/s]
+
+ 31%|███       | 312/1000 [00:48<01:14,  9.19it/s]
+
+ 31%|███▏      | 314/1000 [00:48<01:11,  9.63it/s]
+
+ 32%|███▏      | 315/1000 [00:48<01:17,  8.80it/s]
+
+ 32%|███▏      | 316/1000 [00:48<01:28,  7.75it/s]
+
+ 32%|███▏      | 318/1000 [00:49<01:16,  8.89it/s]
+
+ 32%|███▏      | 319/1000 [00:49<01:59,  5.71it/s]
+
+ 32%|███▏      | 321/1000 [00:49<02:01,  5.58it/s]
+
+ 32%|███▏      | 322/1000 [00:50<02:08,  5.26it/s]
+
+ 32%|███▏      | 324/1000 [00:50<01:42,  6.60it/s]
+
+ 32%|███▎      | 325/1000 [00:50<02:21,  4.76it/s]
+
+ 33%|███▎      | 327/1000 [00:50<01:45,  6.35it/s]
+
+ 33%|███▎      | 328/1000 [00:50<01:48,  6.19it/s]
+
+ 33%|███▎      | 330/1000 [00:51<01:53,  5.89it/s]
+
+ 33%|███▎      | 332/1000 [00:51<01:56,  5.73it/s]
+
+ 33%|███▎      | 333/1000 [00:51<01:49,  6.07it/s]
+
+ 33%|███▎      | 334/1000 [00:52<01:59,  5.57it/s]
+
+ 34%|███▎      | 336/1000 [00:52<01:34,  7.06it/s]
+
+ 34%|███▍      | 338/1000 [00:52<01:25,  7.76it/s]
+
+ 34%|███▍      | 340/1000 [00:52<01:14,  8.85it/s]
+
+ 34%|███▍      | 342/1000 [00:52<01:08,  9.60it/s]
+
+ 34%|███▍      | 344/1000 [00:53<01:38,  6.66it/s]
+
+ 34%|███▍      | 345/1000 [00:53<01:36,  6.77it/s]
+
+ 35%|███▍      | 346/1000 [00:53<01:43,  6.31it/s]
+
+ 35%|███▍      | 347/1000 [00:53<01:40,  6.50it/s]
+
+ 35%|███▍      | 349/1000 [00:54<01:47,  6.07it/s]
+
+ 35%|███▌      | 350/1000 [00:54<02:04,  5.21it/s]
+
+ 35%|███▌      | 352/1000 [00:54<01:36,  6.69it/s]
+
+ 35%|███▌      | 354/1000 [00:54<01:37,  6.61it/s]
+
+ 36%|███▌      | 355/1000 [00:55<01:40,  6.44it/s]
+
+ 36%|███▌      | 357/1000 [00:55<01:22,  7.84it/s]
+
+ 36%|███▌      | 359/1000 [00:55<01:08,  9.29it/s]
+
+ 36%|███▌      | 361/1000 [00:55<01:17,  8.26it/s]
+
+ 36%|███▋      | 363/1000 [00:56<01:41,  6.28it/s]
+
+ 36%|███▋      | 364/1000 [00:56<01:55,  5.49it/s]
+
+ 36%|███▋      | 365/1000 [00:56<02:30,  4.21it/s]
+
+ 37%|███▋      | 367/1000 [00:57<02:16,  4.63it/s]
+
+ 37%|███▋      | 368/1000 [00:57<02:28,  4.25it/s]
+
+ 37%|███▋      | 369/1000 [00:57<02:11,  4.79it/s]
+
+ 37%|███▋      | 371/1000 [00:57<01:36,  6.51it/s]
+
+ 37%|███▋      | 372/1000 [00:57<01:34,  6.63it/s]
+
+ 37%|███▋      | 374/1000 [00:58<01:29,  7.00it/s]
+
+ 38%|███▊      | 375/1000 [00:58<01:59,  5.23it/s]
+
+ 38%|███▊      | 376/1000 [00:58<02:13,  4.67it/s]
+
+ 38%|███▊      | 377/1000 [00:58<01:57,  5.30it/s]
+
+ 38%|███▊      | 379/1000 [00:59<01:27,  7.08it/s]
+
+ 38%|███▊      | 381/1000 [00:59<01:19,  7.75it/s]
+
+ 38%|███▊      | 382/1000 [00:59<01:32,  6.71it/s]
+
+ 38%|███▊      | 384/1000 [00:59<01:16,  8.04it/s]
+
+ 38%|███▊      | 385/1000 [00:59<01:18,  7.87it/s]
+
+ 39%|███▊      | 386/1000 [00:59<01:15,  8.10it/s]
+
+ 39%|███▊      | 387/1000 [01:00<01:42,  6.01it/s]
+
+ 39%|███▉      | 388/1000 [01:00<01:44,  5.88it/s]
+
+ 39%|███▉      | 389/1000 [01:00<01:34,  6.47it/s]
+
+ 39%|███▉      | 390/1000 [01:00<01:31,  6.68it/s]
+
+ 39%|███▉      | 391/1000 [01:00<01:36,  6.31it/s]
+
+ 39%|███▉      | 393/1000 [01:01<01:15,  8.03it/s]
+
+ 40%|███▉      | 395/1000 [01:01<01:36,  6.28it/s]
+
+ 40%|███▉      | 397/1000 [01:01<01:20,  7.49it/s]
+
+ 40%|███▉      | 398/1000 [01:01<01:38,  6.11it/s]
+
+ 40%|███▉      | 399/1000 [01:02<01:59,  5.02it/s]
+
+ 40%|████      | 400/1000 [01:02<02:11,  4.57it/s]
+
+ 40%|████      | 402/1000 [01:02<02:03,  4.83it/s]
+
+ 40%|████      | 404/1000 [01:03<01:36,  6.20it/s]
+
+ 41%|████      | 406/1000 [01:03<01:19,  7.43it/s]
+
+ 41%|████      | 407/1000 [01:03<01:36,  6.14it/s]
+
+ 41%|████      | 408/1000 [01:03<01:50,  5.35it/s]
+
+ 41%|████      | 410/1000 [01:04<01:50,  5.32it/s]
+
+ 41%|████      | 411/1000 [01:04<02:02,  4.81it/s]
+
+ 41%|████      | 412/1000 [01:04<02:12,  4.45it/s]
+
+ 41%|████▏     | 413/1000 [01:04<01:54,  5.12it/s]
+
+ 42%|████▏     | 415/1000 [01:04<01:24,  6.96it/s]
+
+ 42%|████▏     | 416/1000 [01:05<01:41,  5.73it/s]
+
+ 42%|████▏     | 418/1000 [01:05<01:19,  7.32it/s]
+
+ 42%|████▏     | 420/1000 [01:05<01:17,  7.47it/s]
+
+ 42%|████▏     | 421/1000 [01:05<01:29,  6.50it/s]
+
+ 42%|████▏     | 423/1000 [01:06<01:42,  5.62it/s]
+
+ 42%|████▏     | 424/1000 [01:06<01:54,  5.03it/s]
+
+ 42%|████▎     | 425/1000 [01:06<01:45,  5.45it/s]
+
+ 43%|████▎     | 427/1000 [01:06<01:23,  6.85it/s]
+
+ 43%|████▎     | 428/1000 [01:07<01:28,  6.47it/s]
+
+ 43%|████▎     | 429/1000 [01:07<01:24,  6.76it/s]
+
+ 43%|████▎     | 431/1000 [01:07<01:08,  8.28it/s]
+
+ 43%|████▎     | 433/1000 [01:07<00:59,  9.45it/s]
+
+ 44%|████▎     | 435/1000 [01:07<00:58,  9.60it/s]
+
+ 44%|████▎     | 436/1000 [01:07<01:06,  8.51it/s]
+
+ 44%|████▍     | 438/1000 [01:08<01:00,  9.36it/s]
+
+ 44%|████▍     | 439/1000 [01:08<01:26,  6.49it/s]
+
+ 44%|████▍     | 441/1000 [01:08<01:11,  7.83it/s]
+
+ 44%|████▍     | 442/1000 [01:08<01:28,  6.27it/s]
+
+ 44%|████▍     | 444/1000 [01:09<01:30,  6.13it/s]
+
+ 45%|████▍     | 446/1000 [01:09<01:14,  7.44it/s]
+
+ 45%|████▍     | 448/1000 [01:09<01:25,  6.43it/s]
+
+ 45%|████▌     | 450/1000 [01:10<01:32,  5.95it/s]
+
+ 45%|████▌     | 451/1000 [01:10<01:43,  5.29it/s]
+
+ 45%|████▌     | 452/1000 [01:10<01:54,  4.80it/s]
+
+ 45%|████▌     | 454/1000 [01:10<01:28,  6.19it/s]
+
+ 46%|████▌     | 456/1000 [01:11<01:13,  7.37it/s]
+
+ 46%|████▌     | 457/1000 [01:11<01:20,  6.78it/s]
+
+ 46%|████▌     | 459/1000 [01:11<01:07,  8.06it/s]
+
+ 46%|████▌     | 461/1000 [01:11<00:57,  9.44it/s]
+
+ 46%|████▋     | 463/1000 [01:12<01:22,  6.49it/s]
+
+ 46%|████▋     | 465/1000 [01:12<01:32,  5.80it/s]
+
+ 47%|████▋     | 466/1000 [01:12<01:42,  5.19it/s]
+
+ 47%|████▋     | 467/1000 [01:13<02:08,  4.16it/s]
+
+ 47%|████▋     | 469/1000 [01:13<01:37,  5.47it/s]
+
+ 47%|████▋     | 471/1000 [01:13<01:17,  6.84it/s]
+
+ 47%|████▋     | 472/1000 [01:13<01:13,  7.17it/s]
+
+ 47%|████▋     | 473/1000 [01:13<01:21,  6.46it/s]
+
+ 47%|████▋     | 474/1000 [01:14<01:37,  5.40it/s]
+
+ 48%|████▊     | 476/1000 [01:14<01:36,  5.41it/s]
+
+ 48%|████▊     | 477/1000 [01:14<01:53,  4.61it/s]
+
+ 48%|████▊     | 479/1000 [01:15<01:24,  6.20it/s]
+
+ 48%|████▊     | 480/1000 [01:15<01:34,  5.52it/s]
+
+ 48%|████▊     | 482/1000 [01:15<01:15,  6.82it/s]
+
+ 48%|████▊     | 484/1000 [01:15<01:21,  6.29it/s]
+
+ 48%|████▊     | 485/1000 [01:16<01:36,  5.32it/s]
+
+ 49%|████▊     | 486/1000 [01:16<01:36,  5.33it/s]
+
+ 49%|████▉     | 488/1000 [01:16<01:14,  6.85it/s]
+
+ 49%|████▉     | 489/1000 [01:16<01:29,  5.71it/s]
+
+ 49%|████▉     | 491/1000 [01:16<01:11,  7.11it/s]
+
+ 49%|████▉     | 492/1000 [01:17<01:10,  7.20it/s]
+
+ 49%|████▉     | 494/1000 [01:17<00:57,  8.73it/s]
+
+ 50%|████▉     | 496/1000 [01:17<01:08,  7.35it/s]
+
+ 50%|████▉     | 498/1000 [01:17<01:08,  7.38it/s]
+
+ 50%|████▉     | 499/1000 [01:18<01:28,  5.66it/s]
+
+ 50%|█████     | 500/1000 [01:18<01:38,  5.06it/s]
+
+ 50%|█████     | 502/1000 [01:18<01:16,  6.48it/s]
+
+ 50%|█████     | 503/1000 [01:18<01:27,  5.71it/s]
+
+ 50%|█████     | 505/1000 [01:19<01:28,  5.62it/s]
+
+ 51%|█████     | 506/1000 [01:19<01:38,  5.02it/s]
+
+ 51%|█████     | 508/1000 [01:19<01:16,  6.46it/s]
+
+ 51%|█████     | 510/1000 [01:19<01:02,  7.78it/s]
+
+ 51%|█████     | 512/1000 [01:19<00:56,  8.64it/s]
+
+ 51%|█████▏    | 514/1000 [01:20<01:16,  6.33it/s]
+
+ 52%|█████▏    | 515/1000 [01:20<01:15,  6.42it/s]
+
+ 52%|█████▏    | 516/1000 [01:20<01:19,  6.08it/s]
+
+ 52%|█████▏    | 518/1000 [01:21<01:09,  6.98it/s]
+
+ 52%|█████▏    | 519/1000 [01:21<01:26,  5.53it/s]
+
+ 52%|█████▏    | 521/1000 [01:21<01:08,  6.98it/s]
+
+ 52%|█████▏    | 522/1000 [01:21<01:22,  5.81it/s]
+
+ 52%|█████▏    | 523/1000 [01:22<01:34,  5.07it/s]
+
+ 52%|█████▏    | 524/1000 [01:22<01:37,  4.86it/s]
+
+ 53%|█████▎    | 526/1000 [01:22<01:22,  5.77it/s]
+
+ 53%|█████▎    | 528/1000 [01:22<01:24,  5.60it/s]
+
+ 53%|█████▎    | 529/1000 [01:23<01:25,  5.51it/s]
+
+ 53%|█████▎    | 531/1000 [01:23<01:09,  6.74it/s]
+
+ 53%|█████▎    | 533/1000 [01:23<00:57,  8.13it/s]
+
+ 53%|█████▎    | 534/1000 [01:23<01:11,  6.51it/s]
+
+ 54%|█████▎    | 535/1000 [01:24<01:26,  5.37it/s]
+
+ 54%|█████▎    | 536/1000 [01:24<01:29,  5.17it/s]
+
+ 54%|█████▎    | 537/1000 [01:24<01:39,  4.64it/s]
+
+ 54%|█████▍    | 539/1000 [01:24<01:33,  4.95it/s]
+
+ 54%|█████▍    | 540/1000 [01:25<01:34,  4.84it/s]
+
+ 54%|█████▍    | 541/1000 [01:25<01:44,  4.37it/s]
+
+ 54%|█████▍    | 543/1000 [01:25<01:17,  5.92it/s]
+
+ 55%|█████▍    | 545/1000 [01:25<01:14,  6.09it/s]
+
+ 55%|█████▍    | 546/1000 [01:26<01:21,  5.60it/s]
+
+ 55%|█████▍    | 547/1000 [01:26<01:34,  4.81it/s]
+
+ 55%|█████▍    | 549/1000 [01:26<01:29,  5.06it/s]
+
+ 55%|█████▌    | 551/1000 [01:26<01:09,  6.45it/s]
+
+ 55%|█████▌    | 552/1000 [01:27<01:13,  6.10it/s]
+
+ 55%|█████▌    | 553/1000 [01:27<01:24,  5.28it/s]
+
+ 55%|█████▌    | 554/1000 [01:27<01:27,  5.12it/s]
+
+ 56%|█████▌    | 556/1000 [01:27<01:06,  6.72it/s]
+
+ 56%|█████▌    | 557/1000 [01:28<01:12,  6.09it/s]
+
+ 56%|█████▌    | 559/1000 [01:28<00:57,  7.70it/s]
+
+ 56%|█████▌    | 561/1000 [01:28<00:50,  8.63it/s]
+
+ 56%|█████▋    | 563/1000 [01:28<01:01,  7.06it/s]
+
+ 56%|█████▋    | 565/1000 [01:29<01:03,  6.81it/s]
+
+ 57%|█████▋    | 566/1000 [01:29<01:14,  5.81it/s]
+
+ 57%|█████▋    | 568/1000 [01:29<01:17,  5.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:29<01:18,  5.51it/s]
+
+ 57%|█████▋    | 571/1000 [01:30<01:17,  5.56it/s]
+
+ 57%|█████▋    | 572/1000 [01:30<01:18,  5.46it/s]
+
+ 57%|█████▋    | 574/1000 [01:30<01:17,  5.52it/s]
+
+ 58%|█████▊    | 576/1000 [01:31<01:03,  6.69it/s]
+
+ 58%|█████▊    | 577/1000 [01:31<01:25,  4.95it/s]
+
+ 58%|█████▊    | 578/1000 [01:31<01:32,  4.57it/s]
+
+ 58%|█████▊    | 580/1000 [01:31<01:09,  6.08it/s]
+
+ 58%|█████▊    | 582/1000 [01:32<00:56,  7.45it/s]
+
+ 58%|█████▊    | 583/1000 [01:32<01:08,  6.11it/s]
+
+ 58%|█████▊    | 584/1000 [01:32<01:03,  6.57it/s]
+
+ 58%|█████▊    | 585/1000 [01:32<01:10,  5.89it/s]
+
+ 59%|█████▊    | 587/1000 [01:32<01:06,  6.18it/s]
+
+ 59%|█████▉    | 589/1000 [01:33<00:54,  7.56it/s]
+
+ 59%|█████▉    | 590/1000 [01:33<01:10,  5.84it/s]
+
+ 59%|█████▉    | 592/1000 [01:33<00:56,  7.16it/s]
+
+ 59%|█████▉    | 593/1000 [01:33<01:08,  5.91it/s]
+
+ 60%|█████▉    | 595/1000 [01:34<00:55,  7.34it/s]
+
+ 60%|█████▉    | 596/1000 [01:34<01:09,  5.83it/s]
+
+ 60%|█████▉    | 597/1000 [01:34<01:11,  5.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:34<01:11,  5.58it/s]
+
+ 60%|██████    | 601/1000 [01:35<01:11,  5.54it/s]
+
+ 60%|██████    | 602/1000 [01:35<01:08,  5.82it/s]
+
+ 60%|██████    | 604/1000 [01:35<01:20,  4.95it/s]
+
+ 61%|██████    | 606/1000 [01:36<01:04,  6.08it/s]
+
+ 61%|██████    | 608/1000 [01:36<00:57,  6.76it/s]
+
+ 61%|██████    | 610/1000 [01:36<00:54,  7.09it/s]
+
+ 61%|██████    | 612/1000 [01:36<00:46,  8.28it/s]
+
+ 61%|██████▏   | 614/1000 [01:36<00:41,  9.25it/s]
+
+ 62%|██████▏   | 616/1000 [01:37<00:58,  6.59it/s]
+
+ 62%|██████▏   | 617/1000 [01:37<00:56,  6.72it/s]
+
+ 62%|██████▏   | 619/1000 [01:37<00:48,  7.90it/s]
+
+ 62%|██████▏   | 621/1000 [01:38<00:54,  7.00it/s]
+
+ 62%|██████▏   | 623/1000 [01:38<00:53,  7.02it/s]
+
+ 62%|██████▎   | 625/1000 [01:38<00:58,  6.42it/s]
+
+ 63%|██████▎   | 627/1000 [01:38<00:49,  7.48it/s]
+
+ 63%|██████▎   | 628/1000 [01:39<00:59,  6.25it/s]
+
+ 63%|██████▎   | 629/1000 [01:39<01:08,  5.42it/s]
+
+ 63%|██████▎   | 631/1000 [01:39<00:53,  6.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:39<00:44,  8.31it/s]
+
+ 63%|██████▎   | 634/1000 [01:40<01:05,  5.62it/s]
+
+ 64%|██████▎   | 635/1000 [01:40<01:01,  5.96it/s]
+
+ 64%|██████▎   | 636/1000 [01:40<00:57,  6.35it/s]
+
+ 64%|██████▍   | 638/1000 [01:40<00:51,  7.04it/s]
+
+ 64%|██████▍   | 640/1000 [01:40<00:44,  8.18it/s]
+
+ 64%|██████▍   | 641/1000 [01:41<00:55,  6.47it/s]
+
+ 64%|██████▍   | 643/1000 [01:41<00:52,  6.79it/s]
+
+ 64%|██████▍   | 645/1000 [01:41<00:56,  6.30it/s]
+
+ 65%|██████▍   | 647/1000 [01:41<00:52,  6.70it/s]
+
+ 65%|██████▍   | 648/1000 [01:42<01:01,  5.75it/s]
+
+ 65%|██████▌   | 650/1000 [01:42<01:02,  5.64it/s]
+
+ 65%|██████▌   | 652/1000 [01:43<01:03,  5.45it/s]
+
+ 65%|██████▌   | 654/1000 [01:43<00:52,  6.65it/s]
+
+ 66%|██████▌   | 656/1000 [01:43<00:45,  7.55it/s]
+
+ 66%|██████▌   | 657/1000 [01:43<00:54,  6.33it/s]
+
+ 66%|██████▌   | 659/1000 [01:44<01:03,  5.35it/s]
+
+ 66%|██████▌   | 661/1000 [01:44<00:50,  6.67it/s]
+
+ 66%|██████▌   | 662/1000 [01:44<00:59,  5.72it/s]
+
+ 66%|██████▋   | 664/1000 [01:45<01:07,  4.98it/s]
+
+ 66%|██████▋   | 665/1000 [01:45<01:12,  4.63it/s]
+
+ 67%|██████▋   | 667/1000 [01:45<01:08,  4.88it/s]
+
+ 67%|██████▋   | 668/1000 [01:45<01:13,  4.54it/s]
+
+ 67%|██████▋   | 669/1000 [01:46<01:06,  4.99it/s]
+
+ 67%|██████▋   | 671/1000 [01:46<01:03,  5.15it/s]
+
+ 67%|██████▋   | 673/1000 [01:46<00:56,  5.81it/s]
+
+ 68%|██████▊   | 675/1000 [01:47<00:50,  6.38it/s]
+
+ 68%|██████▊   | 677/1000 [01:47<00:42,  7.60it/s]
+
+ 68%|██████▊   | 679/1000 [01:47<00:37,  8.53it/s]
+
+ 68%|██████▊   | 681/1000 [01:47<00:32,  9.76it/s]
+
+ 68%|██████▊   | 683/1000 [01:47<00:40,  7.90it/s]
+
+ 68%|██████▊   | 685/1000 [01:48<00:36,  8.75it/s]
+
+ 69%|██████▊   | 686/1000 [01:48<00:45,  6.90it/s]
+
+ 69%|██████▊   | 687/1000 [01:48<00:53,  5.80it/s]
+
+ 69%|██████▉   | 688/1000 [01:48<00:50,  6.18it/s]
+
+ 69%|██████▉   | 690/1000 [01:49<00:52,  5.87it/s]
+
+ 69%|██████▉   | 692/1000 [01:49<00:44,  6.90it/s]
+
+ 69%|██████▉   | 694/1000 [01:49<00:47,  6.38it/s]
+
+ 70%|██████▉   | 695/1000 [01:49<00:46,  6.53it/s]
+
+ 70%|██████▉   | 696/1000 [01:50<00:55,  5.52it/s]
+
+ 70%|██████▉   | 698/1000 [01:50<00:49,  6.07it/s]
+
+ 70%|██████▉   | 699/1000 [01:50<00:57,  5.28it/s]
+
+ 70%|███████   | 701/1000 [01:50<00:44,  6.67it/s]
+
+ 70%|███████   | 702/1000 [01:51<01:00,  4.90it/s]
+
+ 70%|███████   | 704/1000 [01:51<00:47,  6.19it/s]
+
+ 70%|███████   | 705/1000 [01:51<01:03,  4.64it/s]
+
+ 71%|███████   | 707/1000 [01:51<00:48,  6.05it/s]
+
+ 71%|███████   | 709/1000 [01:52<00:44,  6.60it/s]
+
+ 71%|███████   | 710/1000 [01:52<00:51,  5.65it/s]
+
+ 71%|███████   | 712/1000 [01:52<00:43,  6.60it/s]
+
+ 71%|███████▏  | 713/1000 [01:52<00:42,  6.74it/s]
+
+ 72%|███████▏  | 715/1000 [01:52<00:35,  8.09it/s]
+
+ 72%|███████▏  | 716/1000 [01:53<00:44,  6.41it/s]
+
+ 72%|███████▏  | 717/1000 [01:53<00:40,  6.97it/s]
+
+ 72%|███████▏  | 718/1000 [01:53<00:49,  5.67it/s]
+
+ 72%|███████▏  | 719/1000 [01:53<00:57,  4.90it/s]
+
+ 72%|███████▏  | 720/1000 [01:54<00:51,  5.49it/s]
+
+ 72%|███████▏  | 722/1000 [01:54<00:39,  7.11it/s]
+
+ 72%|███████▏  | 723/1000 [01:54<00:38,  7.20it/s]
+
+ 72%|███████▏  | 724/1000 [01:54<00:38,  7.21it/s]
+
+ 72%|███████▎  | 725/1000 [01:54<00:37,  7.29it/s]
+
+ 73%|███████▎  | 727/1000 [01:54<00:30,  8.95it/s]
+
+ 73%|███████▎  | 729/1000 [01:54<00:27,  9.86it/s]
+
+ 73%|███████▎  | 730/1000 [01:55<00:37,  7.18it/s]
+
+ 73%|███████▎  | 731/1000 [01:55<00:37,  7.20it/s]
+
+ 73%|███████▎  | 733/1000 [01:55<00:31,  8.56it/s]
+
+ 73%|███████▎  | 734/1000 [01:55<00:31,  8.45it/s]
+
+ 74%|███████▎  | 736/1000 [01:55<00:33,  7.87it/s]
+
+ 74%|███████▍  | 738/1000 [01:56<00:30,  8.50it/s]
+
+ 74%|███████▍  | 740/1000 [01:56<00:27,  9.35it/s]
+
+ 74%|███████▍  | 741/1000 [01:56<00:36,  7.14it/s]
+
+ 74%|███████▍  | 743/1000 [01:56<00:36,  7.05it/s]
+
+ 74%|███████▍  | 745/1000 [01:57<00:30,  8.30it/s]
+
+ 75%|███████▍  | 747/1000 [01:57<00:27,  9.14it/s]
+
+ 75%|███████▍  | 748/1000 [01:57<00:35,  7.09it/s]
+
+ 75%|███████▌  | 750/1000 [01:57<00:36,  6.80it/s]
+
+ 75%|███████▌  | 751/1000 [01:58<00:46,  5.39it/s]
+
+ 75%|███████▌  | 753/1000 [01:58<00:39,  6.33it/s]
+
+ 75%|███████▌  | 754/1000 [01:58<00:44,  5.48it/s]
+
+ 76%|███████▌  | 756/1000 [01:59<00:44,  5.54it/s]
+
+ 76%|███████▌  | 758/1000 [01:59<00:36,  6.59it/s]
+
+ 76%|███████▌  | 759/1000 [01:59<00:37,  6.37it/s]
+
+ 76%|███████▌  | 761/1000 [01:59<00:31,  7.59it/s]
+
+ 76%|███████▋  | 763/1000 [01:59<00:35,  6.71it/s]
+
+ 76%|███████▋  | 764/1000 [02:00<00:41,  5.75it/s]
+
+ 77%|███████▋  | 766/1000 [02:00<00:41,  5.65it/s]
+
+ 77%|███████▋  | 768/1000 [02:00<00:34,  6.82it/s]
+
+ 77%|███████▋  | 769/1000 [02:01<00:40,  5.68it/s]
+
+ 77%|███████▋  | 770/1000 [02:01<00:40,  5.65it/s]
+
+ 77%|███████▋  | 771/1000 [02:01<00:36,  6.22it/s]
+
+ 77%|███████▋  | 773/1000 [02:01<00:40,  5.62it/s]
+
+ 77%|███████▋  | 774/1000 [02:01<00:38,  5.94it/s]
+
+ 78%|███████▊  | 776/1000 [02:02<00:33,  6.73it/s]
+
+ 78%|███████▊  | 778/1000 [02:02<00:27,  8.21it/s]
+
+ 78%|███████▊  | 780/1000 [02:02<00:36,  6.00it/s]
+
+ 78%|███████▊  | 782/1000 [02:02<00:30,  7.23it/s]
+
+ 78%|███████▊  | 783/1000 [02:03<00:35,  6.05it/s]
+
+ 78%|███████▊  | 784/1000 [02:03<00:34,  6.30it/s]
+
+ 79%|███████▊  | 786/1000 [02:03<00:32,  6.57it/s]
+
+ 79%|███████▉  | 788/1000 [02:03<00:28,  7.38it/s]
+
+ 79%|███████▉  | 790/1000 [02:04<00:26,  7.93it/s]
+
+ 79%|███████▉  | 791/1000 [02:04<00:29,  7.17it/s]
+
+ 79%|███████▉  | 792/1000 [02:04<00:30,  6.75it/s]
+
+ 79%|███████▉  | 793/1000 [02:04<00:29,  7.00it/s]
+
+ 80%|███████▉  | 795/1000 [02:04<00:32,  6.31it/s]
+
+ 80%|███████▉  | 797/1000 [02:05<00:33,  6.01it/s]
+
+ 80%|███████▉  | 798/1000 [02:05<00:38,  5.27it/s]
+
+ 80%|████████  | 800/1000 [02:05<00:33,  5.91it/s]
+
+ 80%|████████  | 802/1000 [02:06<00:29,  6.73it/s]
+
+ 80%|████████  | 804/1000 [02:06<00:28,  6.90it/s]
+
+ 81%|████████  | 806/1000 [02:06<00:24,  8.07it/s]
+
+ 81%|████████  | 807/1000 [02:06<00:24,  7.89it/s]
+
+ 81%|████████  | 809/1000 [02:06<00:21,  9.02it/s]
+
+ 81%|████████  | 811/1000 [02:06<00:19,  9.92it/s]
+
+ 81%|████████▏ | 813/1000 [02:07<00:23,  7.97it/s]
+
+ 81%|████████▏ | 814/1000 [02:07<00:23,  7.82it/s]
+
+ 82%|████████▏ | 816/1000 [02:07<00:21,  8.74it/s]
+
+ 82%|████████▏ | 817/1000 [02:07<00:26,  6.86it/s]
+
+ 82%|████████▏ | 819/1000 [02:08<00:22,  8.06it/s]
+
+ 82%|████████▏ | 820/1000 [02:08<00:28,  6.33it/s]
+
+ 82%|████████▏ | 822/1000 [02:08<00:23,  7.46it/s]
+
+ 82%|████████▏ | 823/1000 [02:08<00:27,  6.53it/s]
+
+ 82%|████████▏ | 824/1000 [02:09<00:31,  5.51it/s]
+
+ 82%|████████▎ | 825/1000 [02:09<00:29,  5.86it/s]
+
+ 83%|████████▎ | 826/1000 [02:09<00:36,  4.76it/s]
+
+ 83%|████████▎ | 828/1000 [02:09<00:39,  4.37it/s]
+
+ 83%|████████▎ | 829/1000 [02:10<00:35,  4.86it/s]
+
+ 83%|████████▎ | 831/1000 [02:10<00:28,  5.91it/s]
+
+ 83%|████████▎ | 832/1000 [02:10<00:34,  4.85it/s]
+
+ 83%|████████▎ | 833/1000 [02:10<00:31,  5.30it/s]
+
+ 83%|████████▎ | 834/1000 [02:11<00:33,  5.00it/s]
+
+ 84%|████████▎ | 835/1000 [02:11<00:30,  5.49it/s]
+
+ 84%|████████▎ | 836/1000 [02:11<00:31,  5.18it/s]
+
+ 84%|████████▍ | 838/1000 [02:11<00:25,  6.39it/s]
+
+ 84%|████████▍ | 840/1000 [02:12<00:28,  5.60it/s]
+
+ 84%|████████▍ | 842/1000 [02:12<00:22,  6.93it/s]
+
+ 84%|████████▍ | 844/1000 [02:12<00:28,  5.39it/s]
+
+ 84%|████████▍ | 845/1000 [02:13<00:31,  4.90it/s]
+
+ 85%|████████▍ | 847/1000 [02:13<00:29,  5.25it/s]
+
+ 85%|████████▍ | 849/1000 [02:13<00:23,  6.41it/s]
+
+ 85%|████████▌ | 850/1000 [02:13<00:26,  5.58it/s]
+
+ 85%|████████▌ | 851/1000 [02:13<00:25,  5.94it/s]
+
+ 85%|████████▌ | 853/1000 [02:14<00:19,  7.35it/s]
+
+ 86%|████████▌ | 855/1000 [02:14<00:16,  8.72it/s]
+
+ 86%|████████▌ | 856/1000 [02:14<00:16,  8.54it/s]
+
+ 86%|████████▌ | 858/1000 [02:14<00:20,  7.09it/s]
+
+ 86%|████████▌ | 860/1000 [02:14<00:17,  8.20it/s]
+
+ 86%|████████▌ | 862/1000 [02:15<00:15,  8.92it/s]
+
+ 86%|████████▋ | 864/1000 [02:15<00:13, 10.06it/s]
+
+ 87%|████████▋ | 866/1000 [02:15<00:12, 10.64it/s]
+
+ 87%|████████▋ | 868/1000 [02:15<00:16,  8.21it/s]
+
+ 87%|████████▋ | 870/1000 [02:15<00:14,  8.75it/s]
+
+ 87%|████████▋ | 872/1000 [02:16<00:15,  8.17it/s]
+
+ 87%|████████▋ | 873/1000 [02:16<00:19,  6.58it/s]
+
+ 87%|████████▋ | 874/1000 [02:16<00:21,  5.95it/s]
+
+ 88%|████████▊ | 876/1000 [02:16<00:16,  7.37it/s]
+
+ 88%|████████▊ | 878/1000 [02:17<00:18,  6.58it/s]
+
+ 88%|████████▊ | 879/1000 [02:17<00:20,  5.98it/s]
+
+ 88%|████████▊ | 880/1000 [02:17<00:18,  6.49it/s]
+
+ 88%|████████▊ | 881/1000 [02:17<00:17,  6.65it/s]
+
+ 88%|████████▊ | 882/1000 [02:18<00:21,  5.39it/s]
+
+ 88%|████████▊ | 884/1000 [02:18<00:16,  7.07it/s]
+
+ 89%|████████▊ | 886/1000 [02:18<00:13,  8.35it/s]
+
+ 89%|████████▉ | 888/1000 [02:18<00:11,  9.45it/s]
+
+ 89%|████████▉ | 890/1000 [02:18<00:12,  8.73it/s]
+
+ 89%|████████▉ | 892/1000 [02:18<00:11,  9.62it/s]
+
+ 89%|████████▉ | 894/1000 [02:19<00:15,  6.67it/s]
+
+ 90%|████████▉ | 896/1000 [02:19<00:13,  7.78it/s]
+
+ 90%|████████▉ | 897/1000 [02:19<00:16,  6.41it/s]
+
+ 90%|████████▉ | 898/1000 [02:20<00:19,  5.31it/s]
+
+ 90%|█████████ | 900/1000 [02:20<00:15,  6.65it/s]
+
+ 90%|█████████ | 901/1000 [02:20<00:14,  6.66it/s]
+
+ 90%|█████████ | 902/1000 [02:20<00:14,  6.92it/s]
+
+ 90%|█████████ | 904/1000 [02:20<00:11,  8.37it/s]
+
+ 90%|█████████ | 905/1000 [02:21<00:13,  7.14it/s]
+
+ 91%|█████████ | 906/1000 [02:21<00:16,  5.77it/s]
+
+ 91%|█████████ | 908/1000 [02:21<00:12,  7.38it/s]
+
+ 91%|█████████ | 910/1000 [02:21<00:13,  6.58it/s]
+
+ 91%|█████████ | 912/1000 [02:22<00:11,  7.84it/s]
+
+ 91%|█████████▏| 913/1000 [02:22<00:15,  5.75it/s]
+
+ 92%|█████████▏| 915/1000 [02:22<00:12,  7.03it/s]
+
+ 92%|█████████▏| 916/1000 [02:22<00:14,  5.92it/s]
+
+ 92%|█████████▏| 917/1000 [02:22<00:13,  6.30it/s]
+
+ 92%|█████████▏| 918/1000 [02:23<00:17,  4.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:23<00:16,  4.77it/s]
+
+ 92%|█████████▏| 921/1000 [02:23<00:16,  4.72it/s]
+
+ 92%|█████████▏| 922/1000 [02:24<00:15,  4.97it/s]
+
+ 92%|█████████▏| 924/1000 [02:24<00:12,  6.14it/s]
+
+ 93%|█████████▎| 926/1000 [02:24<00:09,  7.47it/s]
+
+ 93%|█████████▎| 927/1000 [02:24<00:10,  7.01it/s]
+
+ 93%|█████████▎| 928/1000 [02:24<00:12,  5.75it/s]
+
+ 93%|█████████▎| 930/1000 [02:25<00:12,  5.49it/s]
+
+ 93%|█████████▎| 931/1000 [02:25<00:11,  5.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:25<00:11,  5.59it/s]
+
+ 93%|█████████▎| 934/1000 [02:26<00:12,  5.27it/s]
+
+ 94%|█████████▎| 936/1000 [02:26<00:10,  6.28it/s]
+
+ 94%|█████████▍| 938/1000 [02:26<00:08,  7.63it/s]
+
+ 94%|█████████▍| 939/1000 [02:26<00:08,  6.79it/s]
+
+ 94%|█████████▍| 940/1000 [02:26<00:08,  6.89it/s]
+
+ 94%|█████████▍| 942/1000 [02:27<00:10,  5.28it/s]
+
+ 94%|█████████▍| 944/1000 [02:27<00:09,  5.99it/s]
+
+ 95%|█████████▍| 946/1000 [02:27<00:07,  7.15it/s]
+
+ 95%|█████████▍| 948/1000 [02:27<00:06,  8.32it/s]
+
+ 95%|█████████▍| 949/1000 [02:28<00:07,  7.19it/s]
+
+ 95%|█████████▌| 951/1000 [02:28<00:05,  8.27it/s]
+
+ 95%|█████████▌| 952/1000 [02:28<00:07,  6.11it/s]
+
+ 95%|█████████▌| 954/1000 [02:29<00:09,  4.96it/s]
+
+ 96%|█████████▌| 956/1000 [02:29<00:07,  6.22it/s]
+
+ 96%|█████████▌| 957/1000 [02:29<00:08,  5.37it/s]
+
+ 96%|█████████▌| 959/1000 [02:30<00:08,  4.88it/s]
+
+ 96%|█████████▌| 961/1000 [02:30<00:06,  5.87it/s]
+
+ 96%|█████████▌| 962/1000 [02:30<00:07,  5.22it/s]
+
+ 96%|█████████▋| 964/1000 [02:30<00:05,  6.52it/s]
+
+ 96%|█████████▋| 965/1000 [02:31<00:06,  5.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:31<00:06,  5.40it/s]
+
+ 97%|█████████▋| 968/1000 [02:31<00:04,  6.92it/s]
+
+ 97%|█████████▋| 970/1000 [02:31<00:03,  8.18it/s]
+
+ 97%|█████████▋| 971/1000 [02:31<00:04,  6.48it/s]
+
+ 97%|█████████▋| 973/1000 [02:32<00:03,  7.75it/s]
+
+ 98%|█████████▊| 975/1000 [02:32<00:03,  8.18it/s]
+
+ 98%|█████████▊| 977/1000 [02:32<00:02,  9.38it/s]
+
+ 98%|█████████▊| 979/1000 [02:32<00:02,  9.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:33<00:03,  6.09it/s]
+
+ 98%|█████████▊| 982/1000 [02:33<00:03,  4.86it/s]
+
+ 98%|█████████▊| 984/1000 [02:33<00:02,  6.04it/s]
+
+ 99%|█████████▊| 986/1000 [02:33<00:01,  7.23it/s]
+
+ 99%|█████████▊| 987/1000 [02:34<00:01,  6.76it/s]
+
+ 99%|█████████▉| 988/1000 [02:34<00:02,  5.64it/s]
+
+ 99%|█████████▉| 989/1000 [02:34<00:02,  4.97it/s]
+
+ 99%|█████████▉| 991/1000 [02:35<00:01,  5.22it/s]
+
+ 99%|█████████▉| 992/1000 [02:35<00:01,  4.19it/s]
+
+ 99%|█████████▉| 994/1000 [02:35<00:01,  5.49it/s]
+
+100%|█████████▉| 995/1000 [02:35<00:01,  4.65it/s]
+
+100%|█████████▉| 997/1000 [02:36<00:00,  6.03it/s]
+
+100%|█████████▉| 999/1000 [02:36<00:00,  5.82it/s]
+
+100%|██████████| 1000/1000 [02:36<00:00,  5.76it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7433', 'eval_runtime': '156.9', 'eval_samples_per_second': '6.374', 'eval_steps_per_second': '6.374', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904}
+
+ 83%|████████▎ | 1500/1813 [2:48:11<36:02,  6.91s/it]
+
+100%|██████████| 1000/1000 [02:36<00:00,  5.76it/s]
+
+                                                   
+ 83%|████████▎ | 1501/1813 [2:48:18<4:41:15, 54.09s/it]
+ 83%|████████▎ | 1502/1813 [2:48:23<3:24:09, 39.39s/it]
+ 83%|████████▎ | 1503/1813 [2:48:30<2:33:31, 29.71s/it]
+ 83%|████████▎ | 1504/1813 [2:48:36<1:55:58, 22.52s/it]
+ 83%|████████▎ | 1505/1813 [2:48:42<1:30:30, 17.63s/it]
+ 83%|████████▎ | 1506/1813 [2:48:50<1:15:15, 14.71s/it]
+ 83%|████████▎ | 1507/1813 [2:48:56<1:01:08, 11.99s/it]
+ 83%|████████▎ | 1508/1813 [2:49:04<54:52, 10.80s/it]  
+ 83%|████████▎ | 1509/1813 [2:49:10<48:08,  9.50s/it]
+ 83%|████████▎ | 1510/1813 [2:49:17<43:40,  8.65s/it]
+                                                     
+{'loss': '0.8178', 'grad_norm': '1.216', 'learning_rate': '1.604e-05', 'epoch': '0.8331', 'num_input_tokens_seen': 10151928, 'train_runtime': '1.016e+04', 'train_tokens_per_second': '999.4'}
+
+ 83%|████████▎ | 1510/1813 [2:49:17<43:40,  8.65s/it]
+ 83%|████████▎ | 1511/1813 [2:49:24<41:29,  8.24s/it]
+ 83%|████████▎ | 1512/1813 [2:49:31<38:37,  7.70s/it]
+ 83%|████████▎ | 1513/1813 [2:49:37<36:37,  7.33s/it]
+ 84%|████████▎ | 1514/1813 [2:49:45<36:44,  7.37s/it]
+ 84%|████████▎ | 1515/1813 [2:49:53<38:21,  7.72s/it]
+ 84%|████████▎ | 1516/1813 [2:50:01<38:38,  7.81s/it]
+ 84%|████████▎ | 1517/1813 [2:50:09<38:02,  7.71s/it]
+ 84%|████████▎ | 1518/1813 [2:50:18<39:25,  8.02s/it]
+ 84%|████████▍ | 1519/1813 [2:50:26<39:32,  8.07s/it]
+ 84%|████████▍ | 1520/1813 [2:50:33<37:54,  7.76s/it]
+                                                     
+{'loss': '0.8583', 'grad_norm': '1.18', 'learning_rate': '1.503e-05', 'epoch': '0.8386', 'num_input_tokens_seen': 10225024, 'train_runtime': '1.023e+04', 'train_tokens_per_second': '999.2'}
+
+ 84%|████████▍ | 1520/1813 [2:50:33<37:54,  7.76s/it]
+ 84%|████████▍ | 1521/1813 [2:50:41<37:54,  7.79s/it]
+ 84%|████████▍ | 1522/1813 [2:50:48<36:47,  7.59s/it]
+ 84%|████████▍ | 1523/1813 [2:50:56<37:47,  7.82s/it]
+ 84%|████████▍ | 1524/1813 [2:51:03<36:48,  7.64s/it]
+ 84%|████████▍ | 1525/1813 [2:51:09<33:32,  6.99s/it]
+ 84%|████████▍ | 1526/1813 [2:51:16<33:56,  7.10s/it]
+ 84%|████████▍ | 1527/1813 [2:51:24<34:34,  7.25s/it]
+ 84%|████████▍ | 1528/1813 [2:51:31<34:32,  7.27s/it]
+ 84%|████████▍ | 1529/1813 [2:51:39<35:26,  7.49s/it]
+ 84%|████████▍ | 1530/1813 [2:51:45<33:48,  7.17s/it]
+                                                     
+{'loss': '0.9037', 'grad_norm': '0.9507', 'learning_rate': '1.405e-05', 'epoch': '0.8441', 'num_input_tokens_seen': 10294368, 'train_runtime': '1.031e+04', 'train_tokens_per_second': '998.9'}
+
+ 84%|████████▍ | 1530/1813 [2:51:45<33:48,  7.17s/it]
+ 84%|████████▍ | 1531/1813 [2:51:51<31:54,  6.79s/it]
+ 85%|████████▍ | 1532/1813 [2:51:59<33:19,  7.12s/it]
+ 85%|████████▍ | 1533/1813 [2:52:05<31:19,  6.71s/it]
+ 85%|████████▍ | 1534/1813 [2:52:14<34:42,  7.46s/it]
+ 85%|████████▍ | 1535/1813 [2:52:20<32:50,  7.09s/it]
+ 85%|████████▍ | 1536/1813 [2:52:29<34:49,  7.54s/it]
+ 85%|████████▍ | 1537/1813 [2:52:37<34:55,  7.59s/it]
+ 85%|████████▍ | 1538/1813 [2:52:42<31:35,  6.89s/it]
+ 85%|████████▍ | 1539/1813 [2:52:49<30:53,  6.76s/it]
+ 85%|████████▍ | 1540/1813 [2:52:55<30:11,  6.64s/it]
+                                                     
+{'loss': '0.9012', 'grad_norm': '1.3', 'learning_rate': '1.31e-05', 'epoch': '0.8497', 'num_input_tokens_seen': 10359416, 'train_runtime': '1.038e+04', 'train_tokens_per_second': '998.5'}
+
+ 85%|████████▍ | 1540/1813 [2:52:55<30:11,  6.64s/it]
+ 85%|████████▍ | 1541/1813 [2:53:02<31:17,  6.90s/it]
+ 85%|████████▌ | 1542/1813 [2:53:10<31:29,  6.97s/it]
+ 85%|████████▌ | 1543/1813 [2:53:15<29:24,  6.53s/it]
+ 85%|████████▌ | 1544/1813 [2:53:21<28:08,  6.28s/it]
+ 85%|████████▌ | 1545/1813 [2:53:30<31:45,  7.11s/it]
+ 85%|████████▌ | 1546/1813 [2:53:37<31:17,  7.03s/it]
+ 85%|████████▌ | 1547/1813 [2:53:43<29:43,  6.71s/it]
+ 85%|████████▌ | 1548/1813 [2:53:52<33:07,  7.50s/it]
+ 85%|████████▌ | 1549/1813 [2:53:58<31:02,  7.06s/it]
+ 85%|████████▌ | 1550/1813 [2:54:06<32:37,  7.44s/it]
+                                                     
+{'loss': '0.9699', 'grad_norm': '0.9705', 'learning_rate': '1.218e-05', 'epoch': '0.8552', 'num_input_tokens_seen': 10427368, 'train_runtime': '1.045e+04', 'train_tokens_per_second': '998.1'}
+
+ 85%|████████▌ | 1550/1813 [2:54:06<32:37,  7.44s/it]
+ 86%|████████▌ | 1551/1813 [2:54:15<34:13,  7.84s/it]
+ 86%|████████▌ | 1552/1813 [2:54:22<32:38,  7.50s/it]
+ 86%|████████▌ | 1553/1813 [2:54:28<31:19,  7.23s/it]
+ 86%|████████▌ | 1554/1813 [2:54:36<31:38,  7.33s/it]
+ 86%|████████▌ | 1555/1813 [2:54:43<31:49,  7.40s/it]
+ 86%|████████▌ | 1556/1813 [2:54:50<31:09,  7.27s/it]
+ 86%|████████▌ | 1557/1813 [2:54:59<32:59,  7.73s/it]
+ 86%|████████▌ | 1558/1813 [2:55:06<31:43,  7.46s/it]
+ 86%|████████▌ | 1559/1813 [2:55:13<31:13,  7.38s/it]
+ 86%|████████▌ | 1560/1813 [2:55:19<29:16,  6.94s/it]
+                                                     
+{'loss': '0.9822', 'grad_norm': '0.9182', 'learning_rate': '1.129e-05', 'epoch': '0.8607', 'num_input_tokens_seen': 10497472, 'train_runtime': '1.052e+04', 'train_tokens_per_second': '997.9'}
+
+ 86%|████████▌ | 1560/1813 [2:55:19<29:16,  6.94s/it]
+ 86%|████████▌ | 1561/1813 [2:55:27<30:28,  7.26s/it]
+ 86%|████████▌ | 1562/1813 [2:55:34<30:22,  7.26s/it]
+ 86%|████████▌ | 1563/1813 [2:55:41<29:51,  7.17s/it]
+ 86%|████████▋ | 1564/1813 [2:55:48<29:35,  7.13s/it]
+ 86%|████████▋ | 1565/1813 [2:55:53<26:47,  6.48s/it]
+ 86%|████████▋ | 1566/1813 [2:56:00<26:58,  6.55s/it]
+ 86%|████████▋ | 1567/1813 [2:56:07<27:24,  6.68s/it]
+ 86%|████████▋ | 1568/1813 [2:56:13<25:52,  6.34s/it]
+ 87%|████████▋ | 1569/1813 [2:56:21<28:34,  7.03s/it]
+ 87%|████████▋ | 1570/1813 [2:56:30<30:09,  7.45s/it]
+                                                     
+{'loss': '0.799', 'grad_norm': '1.071', 'learning_rate': '1.044e-05', 'epoch': '0.8662', 'num_input_tokens_seen': 10564336, 'train_runtime': '1.059e+04', 'train_tokens_per_second': '997.6'}
+
+ 87%|████████▋ | 1570/1813 [2:56:30<30:09,  7.45s/it]
+ 87%|████████▋ | 1571/1813 [2:56:36<28:12,  6.99s/it]
+ 87%|████████▋ | 1572/1813 [2:56:42<27:07,  6.75s/it]
+ 87%|████████▋ | 1573/1813 [2:56:48<26:36,  6.65s/it]
+ 87%|████████▋ | 1574/1813 [2:56:56<27:14,  6.84s/it]
+ 87%|████████▋ | 1575/1813 [2:57:02<26:45,  6.75s/it]
+ 87%|████████▋ | 1576/1813 [2:57:10<28:38,  7.25s/it]
+ 87%|████████▋ | 1577/1813 [2:57:18<29:07,  7.40s/it]
+ 87%|████████▋ | 1578/1813 [2:57:25<28:41,  7.33s/it]
+ 87%|████████▋ | 1579/1813 [2:57:32<28:07,  7.21s/it]
+ 87%|████████▋ | 1580/1813 [2:57:38<26:11,  6.74s/it]
+                                                     
+{'loss': '0.9316', 'grad_norm': '1.203', 'learning_rate': '9.612e-06', 'epoch': '0.8717', 'num_input_tokens_seen': 10628400, 'train_runtime': '1.066e+04', 'train_tokens_per_second': '997.2'}
+
+ 87%|████████▋ | 1580/1813 [2:57:38<26:11,  6.74s/it]
+ 87%|████████▋ | 1581/1813 [2:57:45<25:55,  6.71s/it]
+ 87%|████████▋ | 1582/1813 [2:57:49<23:34,  6.12s/it]
+ 87%|████████▋ | 1583/1813 [2:57:57<24:41,  6.44s/it]
+ 87%|████████▋ | 1584/1813 [2:58:04<25:31,  6.69s/it]
+ 87%|████████▋ | 1585/1813 [2:58:12<27:20,  7.20s/it]
+ 87%|████████▋ | 1586/1813 [2:58:21<28:38,  7.57s/it]
+ 88%|████████▊ | 1587/1813 [2:58:29<29:36,  7.86s/it]
+ 88%|████████▊ | 1588/1813 [2:58:36<28:07,  7.50s/it]
+ 88%|████████▊ | 1589/1813 [2:58:42<25:58,  6.96s/it]
+ 88%|████████▊ | 1590/1813 [2:58:49<26:35,  7.15s/it]
+                                                     
+{'loss': '0.8658', 'grad_norm': '1.09', 'learning_rate': '8.82e-06', 'epoch': '0.8772', 'num_input_tokens_seen': 10695288, 'train_runtime': '1.073e+04', 'train_tokens_per_second': '996.8'}
+
+ 88%|████████▊ | 1590/1813 [2:58:49<26:35,  7.15s/it]
+ 88%|████████▊ | 1591/1813 [2:58:56<26:12,  7.08s/it]
+ 88%|████████▊ | 1592/1813 [2:59:03<26:25,  7.18s/it]
+ 88%|████████▊ | 1593/1813 [2:59:11<26:46,  7.30s/it]
+ 88%|████████▊ | 1594/1813 [2:59:19<27:03,  7.41s/it]
+ 88%|████████▊ | 1595/1813 [2:59:27<28:02,  7.72s/it]
+ 88%|████████▊ | 1596/1813 [2:59:37<30:23,  8.40s/it]
+ 88%|████████▊ | 1597/1813 [2:59:45<29:36,  8.23s/it]
+ 88%|████████▊ | 1598/1813 [2:59:52<28:42,  8.01s/it]
+ 88%|████████▊ | 1599/1813 [2:59:59<26:48,  7.52s/it]
+ 88%|████████▊ | 1600/1813 [3:00:04<24:10,  6.81s/it]
+                                                     
+{'loss': '0.8727', 'grad_norm': '1.014', 'learning_rate': '8.061e-06', 'epoch': '0.8828', 'num_input_tokens_seen': 10767512, 'train_runtime': '1.08e+04', 'train_tokens_per_second': '996.6'}
+
+ 88%|████████▊ | 1600/1813 [3:00:04<24:10,  6.81s/it]
+ 88%|████████▊ | 1601/1813 [3:00:13<26:34,  7.52s/it]
+ 88%|████████▊ | 1602/1813 [3:00:20<25:52,  7.36s/it]
+ 88%|████████▊ | 1603/1813 [3:00:28<26:16,  7.51s/it]
+ 88%|████████▊ | 1604/1813 [3:00:36<26:28,  7.60s/it]
+ 89%|████████▊ | 1605/1813 [3:00:45<28:13,  8.14s/it]
+ 89%|████████▊ | 1606/1813 [3:00:51<25:47,  7.47s/it]
+ 89%|████████▊ | 1607/1813 [3:00:59<25:53,  7.54s/it]
+ 89%|████████▊ | 1608/1813 [3:01:07<26:36,  7.79s/it]
+ 89%|████████▊ | 1609/1813 [3:01:13<24:45,  7.28s/it]
+ 89%|████████▉ | 1610/1813 [3:01:20<24:11,  7.15s/it]
+                                                     
+{'loss': '0.8924', 'grad_norm': '1.515', 'learning_rate': '7.334e-06', 'epoch': '0.8883', 'num_input_tokens_seen': 10840800, 'train_runtime': '1.088e+04', 'train_tokens_per_second': '996.3'}
+
+ 89%|████████▉ | 1610/1813 [3:01:20<24:11,  7.15s/it]
+ 89%|████████▉ | 1611/1813 [3:01:26<22:34,  6.70s/it]
+ 89%|████████▉ | 1612/1813 [3:01:32<22:24,  6.69s/it]
+ 89%|████████▉ | 1613/1813 [3:01:38<20:48,  6.24s/it]
+ 89%|████████▉ | 1614/1813 [3:01:44<21:05,  6.36s/it]
+ 89%|████████▉ | 1615/1813 [3:01:52<21:55,  6.64s/it]
+ 89%|████████▉ | 1616/1813 [3:01:59<22:32,  6.86s/it]
+ 89%|████████▉ | 1617/1813 [3:02:07<23:11,  7.10s/it]
+ 89%|████████▉ | 1618/1813 [3:02:12<21:43,  6.69s/it]
+ 89%|████████▉ | 1619/1813 [3:02:19<21:21,  6.61s/it]
+ 89%|████████▉ | 1620/1813 [3:02:24<20:12,  6.28s/it]
+                                                     
+{'loss': '0.9038', 'grad_norm': '1.104', 'learning_rate': '6.641e-06', 'epoch': '0.8938', 'num_input_tokens_seen': 10899640, 'train_runtime': '1.094e+04', 'train_tokens_per_second': '995.9'}
+
+ 89%|████████▉ | 1620/1813 [3:02:24<20:12,  6.28s/it]
+ 89%|████████▉ | 1621/1813 [3:02:30<19:58,  6.24s/it]
+ 89%|████████▉ | 1622/1813 [3:02:39<22:10,  6.97s/it]
+ 90%|████████▉ | 1623/1813 [3:02:47<22:56,  7.24s/it]
+ 90%|████████▉ | 1624/1813 [3:02:54<22:18,  7.08s/it]
+ 90%|████████▉ | 1625/1813 [3:03:02<23:24,  7.47s/it]
+ 90%|████████▉ | 1626/1813 [3:03:09<22:55,  7.35s/it]
+ 90%|████████▉ | 1627/1813 [3:03:16<22:09,  7.15s/it]
+ 90%|████████▉ | 1628/1813 [3:03:22<21:20,  6.92s/it]
+ 90%|████████▉ | 1629/1813 [3:03:30<22:28,  7.33s/it]
+ 90%|████████▉ | 1630/1813 [3:03:37<21:11,  6.95s/it]
+                                                     
+{'loss': '0.8923', 'grad_norm': '1.513', 'learning_rate': '5.981e-06', 'epoch': '0.8993', 'num_input_tokens_seen': 10968528, 'train_runtime': '1.102e+04', 'train_tokens_per_second': '995.6'}
+
+ 90%|████████▉ | 1630/1813 [3:03:37<21:11,  6.95s/it]
+ 90%|████████▉ | 1631/1813 [3:03:43<20:28,  6.75s/it]
+ 90%|█████████ | 1632/1813 [3:03:48<19:19,  6.41s/it]
+ 90%|█████████ | 1633/1813 [3:03:57<21:09,  7.05s/it]
+ 90%|█████████ | 1634/1813 [3:04:03<19:48,  6.64s/it]
+ 90%|█████████ | 1635/1813 [3:04:10<20:39,  6.96s/it]
+ 90%|█████████ | 1636/1813 [3:04:17<19:51,  6.73s/it]
+ 90%|█████████ | 1637/1813 [3:04:24<20:11,  6.88s/it]
+ 90%|█████████ | 1638/1813 [3:04:30<19:18,  6.62s/it]
+ 90%|█████████ | 1639/1813 [3:04:38<20:09,  6.95s/it]
+ 90%|█████████ | 1640/1813 [3:04:43<19:08,  6.64s/it]
+                                                     
+{'loss': '0.903', 'grad_norm': '0.9351', 'learning_rate': '5.354e-06', 'epoch': '0.9048', 'num_input_tokens_seen': 11030048, 'train_runtime': '1.108e+04', 'train_tokens_per_second': '995.1'}
+
+ 90%|█████████ | 1640/1813 [3:04:43<19:08,  6.64s/it]
+ 91%|█████████ | 1641/1813 [3:04:50<18:57,  6.61s/it]
+ 91%|█████████ | 1642/1813 [3:04:57<19:33,  6.86s/it]
+ 91%|█████████ | 1643/1813 [3:05:04<19:26,  6.86s/it]
+ 91%|█████████ | 1644/1813 [3:05:12<19:59,  7.10s/it]
+ 91%|█████████ | 1645/1813 [3:05:23<22:55,  8.19s/it]
+ 91%|█████████ | 1646/1813 [3:05:30<22:11,  7.97s/it]
+ 91%|█████████ | 1647/1813 [3:05:37<20:55,  7.57s/it]
+ 91%|█████████ | 1648/1813 [3:05:44<20:41,  7.53s/it]
+ 91%|█████████ | 1649/1813 [3:05:50<19:11,  7.02s/it]
+ 91%|█████████ | 1650/1813 [3:05:56<17:58,  6.62s/it]
+                                                     
+{'loss': '0.8583', 'grad_norm': '1.028', 'learning_rate': '4.761e-06', 'epoch': '0.9103', 'num_input_tokens_seen': 11098768, 'train_runtime': '1.116e+04', 'train_tokens_per_second': '994.8'}
+
+ 91%|█████████ | 1650/1813 [3:05:56<17:58,  6.62s/it]
+ 91%|█████████ | 1651/1813 [3:06:03<18:16,  6.77s/it]
+ 91%|█████████ | 1652/1813 [3:06:10<18:35,  6.93s/it]
+ 91%|█████████ | 1653/1813 [3:06:18<19:01,  7.13s/it]
+ 91%|█████████ | 1654/1813 [3:06:25<18:36,  7.02s/it]
+ 91%|█████████▏| 1655/1813 [3:06:32<18:41,  7.10s/it]
+ 91%|█████████▏| 1656/1813 [3:06:38<18:05,  6.92s/it]
+ 91%|█████████▏| 1657/1813 [3:06:47<19:26,  7.48s/it]
+ 91%|█████████▏| 1658/1813 [3:06:54<19:07,  7.41s/it]
+ 92%|█████████▏| 1659/1813 [3:07:02<19:20,  7.54s/it]
+ 92%|█████████▏| 1660/1813 [3:07:09<18:48,  7.38s/it]
+                                                     
+{'loss': '0.9241', 'grad_norm': '0.9705', 'learning_rate': '4.202e-06', 'epoch': '0.9159', 'num_input_tokens_seen': 11168848, 'train_runtime': '1.123e+04', 'train_tokens_per_second': '994.6'}
+
+ 92%|█████████▏| 1660/1813 [3:07:09<18:48,  7.38s/it]
+ 92%|█████████▏| 1661/1813 [3:07:17<19:10,  7.57s/it]
+ 92%|█████████▏| 1662/1813 [3:07:25<19:02,  7.56s/it]
+ 92%|█████████▏| 1663/1813 [3:07:33<19:08,  7.66s/it]
+ 92%|█████████▏| 1664/1813 [3:07:41<19:14,  7.75s/it]
+ 92%|█████████▏| 1665/1813 [3:07:49<19:52,  8.06s/it]
+ 92%|█████████▏| 1666/1813 [3:07:55<17:38,  7.20s/it]
+ 92%|█████████▏| 1667/1813 [3:08:02<17:47,  7.31s/it]
+ 92%|█████████▏| 1668/1813 [3:08:09<17:23,  7.20s/it]
+ 92%|█████████▏| 1669/1813 [3:08:21<20:19,  8.47s/it]
+ 92%|█████████▏| 1670/1813 [3:08:26<18:20,  7.70s/it]
+                                                     
+{'loss': '0.8751', 'grad_norm': '1.113', 'learning_rate': '3.677e-06', 'epoch': '0.9214', 'num_input_tokens_seen': 11244288, 'train_runtime': '1.131e+04', 'train_tokens_per_second': '994.5'}
+
+ 92%|█████████▏| 1670/1813 [3:08:26<18:20,  7.70s/it]
+ 92%|█████████▏| 1671/1813 [3:08:32<16:50,  7.12s/it]
+ 92%|█████████▏| 1672/1813 [3:08:39<16:18,  6.94s/it]
+ 92%|█████████▏| 1673/1813 [3:08:45<15:56,  6.83s/it]
+ 92%|█████████▏| 1674/1813 [3:08:53<16:37,  7.18s/it]
+ 92%|█████████▏| 1675/1813 [3:09:01<17:13,  7.49s/it]
+ 92%|█████████▏| 1676/1813 [3:09:10<17:43,  7.77s/it]
+ 92%|█████████▏| 1677/1813 [3:09:17<17:17,  7.63s/it]
+ 93%|█████████▎| 1678/1813 [3:09:24<16:54,  7.51s/it]
+ 93%|█████████▎| 1679/1813 [3:09:31<16:15,  7.28s/it]
+ 93%|█████████▎| 1680/1813 [3:09:41<18:01,  8.13s/it]
+                                                     
+{'loss': '0.9481', 'grad_norm': '1.048', 'learning_rate': '3.187e-06', 'epoch': '0.9269', 'num_input_tokens_seen': 11317056, 'train_runtime': '1.138e+04', 'train_tokens_per_second': '994.3'}
+
+ 93%|█████████▎| 1680/1813 [3:09:41<18:01,  8.13s/it]
+ 93%|█████████▎| 1681/1813 [3:09:47<16:08,  7.34s/it]
+ 93%|█████████▎| 1682/1813 [3:09:53<15:07,  6.93s/it]
+ 93%|█████████▎| 1683/1813 [3:09:58<13:51,  6.40s/it]
+ 93%|█████████▎| 1684/1813 [3:10:06<14:43,  6.85s/it]
+ 93%|█████████▎| 1685/1813 [3:10:13<15:05,  7.07s/it]
+ 93%|█████████▎| 1686/1813 [3:10:21<15:29,  7.32s/it]
+ 93%|█████████▎| 1687/1813 [3:10:29<15:24,  7.34s/it]
+ 93%|█████████▎| 1688/1813 [3:10:36<15:35,  7.48s/it]
+ 93%|█████████▎| 1689/1813 [3:10:42<14:08,  6.84s/it]
+ 93%|█████████▎| 1690/1813 [3:10:49<14:26,  7.04s/it]
+                                                     
+{'loss': '0.8746', 'grad_norm': '0.9118', 'learning_rate': '2.731e-06', 'epoch': '0.9324', 'num_input_tokens_seen': 11380560, 'train_runtime': '1.145e+04', 'train_tokens_per_second': '993.9'}
+
+ 93%|█████████▎| 1690/1813 [3:10:49<14:26,  7.04s/it]
+ 93%|█████████▎| 1691/1813 [3:10:56<14:21,  7.06s/it]
+ 93%|█████████▎| 1692/1813 [3:11:05<15:14,  7.56s/it]
+ 93%|█████████▎| 1693/1813 [3:11:12<14:46,  7.39s/it]
+ 93%|█████████▎| 1694/1813 [3:11:22<15:55,  8.03s/it]
+ 93%|█████████▎| 1695/1813 [3:11:29<15:04,  7.66s/it]
+ 94%|█████████▎| 1696/1813 [3:11:35<14:20,  7.35s/it]
+ 94%|█████████▎| 1697/1813 [3:11:42<14:08,  7.31s/it]
+ 94%|█████████▎| 1698/1813 [3:11:48<13:16,  6.93s/it]
+ 94%|█████████▎| 1699/1813 [3:11:56<13:19,  7.02s/it]
+ 94%|█████████▍| 1700/1813 [3:12:03<13:16,  7.05s/it]
+                                                     
+{'loss': '0.8551', 'grad_norm': '1.133', 'learning_rate': '2.31e-06', 'epoch': '0.9379', 'num_input_tokens_seen': 11450712, 'train_runtime': '1.152e+04', 'train_tokens_per_second': '993.7'}
+
+ 94%|█████████▍| 1700/1813 [3:12:03<13:16,  7.05s/it]
+ 94%|█████████▍| 1701/1813 [3:12:08<12:10,  6.52s/it]
+ 94%|█████████▍| 1702/1813 [3:12:14<11:45,  6.35s/it]
+ 94%|█████████▍| 1703/1813 [3:12:20<11:44,  6.40s/it]
+ 94%|█████████▍| 1704/1813 [3:12:29<12:53,  7.09s/it]
+ 94%|█████████▍| 1705/1813 [3:12:37<12:59,  7.22s/it]
+ 94%|█████████▍| 1706/1813 [3:12:44<12:57,  7.27s/it]
+ 94%|█████████▍| 1707/1813 [3:12:51<12:23,  7.02s/it]
+ 94%|█████████▍| 1708/1813 [3:12:56<11:28,  6.55s/it]
+ 94%|█████████▍| 1709/1813 [3:13:03<11:36,  6.69s/it]
+ 94%|█████████▍| 1710/1813 [3:13:09<11:21,  6.62s/it]
+                                                     
+{'loss': '0.9356', 'grad_norm': '1.195', 'learning_rate': '1.924e-06', 'epoch': '0.9434', 'num_input_tokens_seen': 11511984, 'train_runtime': '1.159e+04', 'train_tokens_per_second': '993.3'}
+
+ 94%|█████████▍| 1710/1813 [3:13:09<11:21,  6.62s/it]
+ 94%|█████████▍| 1711/1813 [3:13:16<10:57,  6.45s/it]
+ 94%|█████████▍| 1712/1813 [3:13:22<10:53,  6.47s/it]
+ 94%|█████████▍| 1713/1813 [3:13:29<10:53,  6.53s/it]
+ 95%|█████████▍| 1714/1813 [3:13:35<10:48,  6.55s/it]
+ 95%|█████████▍| 1715/1813 [3:13:42<10:34,  6.47s/it]
+ 95%|█████████▍| 1716/1813 [3:13:47<10:05,  6.25s/it]
+ 95%|█████████▍| 1717/1813 [3:13:53<09:53,  6.19s/it]
+ 95%|█████████▍| 1718/1813 [3:14:01<10:15,  6.48s/it]
+ 95%|█████████▍| 1719/1813 [3:14:08<10:30,  6.71s/it]
+ 95%|█████████▍| 1720/1813 [3:14:15<10:28,  6.75s/it]
+                                                     
+{'loss': '0.852', 'grad_norm': '0.8845', 'learning_rate': '1.573e-06', 'epoch': '0.949', 'num_input_tokens_seen': 11571368, 'train_runtime': '1.166e+04', 'train_tokens_per_second': '992.8'}
+
+ 95%|█████████▍| 1720/1813 [3:14:15<10:28,  6.75s/it]
+ 95%|█████████▍| 1721/1813 [3:14:23<11:07,  7.25s/it]
+ 95%|█████████▍| 1722/1813 [3:14:30<10:50,  7.15s/it]
+ 95%|█████████▌| 1723/1813 [3:14:37<10:37,  7.08s/it]
+ 95%|█████████▌| 1724/1813 [3:14:44<10:25,  7.03s/it]
+ 95%|█████████▌| 1725/1813 [3:14:52<10:51,  7.40s/it]
+ 95%|█████████▌| 1726/1813 [3:15:01<11:13,  7.75s/it]
+ 95%|█████████▌| 1727/1813 [3:15:07<10:43,  7.48s/it]
+ 95%|█████████▌| 1728/1813 [3:15:13<09:46,  6.90s/it]
+ 95%|█████████▌| 1729/1813 [3:15:19<09:18,  6.64s/it]
+ 95%|█████████▌| 1730/1813 [3:15:27<09:35,  6.93s/it]
+                                                     
+{'loss': '0.9187', 'grad_norm': '0.9346', 'learning_rate': '1.256e-06', 'epoch': '0.9545', 'num_input_tokens_seen': 11640392, 'train_runtime': '1.173e+04', 'train_tokens_per_second': '992.6'}
+
+ 95%|█████████▌| 1730/1813 [3:15:27<09:35,  6.93s/it]
+ 95%|█████████▌| 1731/1813 [3:15:34<09:32,  6.98s/it]
+ 96%|█████████▌| 1732/1813 [3:15:42<09:44,  7.21s/it]
+ 96%|█████████▌| 1733/1813 [3:15:47<09:03,  6.79s/it]
+ 96%|█████████▌| 1734/1813 [3:15:54<08:53,  6.75s/it]
+ 96%|█████████▌| 1735/1813 [3:16:01<09:03,  6.97s/it]
+ 96%|█████████▌| 1736/1813 [3:16:09<09:16,  7.22s/it]
+ 96%|█████████▌| 1737/1813 [3:16:16<09:04,  7.17s/it]
+ 96%|█████████▌| 1738/1813 [3:16:22<08:23,  6.71s/it]
+ 96%|█████████▌| 1739/1813 [3:16:31<09:11,  7.45s/it]
+ 96%|█████████▌| 1740/1813 [3:16:38<09:02,  7.43s/it]
+                                                     
+{'loss': '0.9058', 'grad_norm': '1.098', 'learning_rate': '9.755e-07', 'epoch': '0.96', 'num_input_tokens_seen': 11709120, 'train_runtime': '1.18e+04', 'train_tokens_per_second': '992.4'}
+
+ 96%|█████████▌| 1740/1813 [3:16:38<09:02,  7.43s/it]
+ 96%|█████████▌| 1741/1813 [3:16:45<08:35,  7.16s/it]
+ 96%|█████████▌| 1742/1813 [3:16:53<08:44,  7.39s/it]
+ 96%|█████████▌| 1743/1813 [3:17:01<08:51,  7.60s/it]
+ 96%|█████████▌| 1744/1813 [3:17:09<08:53,  7.74s/it]
+ 96%|█████████▌| 1745/1813 [3:17:16<08:26,  7.44s/it]
+ 96%|█████████▋| 1746/1813 [3:17:24<08:37,  7.73s/it]
+ 96%|█████████▋| 1747/1813 [3:17:30<07:47,  7.09s/it]
+ 96%|█████████▋| 1748/1813 [3:17:38<08:07,  7.50s/it]
+ 96%|█████████▋| 1749/1813 [3:17:44<07:33,  7.09s/it]
+ 97%|█████████▋| 1750/1813 [3:17:51<07:07,  6.79s/it]
+                                                     
+{'loss': '0.8536', 'grad_norm': '0.9043', 'learning_rate': '7.3e-07', 'epoch': '0.9655', 'num_input_tokens_seen': 11777768, 'train_runtime': '1.187e+04', 'train_tokens_per_second': '992.1'}
+
+ 97%|█████████▋| 1750/1813 [3:17:51<07:07,  6.79s/it]
+ 97%|█████████▋| 1751/1813 [3:17:57<07:02,  6.82s/it]
+ 97%|█████████▋| 1752/1813 [3:18:04<06:54,  6.79s/it]
+ 97%|█████████▋| 1753/1813 [3:18:13<07:21,  7.36s/it]
+ 97%|█████████▋| 1754/1813 [3:18:21<07:30,  7.64s/it]
+ 97%|█████████▋| 1755/1813 [3:18:29<07:19,  7.58s/it]
+ 97%|█████████▋| 1756/1813 [3:18:35<06:47,  7.15s/it]
+ 97%|█████████▋| 1757/1813 [3:18:41<06:27,  6.92s/it]
+ 97%|█████████▋| 1758/1813 [3:18:47<06:11,  6.75s/it]
+ 97%|█████████▋| 1759/1813 [3:18:54<06:04,  6.75s/it]
+ 97%|█████████▋| 1760/1813 [3:19:00<05:44,  6.51s/it]
+                                                     
+{'loss': '0.8667', 'grad_norm': '1.107', 'learning_rate': '5.199e-07', 'epoch': '0.971', 'num_input_tokens_seen': 11843192, 'train_runtime': '1.194e+04', 'train_tokens_per_second': '991.8'}
+
+ 97%|█████████▋| 1760/1813 [3:19:00<05:44,  6.51s/it]
+ 97%|█████████▋| 1761/1813 [3:19:06<05:32,  6.39s/it]
+ 97%|█████████▋| 1762/1813 [3:19:11<05:06,  6.01s/it]
+ 97%|█████████▋| 1763/1813 [3:19:20<05:42,  6.84s/it]
+ 97%|█████████▋| 1764/1813 [3:19:26<05:20,  6.54s/it]
+ 97%|█████████▋| 1765/1813 [3:19:32<05:08,  6.43s/it]
+ 97%|█████████▋| 1766/1813 [3:19:38<04:59,  6.37s/it]
+ 97%|█████████▋| 1767/1813 [3:19:45<04:52,  6.35s/it]
+ 98%|█████████▊| 1768/1813 [3:19:51<04:50,  6.45s/it]
+ 98%|█████████▊| 1769/1813 [3:19:58<04:44,  6.46s/it]
+ 98%|█████████▊| 1770/1813 [3:20:06<04:54,  6.84s/it]
+                                                     
+{'loss': '0.8791', 'grad_norm': '0.9812', 'learning_rate': '3.453e-07', 'epoch': '0.9766', 'num_input_tokens_seen': 11903728, 'train_runtime': '1.201e+04', 'train_tokens_per_second': '991.5'}
+
+ 98%|█████████▊| 1770/1813 [3:20:06<04:54,  6.84s/it]
+ 98%|█████████▊| 1771/1813 [3:20:13<04:51,  6.94s/it]
+ 98%|█████████▊| 1772/1813 [3:20:20<04:48,  7.04s/it]
+ 98%|█████████▊| 1773/1813 [3:20:28<04:52,  7.31s/it]
+ 98%|█████████▊| 1774/1813 [3:20:33<04:19,  6.66s/it]
+ 98%|█████████▊| 1775/1813 [3:20:42<04:35,  7.26s/it]
+ 98%|█████████▊| 1776/1813 [3:20:50<04:41,  7.61s/it]
+ 98%|█████████▊| 1777/1813 [3:20:59<04:45,  7.92s/it]
+ 98%|█████████▊| 1778/1813 [3:21:05<04:20,  7.46s/it]
+ 98%|█████████▊| 1779/1813 [3:21:12<04:01,  7.11s/it]
+ 98%|█████████▊| 1780/1813 [3:21:18<03:51,  7.02s/it]
+                                                     
+{'loss': '0.8568', 'grad_norm': '0.9052', 'learning_rate': '2.062e-07', 'epoch': '0.9821', 'num_input_tokens_seen': 11972864, 'train_runtime': '1.208e+04', 'train_tokens_per_second': '991.2'}
+
+ 98%|█████████▊| 1780/1813 [3:21:18<03:51,  7.02s/it]
+ 98%|█████████▊| 1781/1813 [3:21:25<03:45,  7.05s/it]
+ 98%|█████████▊| 1782/1813 [3:21:32<03:35,  6.94s/it]
+ 98%|█████████▊| 1783/1813 [3:21:39<03:26,  6.89s/it]
+ 98%|█████████▊| 1784/1813 [3:21:47<03:25,  7.10s/it]
+ 98%|█████████▊| 1785/1813 [3:21:53<03:17,  7.04s/it]
+ 99%|█████████▊| 1786/1813 [3:22:00<03:10,  7.05s/it]
+ 99%|█████████▊| 1787/1813 [3:22:07<02:57,  6.81s/it]
+ 99%|█████████▊| 1788/1813 [3:22:14<02:55,  7.04s/it]
+ 99%|█████████▊| 1789/1813 [3:22:24<03:08,  7.86s/it]
+ 99%|█████████▊| 1790/1813 [3:22:32<03:03,  7.98s/it]
+                                                     
+{'loss': '0.8838', 'grad_norm': '0.8536', 'learning_rate': '1.028e-07', 'epoch': '0.9876', 'num_input_tokens_seen': 12043496, 'train_runtime': '1.215e+04', 'train_tokens_per_second': '991'}
+
+ 99%|█████████▊| 1790/1813 [3:22:32<03:03,  7.98s/it]
+ 99%|█████████▉| 1791/1813 [3:22:38<02:42,  7.39s/it]
+ 99%|█████████▉| 1792/1813 [3:22:45<02:33,  7.29s/it]
+ 99%|█████████▉| 1793/1813 [3:22:53<02:28,  7.44s/it]
+ 99%|█████████▉| 1794/1813 [3:23:01<02:25,  7.65s/it]
+ 99%|█████████▉| 1795/1813 [3:23:09<02:18,  7.67s/it]
+ 99%|█████████▉| 1796/1813 [3:23:17<02:10,  7.69s/it]
+ 99%|█████████▉| 1797/1813 [3:23:24<02:00,  7.53s/it]
+ 99%|█████████▉| 1798/1813 [3:23:35<02:06,  8.45s/it]
+ 99%|█████████▉| 1799/1813 [3:23:42<01:53,  8.12s/it]
+ 99%|█████████▉| 1800/1813 [3:23:47<01:33,  7.22s/it]
+                                                     
+{'loss': '0.859', 'grad_norm': '1.005', 'learning_rate': '3.497e-08', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448, 'train_runtime': '1.223e+04', 'train_tokens_per_second': '990.8'}
+
+ 99%|█████████▉| 1800/1813 [3:23:47<01:33,  7.22s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:11, 13.87it/s]
+
+  0%|          | 5/1000 [00:00<02:37,  6.33it/s]
+
+  1%|          | 7/1000 [00:00<02:15,  7.32it/s]
+
+  1%|          | 9/1000 [00:01<02:55,  5.64it/s]
+
+  1%|          | 11/1000 [00:01<02:19,  7.09it/s]
+
+  1%|          | 12/1000 [00:01<02:17,  7.16it/s]
+
+  1%|▏         | 13/1000 [00:01<02:48,  5.85it/s]
+
+  1%|▏         | 14/1000 [00:02<02:51,  5.74it/s]
+
+  2%|▏         | 15/1000 [00:02<03:17,  4.98it/s]
+
+  2%|▏         | 16/1000 [00:02<03:24,  4.82it/s]
+
+  2%|▏         | 18/1000 [00:03<03:09,  5.19it/s]
+
+  2%|▏         | 19/1000 [00:03<03:43,  4.39it/s]
+
+  2%|▏         | 20/1000 [00:03<04:10,  3.92it/s]
+
+  2%|▏         | 22/1000 [00:03<03:00,  5.41it/s]
+
+  2%|▏         | 23/1000 [00:04<03:21,  4.84it/s]
+
+  2%|▎         | 25/1000 [00:04<02:31,  6.42it/s]
+
+  3%|▎         | 26/1000 [00:04<02:58,  5.47it/s]
+
+  3%|▎         | 28/1000 [00:04<03:06,  5.22it/s]
+
+  3%|▎         | 29/1000 [00:05<02:55,  5.55it/s]
+
+  3%|▎         | 30/1000 [00:05<03:06,  5.20it/s]
+
+  3%|▎         | 31/1000 [00:05<03:27,  4.66it/s]
+
+  3%|▎         | 32/1000 [00:05<03:04,  5.24it/s]
+
+  3%|▎         | 33/1000 [00:05<03:12,  5.03it/s]
+
+  4%|▎         | 35/1000 [00:06<02:24,  6.69it/s]
+
+  4%|▎         | 37/1000 [00:06<02:05,  7.69it/s]
+
+  4%|▍         | 39/1000 [00:06<02:28,  6.48it/s]
+
+  4%|▍         | 40/1000 [00:06<02:24,  6.64it/s]
+
+  4%|▍         | 42/1000 [00:07<02:04,  7.68it/s]
+
+  4%|▍         | 44/1000 [00:07<01:48,  8.77it/s]
+
+  4%|▍         | 45/1000 [00:07<02:11,  7.28it/s]
+
+  5%|▍         | 47/1000 [00:07<02:30,  6.34it/s]
+
+  5%|▍         | 49/1000 [00:08<02:06,  7.49it/s]
+
+  5%|▌         | 51/1000 [00:08<01:51,  8.51it/s]
+
+  5%|▌         | 53/1000 [00:08<01:43,  9.18it/s]
+
+  6%|▌         | 55/1000 [00:08<01:43,  9.15it/s]
+
+  6%|▌         | 56/1000 [00:08<02:05,  7.54it/s]
+
+  6%|▌         | 58/1000 [00:09<01:56,  8.09it/s]
+
+  6%|▌         | 59/1000 [00:09<02:14,  6.99it/s]
+
+  6%|▌         | 61/1000 [00:09<01:54,  8.23it/s]
+
+  6%|▋         | 63/1000 [00:09<01:48,  8.60it/s]
+
+  6%|▋         | 64/1000 [00:09<02:18,  6.78it/s]
+
+  7%|▋         | 66/1000 [00:10<01:57,  7.94it/s]
+
+  7%|▋         | 67/1000 [00:10<02:27,  6.32it/s]
+
+  7%|▋         | 69/1000 [00:10<02:13,  6.97it/s]
+
+  7%|▋         | 70/1000 [00:10<02:41,  5.77it/s]
+
+  7%|▋         | 72/1000 [00:11<02:44,  5.64it/s]
+
+  7%|▋         | 74/1000 [00:11<02:15,  6.82it/s]
+
+  8%|▊         | 75/1000 [00:11<02:39,  5.79it/s]
+
+  8%|▊         | 76/1000 [00:11<02:43,  5.66it/s]
+
+  8%|▊         | 77/1000 [00:12<02:28,  6.21it/s]
+
+  8%|▊         | 78/1000 [00:12<02:59,  5.13it/s]
+
+  8%|▊         | 80/1000 [00:12<02:14,  6.84it/s]
+
+  8%|▊         | 82/1000 [00:12<01:52,  8.15it/s]
+
+  8%|▊         | 83/1000 [00:12<02:26,  6.25it/s]
+
+  8%|▊         | 85/1000 [00:13<02:33,  5.95it/s]
+
+  9%|▊         | 86/1000 [00:13<02:55,  5.21it/s]
+
+  9%|▊         | 87/1000 [00:13<03:18,  4.59it/s]
+
+  9%|▉         | 89/1000 [00:14<02:31,  6.00it/s]
+
+  9%|▉         | 90/1000 [00:14<02:25,  6.24it/s]
+
+  9%|▉         | 91/1000 [00:14<02:52,  5.25it/s]
+
+  9%|▉         | 92/1000 [00:14<02:52,  5.27it/s]
+
+  9%|▉         | 93/1000 [00:14<03:03,  4.94it/s]
+
+  9%|▉         | 94/1000 [00:15<02:40,  5.65it/s]
+
+ 10%|▉         | 95/1000 [00:15<03:24,  4.42it/s]
+
+ 10%|▉         | 97/1000 [00:15<02:35,  5.80it/s]
+
+ 10%|▉         | 99/1000 [00:15<02:02,  7.33it/s]
+
+ 10%|█         | 101/1000 [00:15<01:45,  8.54it/s]
+
+ 10%|█         | 102/1000 [00:16<02:14,  6.68it/s]
+
+ 10%|█         | 103/1000 [00:16<02:10,  6.86it/s]
+
+ 10%|█         | 104/1000 [00:16<02:09,  6.94it/s]
+
+ 11%|█         | 106/1000 [00:16<01:43,  8.62it/s]
+
+ 11%|█         | 108/1000 [00:16<01:32,  9.63it/s]
+
+ 11%|█         | 110/1000 [00:16<01:26, 10.26it/s]
+
+ 11%|█         | 112/1000 [00:17<01:21, 10.93it/s]
+
+ 11%|█▏        | 114/1000 [00:17<01:18, 11.27it/s]
+
+ 12%|█▏        | 116/1000 [00:17<01:17, 11.47it/s]
+
+ 12%|█▏        | 118/1000 [00:17<01:14, 11.78it/s]
+
+ 12%|█▏        | 120/1000 [00:18<02:25,  6.04it/s]
+
+ 12%|█▏        | 122/1000 [00:18<02:04,  7.08it/s]
+
+ 12%|█▏        | 124/1000 [00:18<02:14,  6.53it/s]
+
+ 12%|█▎        | 125/1000 [00:19<02:55,  5.00it/s]
+
+ 13%|█▎        | 126/1000 [00:19<02:42,  5.37it/s]
+
+ 13%|█▎        | 127/1000 [00:19<02:52,  5.05it/s]
+
+ 13%|█▎        | 128/1000 [00:19<03:09,  4.61it/s]
+
+ 13%|█▎        | 130/1000 [00:20<02:20,  6.21it/s]
+
+ 13%|█▎        | 132/1000 [00:20<01:53,  7.63it/s]
+
+ 13%|█▎        | 134/1000 [00:20<01:37,  8.88it/s]
+
+ 14%|█▎        | 136/1000 [00:20<01:42,  8.46it/s]
+
+ 14%|█▎        | 137/1000 [00:20<01:43,  8.30it/s]
+
+ 14%|█▍        | 139/1000 [00:21<01:40,  8.59it/s]
+
+ 14%|█▍        | 141/1000 [00:21<01:39,  8.66it/s]
+
+ 14%|█▍        | 142/1000 [00:21<01:48,  7.93it/s]
+
+ 14%|█▍        | 143/1000 [00:21<01:50,  7.74it/s]
+
+ 14%|█▍        | 145/1000 [00:21<01:35,  8.92it/s]
+
+ 15%|█▍        | 147/1000 [00:22<01:56,  7.34it/s]
+
+ 15%|█▍        | 148/1000 [00:22<01:57,  7.28it/s]
+
+ 15%|█▌        | 150/1000 [00:22<01:40,  8.49it/s]
+
+ 15%|█▌        | 152/1000 [00:22<01:29,  9.49it/s]
+
+ 15%|█▌        | 154/1000 [00:22<01:34,  8.94it/s]
+
+ 16%|█▌        | 155/1000 [00:23<02:04,  6.77it/s]
+
+ 16%|█▌        | 156/1000 [00:23<02:39,  5.29it/s]
+
+ 16%|█▌        | 158/1000 [00:23<02:04,  6.77it/s]
+
+ 16%|█▌        | 160/1000 [00:23<01:52,  7.46it/s]
+
+ 16%|█▌        | 162/1000 [00:24<02:07,  6.58it/s]
+
+ 16%|█▋        | 163/1000 [00:24<02:11,  6.35it/s]
+
+ 16%|█▋        | 164/1000 [00:24<02:33,  5.44it/s]
+
+ 16%|█▋        | 165/1000 [00:24<02:56,  4.73it/s]
+
+ 17%|█▋        | 166/1000 [00:25<03:11,  4.36it/s]
+
+ 17%|█▋        | 167/1000 [00:25<02:49,  4.92it/s]
+
+ 17%|█▋        | 169/1000 [00:25<02:02,  6.77it/s]
+
+ 17%|█▋        | 171/1000 [00:25<02:13,  6.22it/s]
+
+ 17%|█▋        | 172/1000 [00:26<02:34,  5.35it/s]
+
+ 17%|█▋        | 173/1000 [00:26<02:36,  5.28it/s]
+
+ 17%|█▋        | 174/1000 [00:26<02:23,  5.75it/s]
+
+ 18%|█▊        | 176/1000 [00:26<01:53,  7.26it/s]
+
+ 18%|█▊        | 177/1000 [00:26<02:20,  5.87it/s]
+
+ 18%|█▊        | 179/1000 [00:27<01:50,  7.45it/s]
+
+ 18%|█▊        | 180/1000 [00:27<01:50,  7.40it/s]
+
+ 18%|█▊        | 181/1000 [00:27<02:19,  5.89it/s]
+
+ 18%|█▊        | 182/1000 [00:27<02:44,  4.98it/s]
+
+ 18%|█▊        | 183/1000 [00:27<02:41,  5.05it/s]
+
+ 18%|█▊        | 185/1000 [00:28<02:00,  6.76it/s]
+
+ 19%|█▊        | 186/1000 [00:28<01:57,  6.90it/s]
+
+ 19%|█▊        | 187/1000 [00:28<02:25,  5.59it/s]
+
+ 19%|█▉        | 189/1000 [00:28<01:54,  7.11it/s]
+
+ 19%|█▉        | 190/1000 [00:29<02:19,  5.82it/s]
+
+ 19%|█▉        | 192/1000 [00:29<02:21,  5.70it/s]
+
+ 19%|█▉        | 194/1000 [00:29<02:23,  5.62it/s]
+
+ 20%|█▉        | 196/1000 [00:29<01:57,  6.82it/s]
+
+ 20%|█▉        | 198/1000 [00:30<01:39,  8.09it/s]
+
+ 20%|██        | 200/1000 [00:30<01:24,  9.46it/s]
+
+ 20%|██        | 202/1000 [00:30<01:43,  7.73it/s]
+
+ 20%|██        | 204/1000 [00:30<01:44,  7.59it/s]
+
+ 21%|██        | 206/1000 [00:30<01:30,  8.79it/s]
+
+ 21%|██        | 208/1000 [00:31<01:45,  7.51it/s]
+
+ 21%|██        | 210/1000 [00:31<01:45,  7.46it/s]
+
+ 21%|██        | 212/1000 [00:31<01:32,  8.51it/s]
+
+ 21%|██▏       | 213/1000 [00:31<01:42,  7.68it/s]
+
+ 21%|██▏       | 214/1000 [00:32<01:42,  7.68it/s]
+
+ 22%|██▏       | 216/1000 [00:32<01:36,  8.17it/s]
+
+ 22%|██▏       | 217/1000 [00:32<02:01,  6.45it/s]
+
+ 22%|██▏       | 219/1000 [00:32<01:44,  7.50it/s]
+
+ 22%|██▏       | 220/1000 [00:33<02:27,  5.30it/s]
+
+ 22%|██▏       | 221/1000 [00:33<02:25,  5.36it/s]
+
+ 22%|██▏       | 223/1000 [00:33<02:43,  4.74it/s]
+
+ 22%|██▏       | 224/1000 [00:33<02:29,  5.20it/s]
+
+ 22%|██▎       | 225/1000 [00:34<02:27,  5.26it/s]
+
+ 23%|██▎       | 226/1000 [00:34<02:49,  4.57it/s]
+
+ 23%|██▎       | 228/1000 [00:34<02:12,  5.83it/s]
+
+ 23%|██▎       | 229/1000 [00:34<02:31,  5.09it/s]
+
+ 23%|██▎       | 230/1000 [00:35<02:20,  5.48it/s]
+
+ 23%|██▎       | 231/1000 [00:35<02:06,  6.06it/s]
+
+ 23%|██▎       | 233/1000 [00:35<01:48,  7.09it/s]
+
+ 24%|██▎       | 235/1000 [00:35<01:29,  8.59it/s]
+
+ 24%|██▎       | 236/1000 [00:35<01:30,  8.42it/s]
+
+ 24%|██▎       | 237/1000 [00:35<01:33,  8.12it/s]
+
+ 24%|██▍       | 239/1000 [00:36<01:39,  7.67it/s]
+
+ 24%|██▍       | 240/1000 [00:36<01:35,  7.97it/s]
+
+ 24%|██▍       | 241/1000 [00:36<01:53,  6.67it/s]
+
+ 24%|██▍       | 242/1000 [00:36<02:46,  4.54it/s]
+
+ 24%|██▍       | 243/1000 [00:37<02:58,  4.23it/s]
+
+ 24%|██▍       | 244/1000 [00:37<02:32,  4.97it/s]
+
+ 24%|██▍       | 245/1000 [00:37<02:17,  5.51it/s]
+
+ 25%|██▍       | 247/1000 [00:37<01:41,  7.42it/s]
+
+ 25%|██▍       | 248/1000 [00:37<02:07,  5.88it/s]
+
+ 25%|██▍       | 249/1000 [00:38<02:28,  5.04it/s]
+
+ 25%|██▌       | 250/1000 [00:38<02:31,  4.96it/s]
+
+ 25%|██▌       | 252/1000 [00:38<02:26,  5.11it/s]
+
+ 25%|██▌       | 253/1000 [00:38<02:32,  4.89it/s]
+
+ 26%|██▌       | 255/1000 [00:39<01:57,  6.36it/s]
+
+ 26%|██▌       | 256/1000 [00:39<01:51,  6.67it/s]
+
+ 26%|██▌       | 258/1000 [00:39<01:37,  7.58it/s]
+
+ 26%|██▌       | 260/1000 [00:39<01:50,  6.68it/s]
+
+ 26%|██▌       | 261/1000 [00:40<02:13,  5.53it/s]
+
+ 26%|██▌       | 262/1000 [00:40<02:14,  5.48it/s]
+
+ 26%|██▋       | 264/1000 [00:40<02:15,  5.44it/s]
+
+ 26%|██▋       | 265/1000 [00:40<02:09,  5.68it/s]
+
+ 27%|██▋       | 266/1000 [00:40<02:02,  5.98it/s]
+
+ 27%|██▋       | 268/1000 [00:41<01:56,  6.29it/s]
+
+ 27%|██▋       | 270/1000 [00:41<02:01,  6.00it/s]
+
+ 27%|██▋       | 271/1000 [00:42<02:36,  4.65it/s]
+
+ 27%|██▋       | 273/1000 [00:42<02:31,  4.80it/s]
+
+ 28%|██▊       | 275/1000 [00:42<02:07,  5.68it/s]
+
+ 28%|██▊       | 277/1000 [00:42<01:45,  6.88it/s]
+
+ 28%|██▊       | 278/1000 [00:43<02:06,  5.71it/s]
+
+ 28%|██▊       | 280/1000 [00:43<02:07,  5.65it/s]
+
+ 28%|██▊       | 282/1000 [00:43<01:44,  6.87it/s]
+
+ 28%|██▊       | 283/1000 [00:44<02:25,  4.94it/s]
+
+ 28%|██▊       | 284/1000 [00:44<02:28,  4.83it/s]
+
+ 29%|██▊       | 286/1000 [00:44<01:52,  6.33it/s]
+
+ 29%|██▉       | 288/1000 [00:44<01:35,  7.47it/s]
+
+ 29%|██▉       | 290/1000 [00:45<01:48,  6.57it/s]
+
+ 29%|██▉       | 291/1000 [00:45<01:59,  5.93it/s]
+
+ 29%|██▉       | 292/1000 [00:45<02:16,  5.18it/s]
+
+ 29%|██▉       | 293/1000 [00:45<02:01,  5.82it/s]
+
+ 29%|██▉       | 294/1000 [00:45<01:54,  6.17it/s]
+
+ 30%|██▉       | 296/1000 [00:46<01:59,  5.91it/s]
+
+ 30%|██▉       | 298/1000 [00:46<01:37,  7.21it/s]
+
+ 30%|██▉       | 299/1000 [00:46<02:06,  5.55it/s]
+
+ 30%|███       | 300/1000 [00:46<02:01,  5.78it/s]
+
+ 30%|███       | 301/1000 [00:46<01:55,  6.05it/s]
+
+ 30%|███       | 303/1000 [00:47<01:32,  7.57it/s]
+
+ 30%|███       | 304/1000 [00:47<01:39,  6.99it/s]
+
+ 30%|███       | 305/1000 [00:47<01:40,  6.91it/s]
+
+ 31%|███       | 306/1000 [00:47<01:58,  5.86it/s]
+
+ 31%|███       | 307/1000 [00:47<02:00,  5.75it/s]
+
+ 31%|███       | 309/1000 [00:48<01:31,  7.56it/s]
+
+ 31%|███       | 311/1000 [00:48<01:17,  8.84it/s]
+
+ 31%|███       | 312/1000 [00:48<01:16,  9.02it/s]
+
+ 31%|███▏      | 314/1000 [00:48<01:11,  9.58it/s]
+
+ 32%|███▏      | 315/1000 [00:48<01:18,  8.70it/s]
+
+ 32%|███▏      | 316/1000 [00:48<01:29,  7.64it/s]
+
+ 32%|███▏      | 317/1000 [00:48<01:25,  7.97it/s]
+
+ 32%|███▏      | 319/1000 [00:49<01:58,  5.73it/s]
+
+ 32%|███▏      | 321/1000 [00:49<02:01,  5.59it/s]
+
+ 32%|███▏      | 322/1000 [00:50<02:08,  5.29it/s]
+
+ 32%|███▏      | 324/1000 [00:50<01:42,  6.60it/s]
+
+ 32%|███▎      | 325/1000 [00:50<02:23,  4.69it/s]
+
+ 33%|███▎      | 327/1000 [00:50<01:47,  6.26it/s]
+
+ 33%|███▎      | 328/1000 [00:50<01:49,  6.11it/s]
+
+ 33%|███▎      | 330/1000 [00:51<01:54,  5.85it/s]
+
+ 33%|███▎      | 332/1000 [00:51<02:00,  5.57it/s]
+
+ 33%|███▎      | 333/1000 [00:51<01:52,  5.92it/s]
+
+ 33%|███▎      | 334/1000 [00:52<02:01,  5.47it/s]
+
+ 34%|███▎      | 336/1000 [00:52<01:35,  6.94it/s]
+
+ 34%|███▍      | 338/1000 [00:52<01:26,  7.66it/s]
+
+ 34%|███▍      | 340/1000 [00:52<01:15,  8.75it/s]
+
+ 34%|███▍      | 342/1000 [00:52<01:09,  9.50it/s]
+
+ 34%|███▍      | 344/1000 [00:53<01:41,  6.49it/s]
+
+ 34%|███▍      | 345/1000 [00:53<01:38,  6.62it/s]
+
+ 35%|███▍      | 346/1000 [00:53<01:45,  6.23it/s]
+
+ 35%|███▍      | 347/1000 [00:53<01:41,  6.44it/s]
+
+ 35%|███▍      | 349/1000 [00:54<01:48,  6.01it/s]
+
+ 35%|███▌      | 350/1000 [00:54<02:08,  5.06it/s]
+
+ 35%|███▌      | 352/1000 [00:54<01:39,  6.52it/s]
+
+ 35%|███▌      | 354/1000 [00:54<01:38,  6.53it/s]
+
+ 36%|███▌      | 355/1000 [00:55<01:41,  6.38it/s]
+
+ 36%|███▌      | 357/1000 [00:55<01:22,  7.77it/s]
+
+ 36%|███▌      | 359/1000 [00:55<01:09,  9.28it/s]
+
+ 36%|███▌      | 361/1000 [00:55<01:19,  8.03it/s]
+
+ 36%|███▋      | 363/1000 [00:56<01:42,  6.20it/s]
+
+ 36%|███▋      | 364/1000 [00:56<01:56,  5.45it/s]
+
+ 36%|███▋      | 365/1000 [00:56<02:32,  4.17it/s]
+
+ 37%|███▋      | 367/1000 [00:57<02:17,  4.60it/s]
+
+ 37%|███▋      | 368/1000 [00:57<02:25,  4.33it/s]
+
+ 37%|███▋      | 369/1000 [00:57<02:10,  4.83it/s]
+
+ 37%|███▋      | 371/1000 [00:57<01:40,  6.23it/s]
+
+ 37%|███▋      | 372/1000 [00:58<01:37,  6.45it/s]
+
+ 37%|███▋      | 374/1000 [00:58<01:33,  6.70it/s]
+
+ 38%|███▊      | 375/1000 [00:58<01:59,  5.24it/s]
+
+ 38%|███▊      | 376/1000 [00:58<02:17,  4.55it/s]
+
+ 38%|███▊      | 377/1000 [00:59<02:01,  5.13it/s]
+
+ 38%|███▊      | 378/1000 [00:59<01:45,  5.87it/s]
+
+ 38%|███▊      | 380/1000 [00:59<01:18,  7.93it/s]
+
+ 38%|███▊      | 381/1000 [00:59<01:17,  7.96it/s]
+
+ 38%|███▊      | 382/1000 [00:59<01:32,  6.69it/s]
+
+ 38%|███▊      | 384/1000 [00:59<01:16,  8.02it/s]
+
+ 38%|███▊      | 385/1000 [00:59<01:18,  7.88it/s]
+
+ 39%|███▊      | 386/1000 [01:00<01:15,  8.13it/s]
+
+ 39%|███▊      | 387/1000 [01:00<01:45,  5.78it/s]
+
+ 39%|███▉      | 388/1000 [01:00<01:47,  5.71it/s]
+
+ 39%|███▉      | 389/1000 [01:00<01:35,  6.37it/s]
+
+ 39%|███▉      | 390/1000 [01:00<01:32,  6.60it/s]
+
+ 39%|███▉      | 391/1000 [01:01<01:41,  6.02it/s]
+
+ 39%|███▉      | 393/1000 [01:01<01:17,  7.78it/s]
+
+ 40%|███▉      | 395/1000 [01:01<01:35,  6.35it/s]
+
+ 40%|███▉      | 397/1000 [01:01<01:18,  7.64it/s]
+
+ 40%|███▉      | 398/1000 [01:02<01:38,  6.09it/s]
+
+ 40%|███▉      | 399/1000 [01:02<02:00,  5.00it/s]
+
+ 40%|████      | 400/1000 [01:02<02:11,  4.56it/s]
+
+ 40%|████      | 402/1000 [01:03<02:03,  4.84it/s]
+
+ 40%|████      | 404/1000 [01:03<01:36,  6.18it/s]
+
+ 41%|████      | 406/1000 [01:03<01:20,  7.38it/s]
+
+ 41%|████      | 407/1000 [01:03<01:36,  6.12it/s]
+
+ 41%|████      | 408/1000 [01:03<01:46,  5.55it/s]
+
+ 41%|████      | 410/1000 [01:04<01:46,  5.56it/s]
+
+ 41%|████      | 411/1000 [01:04<01:58,  4.98it/s]
+
+ 41%|████      | 412/1000 [01:04<02:09,  4.55it/s]
+
+ 41%|████▏     | 414/1000 [01:04<01:34,  6.20it/s]
+
+ 42%|████▏     | 416/1000 [01:05<01:37,  5.97it/s]
+
+ 42%|████▏     | 418/1000 [01:05<01:19,  7.29it/s]
+
+ 42%|████▏     | 420/1000 [01:05<01:19,  7.27it/s]
+
+ 42%|████▏     | 421/1000 [01:05<01:29,  6.46it/s]
+
+ 42%|████▏     | 423/1000 [01:06<01:41,  5.66it/s]
+
+ 42%|████▏     | 424/1000 [01:06<01:53,  5.06it/s]
+
+ 42%|████▎     | 425/1000 [01:06<01:45,  5.47it/s]
+
+ 43%|████▎     | 427/1000 [01:06<01:23,  6.85it/s]
+
+ 43%|████▎     | 428/1000 [01:07<01:29,  6.36it/s]
+
+ 43%|████▎     | 429/1000 [01:07<01:26,  6.63it/s]
+
+ 43%|████▎     | 431/1000 [01:07<01:09,  8.16it/s]
+
+ 43%|████▎     | 433/1000 [01:07<01:01,  9.23it/s]
+
+ 44%|████▎     | 435/1000 [01:07<00:56, 10.01it/s]
+
+ 44%|████▎     | 437/1000 [01:08<01:02,  9.03it/s]
+
+ 44%|████▍     | 439/1000 [01:08<01:20,  6.96it/s]
+
+ 44%|████▍     | 441/1000 [01:08<01:09,  8.05it/s]
+
+ 44%|████▍     | 442/1000 [01:08<01:25,  6.56it/s]
+
+ 44%|████▍     | 444/1000 [01:09<01:27,  6.36it/s]
+
+ 45%|████▍     | 446/1000 [01:09<01:12,  7.60it/s]
+
+ 45%|████▍     | 448/1000 [01:09<01:24,  6.51it/s]
+
+ 45%|████▌     | 450/1000 [01:10<01:29,  6.12it/s]
+
+ 45%|████▌     | 451/1000 [01:10<01:43,  5.30it/s]
+
+ 45%|████▌     | 452/1000 [01:10<01:54,  4.80it/s]
+
+ 45%|████▌     | 454/1000 [01:10<01:30,  6.06it/s]
+
+ 46%|████▌     | 456/1000 [01:11<01:16,  7.14it/s]
+
+ 46%|████▌     | 457/1000 [01:11<01:24,  6.40it/s]
+
+ 46%|████▌     | 459/1000 [01:11<01:10,  7.64it/s]
+
+ 46%|████▌     | 461/1000 [01:11<00:59,  9.02it/s]
+
+ 46%|████▋     | 463/1000 [01:12<01:24,  6.35it/s]
+
+ 46%|████▋     | 465/1000 [01:12<01:33,  5.71it/s]
+
+ 47%|████▋     | 466/1000 [01:12<01:46,  5.03it/s]
+
+ 47%|████▋     | 467/1000 [01:13<02:10,  4.08it/s]
+
+ 47%|████▋     | 469/1000 [01:13<01:38,  5.37it/s]
+
+ 47%|████▋     | 471/1000 [01:13<01:18,  6.73it/s]
+
+ 47%|████▋     | 472/1000 [01:13<01:14,  7.07it/s]
+
+ 47%|████▋     | 473/1000 [01:14<01:22,  6.39it/s]
+
+ 47%|████▋     | 474/1000 [01:14<01:39,  5.27it/s]
+
+ 48%|████▊     | 476/1000 [01:14<01:38,  5.32it/s]
+
+ 48%|████▊     | 477/1000 [01:15<01:54,  4.56it/s]
+
+ 48%|████▊     | 479/1000 [01:15<01:25,  6.13it/s]
+
+ 48%|████▊     | 480/1000 [01:15<01:32,  5.62it/s]
+
+ 48%|████▊     | 482/1000 [01:15<01:16,  6.74it/s]
+
+ 48%|████▊     | 484/1000 [01:15<01:22,  6.24it/s]
+
+ 48%|████▊     | 485/1000 [01:16<01:35,  5.41it/s]
+
+ 49%|████▊     | 486/1000 [01:16<01:34,  5.42it/s]
+
+ 49%|████▊     | 487/1000 [01:16<01:24,  6.07it/s]
+
+ 49%|████▉     | 489/1000 [01:16<01:27,  5.81it/s]
+
+ 49%|████▉     | 491/1000 [01:17<01:11,  7.15it/s]
+
+ 49%|████▉     | 492/1000 [01:17<01:10,  7.23it/s]
+
+ 49%|████▉     | 494/1000 [01:17<00:58,  8.64it/s]
+
+ 50%|████▉     | 496/1000 [01:17<01:09,  7.30it/s]
+
+ 50%|████▉     | 498/1000 [01:17<01:09,  7.20it/s]
+
+ 50%|████▉     | 499/1000 [01:18<01:27,  5.73it/s]
+
+ 50%|█████     | 500/1000 [01:18<01:37,  5.11it/s]
+
+ 50%|█████     | 502/1000 [01:18<01:18,  6.33it/s]
+
+ 50%|█████     | 503/1000 [01:19<01:25,  5.79it/s]
+
+ 50%|█████     | 505/1000 [01:19<01:27,  5.66it/s]
+
+ 51%|█████     | 506/1000 [01:19<01:40,  4.92it/s]
+
+ 51%|█████     | 508/1000 [01:19<01:17,  6.36it/s]
+
+ 51%|█████     | 510/1000 [01:19<01:03,  7.69it/s]
+
+ 51%|█████     | 512/1000 [01:20<00:56,  8.61it/s]
+
+ 51%|█████▏    | 514/1000 [01:20<01:18,  6.17it/s]
+
+ 52%|█████▏    | 515/1000 [01:20<01:15,  6.43it/s]
+
+ 52%|█████▏    | 516/1000 [01:21<01:19,  6.08it/s]
+
+ 52%|█████▏    | 518/1000 [01:21<01:09,  6.97it/s]
+
+ 52%|█████▏    | 519/1000 [01:21<01:27,  5.53it/s]
+
+ 52%|█████▏    | 521/1000 [01:21<01:08,  6.98it/s]
+
+ 52%|█████▏    | 522/1000 [01:21<01:22,  5.82it/s]
+
+ 52%|█████▏    | 523/1000 [01:22<01:36,  4.93it/s]
+
+ 52%|█████▏    | 524/1000 [01:22<01:39,  4.78it/s]
+
+ 53%|█████▎    | 526/1000 [01:22<01:23,  5.70it/s]
+
+ 53%|█████▎    | 528/1000 [01:23<01:23,  5.69it/s]
+
+ 53%|█████▎    | 529/1000 [01:23<01:24,  5.56it/s]
+
+ 53%|█████▎    | 531/1000 [01:23<01:07,  6.94it/s]
+
+ 53%|█████▎    | 533/1000 [01:23<00:55,  8.39it/s]
+
+ 53%|█████▎    | 534/1000 [01:23<01:09,  6.66it/s]
+
+ 54%|█████▎    | 535/1000 [01:24<01:23,  5.59it/s]
+
+ 54%|█████▎    | 536/1000 [01:24<01:30,  5.14it/s]
+
+ 54%|█████▎    | 537/1000 [01:24<01:40,  4.62it/s]
+
+ 54%|█████▍    | 539/1000 [01:25<01:32,  5.00it/s]
+
+ 54%|█████▍    | 540/1000 [01:25<01:33,  4.89it/s]
+
+ 54%|█████▍    | 541/1000 [01:25<01:42,  4.48it/s]
+
+ 54%|█████▍    | 543/1000 [01:25<01:15,  6.03it/s]
+
+ 54%|█████▍    | 544/1000 [01:25<01:09,  6.60it/s]
+
+ 55%|█████▍    | 545/1000 [01:26<01:17,  5.86it/s]
+
+ 55%|█████▍    | 546/1000 [01:26<01:25,  5.34it/s]
+
+ 55%|█████▍    | 547/1000 [01:26<01:40,  4.50it/s]
+
+ 55%|█████▍    | 549/1000 [01:26<01:32,  4.89it/s]
+
+ 55%|█████▌    | 551/1000 [01:27<01:10,  6.40it/s]
+
+ 55%|█████▌    | 552/1000 [01:27<01:11,  6.23it/s]
+
+ 55%|█████▌    | 553/1000 [01:27<01:24,  5.28it/s]
+
+ 55%|█████▌    | 554/1000 [01:27<01:27,  5.11it/s]
+
+ 56%|█████▌    | 555/1000 [01:27<01:16,  5.84it/s]
+
+ 56%|█████▌    | 557/1000 [01:28<01:11,  6.20it/s]
+
+ 56%|█████▌    | 559/1000 [01:28<00:56,  7.74it/s]
+
+ 56%|█████▌    | 561/1000 [01:28<00:49,  8.81it/s]
+
+ 56%|█████▋    | 563/1000 [01:28<00:59,  7.33it/s]
+
+ 56%|█████▋    | 565/1000 [01:29<01:02,  6.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:29<01:13,  5.91it/s]
+
+ 57%|█████▋    | 568/1000 [01:29<01:15,  5.75it/s]
+
+ 57%|█████▋    | 569/1000 [01:30<01:17,  5.53it/s]
+
+ 57%|█████▋    | 571/1000 [01:30<01:17,  5.55it/s]
+
+ 57%|█████▋    | 572/1000 [01:30<01:18,  5.45it/s]
+
+ 57%|█████▋    | 574/1000 [01:31<01:19,  5.38it/s]
+
+ 58%|█████▊    | 576/1000 [01:31<01:03,  6.67it/s]
+
+ 58%|█████▊    | 577/1000 [01:31<01:26,  4.90it/s]
+
+ 58%|█████▊    | 578/1000 [01:31<01:33,  4.53it/s]
+
+ 58%|█████▊    | 580/1000 [01:32<01:09,  6.03it/s]
+
+ 58%|█████▊    | 582/1000 [01:32<00:56,  7.41it/s]
+
+ 58%|█████▊    | 583/1000 [01:32<01:10,  5.92it/s]
+
+ 58%|█████▊    | 585/1000 [01:32<01:08,  6.06it/s]
+
+ 59%|█████▊    | 587/1000 [01:33<01:05,  6.27it/s]
+
+ 59%|█████▉    | 589/1000 [01:33<00:54,  7.48it/s]
+
+ 59%|█████▉    | 590/1000 [01:33<01:09,  5.90it/s]
+
+ 59%|█████▉    | 592/1000 [01:33<00:57,  7.09it/s]
+
+ 59%|█████▉    | 593/1000 [01:34<01:08,  5.95it/s]
+
+ 60%|█████▉    | 595/1000 [01:34<00:55,  7.35it/s]
+
+ 60%|█████▉    | 596/1000 [01:34<01:08,  5.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:34<01:10,  5.71it/s]
+
+ 60%|█████▉    | 599/1000 [01:35<01:11,  5.59it/s]
+
+ 60%|██████    | 601/1000 [01:35<01:11,  5.56it/s]
+
+ 60%|██████    | 602/1000 [01:35<01:10,  5.68it/s]
+
+ 60%|██████    | 604/1000 [01:36<01:21,  4.87it/s]
+
+ 61%|██████    | 606/1000 [01:36<01:04,  6.12it/s]
+
+ 61%|██████    | 608/1000 [01:36<00:57,  6.81it/s]
+
+ 61%|██████    | 610/1000 [01:36<00:53,  7.36it/s]
+
+ 61%|██████    | 612/1000 [01:36<00:45,  8.45it/s]
+
+ 61%|██████▏   | 614/1000 [01:37<00:41,  9.23it/s]
+
+ 62%|██████▏   | 616/1000 [01:37<00:58,  6.52it/s]
+
+ 62%|██████▏   | 617/1000 [01:37<00:57,  6.65it/s]
+
+ 62%|██████▏   | 619/1000 [01:37<00:48,  7.82it/s]
+
+ 62%|██████▏   | 621/1000 [01:38<00:55,  6.79it/s]
+
+ 62%|██████▏   | 623/1000 [01:38<00:53,  7.00it/s]
+
+ 62%|██████▎   | 625/1000 [01:38<00:58,  6.44it/s]
+
+ 63%|██████▎   | 627/1000 [01:39<00:49,  7.50it/s]
+
+ 63%|██████▎   | 628/1000 [01:39<00:59,  6.25it/s]
+
+ 63%|██████▎   | 629/1000 [01:39<01:08,  5.43it/s]
+
+ 63%|██████▎   | 631/1000 [01:39<00:54,  6.71it/s]
+
+ 63%|██████▎   | 633/1000 [01:39<00:45,  8.08it/s]
+
+ 63%|██████▎   | 634/1000 [01:40<01:06,  5.51it/s]
+
+ 64%|██████▎   | 635/1000 [01:40<01:01,  5.89it/s]
+
+ 64%|██████▎   | 636/1000 [01:40<00:57,  6.28it/s]
+
+ 64%|██████▍   | 638/1000 [01:40<00:50,  7.14it/s]
+
+ 64%|██████▍   | 640/1000 [01:41<00:43,  8.36it/s]
+
+ 64%|██████▍   | 641/1000 [01:41<00:54,  6.58it/s]
+
+ 64%|██████▍   | 643/1000 [01:41<00:53,  6.71it/s]
+
+ 64%|██████▍   | 645/1000 [01:41<00:57,  6.23it/s]
+
+ 65%|██████▍   | 647/1000 [01:42<00:53,  6.65it/s]
+
+ 65%|██████▍   | 648/1000 [01:42<01:01,  5.72it/s]
+
+ 65%|██████▌   | 650/1000 [01:42<01:00,  5.77it/s]
+
+ 65%|██████▌   | 652/1000 [01:43<01:03,  5.52it/s]
+
+ 65%|██████▌   | 654/1000 [01:43<00:51,  6.74it/s]
+
+ 66%|██████▌   | 655/1000 [01:43<00:48,  7.16it/s]
+
+ 66%|██████▌   | 657/1000 [01:43<00:52,  6.50it/s]
+
+ 66%|██████▌   | 659/1000 [01:44<01:02,  5.46it/s]
+
+ 66%|██████▌   | 661/1000 [01:44<00:50,  6.73it/s]
+
+ 66%|██████▌   | 662/1000 [01:44<00:58,  5.79it/s]
+
+ 66%|██████▋   | 664/1000 [01:45<01:05,  5.13it/s]
+
+ 66%|██████▋   | 665/1000 [01:45<01:12,  4.63it/s]
+
+ 67%|██████▋   | 667/1000 [01:45<01:08,  4.86it/s]
+
+ 67%|██████▋   | 668/1000 [01:46<01:13,  4.53it/s]
+
+ 67%|██████▋   | 669/1000 [01:46<01:07,  4.87it/s]
+
+ 67%|██████▋   | 671/1000 [01:46<01:05,  5.06it/s]
+
+ 67%|██████▋   | 673/1000 [01:46<00:58,  5.59it/s]
+
+ 68%|██████▊   | 675/1000 [01:47<00:52,  6.19it/s]
+
+ 68%|██████▊   | 677/1000 [01:47<00:43,  7.39it/s]
+
+ 68%|██████▊   | 679/1000 [01:47<00:39,  8.16it/s]
+
+ 68%|██████▊   | 681/1000 [01:47<00:34,  9.20it/s]
+
+ 68%|██████▊   | 683/1000 [01:48<00:41,  7.65it/s]
+
+ 68%|██████▊   | 685/1000 [01:48<00:37,  8.50it/s]
+
+ 69%|██████▊   | 686/1000 [01:48<00:47,  6.57it/s]
+
+ 69%|██████▊   | 687/1000 [01:48<00:55,  5.62it/s]
+
+ 69%|██████▉   | 688/1000 [01:48<00:51,  6.00it/s]
+
+ 69%|██████▉   | 690/1000 [01:49<00:53,  5.77it/s]
+
+ 69%|██████▉   | 692/1000 [01:49<00:45,  6.77it/s]
+
+ 69%|██████▉   | 694/1000 [01:49<00:49,  6.16it/s]
+
+ 70%|██████▉   | 695/1000 [01:50<00:48,  6.35it/s]
+
+ 70%|██████▉   | 696/1000 [01:50<00:55,  5.43it/s]
+
+ 70%|██████▉   | 698/1000 [01:50<00:50,  6.00it/s]
+
+ 70%|██████▉   | 699/1000 [01:50<00:57,  5.23it/s]
+
+ 70%|███████   | 701/1000 [01:51<00:46,  6.43it/s]
+
+ 70%|███████   | 702/1000 [01:51<01:01,  4.82it/s]
+
+ 70%|███████   | 704/1000 [01:51<00:47,  6.24it/s]
+
+ 70%|███████   | 705/1000 [01:52<01:04,  4.55it/s]
+
+ 71%|███████   | 707/1000 [01:52<00:48,  5.98it/s]
+
+ 71%|███████   | 709/1000 [01:52<00:44,  6.53it/s]
+
+ 71%|███████   | 710/1000 [01:52<00:51,  5.61it/s]
+
+ 71%|███████   | 712/1000 [01:53<00:42,  6.72it/s]
+
+ 71%|███████▏  | 713/1000 [01:53<00:41,  6.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:53<00:34,  8.18it/s]
+
+ 72%|███████▏  | 716/1000 [01:53<00:45,  6.28it/s]
+
+ 72%|███████▏  | 718/1000 [01:53<00:47,  5.99it/s]
+
+ 72%|███████▏  | 719/1000 [01:54<00:53,  5.24it/s]
+
+ 72%|███████▏  | 720/1000 [01:54<00:49,  5.69it/s]
+
+ 72%|███████▏  | 722/1000 [01:54<00:40,  6.91it/s]
+
+ 72%|███████▏  | 723/1000 [01:54<00:39,  7.06it/s]
+
+ 72%|███████▏  | 724/1000 [01:54<00:38,  7.11it/s]
+
+ 72%|███████▎  | 725/1000 [01:54<00:38,  7.19it/s]
+
+ 73%|███████▎  | 727/1000 [01:55<00:31,  8.77it/s]
+
+ 73%|███████▎  | 729/1000 [01:55<00:26, 10.08it/s]
+
+ 73%|███████▎  | 731/1000 [01:55<00:37,  7.25it/s]
+
+ 73%|███████▎  | 733/1000 [01:55<00:32,  8.31it/s]
+
+ 73%|███████▎  | 734/1000 [01:56<00:33,  7.99it/s]
+
+ 74%|███████▎  | 736/1000 [01:56<00:34,  7.67it/s]
+
+ 74%|███████▍  | 738/1000 [01:56<00:31,  8.30it/s]
+
+ 74%|███████▍  | 740/1000 [01:56<00:28,  9.11it/s]
+
+ 74%|███████▍  | 741/1000 [01:56<00:37,  6.88it/s]
+
+ 74%|███████▍  | 743/1000 [01:57<00:36,  7.05it/s]
+
+ 74%|███████▍  | 745/1000 [01:57<00:30,  8.28it/s]
+
+ 75%|███████▍  | 747/1000 [01:57<00:27,  9.11it/s]
+
+ 75%|███████▍  | 748/1000 [01:57<00:36,  6.96it/s]
+
+ 75%|███████▌  | 750/1000 [01:58<00:37,  6.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:58<00:46,  5.37it/s]
+
+ 75%|███████▌  | 753/1000 [01:58<00:39,  6.30it/s]
+
+ 75%|███████▌  | 754/1000 [01:59<00:45,  5.43it/s]
+
+ 76%|███████▌  | 756/1000 [01:59<00:44,  5.49it/s]
+
+ 76%|███████▌  | 758/1000 [01:59<00:37,  6.52it/s]
+
+ 76%|███████▌  | 759/1000 [01:59<00:38,  6.33it/s]
+
+ 76%|███████▌  | 761/1000 [01:59<00:31,  7.56it/s]
+
+ 76%|███████▋  | 763/1000 [02:00<00:35,  6.71it/s]
+
+ 76%|███████▋  | 764/1000 [02:00<00:40,  5.78it/s]
+
+ 77%|███████▋  | 766/1000 [02:00<00:41,  5.67it/s]
+
+ 77%|███████▋  | 768/1000 [02:01<00:33,  6.84it/s]
+
+ 77%|███████▋  | 769/1000 [02:01<00:39,  5.84it/s]
+
+ 77%|███████▋  | 770/1000 [02:01<00:39,  5.76it/s]
+
+ 77%|███████▋  | 772/1000 [02:01<00:31,  7.17it/s]
+
+ 77%|███████▋  | 773/1000 [02:02<00:40,  5.58it/s]
+
+ 77%|███████▋  | 774/1000 [02:02<00:38,  5.95it/s]
+
+ 78%|███████▊  | 776/1000 [02:02<00:32,  6.93it/s]
+
+ 78%|███████▊  | 778/1000 [02:02<00:26,  8.50it/s]
+
+ 78%|███████▊  | 780/1000 [02:03<00:36,  6.08it/s]
+
+ 78%|███████▊  | 782/1000 [02:03<00:29,  7.32it/s]
+
+ 78%|███████▊  | 783/1000 [02:03<00:35,  6.10it/s]
+
+ 78%|███████▊  | 784/1000 [02:03<00:34,  6.34it/s]
+
+ 79%|███████▊  | 786/1000 [02:03<00:31,  6.74it/s]
+
+ 79%|███████▉  | 788/1000 [02:04<00:28,  7.54it/s]
+
+ 79%|███████▉  | 790/1000 [02:04<00:25,  8.09it/s]
+
+ 79%|███████▉  | 791/1000 [02:04<00:28,  7.28it/s]
+
+ 79%|███████▉  | 792/1000 [02:04<00:30,  6.82it/s]
+
+ 79%|███████▉  | 793/1000 [02:04<00:29,  7.06it/s]
+
+ 80%|███████▉  | 795/1000 [02:05<00:32,  6.34it/s]
+
+ 80%|███████▉  | 797/1000 [02:05<00:33,  6.02it/s]
+
+ 80%|███████▉  | 798/1000 [02:05<00:38,  5.28it/s]
+
+ 80%|████████  | 800/1000 [02:06<00:33,  5.93it/s]
+
+ 80%|████████  | 802/1000 [02:06<00:29,  6.74it/s]
+
+ 80%|████████  | 804/1000 [02:06<00:28,  6.86it/s]
+
+ 81%|████████  | 806/1000 [02:06<00:24,  7.86it/s]
+
+ 81%|████████  | 807/1000 [02:06<00:24,  7.77it/s]
+
+ 81%|████████  | 809/1000 [02:07<00:21,  8.86it/s]
+
+ 81%|████████  | 811/1000 [02:07<00:19,  9.75it/s]
+
+ 81%|████████▏ | 813/1000 [02:07<00:24,  7.65it/s]
+
+ 81%|████████▏ | 814/1000 [02:07<00:24,  7.63it/s]
+
+ 82%|████████▏ | 816/1000 [02:07<00:21,  8.62it/s]
+
+ 82%|████████▏ | 817/1000 [02:08<00:26,  6.81it/s]
+
+ 82%|████████▏ | 819/1000 [02:08<00:22,  7.97it/s]
+
+ 82%|████████▏ | 820/1000 [02:08<00:28,  6.34it/s]
+
+ 82%|████████▏ | 822/1000 [02:08<00:23,  7.58it/s]
+
+ 82%|████████▏ | 823/1000 [02:09<00:26,  6.61it/s]
+
+ 82%|████████▏ | 824/1000 [02:09<00:31,  5.54it/s]
+
+ 82%|████████▎ | 825/1000 [02:09<00:30,  5.73it/s]
+
+ 83%|████████▎ | 826/1000 [02:09<00:37,  4.69it/s]
+
+ 83%|████████▎ | 828/1000 [02:10<00:39,  4.35it/s]
+
+ 83%|████████▎ | 829/1000 [02:10<00:35,  4.83it/s]
+
+ 83%|████████▎ | 831/1000 [02:10<00:28,  6.01it/s]
+
+ 83%|████████▎ | 832/1000 [02:10<00:34,  4.91it/s]
+
+ 83%|████████▎ | 833/1000 [02:11<00:31,  5.36it/s]
+
+ 83%|████████▎ | 834/1000 [02:11<00:32,  5.06it/s]
+
+ 84%|████████▎ | 835/1000 [02:11<00:29,  5.52it/s]
+
+ 84%|████████▎ | 836/1000 [02:11<00:31,  5.21it/s]
+
+ 84%|████████▍ | 838/1000 [02:11<00:25,  6.43it/s]
+
+ 84%|████████▍ | 840/1000 [02:12<00:27,  5.72it/s]
+
+ 84%|████████▍ | 842/1000 [02:12<00:22,  7.07it/s]
+
+ 84%|████████▍ | 844/1000 [02:13<00:28,  5.55it/s]
+
+ 84%|████████▍ | 845/1000 [02:13<00:30,  5.03it/s]
+
+ 85%|████████▍ | 846/1000 [02:13<00:27,  5.60it/s]
+
+ 85%|████████▍ | 847/1000 [02:13<00:29,  5.27it/s]
+
+ 85%|████████▍ | 849/1000 [02:13<00:22,  6.68it/s]
+
+ 85%|████████▌ | 850/1000 [02:14<00:26,  5.61it/s]
+
+ 85%|████████▌ | 851/1000 [02:14<00:25,  5.79it/s]
+
+ 85%|████████▌ | 853/1000 [02:14<00:19,  7.39it/s]
+
+ 86%|████████▌ | 855/1000 [02:14<00:16,  8.86it/s]
+
+ 86%|████████▌ | 856/1000 [02:14<00:16,  8.64it/s]
+
+ 86%|████████▌ | 858/1000 [02:15<00:20,  7.08it/s]
+
+ 86%|████████▌ | 860/1000 [02:15<00:17,  8.20it/s]
+
+ 86%|████████▌ | 862/1000 [02:15<00:14,  9.25it/s]
+
+ 86%|████████▋ | 864/1000 [02:15<00:13, 10.40it/s]
+
+ 87%|████████▋ | 866/1000 [02:15<00:12, 10.58it/s]
+
+ 87%|████████▋ | 868/1000 [02:16<00:16,  8.25it/s]
+
+ 87%|████████▋ | 870/1000 [02:16<00:14,  9.02it/s]
+
+ 87%|████████▋ | 872/1000 [02:16<00:15,  8.26it/s]
+
+ 87%|████████▋ | 873/1000 [02:16<00:19,  6.46it/s]
+
+ 87%|████████▋ | 874/1000 [02:17<00:21,  5.87it/s]
+
+ 88%|████████▊ | 876/1000 [02:17<00:16,  7.33it/s]
+
+ 88%|████████▊ | 878/1000 [02:17<00:18,  6.53it/s]
+
+ 88%|████████▊ | 879/1000 [02:17<00:20,  5.95it/s]
+
+ 88%|████████▊ | 880/1000 [02:17<00:18,  6.46it/s]
+
+ 88%|████████▊ | 881/1000 [02:18<00:17,  6.62it/s]
+
+ 88%|████████▊ | 882/1000 [02:18<00:22,  5.36it/s]
+
+ 88%|████████▊ | 883/1000 [02:18<00:19,  6.03it/s]
+
+ 88%|████████▊ | 885/1000 [02:18<00:14,  7.74it/s]
+
+ 89%|████████▊ | 886/1000 [02:18<00:14,  7.97it/s]
+
+ 89%|████████▉ | 888/1000 [02:18<00:12,  9.28it/s]
+
+ 89%|████████▉ | 889/1000 [02:19<00:13,  8.08it/s]
+
+ 89%|████████▉ | 891/1000 [02:19<00:11,  9.25it/s]
+
+ 89%|████████▉ | 893/1000 [02:19<00:13,  8.07it/s]
+
+ 89%|████████▉ | 894/1000 [02:19<00:16,  6.42it/s]
+
+ 90%|████████▉ | 896/1000 [02:20<00:13,  7.52it/s]
+
+ 90%|████████▉ | 897/1000 [02:20<00:16,  6.11it/s]
+
+ 90%|████████▉ | 898/1000 [02:20<00:19,  5.25it/s]
+
+ 90%|████████▉ | 899/1000 [02:20<00:17,  5.93it/s]
+
+ 90%|█████████ | 901/1000 [02:20<00:14,  6.81it/s]
+
+ 90%|█████████ | 902/1000 [02:21<00:13,  7.05it/s]
+
+ 90%|█████████ | 904/1000 [02:21<00:11,  8.17it/s]
+
+ 90%|█████████ | 905/1000 [02:21<00:13,  7.01it/s]
+
+ 91%|█████████ | 906/1000 [02:21<00:16,  5.72it/s]
+
+ 91%|█████████ | 908/1000 [02:21<00:12,  7.34it/s]
+
+ 91%|█████████ | 910/1000 [02:22<00:13,  6.54it/s]
+
+ 91%|█████████ | 912/1000 [02:22<00:11,  7.67it/s]
+
+ 91%|█████████▏| 913/1000 [02:22<00:15,  5.70it/s]
+
+ 92%|█████████▏| 915/1000 [02:22<00:12,  6.98it/s]
+
+ 92%|█████████▏| 916/1000 [02:23<00:14,  5.90it/s]
+
+ 92%|█████████▏| 917/1000 [02:23<00:13,  6.25it/s]
+
+ 92%|█████████▏| 918/1000 [02:23<00:17,  4.58it/s]
+
+ 92%|█████████▏| 919/1000 [02:23<00:17,  4.75it/s]
+
+ 92%|█████████▏| 921/1000 [02:24<00:16,  4.70it/s]
+
+ 92%|█████████▏| 922/1000 [02:24<00:15,  5.06it/s]
+
+ 92%|█████████▏| 924/1000 [02:24<00:12,  6.07it/s]
+
+ 93%|█████████▎| 926/1000 [02:24<00:10,  7.24it/s]
+
+ 93%|█████████▎| 927/1000 [02:25<00:10,  6.86it/s]
+
+ 93%|█████████▎| 928/1000 [02:25<00:12,  5.66it/s]
+
+ 93%|█████████▎| 930/1000 [02:25<00:12,  5.57it/s]
+
+ 93%|█████████▎| 931/1000 [02:25<00:11,  5.89it/s]
+
+ 93%|█████████▎| 933/1000 [02:26<00:11,  5.63it/s]
+
+ 93%|█████████▎| 934/1000 [02:26<00:12,  5.29it/s]
+
+ 94%|█████████▎| 936/1000 [02:26<00:10,  6.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:26<00:08,  7.64it/s]
+
+ 94%|█████████▍| 939/1000 [02:27<00:08,  6.99it/s]
+
+ 94%|█████████▍| 940/1000 [02:27<00:08,  7.05it/s]
+
+ 94%|█████████▍| 942/1000 [02:27<00:10,  5.48it/s]
+
+ 94%|█████████▍| 944/1000 [02:27<00:09,  6.02it/s]
+
+ 95%|█████████▍| 946/1000 [02:28<00:07,  7.16it/s]
+
+ 95%|█████████▍| 948/1000 [02:28<00:06,  8.33it/s]
+
+ 95%|█████████▍| 949/1000 [02:28<00:07,  7.19it/s]
+
+ 95%|█████████▌| 951/1000 [02:28<00:05,  8.23it/s]
+
+ 95%|█████████▌| 952/1000 [02:29<00:08,  5.92it/s]
+
+ 95%|█████████▌| 954/1000 [02:29<00:09,  4.96it/s]
+
+ 96%|█████████▌| 956/1000 [02:29<00:07,  6.21it/s]
+
+ 96%|█████████▌| 957/1000 [02:30<00:08,  5.33it/s]
+
+ 96%|█████████▌| 959/1000 [02:30<00:08,  4.80it/s]
+
+ 96%|█████████▌| 961/1000 [02:30<00:06,  5.79it/s]
+
+ 96%|█████████▌| 962/1000 [02:31<00:07,  5.16it/s]
+
+ 96%|█████████▋| 964/1000 [02:31<00:05,  6.26it/s]
+
+ 96%|█████████▋| 965/1000 [02:31<00:06,  5.47it/s]
+
+ 97%|█████████▋| 966/1000 [02:31<00:06,  5.42it/s]
+
+ 97%|█████████▋| 968/1000 [02:31<00:04,  6.90it/s]
+
+ 97%|█████████▋| 970/1000 [02:32<00:03,  7.99it/s]
+
+ 97%|█████████▋| 971/1000 [02:32<00:04,  6.38it/s]
+
+ 97%|█████████▋| 973/1000 [02:32<00:03,  7.51it/s]
+
+ 98%|█████████▊| 975/1000 [02:32<00:03,  8.00it/s]
+
+ 98%|█████████▊| 977/1000 [02:32<00:02,  9.25it/s]
+
+ 98%|█████████▊| 979/1000 [02:33<00:02,  9.30it/s]
+
+ 98%|█████████▊| 980/1000 [02:33<00:02,  7.15it/s]
+
+ 98%|█████████▊| 981/1000 [02:33<00:03,  5.91it/s]
+
+ 98%|█████████▊| 982/1000 [02:34<00:04,  4.40it/s]
+
+ 98%|█████████▊| 984/1000 [02:34<00:02,  5.82it/s]
+
+ 99%|█████████▊| 986/1000 [02:34<00:01,  7.17it/s]
+
+ 99%|█████████▊| 987/1000 [02:34<00:01,  6.66it/s]
+
+ 99%|█████████▉| 988/1000 [02:34<00:02,  5.52it/s]
+
+ 99%|█████████▉| 989/1000 [02:35<00:02,  4.74it/s]
+
+ 99%|█████████▉| 991/1000 [02:35<00:01,  5.06it/s]
+
+ 99%|█████████▉| 992/1000 [02:35<00:01,  4.00it/s]
+
+ 99%|█████████▉| 994/1000 [02:36<00:01,  5.49it/s]
+
+100%|█████████▉| 995/1000 [02:36<00:01,  4.62it/s]
+
+100%|█████████▉| 997/1000 [02:36<00:00,  5.88it/s]
+
+100%|█████████▉| 999/1000 [02:37<00:00,  5.71it/s]
+
+100%|██████████| 1000/1000 [02:37<00:00,  5.67it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7405', 'eval_runtime': '157.4', 'eval_samples_per_second': '6.354', 'eval_steps_per_second': '6.354', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448}
+
+ 99%|█████████▉| 1800/1813 [3:26:24<01:33,  7.22s/it]
+
+100%|██████████| 1000/1000 [02:37<00:00,  5.67it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_003_lr_2.000e-04/checkpoint-1800
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+ 99%|█████████▉| 1801/1813 [3:26:34<11:03, 55.27s/it]
+ 99%|█████████▉| 1802/1813 [3:26:40<07:25, 40.48s/it]
+ 99%|█████████▉| 1803/1813 [3:26:47<05:02, 30.24s/it]
+100%|█████████▉| 1804/1813 [3:26:55<03:33, 23.75s/it]
+100%|█████████▉| 1805/1813 [3:27:02<02:29, 18.68s/it]
+100%|█████████▉| 1806/1813 [3:27:10<01:46, 15.28s/it]
+100%|█████████▉| 1807/1813 [3:27:17<01:18, 13.09s/it]
+100%|█████████▉| 1808/1813 [3:27:26<00:58, 11.71s/it]
+100%|█████████▉| 1809/1813 [3:27:33<00:41, 10.38s/it]
+100%|█████████▉| 1810/1813 [3:27:40<00:27,  9.20s/it]
+                                                     
+{'loss': '0.8548', 'grad_norm': '0.8624', 'learning_rate': '2.855e-09', 'epoch': '0.9986', 'num_input_tokens_seen': 12187432, 'train_runtime': '1.246e+04', 'train_tokens_per_second': '978.1'}
+
+100%|█████████▉| 1810/1813 [3:27:40<00:27,  9.20s/it]
+100%|█████████▉| 1811/1813 [3:27:46<00:16,  8.33s/it]
+100%|█████████▉| 1812/1813 [3:27:52<00:07,  7.66s/it]
+100%|██████████| 1813/1813 [3:27:56<00:00,  6.40s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:13, 13.53it/s]
+
+  0%|          | 5/1000 [00:00<02:43,  6.07it/s]
+
+  1%|          | 7/1000 [00:00<02:19,  7.13it/s]
+
+  1%|          | 9/1000 [00:01<02:58,  5.56it/s]
+
+  1%|          | 11/1000 [00:01<02:27,  6.71it/s]
+
+  1%|          | 12/1000 [00:01<02:23,  6.87it/s]
+
+  1%|▏         | 13/1000 [00:02<02:56,  5.59it/s]
+
+  1%|▏         | 14/1000 [00:02<02:57,  5.57it/s]
+
+  2%|▏         | 15/1000 [00:02<03:22,  4.87it/s]
+
+  2%|▏         | 16/1000 [00:02<03:27,  4.75it/s]
+
+  2%|▏         | 18/1000 [00:03<03:11,  5.14it/s]
+
+  2%|▏         | 19/1000 [00:03<03:45,  4.35it/s]
+
+  2%|▏         | 20/1000 [00:03<04:11,  3.90it/s]
+
+  2%|▏         | 22/1000 [00:03<03:01,  5.39it/s]
+
+  2%|▏         | 23/1000 [00:04<03:21,  4.84it/s]
+
+  2%|▎         | 25/1000 [00:04<02:31,  6.44it/s]
+
+  3%|▎         | 26/1000 [00:04<02:57,  5.47it/s]
+
+  3%|▎         | 28/1000 [00:05<03:05,  5.24it/s]
+
+  3%|▎         | 29/1000 [00:05<02:54,  5.55it/s]
+
+  3%|▎         | 30/1000 [00:05<03:06,  5.20it/s]
+
+  3%|▎         | 31/1000 [00:05<03:27,  4.68it/s]
+
+  3%|▎         | 32/1000 [00:05<03:04,  5.25it/s]
+
+  3%|▎         | 33/1000 [00:06<03:11,  5.05it/s]
+
+  4%|▎         | 35/1000 [00:06<02:23,  6.72it/s]
+
+  4%|▎         | 37/1000 [00:06<02:00,  7.98it/s]
+
+  4%|▍         | 39/1000 [00:06<02:25,  6.62it/s]
+
+  4%|▍         | 40/1000 [00:06<02:21,  6.76it/s]
+
+  4%|▍         | 42/1000 [00:07<01:59,  7.99it/s]
+
+  4%|▍         | 44/1000 [00:07<01:45,  9.08it/s]
+
+  4%|▍         | 45/1000 [00:07<02:07,  7.47it/s]
+
+  5%|▍         | 47/1000 [00:07<02:24,  6.61it/s]
+
+  5%|▍         | 49/1000 [00:08<02:02,  7.77it/s]
+
+  5%|▌         | 51/1000 [00:08<01:48,  8.75it/s]
+
+  5%|▌         | 53/1000 [00:08<01:38,  9.63it/s]
+
+  6%|▌         | 55/1000 [00:08<01:39,  9.47it/s]
+
+  6%|▌         | 57/1000 [00:08<01:55,  8.16it/s]
+
+  6%|▌         | 58/1000 [00:09<01:56,  8.06it/s]
+
+  6%|▌         | 59/1000 [00:09<02:15,  6.93it/s]
+
+  6%|▌         | 61/1000 [00:09<01:54,  8.18it/s]
+
+  6%|▋         | 63/1000 [00:09<01:48,  8.62it/s]
+
+  6%|▋         | 64/1000 [00:09<02:19,  6.72it/s]
+
+  7%|▋         | 66/1000 [00:10<02:01,  7.70it/s]
+
+  7%|▋         | 67/1000 [00:10<02:29,  6.26it/s]
+
+  7%|▋         | 69/1000 [00:10<02:11,  7.10it/s]
+
+  7%|▋         | 70/1000 [00:10<02:37,  5.89it/s]
+
+  7%|▋         | 72/1000 [00:11<02:41,  5.74it/s]
+
+  7%|▋         | 74/1000 [00:11<02:10,  7.08it/s]
+
+  8%|▊         | 75/1000 [00:11<02:39,  5.80it/s]
+
+  8%|▊         | 76/1000 [00:11<02:43,  5.65it/s]
+
+  8%|▊         | 77/1000 [00:12<02:28,  6.22it/s]
+
+  8%|▊         | 78/1000 [00:12<02:56,  5.21it/s]
+
+  8%|▊         | 80/1000 [00:12<02:18,  6.64it/s]
+
+  8%|▊         | 82/1000 [00:12<01:54,  8.02it/s]
+
+  8%|▊         | 83/1000 [00:12<02:24,  6.35it/s]
+
+  8%|▊         | 85/1000 [00:13<02:32,  6.00it/s]
+
+  9%|▊         | 86/1000 [00:13<02:54,  5.24it/s]
+
+  9%|▊         | 87/1000 [00:13<03:12,  4.73it/s]
+
+  9%|▉         | 89/1000 [00:14<02:27,  6.18it/s]
+
+  9%|▉         | 90/1000 [00:14<02:27,  6.19it/s]
+
+  9%|▉         | 91/1000 [00:14<02:54,  5.22it/s]
+
+  9%|▉         | 92/1000 [00:14<02:50,  5.32it/s]
+
+  9%|▉         | 93/1000 [00:14<03:02,  4.97it/s]
+
+  9%|▉         | 94/1000 [00:14<02:39,  5.68it/s]
+
+ 10%|▉         | 95/1000 [00:15<03:18,  4.56it/s]
+
+ 10%|▉         | 97/1000 [00:15<02:31,  5.94it/s]
+
+ 10%|▉         | 99/1000 [00:15<02:03,  7.32it/s]
+
+ 10%|█         | 101/1000 [00:15<01:46,  8.47it/s]
+
+ 10%|█         | 102/1000 [00:16<02:15,  6.62it/s]
+
+ 10%|█         | 103/1000 [00:16<02:12,  6.78it/s]
+
+ 10%|█         | 104/1000 [00:16<02:14,  6.64it/s]
+
+ 11%|█         | 106/1000 [00:16<01:47,  8.31it/s]
+
+ 11%|█         | 108/1000 [00:16<01:35,  9.36it/s]
+
+ 11%|█         | 110/1000 [00:16<01:32,  9.61it/s]
+
+ 11%|█         | 112/1000 [00:17<01:24, 10.47it/s]
+
+ 11%|█▏        | 114/1000 [00:17<01:21, 10.93it/s]
+
+ 12%|█▏        | 116/1000 [00:17<01:19, 11.16it/s]
+
+ 12%|█▏        | 118/1000 [00:17<01:16, 11.59it/s]
+
+ 12%|█▏        | 120/1000 [00:18<02:29,  5.87it/s]
+
+ 12%|█▏        | 122/1000 [00:18<02:06,  6.93it/s]
+
+ 12%|█▏        | 124/1000 [00:18<02:16,  6.41it/s]
+
+ 12%|█▎        | 125/1000 [00:19<03:00,  4.86it/s]
+
+ 13%|█▎        | 126/1000 [00:19<02:46,  5.24it/s]
+
+ 13%|█▎        | 127/1000 [00:19<02:56,  4.95it/s]
+
+ 13%|█▎        | 128/1000 [00:19<03:16,  4.43it/s]
+
+ 13%|█▎        | 130/1000 [00:20<02:24,  6.01it/s]
+
+ 13%|█▎        | 132/1000 [00:20<01:56,  7.45it/s]
+
+ 13%|█▎        | 134/1000 [00:20<01:39,  8.70it/s]
+
+ 14%|█▎        | 136/1000 [00:20<01:46,  8.14it/s]
+
+ 14%|█▎        | 137/1000 [00:20<01:47,  8.00it/s]
+
+ 14%|█▍        | 139/1000 [00:21<01:42,  8.36it/s]
+
+ 14%|█▍        | 141/1000 [00:21<01:44,  8.18it/s]
+
+ 14%|█▍        | 142/1000 [00:21<01:52,  7.65it/s]
+
+ 14%|█▍        | 143/1000 [00:21<01:53,  7.58it/s]
+
+ 14%|█▍        | 145/1000 [00:21<01:37,  8.79it/s]
+
+ 15%|█▍        | 147/1000 [00:22<01:57,  7.23it/s]
+
+ 15%|█▍        | 148/1000 [00:22<01:56,  7.33it/s]
+
+ 15%|█▍        | 149/1000 [00:22<01:49,  7.74it/s]
+
+ 15%|█▌        | 151/1000 [00:22<01:31,  9.29it/s]
+
+ 15%|█▌        | 153/1000 [00:22<01:21, 10.35it/s]
+
+ 16%|█▌        | 155/1000 [00:23<01:59,  7.06it/s]
+
+ 16%|█▌        | 156/1000 [00:23<02:35,  5.43it/s]
+
+ 16%|█▌        | 158/1000 [00:23<02:03,  6.82it/s]
+
+ 16%|█▌        | 160/1000 [00:23<01:54,  7.33it/s]
+
+ 16%|█▌        | 162/1000 [00:24<02:05,  6.69it/s]
+
+ 16%|█▋        | 163/1000 [00:24<02:10,  6.40it/s]
+
+ 16%|█▋        | 164/1000 [00:24<02:36,  5.34it/s]
+
+ 16%|█▋        | 165/1000 [00:25<02:54,  4.79it/s]
+
+ 17%|█▋        | 166/1000 [00:25<03:08,  4.42it/s]
+
+ 17%|█▋        | 167/1000 [00:25<02:47,  4.96it/s]
+
+ 17%|█▋        | 169/1000 [00:25<02:02,  6.79it/s]
+
+ 17%|█▋        | 171/1000 [00:25<02:16,  6.07it/s]
+
+ 17%|█▋        | 172/1000 [00:26<02:38,  5.23it/s]
+
+ 17%|█▋        | 173/1000 [00:26<02:40,  5.14it/s]
+
+ 17%|█▋        | 174/1000 [00:26<02:26,  5.65it/s]
+
+ 18%|█▊        | 176/1000 [00:26<01:54,  7.17it/s]
+
+ 18%|█▊        | 177/1000 [00:27<02:21,  5.82it/s]
+
+ 18%|█▊        | 179/1000 [00:27<01:52,  7.31it/s]
+
+ 18%|█▊        | 180/1000 [00:27<01:52,  7.27it/s]
+
+ 18%|█▊        | 181/1000 [00:27<02:24,  5.65it/s]
+
+ 18%|█▊        | 182/1000 [00:27<02:48,  4.84it/s]
+
+ 18%|█▊        | 183/1000 [00:28<02:45,  4.94it/s]
+
+ 18%|█▊        | 185/1000 [00:28<02:02,  6.65it/s]
+
+ 19%|█▊        | 186/1000 [00:28<01:59,  6.81it/s]
+
+ 19%|█▊        | 187/1000 [00:28<02:31,  5.36it/s]
+
+ 19%|█▉        | 189/1000 [00:28<01:57,  6.88it/s]
+
+ 19%|█▉        | 190/1000 [00:29<02:22,  5.69it/s]
+
+ 19%|█▉        | 191/1000 [00:29<02:07,  6.35it/s]
+
+ 19%|█▉        | 192/1000 [00:29<02:34,  5.25it/s]
+
+ 19%|█▉        | 194/1000 [00:29<02:31,  5.31it/s]
+
+ 20%|█▉        | 195/1000 [00:30<02:16,  5.92it/s]
+
+ 20%|█▉        | 197/1000 [00:30<01:49,  7.37it/s]
+
+ 20%|█▉        | 199/1000 [00:30<01:30,  8.85it/s]
+
+ 20%|██        | 201/1000 [00:30<01:21,  9.80it/s]
+
+ 20%|██        | 203/1000 [00:30<01:44,  7.64it/s]
+
+ 20%|██        | 204/1000 [00:31<01:52,  7.09it/s]
+
+ 21%|██        | 206/1000 [00:31<01:35,  8.32it/s]
+
+ 21%|██        | 207/1000 [00:31<01:58,  6.67it/s]
+
+ 21%|██        | 209/1000 [00:31<01:40,  7.91it/s]
+
+ 21%|██        | 210/1000 [00:31<01:49,  7.21it/s]
+
+ 21%|██        | 212/1000 [00:32<01:31,  8.57it/s]
+
+ 21%|██▏       | 213/1000 [00:32<01:43,  7.63it/s]
+
+ 21%|██▏       | 214/1000 [00:32<01:46,  7.37it/s]
+
+ 22%|██▏       | 216/1000 [00:32<01:38,  7.93it/s]
+
+ 22%|██▏       | 217/1000 [00:32<02:05,  6.26it/s]
+
+ 22%|██▏       | 219/1000 [00:33<01:42,  7.64it/s]
+
+ 22%|██▏       | 220/1000 [00:33<02:27,  5.29it/s]
+
+ 22%|██▏       | 221/1000 [00:33<02:26,  5.32it/s]
+
+ 22%|██▏       | 222/1000 [00:33<02:11,  5.93it/s]
+
+ 22%|██▏       | 223/1000 [00:34<02:58,  4.35it/s]
+
+ 22%|██▏       | 224/1000 [00:34<02:36,  4.96it/s]
+
+ 22%|██▎       | 225/1000 [00:34<02:38,  4.89it/s]
+
+ 23%|██▎       | 226/1000 [00:34<02:53,  4.46it/s]
+
+ 23%|██▎       | 228/1000 [00:35<02:11,  5.86it/s]
+
+ 23%|██▎       | 229/1000 [00:35<02:31,  5.08it/s]
+
+ 23%|██▎       | 230/1000 [00:35<02:19,  5.51it/s]
+
+ 23%|██▎       | 232/1000 [00:35<01:49,  7.02it/s]
+
+ 23%|██▎       | 233/1000 [00:35<01:46,  7.19it/s]
+
+ 24%|██▎       | 235/1000 [00:35<01:27,  8.78it/s]
+
+ 24%|██▎       | 236/1000 [00:36<01:29,  8.58it/s]
+
+ 24%|██▎       | 237/1000 [00:36<01:32,  8.21it/s]
+
+ 24%|██▍       | 239/1000 [00:36<01:41,  7.51it/s]
+
+ 24%|██▍       | 241/1000 [00:36<01:47,  7.04it/s]
+
+ 24%|██▍       | 242/1000 [00:37<02:31,  5.00it/s]
+
+ 24%|██▍       | 243/1000 [00:37<02:45,  4.58it/s]
+
+ 24%|██▍       | 245/1000 [00:37<02:15,  5.57it/s]
+
+ 25%|██▍       | 247/1000 [00:37<01:47,  7.00it/s]
+
+ 25%|██▍       | 248/1000 [00:38<02:08,  5.86it/s]
+
+ 25%|██▍       | 249/1000 [00:38<02:30,  4.99it/s]
+
+ 25%|██▌       | 250/1000 [00:38<02:30,  4.98it/s]
+
+ 25%|██▌       | 252/1000 [00:38<02:24,  5.18it/s]
+
+ 25%|██▌       | 253/1000 [00:39<02:31,  4.93it/s]
+
+ 25%|██▌       | 254/1000 [00:39<02:13,  5.59it/s]
+
+ 26%|██▌       | 256/1000 [00:39<01:50,  6.71it/s]
+
+ 26%|██▌       | 258/1000 [00:39<01:37,  7.61it/s]
+
+ 26%|██▌       | 260/1000 [00:40<01:49,  6.75it/s]
+
+ 26%|██▌       | 261/1000 [00:40<02:08,  5.74it/s]
+
+ 26%|██▌       | 262/1000 [00:40<02:14,  5.50it/s]
+
+ 26%|██▋       | 264/1000 [00:40<02:14,  5.48it/s]
+
+ 26%|██▋       | 265/1000 [00:41<02:08,  5.71it/s]
+
+ 27%|██▋       | 266/1000 [00:41<02:06,  5.82it/s]
+
+ 27%|██▋       | 268/1000 [00:41<01:53,  6.45it/s]
+
+ 27%|██▋       | 270/1000 [00:41<02:00,  6.08it/s]
+
+ 27%|██▋       | 271/1000 [00:42<02:38,  4.61it/s]
+
+ 27%|██▋       | 273/1000 [00:42<02:28,  4.88it/s]
+
+ 28%|██▊       | 275/1000 [00:42<02:05,  5.77it/s]
+
+ 28%|██▊       | 277/1000 [00:43<01:46,  6.79it/s]
+
+ 28%|██▊       | 278/1000 [00:43<02:04,  5.81it/s]
+
+ 28%|██▊       | 280/1000 [00:43<02:06,  5.69it/s]
+
+ 28%|██▊       | 282/1000 [00:43<01:44,  6.86it/s]
+
+ 28%|██▊       | 283/1000 [00:44<02:22,  5.02it/s]
+
+ 28%|██▊       | 284/1000 [00:44<02:25,  4.92it/s]
+
+ 28%|██▊       | 285/1000 [00:44<02:08,  5.58it/s]
+
+ 29%|██▊       | 287/1000 [00:44<01:36,  7.41it/s]
+
+ 29%|██▉       | 288/1000 [00:44<01:31,  7.75it/s]
+
+ 29%|██▉       | 290/1000 [00:45<01:45,  6.71it/s]
+
+ 29%|██▉       | 291/1000 [00:45<01:59,  5.96it/s]
+
+ 29%|██▉       | 292/1000 [00:45<02:21,  5.01it/s]
+
+ 29%|██▉       | 294/1000 [00:46<01:55,  6.10it/s]
+
+ 30%|██▉       | 296/1000 [00:46<02:01,  5.78it/s]
+
+ 30%|██▉       | 298/1000 [00:46<01:40,  6.96it/s]
+
+ 30%|██▉       | 299/1000 [00:46<02:07,  5.51it/s]
+
+ 30%|███       | 300/1000 [00:47<01:59,  5.87it/s]
+
+ 30%|███       | 301/1000 [00:47<01:53,  6.16it/s]
+
+ 30%|███       | 303/1000 [00:47<01:31,  7.64it/s]
+
+ 30%|███       | 304/1000 [00:47<01:38,  7.07it/s]
+
+ 30%|███       | 305/1000 [00:47<01:37,  7.11it/s]
+
+ 31%|███       | 306/1000 [00:47<01:54,  6.07it/s]
+
+ 31%|███       | 307/1000 [00:48<01:56,  5.93it/s]
+
+ 31%|███       | 309/1000 [00:48<01:30,  7.67it/s]
+
+ 31%|███       | 311/1000 [00:48<01:16,  8.97it/s]
+
+ 31%|███▏      | 313/1000 [00:48<01:09,  9.92it/s]
+
+ 32%|███▏      | 315/1000 [00:48<01:15,  9.04it/s]
+
+ 32%|███▏      | 316/1000 [00:49<01:25,  8.03it/s]
+
+ 32%|███▏      | 317/1000 [00:49<01:22,  8.30it/s]
+
+ 32%|███▏      | 319/1000 [00:49<01:53,  5.99it/s]
+
+ 32%|███▏      | 321/1000 [00:49<01:57,  5.79it/s]
+
+ 32%|███▏      | 322/1000 [00:50<02:05,  5.42it/s]
+
+ 32%|███▏      | 324/1000 [00:50<01:41,  6.69it/s]
+
+ 32%|███▎      | 325/1000 [00:50<02:18,  4.86it/s]
+
+ 33%|███▎      | 327/1000 [00:50<01:45,  6.40it/s]
+
+ 33%|███▎      | 328/1000 [00:51<01:48,  6.21it/s]
+
+ 33%|███▎      | 330/1000 [00:51<01:53,  5.90it/s]
+
+ 33%|███▎      | 332/1000 [00:51<01:56,  5.74it/s]
+
+ 33%|███▎      | 333/1000 [00:51<01:49,  6.08it/s]
+
+ 33%|███▎      | 334/1000 [00:52<01:59,  5.58it/s]
+
+ 34%|███▎      | 336/1000 [00:52<01:34,  7.06it/s]
+
+ 34%|███▎      | 337/1000 [00:52<01:28,  7.50it/s]
+
+ 34%|███▍      | 338/1000 [00:52<01:27,  7.59it/s]
+
+ 34%|███▍      | 340/1000 [00:52<01:13,  8.94it/s]
+
+ 34%|███▍      | 341/1000 [00:52<01:12,  9.10it/s]
+
+ 34%|███▍      | 343/1000 [00:53<01:30,  7.26it/s]
+
+ 34%|███▍      | 344/1000 [00:53<01:41,  6.46it/s]
+
+ 34%|███▍      | 345/1000 [00:53<01:38,  6.63it/s]
+
+ 35%|███▍      | 346/1000 [00:53<01:46,  6.16it/s]
+
+ 35%|███▍      | 347/1000 [00:53<01:41,  6.41it/s]
+
+ 35%|███▍      | 349/1000 [00:54<01:48,  5.97it/s]
+
+ 35%|███▌      | 350/1000 [00:54<02:07,  5.11it/s]
+
+ 35%|███▌      | 352/1000 [00:54<01:37,  6.66it/s]
+
+ 35%|███▌      | 354/1000 [00:55<01:38,  6.59it/s]
+
+ 36%|███▌      | 355/1000 [00:55<01:40,  6.42it/s]
+
+ 36%|███▌      | 357/1000 [00:55<01:21,  7.85it/s]
+
+ 36%|███▌      | 359/1000 [00:55<01:08,  9.40it/s]
+
+ 36%|███▌      | 361/1000 [00:55<01:17,  8.27it/s]
+
+ 36%|███▋      | 363/1000 [00:56<01:41,  6.29it/s]
+
+ 36%|███▋      | 364/1000 [00:56<01:55,  5.51it/s]
+
+ 36%|███▋      | 365/1000 [00:57<02:28,  4.28it/s]
+
+ 37%|███▋      | 367/1000 [00:57<02:14,  4.69it/s]
+
+ 37%|███▋      | 368/1000 [00:57<02:23,  4.40it/s]
+
+ 37%|███▋      | 369/1000 [00:57<02:08,  4.91it/s]
+
+ 37%|███▋      | 371/1000 [00:57<01:34,  6.67it/s]
+
+ 37%|███▋      | 372/1000 [00:58<01:32,  6.79it/s]
+
+ 37%|███▋      | 374/1000 [00:58<01:30,  6.93it/s]
+
+ 38%|███▊      | 375/1000 [00:58<01:57,  5.34it/s]
+
+ 38%|███▊      | 376/1000 [00:58<02:12,  4.73it/s]
+
+ 38%|███▊      | 377/1000 [00:59<01:56,  5.33it/s]
+
+ 38%|███▊      | 379/1000 [00:59<01:27,  7.11it/s]
+
+ 38%|███▊      | 381/1000 [00:59<01:17,  8.02it/s]
+
+ 38%|███▊      | 382/1000 [00:59<01:32,  6.66it/s]
+
+ 38%|███▊      | 384/1000 [00:59<01:17,  7.99it/s]
+
+ 38%|███▊      | 385/1000 [00:59<01:18,  7.84it/s]
+
+ 39%|███▊      | 386/1000 [01:00<01:16,  8.05it/s]
+
+ 39%|███▊      | 387/1000 [01:00<01:42,  5.98it/s]
+
+ 39%|███▉      | 388/1000 [01:00<01:44,  5.87it/s]
+
+ 39%|███▉      | 389/1000 [01:00<01:34,  6.47it/s]
+
+ 39%|███▉      | 390/1000 [01:00<01:31,  6.68it/s]
+
+ 39%|███▉      | 391/1000 [01:01<01:39,  6.10it/s]
+
+ 39%|███▉      | 393/1000 [01:01<01:17,  7.87it/s]
+
+ 40%|███▉      | 395/1000 [01:01<01:35,  6.36it/s]
+
+ 40%|███▉      | 397/1000 [01:01<01:18,  7.65it/s]
+
+ 40%|███▉      | 398/1000 [01:02<01:39,  6.04it/s]
+
+ 40%|███▉      | 399/1000 [01:02<02:00,  4.97it/s]
+
+ 40%|████      | 400/1000 [01:02<02:12,  4.54it/s]
+
+ 40%|████      | 402/1000 [01:03<02:04,  4.82it/s]
+
+ 40%|████      | 404/1000 [01:03<01:36,  6.15it/s]
+
+ 41%|████      | 406/1000 [01:03<01:20,  7.37it/s]
+
+ 41%|████      | 407/1000 [01:03<01:37,  6.10it/s]
+
+ 41%|████      | 408/1000 [01:03<01:51,  5.32it/s]
+
+ 41%|████      | 409/1000 [01:04<01:39,  5.94it/s]
+
+ 41%|████      | 410/1000 [01:04<01:56,  5.06it/s]
+
+ 41%|████      | 411/1000 [01:04<02:12,  4.44it/s]
+
+ 41%|████      | 412/1000 [01:04<02:21,  4.16it/s]
+
+ 41%|████▏     | 414/1000 [01:05<01:38,  5.98it/s]
+
+ 42%|████▏     | 415/1000 [01:05<01:28,  6.61it/s]
+
+ 42%|████▏     | 416/1000 [01:05<01:47,  5.46it/s]
+
+ 42%|████▏     | 418/1000 [01:05<01:21,  7.18it/s]
+
+ 42%|████▏     | 420/1000 [01:05<01:18,  7.40it/s]
+
+ 42%|████▏     | 421/1000 [01:06<01:30,  6.39it/s]
+
+ 42%|████▏     | 422/1000 [01:06<01:24,  6.84it/s]
+
+ 42%|████▏     | 423/1000 [01:06<01:50,  5.23it/s]
+
+ 42%|████▏     | 424/1000 [01:06<02:03,  4.67it/s]
+
+ 42%|████▎     | 425/1000 [01:06<01:50,  5.21it/s]
+
+ 43%|████▎     | 427/1000 [01:07<01:23,  6.84it/s]
+
+ 43%|████▎     | 428/1000 [01:07<01:28,  6.47it/s]
+
+ 43%|████▎     | 429/1000 [01:07<01:27,  6.50it/s]
+
+ 43%|████▎     | 431/1000 [01:07<01:09,  8.21it/s]
+
+ 43%|████▎     | 433/1000 [01:07<00:59,  9.57it/s]
+
+ 44%|████▎     | 435/1000 [01:07<00:54, 10.28it/s]
+
+ 44%|████▎     | 437/1000 [01:08<01:03,  8.87it/s]
+
+ 44%|████▍     | 439/1000 [01:08<01:21,  6.89it/s]
+
+ 44%|████▍     | 441/1000 [01:08<01:09,  8.00it/s]
+
+ 44%|████▍     | 442/1000 [01:09<01:25,  6.51it/s]
+
+ 44%|████▍     | 444/1000 [01:09<01:28,  6.30it/s]
+
+ 45%|████▍     | 446/1000 [01:09<01:13,  7.56it/s]
+
+ 45%|████▍     | 448/1000 [01:09<01:24,  6.50it/s]
+
+ 45%|████▌     | 450/1000 [01:10<01:29,  6.12it/s]
+
+ 45%|████▌     | 451/1000 [01:10<01:41,  5.42it/s]
+
+ 45%|████▌     | 452/1000 [01:10<01:51,  4.91it/s]
+
+ 45%|████▌     | 454/1000 [01:11<01:26,  6.30it/s]
+
+ 46%|████▌     | 456/1000 [01:11<01:12,  7.45it/s]
+
+ 46%|████▌     | 457/1000 [01:11<01:19,  6.84it/s]
+
+ 46%|████▌     | 459/1000 [01:11<01:06,  8.14it/s]
+
+ 46%|████▌     | 461/1000 [01:11<00:56,  9.50it/s]
+
+ 46%|████▋     | 463/1000 [01:12<01:20,  6.68it/s]
+
+ 46%|████▋     | 465/1000 [01:12<01:30,  5.89it/s]
+
+ 47%|████▋     | 466/1000 [01:12<01:41,  5.25it/s]
+
+ 47%|████▋     | 467/1000 [01:13<02:06,  4.20it/s]
+
+ 47%|████▋     | 469/1000 [01:13<01:36,  5.50it/s]
+
+ 47%|████▋     | 471/1000 [01:13<01:17,  6.86it/s]
+
+ 47%|████▋     | 472/1000 [01:13<01:13,  7.18it/s]
+
+ 47%|████▋     | 473/1000 [01:13<01:21,  6.48it/s]
+
+ 47%|████▋     | 474/1000 [01:14<01:37,  5.42it/s]
+
+ 48%|████▊     | 476/1000 [01:14<01:36,  5.45it/s]
+
+ 48%|████▊     | 477/1000 [01:14<01:52,  4.63it/s]
+
+ 48%|████▊     | 479/1000 [01:15<01:24,  6.20it/s]
+
+ 48%|████▊     | 480/1000 [01:15<01:31,  5.68it/s]
+
+ 48%|████▊     | 482/1000 [01:15<01:13,  7.02it/s]
+
+ 48%|████▊     | 484/1000 [01:15<01:20,  6.39it/s]
+
+ 48%|████▊     | 485/1000 [01:16<01:33,  5.51it/s]
+
+ 49%|████▊     | 486/1000 [01:16<01:33,  5.51it/s]
+
+ 49%|████▉     | 488/1000 [01:16<01:13,  7.00it/s]
+
+ 49%|████▉     | 489/1000 [01:16<01:28,  5.80it/s]
+
+ 49%|████▉     | 490/1000 [01:16<01:20,  6.33it/s]
+
+ 49%|████▉     | 492/1000 [01:17<01:10,  7.25it/s]
+
+ 49%|████▉     | 494/1000 [01:17<00:58,  8.68it/s]
+
+ 50%|████▉     | 496/1000 [01:17<01:10,  7.12it/s]
+
+ 50%|████▉     | 498/1000 [01:17<01:09,  7.21it/s]
+
+ 50%|████▉     | 499/1000 [01:18<01:27,  5.73it/s]
+
+ 50%|█████     | 500/1000 [01:18<01:39,  5.02it/s]
+
+ 50%|█████     | 502/1000 [01:18<01:17,  6.41it/s]
+
+ 50%|█████     | 503/1000 [01:18<01:25,  5.81it/s]
+
+ 50%|█████     | 504/1000 [01:19<01:18,  6.32it/s]
+
+ 50%|█████     | 505/1000 [01:19<01:32,  5.34it/s]
+
+ 51%|█████     | 506/1000 [01:19<01:44,  4.73it/s]
+
+ 51%|█████     | 508/1000 [01:19<01:16,  6.43it/s]
+
+ 51%|█████     | 510/1000 [01:19<01:02,  7.87it/s]
+
+ 51%|█████     | 512/1000 [01:20<00:55,  8.80it/s]
+
+ 51%|█████▏    | 514/1000 [01:20<01:18,  6.16it/s]
+
+ 52%|█████▏    | 515/1000 [01:20<01:14,  6.47it/s]
+
+ 52%|█████▏    | 516/1000 [01:20<01:19,  6.12it/s]
+
+ 52%|█████▏    | 518/1000 [01:21<01:08,  7.04it/s]
+
+ 52%|█████▏    | 519/1000 [01:21<01:26,  5.54it/s]
+
+ 52%|█████▏    | 521/1000 [01:21<01:08,  6.98it/s]
+
+ 52%|█████▏    | 522/1000 [01:21<01:22,  5.80it/s]
+
+ 52%|█████▏    | 523/1000 [01:22<01:36,  4.93it/s]
+
+ 52%|█████▏    | 524/1000 [01:22<01:39,  4.76it/s]
+
+ 53%|█████▎    | 526/1000 [01:22<01:23,  5.70it/s]
+
+ 53%|█████▎    | 528/1000 [01:23<01:23,  5.67it/s]
+
+ 53%|█████▎    | 529/1000 [01:23<01:24,  5.55it/s]
+
+ 53%|█████▎    | 531/1000 [01:23<01:06,  7.08it/s]
+
+ 53%|█████▎    | 533/1000 [01:23<00:55,  8.35it/s]
+
+ 53%|█████▎    | 534/1000 [01:23<01:10,  6.60it/s]
+
+ 54%|█████▎    | 535/1000 [01:24<01:23,  5.57it/s]
+
+ 54%|█████▎    | 536/1000 [01:24<01:27,  5.29it/s]
+
+ 54%|█████▎    | 537/1000 [01:24<01:38,  4.70it/s]
+
+ 54%|█████▍    | 539/1000 [01:24<01:31,  5.05it/s]
+
+ 54%|█████▍    | 540/1000 [01:25<01:33,  4.92it/s]
+
+ 54%|█████▍    | 541/1000 [01:25<01:44,  4.40it/s]
+
+ 54%|█████▍    | 543/1000 [01:25<01:16,  5.95it/s]
+
+ 55%|█████▍    | 545/1000 [01:25<01:15,  6.02it/s]
+
+ 55%|█████▍    | 546/1000 [01:26<01:21,  5.56it/s]
+
+ 55%|█████▍    | 547/1000 [01:26<01:32,  4.89it/s]
+
+ 55%|█████▍    | 549/1000 [01:26<01:29,  5.04it/s]
+
+ 55%|█████▌    | 551/1000 [01:27<01:09,  6.42it/s]
+
+ 55%|█████▌    | 552/1000 [01:27<01:11,  6.24it/s]
+
+ 55%|█████▌    | 553/1000 [01:27<01:23,  5.34it/s]
+
+ 55%|█████▌    | 554/1000 [01:27<01:28,  5.02it/s]
+
+ 56%|█████▌    | 556/1000 [01:27<01:07,  6.60it/s]
+
+ 56%|█████▌    | 557/1000 [01:28<01:13,  6.01it/s]
+
+ 56%|█████▌    | 559/1000 [01:28<00:57,  7.61it/s]
+
+ 56%|█████▌    | 561/1000 [01:28<00:50,  8.75it/s]
+
+ 56%|█████▌    | 562/1000 [01:28<00:49,  8.94it/s]
+
+ 56%|█████▋    | 563/1000 [01:28<01:05,  6.66it/s]
+
+ 56%|█████▋    | 565/1000 [01:29<01:06,  6.55it/s]
+
+ 57%|█████▋    | 566/1000 [01:29<01:18,  5.55it/s]
+
+ 57%|█████▋    | 568/1000 [01:29<01:18,  5.52it/s]
+
+ 57%|█████▋    | 569/1000 [01:29<01:18,  5.48it/s]
+
+ 57%|█████▋    | 571/1000 [01:30<01:17,  5.55it/s]
+
+ 57%|█████▋    | 572/1000 [01:30<01:18,  5.46it/s]
+
+ 57%|█████▋    | 574/1000 [01:30<01:17,  5.53it/s]
+
+ 58%|█████▊    | 576/1000 [01:31<01:01,  6.86it/s]
+
+ 58%|█████▊    | 577/1000 [01:31<01:24,  4.98it/s]
+
+ 58%|█████▊    | 578/1000 [01:31<01:32,  4.58it/s]
+
+ 58%|█████▊    | 580/1000 [01:31<01:08,  6.10it/s]
+
+ 58%|█████▊    | 582/1000 [01:32<00:55,  7.51it/s]
+
+ 58%|█████▊    | 583/1000 [01:32<01:07,  6.16it/s]
+
+ 58%|█████▊    | 585/1000 [01:32<01:06,  6.23it/s]
+
+ 59%|█████▊    | 587/1000 [01:32<01:04,  6.39it/s]
+
+ 59%|█████▉    | 589/1000 [01:33<00:54,  7.59it/s]
+
+ 59%|█████▉    | 590/1000 [01:33<01:08,  5.98it/s]
+
+ 59%|█████▉    | 592/1000 [01:33<00:56,  7.21it/s]
+
+ 59%|█████▉    | 593/1000 [01:33<01:07,  5.99it/s]
+
+ 60%|█████▉    | 595/1000 [01:34<00:54,  7.37it/s]
+
+ 60%|█████▉    | 596/1000 [01:34<01:06,  6.05it/s]
+
+ 60%|█████▉    | 597/1000 [01:34<01:08,  5.88it/s]
+
+ 60%|█████▉    | 599/1000 [01:34<01:10,  5.69it/s]
+
+ 60%|██████    | 601/1000 [01:35<01:11,  5.61it/s]
+
+ 60%|██████    | 602/1000 [01:35<01:07,  5.89it/s]
+
+ 60%|██████    | 604/1000 [01:35<01:19,  4.97it/s]
+
+ 61%|██████    | 606/1000 [01:36<01:03,  6.22it/s]
+
+ 61%|██████    | 608/1000 [01:36<00:57,  6.87it/s]
+
+ 61%|██████    | 610/1000 [01:36<00:52,  7.43it/s]
+
+ 61%|██████    | 612/1000 [01:36<00:45,  8.53it/s]
+
+ 61%|██████▏   | 613/1000 [01:36<00:44,  8.70it/s]
+
+ 62%|██████▏   | 615/1000 [01:37<00:52,  7.36it/s]
+
+ 62%|██████▏   | 616/1000 [01:37<00:58,  6.55it/s]
+
+ 62%|██████▏   | 617/1000 [01:37<00:57,  6.69it/s]
+
+ 62%|██████▏   | 619/1000 [01:37<00:47,  7.98it/s]
+
+ 62%|██████▏   | 621/1000 [01:38<00:55,  6.84it/s]
+
+ 62%|██████▏   | 623/1000 [01:38<00:53,  7.05it/s]
+
+ 62%|██████▏   | 624/1000 [01:38<00:50,  7.44it/s]
+
+ 62%|██████▎   | 625/1000 [01:38<01:02,  6.04it/s]
+
+ 63%|██████▎   | 627/1000 [01:38<00:50,  7.41it/s]
+
+ 63%|██████▎   | 628/1000 [01:39<01:01,  6.03it/s]
+
+ 63%|██████▎   | 629/1000 [01:39<01:13,  5.07it/s]
+
+ 63%|██████▎   | 631/1000 [01:39<00:55,  6.65it/s]
+
+ 63%|██████▎   | 633/1000 [01:39<00:44,  8.16it/s]
+
+ 63%|██████▎   | 634/1000 [01:40<01:08,  5.33it/s]
+
+ 64%|██████▎   | 635/1000 [01:40<01:03,  5.73it/s]
+
+ 64%|██████▎   | 636/1000 [01:40<00:59,  6.14it/s]
+
+ 64%|██████▍   | 638/1000 [01:40<00:51,  7.07it/s]
+
+ 64%|██████▍   | 640/1000 [01:40<00:44,  8.06it/s]
+
+ 64%|██████▍   | 641/1000 [01:41<00:56,  6.39it/s]
+
+ 64%|██████▍   | 643/1000 [01:41<00:53,  6.72it/s]
+
+ 64%|██████▍   | 645/1000 [01:41<00:58,  6.07it/s]
+
+ 65%|██████▍   | 647/1000 [01:42<00:54,  6.53it/s]
+
+ 65%|██████▍   | 648/1000 [01:42<01:02,  5.64it/s]
+
+ 65%|██████▌   | 650/1000 [01:42<01:02,  5.62it/s]
+
+ 65%|██████▌   | 652/1000 [01:43<01:03,  5.45it/s]
+
+ 65%|██████▌   | 654/1000 [01:43<00:52,  6.65it/s]
+
+ 66%|██████▌   | 656/1000 [01:43<00:44,  7.73it/s]
+
+ 66%|██████▌   | 657/1000 [01:43<00:53,  6.42it/s]
+
+ 66%|██████▌   | 659/1000 [01:44<01:04,  5.30it/s]
+
+ 66%|██████▌   | 661/1000 [01:44<00:51,  6.62it/s]
+
+ 66%|██████▌   | 662/1000 [01:44<00:59,  5.70it/s]
+
+ 66%|██████▋   | 664/1000 [01:45<01:06,  5.05it/s]
+
+ 66%|██████▋   | 665/1000 [01:45<01:11,  4.68it/s]
+
+ 67%|██████▋   | 667/1000 [01:45<01:09,  4.82it/s]
+
+ 67%|██████▋   | 668/1000 [01:46<01:13,  4.49it/s]
+
+ 67%|██████▋   | 669/1000 [01:46<01:06,  4.94it/s]
+
+ 67%|██████▋   | 671/1000 [01:46<01:04,  5.10it/s]
+
+ 67%|██████▋   | 673/1000 [01:46<00:58,  5.62it/s]
+
+ 68%|██████▊   | 675/1000 [01:47<00:52,  6.25it/s]
+
+ 68%|██████▊   | 677/1000 [01:47<00:43,  7.48it/s]
+
+ 68%|██████▊   | 679/1000 [01:47<00:38,  8.41it/s]
+
+ 68%|██████▊   | 681/1000 [01:47<00:33,  9.48it/s]
+
+ 68%|██████▊   | 683/1000 [01:47<00:40,  7.74it/s]
+
+ 68%|██████▊   | 685/1000 [01:48<00:36,  8.58it/s]
+
+ 69%|██████▊   | 686/1000 [01:48<00:45,  6.83it/s]
+
+ 69%|██████▊   | 687/1000 [01:48<00:55,  5.63it/s]
+
+ 69%|██████▉   | 688/1000 [01:48<00:52,  5.99it/s]
+
+ 69%|██████▉   | 690/1000 [01:49<00:53,  5.74it/s]
+
+ 69%|██████▉   | 692/1000 [01:49<00:45,  6.77it/s]
+
+ 69%|██████▉   | 693/1000 [01:49<00:42,  7.21it/s]
+
+ 69%|██████▉   | 694/1000 [01:49<00:52,  5.85it/s]
+
+ 70%|██████▉   | 695/1000 [01:49<00:49,  6.15it/s]
+
+ 70%|██████▉   | 696/1000 [01:50<00:58,  5.19it/s]
+
+ 70%|██████▉   | 698/1000 [01:50<00:51,  5.88it/s]
+
+ 70%|██████▉   | 699/1000 [01:50<00:59,  5.02it/s]
+
+ 70%|███████   | 701/1000 [01:50<00:45,  6.52it/s]
+
+ 70%|███████   | 702/1000 [01:51<01:02,  4.78it/s]
+
+ 70%|███████   | 704/1000 [01:51<00:47,  6.21it/s]
+
+ 70%|███████   | 705/1000 [01:51<01:04,  4.57it/s]
+
+ 71%|███████   | 707/1000 [01:52<00:48,  5.99it/s]
+
+ 71%|███████   | 709/1000 [01:52<00:44,  6.54it/s]
+
+ 71%|███████   | 710/1000 [01:52<00:51,  5.60it/s]
+
+ 71%|███████   | 712/1000 [01:52<00:43,  6.59it/s]
+
+ 71%|███████▏  | 713/1000 [01:52<00:42,  6.75it/s]
+
+ 72%|███████▏  | 715/1000 [01:53<00:35,  8.10it/s]
+
+ 72%|███████▏  | 716/1000 [01:53<00:44,  6.40it/s]
+
+ 72%|███████▏  | 718/1000 [01:53<00:46,  6.08it/s]
+
+ 72%|███████▏  | 719/1000 [01:54<00:54,  5.15it/s]
+
+ 72%|███████▏  | 720/1000 [01:54<00:49,  5.61it/s]
+
+ 72%|███████▏  | 722/1000 [01:54<00:39,  7.07it/s]
+
+ 72%|███████▏  | 723/1000 [01:54<00:38,  7.13it/s]
+
+ 72%|███████▏  | 724/1000 [01:54<00:38,  7.16it/s]
+
+ 72%|███████▎  | 725/1000 [01:54<00:38,  7.22it/s]
+
+ 73%|███████▎  | 727/1000 [01:54<00:31,  8.59it/s]
+
+ 73%|███████▎  | 729/1000 [01:55<00:27,  9.89it/s]
+
+ 73%|███████▎  | 731/1000 [01:55<00:37,  7.20it/s]
+
+ 73%|███████▎  | 733/1000 [01:55<00:32,  8.27it/s]
+
+ 73%|███████▎  | 734/1000 [01:55<00:32,  8.24it/s]
+
+ 74%|███████▎  | 736/1000 [01:56<00:34,  7.72it/s]
+
+ 74%|███████▍  | 738/1000 [01:56<00:31,  8.28it/s]
+
+ 74%|███████▍  | 740/1000 [01:56<00:28,  9.10it/s]
+
+ 74%|███████▍  | 741/1000 [01:56<00:36,  7.07it/s]
+
+ 74%|███████▍  | 743/1000 [01:57<00:36,  7.01it/s]
+
+ 74%|███████▍  | 745/1000 [01:57<00:30,  8.26it/s]
+
+ 75%|███████▍  | 747/1000 [01:57<00:27,  9.09it/s]
+
+ 75%|███████▍  | 748/1000 [01:57<00:35,  7.08it/s]
+
+ 75%|███████▍  | 749/1000 [01:57<00:34,  7.37it/s]
+
+ 75%|███████▌  | 750/1000 [01:57<00:38,  6.43it/s]
+
+ 75%|███████▌  | 751/1000 [01:58<00:49,  4.99it/s]
+
+ 75%|███████▌  | 753/1000 [01:58<00:40,  6.14it/s]
+
+ 75%|███████▌  | 754/1000 [01:58<00:46,  5.27it/s]
+
+ 76%|███████▌  | 756/1000 [01:59<00:44,  5.52it/s]
+
+ 76%|███████▌  | 758/1000 [01:59<00:36,  6.62it/s]
+
+ 76%|███████▌  | 759/1000 [01:59<00:38,  6.19it/s]
+
+ 76%|███████▌  | 761/1000 [01:59<00:32,  7.47it/s]
+
+ 76%|███████▋  | 763/1000 [02:00<00:35,  6.62it/s]
+
+ 76%|███████▋  | 764/1000 [02:00<00:41,  5.71it/s]
+
+ 77%|███████▋  | 766/1000 [02:00<00:42,  5.49it/s]
+
+ 77%|███████▋  | 768/1000 [02:00<00:34,  6.66it/s]
+
+ 77%|███████▋  | 769/1000 [02:01<00:40,  5.74it/s]
+
+ 77%|███████▋  | 770/1000 [02:01<00:40,  5.68it/s]
+
+ 77%|███████▋  | 771/1000 [02:01<00:36,  6.28it/s]
+
+ 77%|███████▋  | 773/1000 [02:01<00:40,  5.62it/s]
+
+ 77%|███████▋  | 774/1000 [02:02<00:37,  5.96it/s]
+
+ 78%|███████▊  | 776/1000 [02:02<00:32,  6.91it/s]
+
+ 78%|███████▊  | 778/1000 [02:02<00:26,  8.45it/s]
+
+ 78%|███████▊  | 780/1000 [02:02<00:36,  6.08it/s]
+
+ 78%|███████▊  | 782/1000 [02:03<00:29,  7.31it/s]
+
+ 78%|███████▊  | 783/1000 [02:03<00:35,  6.08it/s]
+
+ 78%|███████▊  | 784/1000 [02:03<00:34,  6.32it/s]
+
+ 79%|███████▊  | 786/1000 [02:03<00:31,  6.73it/s]
+
+ 79%|███████▉  | 788/1000 [02:03<00:28,  7.55it/s]
+
+ 79%|███████▉  | 790/1000 [02:04<00:25,  8.11it/s]
+
+ 79%|███████▉  | 791/1000 [02:04<00:28,  7.28it/s]
+
+ 79%|███████▉  | 792/1000 [02:04<00:30,  6.83it/s]
+
+ 79%|███████▉  | 793/1000 [02:04<00:29,  7.06it/s]
+
+ 80%|███████▉  | 795/1000 [02:05<00:32,  6.34it/s]
+
+ 80%|███████▉  | 797/1000 [02:05<00:33,  6.02it/s]
+
+ 80%|███████▉  | 798/1000 [02:05<00:38,  5.28it/s]
+
+ 80%|████████  | 800/1000 [02:05<00:33,  5.93it/s]
+
+ 80%|████████  | 802/1000 [02:06<00:29,  6.74it/s]
+
+ 80%|████████  | 804/1000 [02:06<00:28,  6.88it/s]
+
+ 81%|████████  | 806/1000 [02:06<00:24,  8.05it/s]
+
+ 81%|████████  | 807/1000 [02:06<00:24,  7.87it/s]
+
+ 81%|████████  | 809/1000 [02:06<00:21,  8.97it/s]
+
+ 81%|████████  | 811/1000 [02:07<00:19,  9.88it/s]
+
+ 81%|████████▏ | 813/1000 [02:07<00:23,  7.97it/s]
+
+ 81%|████████▏ | 814/1000 [02:07<00:23,  7.82it/s]
+
+ 82%|████████▏ | 816/1000 [02:07<00:21,  8.75it/s]
+
+ 82%|████████▏ | 817/1000 [02:08<00:26,  6.85it/s]
+
+ 82%|████████▏ | 819/1000 [02:08<00:22,  7.99it/s]
+
+ 82%|████████▏ | 820/1000 [02:08<00:28,  6.37it/s]
+
+ 82%|████████▏ | 821/1000 [02:08<00:26,  6.86it/s]
+
+ 82%|████████▏ | 823/1000 [02:08<00:26,  6.66it/s]
+
+ 82%|████████▏ | 824/1000 [02:09<00:31,  5.60it/s]
+
+ 82%|████████▎ | 825/1000 [02:09<00:29,  5.91it/s]
+
+ 83%|████████▎ | 826/1000 [02:09<00:36,  4.71it/s]
+
+ 83%|████████▎ | 828/1000 [02:10<00:39,  4.36it/s]
+
+ 83%|████████▎ | 829/1000 [02:10<00:35,  4.84it/s]
+
+ 83%|████████▎ | 831/1000 [02:10<00:28,  6.02it/s]
+
+ 83%|████████▎ | 832/1000 [02:10<00:34,  4.92it/s]
+
+ 83%|████████▎ | 833/1000 [02:10<00:31,  5.22it/s]
+
+ 83%|████████▎ | 834/1000 [02:11<00:32,  5.11it/s]
+
+ 84%|████████▎ | 835/1000 [02:11<00:29,  5.57it/s]
+
+ 84%|████████▎ | 836/1000 [02:11<00:31,  5.23it/s]
+
+ 84%|████████▍ | 838/1000 [02:11<00:25,  6.45it/s]
+
+ 84%|████████▍ | 839/1000 [02:11<00:23,  6.89it/s]
+
+ 84%|████████▍ | 840/1000 [02:12<00:30,  5.28it/s]
+
+ 84%|████████▍ | 842/1000 [02:12<00:22,  6.91it/s]
+
+ 84%|████████▍ | 844/1000 [02:12<00:28,  5.38it/s]
+
+ 84%|████████▍ | 845/1000 [02:13<00:32,  4.75it/s]
+
+ 85%|████████▍ | 847/1000 [02:13<00:29,  5.25it/s]
+
+ 85%|████████▍ | 849/1000 [02:13<00:23,  6.43it/s]
+
+ 85%|████████▌ | 850/1000 [02:13<00:27,  5.49it/s]
+
+ 85%|████████▌ | 851/1000 [02:14<00:25,  5.78it/s]
+
+ 85%|████████▌ | 853/1000 [02:14<00:20,  7.24it/s]
+
+ 86%|████████▌ | 855/1000 [02:14<00:16,  8.67it/s]
+
+ 86%|████████▌ | 856/1000 [02:14<00:16,  8.52it/s]
+
+ 86%|████████▌ | 858/1000 [02:14<00:20,  7.07it/s]
+
+ 86%|████████▌ | 860/1000 [02:15<00:17,  8.18it/s]
+
+ 86%|████████▌ | 862/1000 [02:15<00:15,  9.18it/s]
+
+ 86%|████████▋ | 864/1000 [02:15<00:13, 10.32it/s]
+
+ 87%|████████▋ | 866/1000 [02:15<00:12, 10.44it/s]
+
+ 87%|████████▋ | 868/1000 [02:15<00:16,  8.16it/s]
+
+ 87%|████████▋ | 870/1000 [02:16<00:14,  8.96it/s]
+
+ 87%|████████▋ | 872/1000 [02:16<00:15,  8.04it/s]
+
+ 87%|████████▋ | 873/1000 [02:16<00:19,  6.48it/s]
+
+ 87%|████████▋ | 874/1000 [02:16<00:21,  5.88it/s]
+
+ 88%|████████▊ | 875/1000 [02:17<00:19,  6.43it/s]
+
+ 88%|████████▊ | 877/1000 [02:17<00:15,  7.92it/s]
+
+ 88%|████████▊ | 878/1000 [02:17<00:19,  6.24it/s]
+
+ 88%|████████▊ | 879/1000 [02:17<00:21,  5.67it/s]
+
+ 88%|████████▊ | 881/1000 [02:17<00:17,  6.64it/s]
+
+ 88%|████████▊ | 882/1000 [02:18<00:21,  5.48it/s]
+
+ 88%|████████▊ | 884/1000 [02:18<00:17,  6.80it/s]
+
+ 89%|████████▊ | 886/1000 [02:18<00:14,  8.01it/s]
+
+ 89%|████████▉ | 888/1000 [02:18<00:12,  9.13it/s]
+
+ 89%|████████▉ | 889/1000 [02:18<00:14,  7.92it/s]
+
+ 89%|████████▉ | 891/1000 [02:19<00:11,  9.14it/s]
+
+ 89%|████████▉ | 893/1000 [02:19<00:13,  8.07it/s]
+
+ 89%|████████▉ | 894/1000 [02:19<00:16,  6.30it/s]
+
+ 90%|████████▉ | 896/1000 [02:19<00:13,  7.62it/s]
+
+ 90%|████████▉ | 897/1000 [02:20<00:16,  6.20it/s]
+
+ 90%|████████▉ | 898/1000 [02:20<00:19,  5.27it/s]
+
+ 90%|█████████ | 900/1000 [02:20<00:15,  6.65it/s]
+
+ 90%|█████████ | 901/1000 [02:20<00:14,  6.67it/s]
+
+ 90%|█████████ | 902/1000 [02:20<00:14,  6.65it/s]
+
+ 90%|█████████ | 904/1000 [02:21<00:11,  8.16it/s]
+
+ 90%|█████████ | 905/1000 [02:21<00:13,  7.01it/s]
+
+ 91%|█████████ | 906/1000 [02:21<00:17,  5.50it/s]
+
+ 91%|█████████ | 908/1000 [02:21<00:12,  7.16it/s]
+
+ 91%|█████████ | 910/1000 [02:22<00:13,  6.46it/s]
+
+ 91%|█████████ | 912/1000 [02:22<00:11,  7.70it/s]
+
+ 91%|█████████▏| 913/1000 [02:22<00:14,  5.85it/s]
+
+ 92%|█████████▏| 915/1000 [02:22<00:11,  7.15it/s]
+
+ 92%|█████████▏| 916/1000 [02:23<00:14,  5.96it/s]
+
+ 92%|█████████▏| 917/1000 [02:23<00:13,  6.33it/s]
+
+ 92%|█████████▏| 918/1000 [02:23<00:18,  4.49it/s]
+
+ 92%|█████████▏| 919/1000 [02:23<00:17,  4.69it/s]
+
+ 92%|█████████▏| 921/1000 [02:24<00:16,  4.67it/s]
+
+ 92%|█████████▏| 922/1000 [02:24<00:15,  4.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:24<00:12,  6.04it/s]
+
+ 93%|█████████▎| 926/1000 [02:24<00:09,  7.41it/s]
+
+ 93%|█████████▎| 927/1000 [02:24<00:10,  6.97it/s]
+
+ 93%|█████████▎| 928/1000 [02:25<00:12,  5.74it/s]
+
+ 93%|█████████▎| 930/1000 [02:25<00:12,  5.52it/s]
+
+ 93%|█████████▎| 931/1000 [02:25<00:11,  5.86it/s]
+
+ 93%|█████████▎| 933/1000 [02:26<00:11,  5.75it/s]
+
+ 93%|█████████▎| 934/1000 [02:26<00:12,  5.37it/s]
+
+ 94%|█████████▎| 936/1000 [02:26<00:10,  6.38it/s]
+
+ 94%|█████████▍| 938/1000 [02:26<00:08,  7.74it/s]
+
+ 94%|█████████▍| 939/1000 [02:26<00:08,  7.05it/s]
+
+ 94%|█████████▍| 940/1000 [02:27<00:08,  7.15it/s]
+
+ 94%|█████████▍| 941/1000 [02:27<00:07,  7.49it/s]
+
+ 94%|█████████▍| 942/1000 [02:27<00:11,  4.96it/s]
+
+ 94%|█████████▍| 944/1000 [02:27<00:09,  5.70it/s]
+
+ 95%|█████████▍| 946/1000 [02:28<00:07,  7.05it/s]
+
+ 95%|█████████▍| 948/1000 [02:28<00:06,  8.36it/s]
+
+ 95%|█████████▍| 949/1000 [02:28<00:07,  7.12it/s]
+
+ 95%|█████████▌| 951/1000 [02:28<00:06,  8.03it/s]
+
+ 95%|█████████▌| 952/1000 [02:28<00:08,  5.92it/s]
+
+ 95%|█████████▌| 954/1000 [02:29<00:09,  4.94it/s]
+
+ 96%|█████████▌| 956/1000 [02:29<00:07,  6.20it/s]
+
+ 96%|█████████▌| 957/1000 [02:29<00:08,  5.23it/s]
+
+ 96%|█████████▌| 959/1000 [02:30<00:08,  4.81it/s]
+
+ 96%|█████████▌| 961/1000 [02:30<00:06,  5.76it/s]
+
+ 96%|█████████▌| 962/1000 [02:30<00:07,  5.15it/s]
+
+ 96%|█████████▋| 964/1000 [02:31<00:05,  6.28it/s]
+
+ 96%|█████████▋| 965/1000 [02:31<00:06,  5.47it/s]
+
+ 97%|█████████▋| 966/1000 [02:31<00:06,  5.40it/s]
+
+ 97%|█████████▋| 968/1000 [02:31<00:04,  6.90it/s]
+
+ 97%|█████████▋| 970/1000 [02:31<00:03,  8.15it/s]
+
+ 97%|█████████▋| 971/1000 [02:32<00:04,  6.28it/s]
+
+ 97%|█████████▋| 973/1000 [02:32<00:03,  7.60it/s]
+
+ 98%|█████████▊| 975/1000 [02:32<00:03,  8.08it/s]
+
+ 98%|█████████▊| 977/1000 [02:32<00:02,  8.96it/s]
+
+ 98%|█████████▊| 979/1000 [02:32<00:02,  9.06it/s]
+
+ 98%|█████████▊| 980/1000 [02:33<00:02,  7.03it/s]
+
+ 98%|█████████▊| 981/1000 [02:33<00:03,  5.84it/s]
+
+ 98%|█████████▊| 982/1000 [02:33<00:04,  4.45it/s]
+
+ 98%|█████████▊| 984/1000 [02:34<00:02,  5.76it/s]
+
+ 99%|█████████▊| 986/1000 [02:34<00:01,  7.08it/s]
+
+ 99%|█████████▊| 987/1000 [02:34<00:01,  6.64it/s]
+
+ 99%|█████████▉| 988/1000 [02:34<00:02,  5.52it/s]
+
+ 99%|█████████▉| 989/1000 [02:35<00:02,  4.86it/s]
+
+ 99%|█████████▉| 991/1000 [02:35<00:01,  5.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:35<00:01,  4.13it/s]
+
+ 99%|█████████▉| 994/1000 [02:35<00:01,  5.48it/s]
+
+100%|█████████▉| 995/1000 [02:36<00:01,  4.64it/s]
+
+100%|█████████▉| 997/1000 [02:36<00:00,  6.03it/s]
+
+100%|█████████▉| 999/1000 [02:36<00:00,  5.81it/s]
+
+100%|██████████| 1000/1000 [02:37<00:00,  5.61it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7404', 'eval_runtime': '157.2', 'eval_samples_per_second': '6.36', 'eval_steps_per_second': '6.36', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:30:33<00:00,  6.40s/it]
+
+100%|██████████| 1000/1000 [02:37<00:00,  5.61it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_003_lr_2.000e-04/checkpoint-1813
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+
+Training completed. Do not forget to share your model on huggingface.co/models =)
+
+
+
+                                                     
+{'train_runtime': '1.263e+04', 'train_samples_per_second': '2.295', 'train_steps_per_second': '0.143', 'train_loss': '0.952', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:30:34<00:00,  6.40s/it]
+100%|██████████| 1813/1813 [3:30:34<00:00,  6.97s/it]
+Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_003_lr_2.000e-04
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+***** train metrics *****
+  epoch                    =         1.0
+  num_input_tokens_seen    =    12201816
+  total_flos               = 487685654GF
+  train_loss               =       0.952
+  train_runtime            =  3:30:34.35
+  train_samples_per_second =       2.295
+  train_steps_per_second   =       0.143
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_003_lr_2.000e-04/training_loss.png
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_003_lr_2.000e-04/training_eval_loss.png
+[WARNING|2026-10-13 09:55:10] llamafactory.extras.ploting:149 >> No metric eval_accuracy to plot.
+
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+  0%|          | 3/1000 [00:00<01:14, 13.35it/s]
+  0%|          | 5/1000 [00:00<02:37,  6.30it/s]
+  1%|          | 7/1000 [00:00<02:16,  7.30it/s]
+  1%|          | 9/1000 [00:01<02:56,  5.63it/s]
+  1%|          | 11/1000 [00:01<02:19,  7.09it/s]
+  1%|          | 12/1000 [00:01<02:17,  7.17it/s]
+  1%|▏         | 13/1000 [00:01<02:48,  5.87it/s]
+  1%|▏         | 14/1000 [00:02<02:56,  5.60it/s]
+  2%|▏         | 15/1000 [00:02<03:21,  4.88it/s]
+  2%|▏         | 16/1000 [00:02<03:26,  4.76it/s]
+  2%|▏         | 18/1000 [00:03<03:15,  5.03it/s]
+  2%|▏         | 19/1000 [00:03<03:48,  4.29it/s]
+  2%|▏         | 20/1000 [00:03<04:14,  3.85it/s]
+  2%|▏         | 22/1000 [00:03<03:03,  5.33it/s]
+  2%|▏         | 23/1000 [00:04<03:29,  4.66it/s]
+  2%|▎         | 25/1000 [00:04<02:36,  6.23it/s]
+  3%|▎         | 26/1000 [00:04<03:01,  5.36it/s]
+  3%|▎         | 28/1000 [00:05<03:07,  5.17it/s]
+  3%|▎         | 29/1000 [00:05<02:57,  5.49it/s]
+  3%|▎         | 30/1000 [00:05<03:12,  5.05it/s]
+  3%|▎         | 31/1000 [00:05<03:31,  4.58it/s]
+  3%|▎         | 32/1000 [00:05<03:07,  5.15it/s]
+  3%|▎         | 33/1000 [00:06<03:19,  4.86it/s]
+  4%|▎         | 35/1000 [00:06<02:27,  6.55it/s]
+  4%|▎         | 37/1000 [00:06<02:03,  7.81it/s]
+  4%|▍         | 39/1000 [00:06<02:30,  6.39it/s]
+  4%|▍         | 40/1000 [00:06<02:26,  6.57it/s]
+  4%|▍         | 42/1000 [00:07<02:02,  7.80it/s]
+  4%|▍         | 44/1000 [00:07<01:48,  8.81it/s]
+  4%|▍         | 45/1000 [00:07<02:12,  7.20it/s]
+  5%|▍         | 47/1000 [00:07<02:27,  6.46it/s]
+  5%|▍         | 49/1000 [00:08<02:04,  7.64it/s]
+  5%|▌         | 50/1000 [00:08<01:58,  8.00it/s]
+  5%|▌         | 52/1000 [00:08<01:44,  9.10it/s]
+  5%|▌         | 54/1000 [00:08<01:36,  9.79it/s]
+  6%|▌         | 56/1000 [00:08<02:05,  7.55it/s]
+  6%|▌         | 58/1000 [00:09<01:57,  7.99it/s]
+  6%|▌         | 59/1000 [00:09<02:14,  6.99it/s]
+  6%|▌         | 61/1000 [00:09<01:54,  8.20it/s]
+  6%|▌         | 62/1000 [00:09<01:51,  8.43it/s]
+  6%|▋         | 63/1000 [00:09<01:52,  8.30it/s]
+  6%|▋         | 64/1000 [00:10<02:28,  6.30it/s]
+  7%|▋         | 66/1000 [00:10<02:02,  7.63it/s]
+  7%|▋         | 67/1000 [00:10<02:34,  6.03it/s]
+  7%|▋         | 68/1000 [00:10<02:20,  6.64it/s]
+  7%|▋         | 69/1000 [00:10<02:15,  6.87it/s]
+  7%|▋         | 70/1000 [00:11<02:49,  5.50it/s]
+  7%|▋         | 72/1000 [00:11<02:49,  5.47it/s]
+  7%|▋         | 74/1000 [00:11<02:16,  6.79it/s]
+  8%|▊         | 75/1000 [00:11<02:41,  5.74it/s]
+  8%|▊         | 76/1000 [00:12<02:44,  5.60it/s]
+  8%|▊         | 77/1000 [00:12<02:33,  6.01it/s]
+  8%|▊         | 78/1000 [00:12<03:01,  5.09it/s]
+  8%|▊         | 79/1000 [00:12<02:36,  5.87it/s]
+  8%|▊         | 81/1000 [00:12<02:00,  7.61it/s]
+  8%|▊         | 83/1000 [00:13<02:23,  6.40it/s]
+  8%|▊         | 85/1000 [00:13<02:30,  6.08it/s]
+  9%|▊         | 86/1000 [00:13<02:51,  5.33it/s]
+  9%|▊         | 87/1000 [00:14<03:10,  4.80it/s]
+  9%|▉         | 88/1000 [00:14<02:48,  5.41it/s]
+  9%|▉         | 90/1000 [00:14<02:21,  6.43it/s]
+  9%|▉         | 91/1000 [00:14<02:48,  5.40it/s]
+  9%|▉         | 92/1000 [00:14<02:46,  5.44it/s]
+  9%|▉         | 93/1000 [00:15<03:03,  4.95it/s]
+  9%|▉         | 94/1000 [00:15<02:40,  5.65it/s]
+ 10%|▉         | 95/1000 [00:15<03:18,  4.55it/s]
+ 10%|▉         | 97/1000 [00:15<02:32,  5.91it/s]
+ 10%|▉         | 99/1000 [00:15<02:03,  7.28it/s]
+ 10%|█         | 101/1000 [00:16<01:46,  8.45it/s]
+ 10%|█         | 102/1000 [00:16<02:15,  6.61it/s]
+ 10%|█         | 103/1000 [00:16<02:12,  6.77it/s]
+ 10%|█         | 104/1000 [00:16<02:10,  6.86it/s]
+ 10%|█         | 105/1000 [00:16<02:03,  7.27it/s]
+ 11%|█         | 107/1000 [00:16<01:38,  9.06it/s]
+ 11%|█         | 109/1000 [00:17<01:29,  9.98it/s]
+ 11%|█         | 111/1000 [00:17<01:24, 10.49it/s]
+ 11%|█▏        | 113/1000 [00:17<01:19, 11.17it/s]
+ 12%|█▏        | 115/1000 [00:17<01:17, 11.47it/s]
+ 12%|█▏        | 117/1000 [00:17<01:15, 11.64it/s]
+ 12%|█▏        | 119/1000 [00:18<02:07,  6.89it/s]
+ 12%|█▏        | 120/1000 [00:18<02:28,  5.91it/s]
+ 12%|█▏        | 122/1000 [00:18<02:04,  7.06it/s]
+ 12%|█▏        | 124/1000 [00:19<02:18,  6.32it/s]
+ 12%|█▎        | 125/1000 [00:19<03:02,  4.80it/s]
+ 13%|█▎        | 126/1000 [00:19<02:48,  5.20it/s]
+ 13%|█▎        | 127/1000 [00:19<03:00,  4.83it/s]
+ 13%|█▎        | 128/1000 [00:20<03:15,  4.45it/s]
+ 13%|█▎        | 130/1000 [00:20<02:26,  5.94it/s]
+ 13%|█▎        | 132/1000 [00:20<01:58,  7.35it/s]
+ 13%|█▎        | 134/1000 [00:20<01:39,  8.67it/s]
+ 14%|█▎        | 135/1000 [00:20<01:51,  7.79it/s]
+ 14%|█▎        | 137/1000 [00:21<01:45,  8.17it/s]
+ 14%|█▍        | 139/1000 [00:21<01:45,  8.20it/s]
+ 14%|█▍        | 141/1000 [00:21<01:43,  8.32it/s]
+ 14%|█▍        | 142/1000 [00:21<01:50,  7.73it/s]
+ 14%|█▍        | 143/1000 [00:21<01:52,  7.60it/s]
+ 14%|█▍        | 145/1000 [00:22<01:37,  8.79it/s]
+ 15%|█▍        | 147/1000 [00:22<02:00,  7.08it/s]
+ 15%|█▍        | 148/1000 [00:22<01:57,  7.22it/s]
+ 15%|█▌        | 150/1000 [00:22<01:41,  8.36it/s]
+ 15%|█▌        | 152/1000 [00:22<01:31,  9.25it/s]
+ 15%|█▌        | 154/1000 [00:23<01:36,  8.77it/s]
+ 16%|█▌        | 155/1000 [00:23<02:07,  6.63it/s]
+ 16%|█▌        | 156/1000 [00:23<02:40,  5.26it/s]
+ 16%|█▌        | 158/1000 [00:23<02:05,  6.71it/s]
+ 16%|█▌        | 159/1000 [00:24<01:58,  7.09it/s]
+ 16%|█▌        | 160/1000 [00:24<01:55,  7.27it/s]
+ 16%|█▌        | 162/1000 [00:24<02:08,  6.53it/s]
+ 16%|█▋        | 163/1000 [00:24<02:12,  6.31it/s]
+ 16%|█▋        | 164/1000 [00:25<02:37,  5.32it/s]
+ 16%|█▋        | 165/1000 [00:25<03:02,  4.59it/s]
+ 17%|█▋        | 166/1000 [00:25<03:15,  4.26it/s]
+ 17%|█▋        | 167/1000 [00:25<02:51,  4.85it/s]
+ 17%|█▋        | 169/1000 [00:25<02:03,  6.73it/s]
+ 17%|█▋        | 171/1000 [00:26<02:17,  6.01it/s]
+ 17%|█▋        | 172/1000 [00:26<02:38,  5.21it/s]
+ 17%|█▋        | 173/1000 [00:26<02:35,  5.33it/s]
+ 17%|█▋        | 174/1000 [00:26<02:22,  5.79it/s]
+ 18%|█▊        | 176/1000 [00:27<01:53,  7.29it/s]
+ 18%|█▊        | 177/1000 [00:27<02:19,  5.91it/s]
+ 18%|█▊        | 179/1000 [00:27<01:49,  7.49it/s]
+ 18%|█▊        | 180/1000 [00:27<01:49,  7.47it/s]
+ 18%|█▊        | 181/1000 [00:27<02:22,  5.75it/s]
+ 18%|█▊        | 182/1000 [00:28<02:46,  4.90it/s]
+ 18%|█▊        | 183/1000 [00:28<02:44,  4.98it/s]
+ 18%|█▊        | 185/1000 [00:28<02:03,  6.59it/s]
+ 19%|█▊        | 186/1000 [00:28<02:01,  6.73it/s]
+ 19%|█▊        | 187/1000 [00:29<02:27,  5.50it/s]
+ 19%|█▉        | 189/1000 [00:29<01:56,  6.99it/s]
+ 19%|█▉        | 190/1000 [00:29<02:20,  5.76it/s]
+ 19%|█▉        | 192/1000 [00:29<02:25,  5.55it/s]
+ 19%|█▉        | 194/1000 [00:30<02:26,  5.50it/s]
+ 20%|█▉        | 196/1000 [00:30<02:00,  6.67it/s]
+ 20%|█▉        | 198/1000 [00:30<01:40,  7.95it/s]
+ 20%|██        | 200/1000 [00:30<01:28,  9.05it/s]
+ 20%|██        | 202/1000 [00:31<01:45,  7.54it/s]
+ 20%|██        | 204/1000 [00:31<01:50,  7.22it/s]
+ 21%|██        | 206/1000 [00:31<01:36,  8.20it/s]
+ 21%|██        | 207/1000 [00:31<01:58,  6.67it/s]
+ 21%|██        | 208/1000 [00:31<01:52,  7.04it/s]
+ 21%|██        | 210/1000 [00:32<01:55,  6.82it/s]
+ 21%|██        | 212/1000 [00:32<01:38,  7.98it/s]
+ 21%|██▏       | 213/1000 [00:32<01:48,  7.27it/s]
+ 21%|██▏       | 214/1000 [00:32<01:48,  7.27it/s]
+ 22%|██▏       | 216/1000 [00:32<01:39,  7.86it/s]
+ 22%|██▏       | 217/1000 [00:33<02:05,  6.24it/s]
+ 22%|██▏       | 219/1000 [00:33<01:43,  7.57it/s]
+ 22%|██▏       | 220/1000 [00:33<02:27,  5.30it/s]
+ 22%|██▏       | 221/1000 [00:33<02:29,  5.21it/s]
+ 22%|██▏       | 223/1000 [00:34<02:46,  4.66it/s]
+ 22%|██▏       | 224/1000 [00:34<02:31,  5.12it/s]
+ 22%|██▎       | 225/1000 [00:34<02:29,  5.19it/s]
+ 23%|██▎       | 226/1000 [00:35<02:46,  4.66it/s]
+ 23%|██▎       | 227/1000 [00:35<02:22,  5.41it/s]
+ 23%|██▎       | 228/1000 [00:35<02:11,  5.89it/s]
+ 23%|██▎       | 229/1000 [00:35<02:35,  4.97it/s]
+ 23%|██▎       | 230/1000 [00:35<02:20,  5.49it/s]
+ 23%|██▎       | 232/1000 [00:35<01:51,  6.87it/s]
+ 23%|██▎       | 233/1000 [00:36<01:48,  7.10it/s]
+ 24%|██▎       | 235/1000 [00:36<01:27,  8.76it/s]
+ 24%|██▎       | 236/1000 [00:36<01:29,  8.58it/s]
+ 24%|██▎       | 237/1000 [00:36<01:32,  8.22it/s]
+ 24%|██▍       | 239/1000 [00:36<01:41,  7.47it/s]
+ 24%|██▍       | 241/1000 [00:37<01:48,  7.02it/s]
+ 24%|██▍       | 242/1000 [00:37<02:32,  4.98it/s]
+ 24%|██▍       | 243/1000 [00:37<02:49,  4.46it/s]
+ 24%|██▍       | 245/1000 [00:38<02:15,  5.58it/s]
+ 25%|██▍       | 247/1000 [00:38<01:47,  7.03it/s]
+ 25%|██▍       | 248/1000 [00:38<02:07,  5.91it/s]
+ 25%|██▍       | 249/1000 [00:38<02:29,  5.02it/s]
+ 25%|██▌       | 250/1000 [00:38<02:30,  4.98it/s]
+ 25%|██▌       | 252/1000 [00:39<02:23,  5.20it/s]
+ 25%|██▌       | 253/1000 [00:39<02:30,  4.96it/s]
+ 26%|██▌       | 255/1000 [00:39<01:55,  6.42it/s]
+ 26%|██▌       | 256/1000 [00:39<01:51,  6.70it/s]
+ 26%|██▌       | 258/1000 [00:40<01:37,  7.58it/s]
+ 26%|██▌       | 260/1000 [00:40<01:50,  6.71it/s]
+ 26%|██▌       | 261/1000 [00:40<02:12,  5.56it/s]
+ 26%|██▌       | 262/1000 [00:40<02:13,  5.53it/s]
+ 26%|██▋       | 264/1000 [00:41<02:14,  5.49it/s]
+ 26%|██▋       | 265/1000 [00:41<02:08,  5.73it/s]
+ 27%|██▋       | 266/1000 [00:41<02:06,  5.80it/s]
+ 27%|██▋       | 268/1000 [00:41<01:53,  6.44it/s]
+ 27%|██▋       | 270/1000 [00:42<02:00,  6.06it/s]
+ 27%|██▋       | 271/1000 [00:42<02:38,  4.59it/s]
+ 27%|██▋       | 273/1000 [00:43<02:29,  4.88it/s]
+ 27%|██▋       | 274/1000 [00:43<02:14,  5.38it/s]
+ 28%|██▊       | 275/1000 [00:43<02:07,  5.71it/s]
+ 28%|██▊       | 277/1000 [00:43<01:40,  7.21it/s]
+ 28%|██▊       | 278/1000 [00:43<02:02,  5.91it/s]
+ 28%|██▊       | 280/1000 [00:44<02:04,  5.76it/s]
+ 28%|██▊       | 282/1000 [00:44<01:43,  6.92it/s]
+ 28%|██▊       | 283/1000 [00:44<02:24,  4.97it/s]
+ 28%|██▊       | 284/1000 [00:44<02:28,  4.81it/s]
+ 29%|██▊       | 286/1000 [00:45<01:52,  6.34it/s]
+ 29%|██▉       | 288/1000 [00:45<01:37,  7.34it/s]
+ 29%|██▉       | 290/1000 [00:45<01:47,  6.63it/s]
+ 29%|██▉       | 291/1000 [00:45<02:01,  5.84it/s]
+ 29%|██▉       | 292/1000 [00:46<02:18,  5.11it/s]
+ 29%|██▉       | 294/1000 [00:46<01:55,  6.11it/s]
+ 30%|██▉       | 296/1000 [00:46<02:01,  5.80it/s]
+ 30%|██▉       | 298/1000 [00:46<01:41,  6.95it/s]
+ 30%|██▉       | 299/1000 [00:47<02:06,  5.52it/s]
+ 30%|███       | 300/1000 [00:47<01:59,  5.86it/s]
+ 30%|███       | 301/1000 [00:47<01:53,  6.16it/s]
+ 30%|███       | 303/1000 [00:47<01:31,  7.64it/s]
+ 30%|███       | 304/1000 [00:47<01:38,  7.07it/s]
+ 30%|███       | 305/1000 [00:48<01:37,  7.12it/s]
+ 31%|███       | 306/1000 [00:48<01:54,  6.05it/s]
+ 31%|███       | 307/1000 [00:48<01:57,  5.90it/s]
+ 31%|███       | 309/1000 [00:48<01:30,  7.66it/s]
+ 31%|███       | 311/1000 [00:48<01:16,  8.96it/s]
+ 31%|███▏      | 313/1000 [00:48<01:09,  9.91it/s]
+ 32%|███▏      | 315/1000 [00:49<01:15,  9.02it/s]
+ 32%|███▏      | 316/1000 [00:49<01:25,  8.03it/s]
+ 32%|███▏      | 318/1000 [00:49<01:15,  9.01it/s]
+ 32%|███▏      | 319/1000 [00:49<01:55,  5.92it/s]
+ 32%|███▏      | 321/1000 [00:50<02:01,  5.57it/s]
+ 32%|███▏      | 322/1000 [00:50<02:09,  5.25it/s]
+ 32%|███▏      | 324/1000 [00:50<01:43,  6.54it/s]
+ 32%|███▎      | 325/1000 [00:51<02:23,  4.69it/s]
+ 33%|███▎      | 327/1000 [00:51<01:47,  6.24it/s]
+ 33%|███▎      | 328/1000 [00:51<01:50,  6.09it/s]
+ 33%|███▎      | 330/1000 [00:51<01:54,  5.84it/s]
+ 33%|███▎      | 332/1000 [00:52<02:00,  5.57it/s]
+ 33%|███▎      | 333/1000 [00:52<01:52,  5.95it/s]
+ 33%|███▎      | 334/1000 [00:52<02:01,  5.47it/s]
+ 34%|███▎      | 336/1000 [00:52<01:35,  6.96it/s]
+ 34%|███▍      | 338/1000 [00:52<01:26,  7.69it/s]
+ 34%|███▍      | 340/1000 [00:53<01:15,  8.78it/s]
+ 34%|███▍      | 342/1000 [00:53<01:08,  9.57it/s]
+ 34%|███▍      | 344/1000 [00:53<01:40,  6.52it/s]
+ 34%|███▍      | 345/1000 [00:53<01:38,  6.62it/s]
+ 35%|███▍      | 346/1000 [00:54<01:44,  6.24it/s]
+ 35%|███▍      | 347/1000 [00:54<01:44,  6.22it/s]
+ 35%|███▍      | 349/1000 [00:54<01:50,  5.91it/s]
+ 35%|███▌      | 350/1000 [00:54<02:07,  5.12it/s]
+ 35%|███▌      | 352/1000 [00:55<01:38,  6.60it/s]
+ 35%|███▌      | 354/1000 [00:55<01:40,  6.44it/s]
+ 36%|███▌      | 355/1000 [00:55<01:42,  6.31it/s]
+ 36%|███▌      | 357/1000 [00:55<01:26,  7.43it/s]
+ 36%|███▌      | 359/1000 [00:55<01:12,  8.86it/s]
+ 36%|███▌      | 360/1000 [00:56<01:25,  7.51it/s]
+ 36%|███▌      | 362/1000 [00:56<01:13,  8.67it/s]
+ 36%|███▋      | 363/1000 [00:56<01:53,  5.62it/s]
+ 36%|███▋      | 364/1000 [00:57<02:07,  4.98it/s]
+ 36%|███▋      | 365/1000 [00:57<02:43,  3.89it/s]
+ 37%|███▋      | 367/1000 [00:57<02:25,  4.36it/s]
+ 37%|███▋      | 368/1000 [00:58<02:31,  4.16it/s]
+ 37%|███▋      | 369/1000 [00:58<02:13,  4.71it/s]
+ 37%|███▋      | 371/1000 [00:58<01:36,  6.53it/s]
+ 37%|███▋      | 372/1000 [00:58<01:34,  6.65it/s]
+ 37%|███▋      | 374/1000 [00:58<01:30,  6.92it/s]
+ 38%|███▊      | 375/1000 [00:59<01:57,  5.30it/s]
+ 38%|███▊      | 376/1000 [00:59<02:15,  4.60it/s]
+ 38%|███▊      | 377/1000 [00:59<01:59,  5.23it/s]
+ 38%|███▊      | 379/1000 [00:59<01:28,  7.06it/s]
+ 38%|███▊      | 381/1000 [00:59<01:17,  7.99it/s]
+ 38%|███▊      | 382/1000 [01:00<01:29,  6.87it/s]
+ 38%|███▊      | 384/1000 [01:00<01:14,  8.22it/s]
+ 38%|███▊      | 385/1000 [01:00<01:16,  7.99it/s]
+ 39%|███▊      | 386/1000 [01:00<01:17,  7.88it/s]
+ 39%|███▊      | 387/1000 [01:00<01:43,  5.91it/s]
+ 39%|███▉      | 388/1000 [01:01<01:45,  5.79it/s]
+ 39%|███▉      | 389/1000 [01:01<01:35,  6.39it/s]
+ 39%|███▉      | 390/1000 [01:01<01:36,  6.33it/s]
+ 39%|███▉      | 391/1000 [01:01<01:40,  6.07it/s]
+ 39%|███▉      | 393/1000 [01:01<01:17,  7.88it/s]
+ 40%|███▉      | 395/1000 [01:02<01:34,  6.40it/s]
+ 40%|███▉      | 397/1000 [01:02<01:18,  7.71it/s]
+ 40%|███▉      | 398/1000 [01:02<01:36,  6.23it/s]
+ 40%|███▉      | 399/1000 [01:02<02:01,  4.96it/s]
+ 40%|████      | 400/1000 [01:03<02:12,  4.53it/s]
+ 40%|████      | 402/1000 [01:03<02:04,  4.81it/s]
+ 40%|████      | 404/1000 [01:03<01:37,  6.13it/s]
+ 41%|████      | 406/1000 [01:03<01:20,  7.35it/s]
+ 41%|████      | 407/1000 [01:04<01:37,  6.09it/s]
+ 41%|████      | 408/1000 [01:04<01:51,  5.31it/s]
+ 41%|████      | 410/1000 [01:04<01:50,  5.35it/s]
+ 41%|████      | 411/1000 [01:05<02:01,  4.84it/s]
+ 41%|████      | 412/1000 [01:05<02:11,  4.47it/s]
+ 41%|████▏     | 414/1000 [01:05<01:36,  6.08it/s]
+ 42%|████▏     | 416/1000 [01:05<01:40,  5.80it/s]
+ 42%|████▏     | 418/1000 [01:06<01:21,  7.14it/s]
+ 42%|████▏     | 420/1000 [01:06<01:20,  7.17it/s]
+ 42%|████▏     | 421/1000 [01:06<01:30,  6.38it/s]
+ 42%|████▏     | 423/1000 [01:06<01:41,  5.71it/s]
+ 42%|████▏     | 424/1000 [01:07<01:55,  4.98it/s]
+ 42%|████▎     | 425/1000 [01:07<01:46,  5.41it/s]
+ 43%|████▎     | 427/1000 [01:07<01:23,  6.83it/s]
+ 43%|████▎     | 428/1000 [01:07<01:27,  6.51it/s]
+ 43%|████▎     | 429/1000 [01:07<01:24,  6.79it/s]
+ 43%|████▎     | 431/1000 [01:08<01:08,  8.26it/s]
+ 43%|████▎     | 433/1000 [01:08<00:59,  9.58it/s]
+ 44%|████▎     | 435/1000 [01:08<00:56,  9.92it/s]
+ 44%|████▎     | 437/1000 [01:08<01:02,  9.00it/s]
+ 44%|████▍     | 439/1000 [01:09<01:20,  6.94it/s]
+ 44%|████▍     | 441/1000 [01:09<01:11,  7.86it/s]
+ 44%|████▍     | 442/1000 [01:09<01:26,  6.44it/s]
+ 44%|████▍     | 444/1000 [01:09<01:27,  6.37it/s]
+ 45%|████▍     | 446/1000 [01:10<01:12,  7.61it/s]
+ 45%|████▍     | 448/1000 [01:10<01:24,  6.55it/s]
+ 45%|████▌     | 450/1000 [01:10<01:29,  6.16it/s]
+ 45%|████▌     | 451/1000 [01:11<01:40,  5.44it/s]
+ 45%|████▌     | 452/1000 [01:11<01:51,  4.91it/s]
+ 45%|████▌     | 454/1000 [01:11<01:26,  6.32it/s]
+ 46%|████▌     | 456/1000 [01:11<01:12,  7.47it/s]
+ 46%|████▌     | 457/1000 [01:11<01:19,  6.84it/s]
+ 46%|████▌     | 459/1000 [01:12<01:06,  8.14it/s]
+ 46%|████▌     | 461/1000 [01:12<00:56,  9.50it/s]
+ 46%|████▋     | 463/1000 [01:12<01:20,  6.66it/s]
+ 46%|████▋     | 465/1000 [01:13<01:30,  5.89it/s]
+ 47%|████▋     | 466/1000 [01:13<01:41,  5.26it/s]
+ 47%|████▋     | 467/1000 [01:13<02:06,  4.21it/s]
+ 47%|████▋     | 469/1000 [01:13<01:36,  5.50it/s]
+ 47%|████▋     | 471/1000 [01:14<01:17,  6.86it/s]
+ 47%|████▋     | 472/1000 [01:14<01:13,  7.18it/s]
+ 47%|████▋     | 473/1000 [01:14<01:21,  6.47it/s]
+ 47%|████▋     | 474/1000 [01:14<01:37,  5.41it/s]
+ 48%|████▊     | 476/1000 [01:15<01:36,  5.42it/s]
+ 48%|████▊     | 477/1000 [01:15<01:53,  4.62it/s]
+ 48%|████▊     | 479/1000 [01:15<01:24,  6.19it/s]
+ 48%|████▊     | 480/1000 [01:15<01:31,  5.68it/s]
+ 48%|████▊     | 482/1000 [01:15<01:13,  7.04it/s]
+ 48%|████▊     | 484/1000 [01:16<01:20,  6.40it/s]
+ 48%|████▊     | 485/1000 [01:16<01:33,  5.51it/s]
+ 49%|████▊     | 486/1000 [01:16<01:33,  5.48it/s]
+ 49%|████▉     | 488/1000 [01:16<01:15,  6.80it/s]
+ 49%|████▉     | 489/1000 [01:17<01:30,  5.67it/s]
+ 49%|████▉     | 491/1000 [01:17<01:11,  7.09it/s]
+ 49%|████▉     | 492/1000 [01:17<01:10,  7.18it/s]
+ 49%|████▉     | 494/1000 [01:17<00:58,  8.71it/s]
+ 50%|████▉     | 496/1000 [01:18<01:08,  7.34it/s]
+ 50%|████▉     | 498/1000 [01:18<01:08,  7.36it/s]
+ 50%|████▉     | 499/1000 [01:18<01:28,  5.64it/s]
+ 50%|█████     | 500/1000 [01:18<01:38,  5.05it/s]
+ 50%|█████     | 502/1000 [01:19<01:18,  6.38it/s]
+ 50%|█████     | 503/1000 [01:19<01:27,  5.70it/s]
+ 50%|█████     | 505/1000 [01:19<01:28,  5.60it/s]
+ 51%|█████     | 506/1000 [01:20<01:40,  4.91it/s]
+ 51%|█████     | 508/1000 [01:20<01:18,  6.25it/s]
+ 51%|█████     | 510/1000 [01:20<01:04,  7.58it/s]
+ 51%|█████     | 512/1000 [01:20<00:57,  8.46it/s]
+ 51%|█████▏    | 514/1000 [01:21<01:20,  6.07it/s]
+ 52%|█████▏    | 515/1000 [01:21<01:16,  6.34it/s]
+ 52%|█████▏    | 516/1000 [01:21<01:20,  6.04it/s]
+ 52%|█████▏    | 517/1000 [01:21<01:15,  6.40it/s]
+ 52%|█████▏    | 518/1000 [01:21<01:12,  6.64it/s]
+ 52%|█████▏    | 519/1000 [01:22<01:34,  5.12it/s]
+ 52%|█████▏    | 521/1000 [01:22<01:11,  6.73it/s]
+ 52%|█████▏    | 522/1000 [01:22<01:27,  5.48it/s]
+ 52%|█████▏    | 523/1000 [01:22<01:38,  4.83it/s]
+ 52%|█████▏    | 524/1000 [01:22<01:41,  4.69it/s]
+ 53%|█████▎    | 526/1000 [01:23<01:23,  5.65it/s]
+ 53%|█████▎    | 528/1000 [01:23<01:25,  5.50it/s]
+ 53%|█████▎    | 529/1000 [01:23<01:26,  5.42it/s]
+ 53%|█████▎    | 531/1000 [01:23<01:07,  6.95it/s]
+ 53%|█████▎    | 533/1000 [01:24<00:55,  8.46it/s]
+ 53%|█████▎    | 534/1000 [01:24<01:10,  6.65it/s]
+ 54%|█████▎    | 535/1000 [01:24<01:26,  5.41it/s]
+ 54%|█████▎    | 536/1000 [01:24<01:29,  5.17it/s]
+ 54%|█████▎    | 537/1000 [01:25<01:39,  4.64it/s]
+ 54%|█████▍    | 539/1000 [01:25<01:32,  5.00it/s]
+ 54%|█████▍    | 540/1000 [01:25<01:34,  4.87it/s]
+ 54%|█████▍    | 541/1000 [01:26<01:45,  4.36it/s]
+ 54%|█████▍    | 543/1000 [01:26<01:17,  5.91it/s]
+ 55%|█████▍    | 545/1000 [01:26<01:14,  6.08it/s]
+ 55%|█████▍    | 546/1000 [01:26<01:21,  5.59it/s]
+ 55%|█████▍    | 547/1000 [01:27<01:32,  4.89it/s]
+ 55%|█████▍    | 549/1000 [01:27<01:29,  5.01it/s]
+ 55%|█████▌    | 551/1000 [01:27<01:09,  6.43it/s]
+ 55%|█████▌    | 552/1000 [01:27<01:11,  6.26it/s]
+ 55%|█████▌    | 553/1000 [01:28<01:23,  5.34it/s]
+ 55%|█████▌    | 554/1000 [01:28<01:29,  5.01it/s]
+ 56%|█████▌    | 556/1000 [01:28<01:07,  6.61it/s]
+ 56%|█████▌    | 557/1000 [01:28<01:13,  6.01it/s]
+ 56%|█████▌    | 559/1000 [01:28<00:58,  7.59it/s]
+ 56%|█████▌    | 561/1000 [01:29<00:52,  8.44it/s]
+ 56%|█████▋    | 563/1000 [01:29<01:01,  7.14it/s]
+ 56%|█████▋    | 565/1000 [01:29<01:03,  6.84it/s]
+ 57%|█████▋    | 566/1000 [01:30<01:16,  5.69it/s]
+ 57%|█████▋    | 568/1000 [01:30<01:17,  5.61it/s]
+ 57%|█████▋    | 569/1000 [01:30<01:17,  5.53it/s]
+ 57%|█████▋    | 571/1000 [01:30<01:17,  5.54it/s]
+ 57%|█████▋    | 572/1000 [01:31<01:20,  5.31it/s]
+ 57%|█████▋    | 574/1000 [01:31<01:18,  5.42it/s]
+ 58%|█████▊    | 576/1000 [01:31<01:03,  6.69it/s]
+ 58%|█████▊    | 577/1000 [01:32<01:26,  4.91it/s]
+ 58%|█████▊    | 578/1000 [01:32<01:35,  4.44it/s]
+ 58%|█████▊    | 580/1000 [01:32<01:12,  5.80it/s]
+ 58%|█████▊    | 582/1000 [01:32<00:58,  7.13it/s]
+ 58%|█████▊    | 583/1000 [01:33<01:11,  5.80it/s]
+ 58%|█████▊    | 585/1000 [01:33<01:09,  5.96it/s]
+ 59%|█████▊    | 587/1000 [01:33<01:07,  6.16it/s]
+ 59%|█████▉    | 589/1000 [01:33<00:55,  7.34it/s]
+ 59%|█████▉    | 590/1000 [01:34<01:11,  5.71it/s]
+ 59%|█████▉    | 592/1000 [01:34<00:58,  6.94it/s]
+ 59%|█████▉    | 593/1000 [01:34<01:09,  5.84it/s]
+ 60%|█████▉    | 595/1000 [01:34<00:55,  7.24it/s]
+ 60%|█████▉    | 596/1000 [01:35<01:10,  5.76it/s]
+ 60%|█████▉    | 597/1000 [01:35<01:11,  5.65it/s]
+ 60%|█████▉    | 599/1000 [01:35<01:12,  5.56it/s]
+ 60%|██████    | 601/1000 [01:36<01:12,  5.53it/s]
+ 60%|██████    | 602/1000 [01:36<01:10,  5.66it/s]
+ 60%|██████    | 604/1000 [01:36<01:21,  4.84it/s]
+ 61%|██████    | 606/1000 [01:36<01:06,  5.96it/s]
+ 61%|██████    | 608/1000 [01:37<00:59,  6.64it/s]
+ 61%|██████    | 610/1000 [01:37<00:53,  7.23it/s]
+ 61%|██████    | 612/1000 [01:37<00:46,  8.36it/s]
+ 61%|██████▏   | 614/1000 [01:37<00:41,  9.36it/s]
+ 62%|██████▏   | 616/1000 [01:38<00:59,  6.47it/s]
+ 62%|██████▏   | 617/1000 [01:38<00:58,  6.59it/s]
+ 62%|██████▏   | 619/1000 [01:38<00:49,  7.71it/s]
+ 62%|██████▏   | 621/1000 [01:38<00:55,  6.86it/s]
+ 62%|██████▏   | 623/1000 [01:39<00:53,  7.03it/s]
+ 62%|██████▎   | 625/1000 [01:39<00:59,  6.31it/s]
+ 63%|██████▎   | 627/1000 [01:39<00:50,  7.33it/s]
+ 63%|██████▎   | 628/1000 [01:39<01:00,  6.15it/s]
+ 63%|██████▎   | 629/1000 [01:40<01:09,  5.36it/s]
+ 63%|██████▎   | 631/1000 [01:40<00:54,  6.79it/s]
+ 63%|██████▎   | 633/1000 [01:40<00:46,  7.96it/s]
+ 63%|██████▎   | 634/1000 [01:40<01:06,  5.47it/s]
+ 64%|██████▎   | 635/1000 [01:41<01:04,  5.66it/s]
+ 64%|██████▎   | 636/1000 [01:41<00:59,  6.15it/s]
+ 64%|██████▍   | 638/1000 [01:41<00:52,  6.96it/s]
+ 64%|██████▍   | 640/1000 [01:41<00:43,  8.21it/s]
+ 64%|██████▍   | 641/1000 [01:41<00:57,  6.29it/s]
+ 64%|██████▍   | 643/1000 [01:42<00:53,  6.62it/s]
+ 64%|██████▍   | 645/1000 [01:42<00:57,  6.19it/s]
+ 65%|██████▍   | 647/1000 [01:42<00:54,  6.47it/s]
+ 65%|██████▍   | 648/1000 [01:43<01:02,  5.61it/s]
+ 65%|██████▌   | 650/1000 [01:43<01:01,  5.68it/s]
+ 65%|██████▌   | 652/1000 [01:43<01:03,  5.46it/s]
+ 65%|██████▌   | 654/1000 [01:44<00:53,  6.51it/s]
+ 66%|██████▌   | 656/1000 [01:44<00:45,  7.56it/s]
+ 66%|██████▌   | 657/1000 [01:44<00:54,  6.32it/s]
+ 66%|██████▌   | 659/1000 [01:45<01:05,  5.22it/s]
+ 66%|██████▌   | 661/1000 [01:45<00:51,  6.52it/s]
+ 66%|██████▌   | 662/1000 [01:45<01:00,  5.63it/s]
+ 66%|██████▋   | 664/1000 [01:45<01:07,  4.94it/s]
+ 66%|██████▋   | 665/1000 [01:46<01:12,  4.59it/s]
+ 67%|██████▋   | 667/1000 [01:46<01:08,  4.83it/s]
+ 67%|██████▋   | 668/1000 [01:46<01:13,  4.52it/s]
+ 67%|██████▋   | 669/1000 [01:47<01:06,  4.95it/s]
+ 67%|██████▋   | 671/1000 [01:47<01:05,  4.99it/s]
+ 67%|██████▋   | 673/1000 [01:47<00:57,  5.70it/s]
+ 68%|██████▊   | 675/1000 [01:47<00:51,  6.27it/s]
+ 68%|██████▊   | 677/1000 [01:48<00:43,  7.47it/s]
+ 68%|██████▊   | 679/1000 [01:48<00:39,  8.15it/s]
+ 68%|██████▊   | 681/1000 [01:48<00:33,  9.43it/s]
+ 68%|██████▊   | 683/1000 [01:48<00:41,  7.70it/s]
+ 68%|██████▊   | 685/1000 [01:48<00:36,  8.56it/s]
+ 69%|██████▊   | 686/1000 [01:49<00:47,  6.62it/s]
+ 69%|██████▊   | 687/1000 [01:49<00:55,  5.67it/s]
+ 69%|██████▉   | 688/1000 [01:49<00:51,  6.05it/s]
+ 69%|██████▉   | 690/1000 [01:50<00:53,  5.76it/s]
+ 69%|██████▉   | 692/1000 [01:50<00:46,  6.63it/s]
+ 69%|██████▉   | 694/1000 [01:50<00:50,  6.04it/s]
+ 70%|██████▉   | 695/1000 [01:50<00:48,  6.26it/s]
+ 70%|██████▉   | 696/1000 [01:51<00:56,  5.37it/s]
+ 70%|██████▉   | 698/1000 [01:51<00:50,  5.94it/s]
+ 70%|██████▉   | 699/1000 [01:51<00:59,  5.07it/s]
+ 70%|███████   | 701/1000 [01:51<00:46,  6.47it/s]
+ 70%|███████   | 702/1000 [01:52<01:01,  4.82it/s]
+ 70%|███████   | 703/1000 [01:52<00:54,  5.41it/s]
+ 70%|███████   | 705/1000 [01:52<01:01,  4.77it/s]
+ 71%|███████   | 707/1000 [01:53<00:48,  6.10it/s]
+ 71%|███████   | 709/1000 [01:53<00:44,  6.47it/s]
+ 71%|███████   | 710/1000 [01:53<00:52,  5.57it/s]
+ 71%|███████   | 712/1000 [01:53<00:44,  6.54it/s]
+ 71%|███████▏  | 713/1000 [01:53<00:44,  6.51it/s]
+ 72%|███████▏  | 715/1000 [01:54<00:36,  7.86it/s]
+ 72%|███████▏  | 716/1000 [01:54<00:45,  6.29it/s]
+ 72%|███████▏  | 717/1000 [01:54<00:41,  6.87it/s]
+ 72%|███████▏  | 718/1000 [01:54<00:49,  5.66it/s]
+ 72%|███████▏  | 719/1000 [01:55<00:57,  4.90it/s]
+ 72%|███████▏  | 720/1000 [01:55<00:51,  5.47it/s]
+ 72%|███████▏  | 722/1000 [01:55<00:39,  7.03it/s]
+ 72%|███████▏  | 723/1000 [01:55<00:39,  7.01it/s]
+ 72%|███████▏  | 724/1000 [01:55<00:39,  7.04it/s]
+ 72%|███████▎  | 725/1000 [01:55<00:38,  7.16it/s]
+ 73%|███████▎  | 727/1000 [01:55<00:30,  8.85it/s]
+ 73%|███████▎  | 729/1000 [01:56<00:26, 10.16it/s]
+ 73%|███████▎  | 731/1000 [01:56<00:37,  7.26it/s]
+ 73%|███████▎  | 733/1000 [01:56<00:31,  8.34it/s]
+ 73%|███████▎  | 734/1000 [01:56<00:32,  8.28it/s]
+ 74%|███████▎  | 736/1000 [01:57<00:34,  7.66it/s]
+ 74%|███████▍  | 738/1000 [01:57<00:31,  8.29it/s]
+ 74%|███████▍  | 740/1000 [01:57<00:28,  9.09it/s]
+ 74%|███████▍  | 741/1000 [01:57<00:36,  7.06it/s]
+ 74%|███████▍  | 743/1000 [01:58<00:36,  7.02it/s]
+ 74%|███████▍  | 745/1000 [01:58<00:30,  8.26it/s]
+ 75%|███████▍  | 747/1000 [01:58<00:27,  9.07it/s]
+ 75%|███████▍  | 748/1000 [01:58<00:36,  6.84it/s]
+ 75%|███████▌  | 750/1000 [01:58<00:37,  6.68it/s]
+ 75%|███████▌  | 751/1000 [01:59<00:46,  5.35it/s]
+ 75%|███████▌  | 753/1000 [01:59<00:39,  6.28it/s]
+ 75%|███████▌  | 754/1000 [01:59<00:46,  5.31it/s]
+ 76%|███████▌  | 756/1000 [02:00<00:44,  5.51it/s]
+ 76%|███████▌  | 758/1000 [02:00<00:36,  6.57it/s]
+ 76%|███████▌  | 759/1000 [02:00<00:37,  6.35it/s]
+ 76%|███████▌  | 761/1000 [02:00<00:31,  7.50it/s]
+ 76%|███████▋  | 763/1000 [02:01<00:36,  6.56it/s]
+ 76%|███████▋  | 764/1000 [02:01<00:41,  5.69it/s]
+ 77%|███████▋  | 766/1000 [02:01<00:42,  5.51it/s]
+ 77%|███████▋  | 768/1000 [02:01<00:34,  6.66it/s]
+ 77%|███████▋  | 769/1000 [02:02<00:41,  5.59it/s]
+ 77%|███████▋  | 770/1000 [02:02<00:41,  5.58it/s]
+ 77%|███████▋  | 772/1000 [02:02<00:32,  6.96it/s]
+ 77%|███████▋  | 773/1000 [02:02<00:41,  5.47it/s]
+ 77%|███████▋  | 774/1000 [02:03<00:38,  5.83it/s]
+ 78%|███████▊  | 776/1000 [02:03<00:33,  6.65it/s]
+ 78%|███████▊  | 778/1000 [02:03<00:27,  8.08it/s]
+ 78%|███████▊  | 780/1000 [02:03<00:37,  5.91it/s]
+ 78%|███████▊  | 782/1000 [02:04<00:30,  7.15it/s]
+ 78%|███████▊  | 783/1000 [02:04<00:37,  5.86it/s]
+ 78%|███████▊  | 784/1000 [02:04<00:35,  6.14it/s]
+ 79%|███████▊  | 786/1000 [02:04<00:32,  6.58it/s]
+ 79%|███████▉  | 788/1000 [02:05<00:28,  7.43it/s]
+ 79%|███████▉  | 789/1000 [02:05<00:27,  7.75it/s]
+ 79%|███████▉  | 790/1000 [02:05<00:26,  7.81it/s]
+ 79%|███████▉  | 791/1000 [02:05<00:30,  6.96it/s]
+ 79%|███████▉  | 792/1000 [02:05<00:31,  6.50it/s]
+ 79%|███████▉  | 793/1000 [02:05<00:30,  6.86it/s]
+ 80%|███████▉  | 795/1000 [02:06<00:33,  6.15it/s]
+ 80%|███████▉  | 796/1000 [02:06<00:30,  6.69it/s]
+ 80%|███████▉  | 797/1000 [02:06<00:36,  5.52it/s]
+ 80%|███████▉  | 798/1000 [02:06<00:41,  4.82it/s]
+ 80%|████████  | 800/1000 [02:07<00:36,  5.54it/s]
+ 80%|████████  | 802/1000 [02:07<00:30,  6.50it/s]
+ 80%|████████  | 804/1000 [02:07<00:29,  6.73it/s]
+ 80%|████████  | 805/1000 [02:07<00:27,  7.10it/s]
+ 81%|████████  | 807/1000 [02:07<00:24,  7.81it/s]
+ 81%|████████  | 809/1000 [02:08<00:21,  8.91it/s]
+ 81%|████████  | 811/1000 [02:08<00:19,  9.82it/s]
+ 81%|████████▏ | 813/1000 [02:08<00:23,  7.95it/s]
+ 81%|████████▏ | 814/1000 [02:08<00:23,  7.80it/s]
+ 82%|████████▏ | 816/1000 [02:08<00:21,  8.70it/s]
+ 82%|████████▏ | 817/1000 [02:09<00:27,  6.62it/s]
+ 82%|████████▏ | 819/1000 [02:09<00:23,  7.80it/s]
+ 82%|████████▏ | 820/1000 [02:09<00:28,  6.28it/s]
+ 82%|████████▏ | 821/1000 [02:09<00:26,  6.73it/s]
+ 82%|████████▏ | 823/1000 [02:10<00:26,  6.64it/s]
+ 82%|████████▏ | 824/1000 [02:10<00:31,  5.57it/s]
+ 82%|████████▎ | 825/1000 [02:10<00:30,  5.82it/s]
+ 83%|████████▎ | 826/1000 [02:10<00:36,  4.79it/s]
+ 83%|████████▎ | 828/1000 [02:11<00:39,  4.30it/s]
+ 83%|████████▎ | 829/1000 [02:11<00:35,  4.78it/s]
+ 83%|████████▎ | 831/1000 [02:11<00:28,  5.98it/s]
+ 83%|████████▎ | 832/1000 [02:12<00:34,  4.81it/s]
+ 83%|████████▎ | 833/1000 [02:12<00:31,  5.25it/s]
+ 83%|████████▎ | 834/1000 [02:12<00:32,  5.13it/s]
+ 84%|████████▎ | 835/1000 [02:12<00:29,  5.57it/s]
+ 84%|████████▎ | 836/1000 [02:12<00:32,  5.10it/s]
+ 84%|████████▍ | 838/1000 [02:12<00:25,  6.33it/s]
+ 84%|████████▍ | 840/1000 [02:13<00:28,  5.69it/s]
+ 84%|████████▍ | 842/1000 [02:13<00:22,  7.02it/s]
+ 84%|████████▍ | 843/1000 [02:13<00:21,  7.32it/s]
+ 84%|████████▍ | 844/1000 [02:14<00:31,  5.02it/s]
+ 84%|████████▍ | 845/1000 [02:14<00:34,  4.47it/s]
+ 85%|████████▍ | 847/1000 [02:14<00:29,  5.10it/s]
+ 85%|████████▍ | 849/1000 [02:14<00:23,  6.36it/s]
+ 85%|████████▌ | 850/1000 [02:15<00:28,  5.34it/s]
+ 85%|████████▌ | 851/1000 [02:15<00:25,  5.74it/s]
+ 85%|████████▌ | 853/1000 [02:15<00:20,  7.25it/s]
+ 86%|████████▌ | 855/1000 [02:15<00:16,  8.68it/s]
+ 86%|████████▌ | 856/1000 [02:15<00:16,  8.49it/s]
+ 86%|████████▌ | 858/1000 [02:16<00:20,  6.84it/s]
+ 86%|████████▌ | 860/1000 [02:16<00:17,  7.99it/s]
+ 86%|████████▌ | 862/1000 [02:16<00:15,  9.03it/s]
+ 86%|████████▋ | 864/1000 [02:16<00:13, 10.14it/s]
+ 87%|████████▋ | 866/1000 [02:16<00:13, 10.23it/s]
+ 87%|████████▋ | 868/1000 [02:17<00:16,  8.04it/s]
+ 87%|████████▋ | 870/1000 [02:17<00:14,  8.87it/s]
+ 87%|████████▋ | 872/1000 [02:17<00:16,  7.99it/s]
+ 87%|████████▋ | 873/1000 [02:17<00:19,  6.46it/s]
+ 87%|████████▋ | 874/1000 [02:18<00:21,  5.75it/s]
+ 88%|████████▊ | 876/1000 [02:18<00:17,  7.18it/s]
+ 88%|████████▊ | 878/1000 [02:18<00:18,  6.49it/s]
+ 88%|████████▊ | 879/1000 [02:18<00:20,  5.78it/s]
+ 88%|████████▊ | 881/1000 [02:19<00:18,  6.59it/s]
+ 88%|████████▊ | 882/1000 [02:19<00:21,  5.54it/s]
+ 88%|████████▊ | 884/1000 [02:19<00:16,  6.95it/s]
+ 89%|████████▊ | 886/1000 [02:19<00:14,  8.10it/s]
+ 89%|████████▊ | 887/1000 [02:19<00:13,  8.33it/s]
+ 89%|████████▉ | 889/1000 [02:20<00:13,  8.16it/s]
+ 89%|████████▉ | 891/1000 [02:20<00:11,  9.21it/s]
+ 89%|████████▉ | 893/1000 [02:20<00:13,  8.14it/s]
+ 89%|████████▉ | 894/1000 [02:20<00:16,  6.54it/s]
+ 90%|████████▉ | 896/1000 [02:21<00:13,  7.84it/s]
+ 90%|████████▉ | 897/1000 [02:21<00:16,  6.13it/s]
+ 90%|████████▉ | 898/1000 [02:21<00:19,  5.30it/s]
+ 90%|█████████ | 900/1000 [02:21<00:14,  6.72it/s]
+ 90%|█████████ | 901/1000 [02:21<00:14,  6.71it/s]
+ 90%|█████████ | 902/1000 [02:22<00:14,  6.98it/s]
+ 90%|█████████ | 904/1000 [02:22<00:11,  8.48it/s]
+ 90%|█████████ | 905/1000 [02:22<00:13,  7.19it/s]
+ 91%|█████████ | 906/1000 [02:22<00:16,  5.78it/s]
+ 91%|█████████ | 908/1000 [02:22<00:12,  7.41it/s]
+ 91%|█████████ | 910/1000 [02:23<00:13,  6.61it/s]
+ 91%|█████████ | 912/1000 [02:23<00:11,  7.84it/s]
+ 91%|█████████▏| 913/1000 [02:23<00:14,  5.91it/s]
+ 92%|█████████▏| 915/1000 [02:23<00:11,  7.20it/s]
+ 92%|█████████▏| 916/1000 [02:24<00:13,  6.01it/s]
+ 92%|█████████▏| 917/1000 [02:24<00:13,  6.36it/s]
+ 92%|█████████▏| 918/1000 [02:24<00:17,  4.61it/s]
+ 92%|█████████▏| 919/1000 [02:24<00:16,  4.80it/s]
+ 92%|█████████▏| 921/1000 [02:25<00:16,  4.74it/s]
+ 92%|█████████▏| 922/1000 [02:25<00:15,  5.09it/s]
+ 92%|█████████▏| 924/1000 [02:25<00:12,  6.26it/s]
+ 93%|█████████▎| 926/1000 [02:25<00:09,  7.61it/s]
+ 93%|█████████▎| 927/1000 [02:26<00:10,  7.11it/s]
+ 93%|█████████▎| 928/1000 [02:26<00:12,  5.82it/s]
+ 93%|█████████▎| 930/1000 [02:26<00:12,  5.68it/s]
+ 93%|█████████▎| 931/1000 [02:26<00:11,  6.00it/s]
+ 93%|█████████▎| 933/1000 [02:27<00:11,  5.81it/s]
+ 93%|█████████▎| 934/1000 [02:27<00:12,  5.42it/s]
+ 94%|█████████▎| 936/1000 [02:27<00:09,  6.43it/s]
+ 94%|█████████▍| 938/1000 [02:27<00:07,  7.77it/s]
+ 94%|█████████▍| 939/1000 [02:28<00:08,  7.07it/s]
+ 94%|█████████▍| 940/1000 [02:28<00:08,  7.16it/s]
+ 94%|█████████▍| 942/1000 [02:28<00:10,  5.54it/s]
+ 94%|█████████▍| 944/1000 [02:28<00:09,  6.18it/s]
+ 95%|█████████▍| 946/1000 [02:29<00:07,  7.24it/s]
+ 95%|█████████▍| 948/1000 [02:29<00:06,  8.31it/s]
+ 95%|█████████▍| 949/1000 [02:29<00:07,  7.15it/s]
+ 95%|█████████▌| 951/1000 [02:29<00:05,  8.20it/s]
+ 95%|█████████▌| 952/1000 [02:29<00:08,  5.94it/s]
+ 95%|█████████▌| 954/1000 [02:30<00:09,  4.95it/s]
+ 96%|█████████▌| 956/1000 [02:30<00:07,  6.21it/s]
+ 96%|█████████▌| 957/1000 [02:30<00:08,  5.25it/s]
+ 96%|█████████▌| 959/1000 [02:31<00:08,  4.81it/s]
+ 96%|█████████▌| 961/1000 [02:31<00:06,  5.78it/s]
+ 96%|█████████▌| 962/1000 [02:31<00:07,  5.07it/s]
+ 96%|█████████▋| 964/1000 [02:32<00:05,  6.37it/s]
+ 96%|█████████▋| 965/1000 [02:32<00:06,  5.50it/s]
+ 97%|█████████▋| 966/1000 [02:32<00:06,  5.43it/s]
+ 97%|█████████▋| 968/1000 [02:32<00:04,  6.93it/s]
+ 97%|█████████▋| 970/1000 [02:32<00:03,  7.97it/s]
+ 97%|█████████▋| 971/1000 [02:33<00:04,  6.36it/s]
+ 97%|█████████▋| 973/1000 [02:33<00:03,  7.64it/s]
+ 98%|█████████▊| 975/1000 [02:33<00:03,  8.10it/s]
+ 98%|█████████▊| 977/1000 [02:33<00:02,  9.29it/s]
+ 98%|█████████▊| 979/1000 [02:34<00:02,  9.09it/s]
+ 98%|█████████▊| 980/1000 [02:34<00:02,  7.04it/s]
+ 98%|█████████▊| 981/1000 [02:34<00:03,  5.86it/s]
+ 98%|█████████▊| 982/1000 [02:34<00:04,  4.45it/s]
+ 98%|█████████▊| 984/1000 [02:35<00:02,  5.85it/s]
+ 99%|█████████▊| 986/1000 [02:35<00:01,  7.18it/s]
+ 99%|█████████▊| 987/1000 [02:35<00:01,  6.70it/s]
+ 99%|█████████▉| 988/1000 [02:35<00:02,  5.56it/s]
+ 99%|█████████▉| 989/1000 [02:36<00:02,  4.77it/s]
+ 99%|█████████▉| 991/1000 [02:36<00:01,  5.04it/s]
+ 99%|█████████▉| 992/1000 [02:36<00:01,  4.07it/s]
+ 99%|█████████▉| 994/1000 [02:37<00:01,  5.55it/s]
+100%|█████████▉| 995/1000 [02:37<00:01,  4.59it/s]
+100%|█████████▉| 997/1000 [02:37<00:00,  5.97it/s]
+100%|█████████▉| 999/1000 [02:37<00:00,  5.79it/s]
+100%|██████████| 1000/1000 [02:38<00:00,  5.55it/s]
+100%|██████████| 1000/1000 [02:38<00:00,  6.32it/s]
+***** eval metrics *****
+  epoch                   =        1.0
+  eval_loss               =     0.7404
+  eval_runtime            = 0:02:38.33
+  eval_samples_per_second =      6.316
+  eval_steps_per_second   =      6.316
+  num_input_tokens_seen   =   12201816
+Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
diff --git a/optuna_30k_experiments/logs/trial_004_lr_1.869e-04.log b/optuna_30k_experiments/logs/trial_004_lr_1.869e-04.log
new file mode 100644
index 0000000..4026033
--- /dev/null
+++ b/optuna_30k_experiments/logs/trial_004_lr_1.869e-04.log
@@ -0,0 +1,12416 @@
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/torch/cuda/__init__.py:734: UserWarning: Can't initialize NVML
+  warnings.warn("Can't initialize NVML")
+[INFO|2026-10-13 09:57:55] llamafactory.hparams.parser:651 >> Process rank: 0, world size: 1, device: cuda:0, distributed training: False, compute dtype: torch.bfloat16
+Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
+[INFO|configuration_utils.py:765] 2026-10-13 09:57:55,703 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-13 09:57:55,706 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|configuration_utils.py:765] 2026-10-13 09:57:57,539 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-13 09:57:57,541 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|configuration_utils.py:765] 2026-10-13 09:57:57,681 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-13 09:57:57,684 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-13 09:57:58] llamafactory.data.loader:144 >> Loading dataset aya_slovak_hpo_30k.jsonl...
+training example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28828, 2176, 28711, 28863, 29114, 10268, 28709, 6612, 28714, 2892, 493, 363, 6294, 10719, 363, 3811, 28723, 7379, 626, 28889, 307, 14637, 311, 28768, 2218, 22885, 3748, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 284, 8341, 6729, 345, 28738, 28739, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28723, 524, 4634, 28711, 570, 1879, 261, 28863, 532, 613, 14483, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 345, 15962, 28739, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 1879, 15173, 537, 284, 8341, 705, 28723, 367, 280, 28879, 570, 4415, 28716, 28797, 9335, 3175, 29826, 626, 323, 412, 8514, 16986, 29826, 28708, 345, 15962, 28739, 1879, 15173, 537, 1221, 4566, 28718, 325, 28707, 28723, 461, 28723, 724, 494, 8891, 24160, 28725, 8582, 28862, 24160, 28725, 20532, 28764, 271, 28725, 438, 29365, 15745, 1319, 12035, 28862, 8269, 284, 28827, 2762, 28709, 264, 4798, 8359, 1450, 8325, 12121, 1798, 420, 762, 1089, 290, 28906, 18765, 724, 2331, 29114, 1221, 4304, 1780, 363, 13881, 29826, 316, 2085, 28708, 28723, 22627, 28873, 263, 335, 20208, 284, 28827, 2762, 28709, 264, 4798, 5424, 10056, 1286, 290, 28906, 18765, 724, 2331, 29114, 11323, 10719, 12490, 363, 13881, 29826, 316, 284, 8341, 2390, 28723, 22127, 280, 28879, 21022, 1150, 424, 724, 28827, 1144, 28873, 340, 17066, 270, 21022, 485, 284, 8341, 705, 28725, 446, 6909, 28797, 637, 28705, 29114, 28708, 28879, 4798, 6612, 28827, 1100, 28768, 28863, 28723, 320, 412, 28879, 637, 15612, 28716, 28889, 28726, 1150, 424, 19523, 412, 28873, 276, 28797, 2390, 284, 8341, 2390, 363, 23407, 11277, 9538, 2085, 28708, 28723, 9532, 2144, 686, 7865, 28718, 341, 14475, 1687, 28706, 18037, 452, 270, 8269, 284, 8341, 705, 28725, 290, 4827, 486, 2710, 2144, 332, 1800, 16344, 29114, 6588, 269, 28862, 28711, 28863, 686, 1653, 5388, 28718, 710, 261, 1296, 28709, 284, 8341, 705, 28723, 351, 28711, 1371, 28797, 1776, 29826, 8269, 284, 8341, 705, 268, 28863, 1776, 29826, 8269, 1879, 3497, 598, 8269, 8631, 28879, 279, 412, 28725, 12121, 7988, 268, 28863, 24270, 12115, 28797, 1879, 5626, 263, 28862, 8269, 8631, 28879, 279, 412, 28723, 318, 15612, 1473, 276, 15690, 2245, 626, 15690, 307, 14637, 311, 28768, 300, 15469, 28711, 570, 446, 450, 4798, 29826, 333, 28729, 1879, 319, 3976, 294, 405, 28768, 4882, 405, 28725, 534, 28724, 2710, 724, 313, 4827, 2245, 626, 28889, 20532, 28764, 271, 28723, 10650, 300, 12235, 28827, 28873, 424, 268, 27172, 2245, 28723, 1054, 15504, 28879, 424, 724, 313, 8768, 4545, 504, 270, 8269, 2124, 2245, 626, 12689, 4584, 10906, 28718, 710, 18782, 28879, 28715, 28797, 1135, 10356, 28723, 2645, 28764, 12275, 28820, 28873, 29114, 13067, 549, 2892, 5872, 3811, 8631, 25991, 3026, 29114, 408, 28906, 28764, 485, 2082, 1403, 12121, 1798, 1773, 29826, 28729, 22160, 710, 408, 28906, 28764, 485, 1135, 10874, 1776, 4536, 28873, 300, 23407, 11277, 9538, 2085, 28708, 28723, 22127, 3361, 13470, 1150, 317, 28873, 424, 2082, 7330, 511, 2085, 28708, 28723, 401, 492, 13591, 28820, 261, 2220, 28862, 10766, 381, 28827, 486, 29114, 363, 28879, 4689, 3418, 28879, 21775, 29114, 280, 28723, 418, 28820, 11277, 28716, 6294, 26177, 903, 12689, 6612, 20122, 28733, 12868, 291, 2124, 2085, 28708, 1879, 6670, 28862, 9348, 493, 686, 375, 5620, 386, 28862, 28827, 28725, 23374, 4536, 25306, 16388, 957, 450, 15612, 28764, 2126, 29114, 511, 2152, 363, 25769, 14110, 17413, 26177, 903, 28873, 15808, 19396, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 7379, 1879, 277, 7939, 1879, 724, 313, 8768, 2245, 28718, 363, 446, 551, 9538, 12121, 1798, 24473, 21694, 300, 261, 5869, 28723, 384, 28728, 491, 28712, 4566, 15469, 28711, 570, 1879, 2245, 28725, 534, 28724, 2710, 7463, 15612, 13335, 28728, 4371, 1144, 28723, 10650, 300, 15469, 28711, 570, 1879, 1160, 28709, 28879, 3907, 13288, 491, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 28725, 534, 28724, 637, 686, 598, 5237, 5945, 17984, 28718, 2117, 18904, 3146, 515, 13288, 25208, 28723, 367, 280, 28879, 570, 12472, 1006, 5872, 28863, 358, 446, 311, 4845, 1879, 6149, 783, 494, 28718, 2245, 28718, 363, 17984, 28718, 405, 13288, 491, 28723, 13343, 29826, 28729, 385, 29114, 284, 8341, 705, 28747, 399, 3711, 3175, 29826, 626, 12990, 17984, 28718, 2117, 268, 6125, 15690, 345, 28738, 28739, 16986, 29826, 28708, 1773, 29826, 10641, 2124, 345, 28738, 28739, 582, 20524, 12214, 1773, 29826, 28729, 385, 29114, 284, 8341, 705, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 15173, 537, 28718, 325, 849, 3748, 28725, 446, 6909, 28820, 3061, 28720, 300, 28827, 1520, 6612, 749, 1780, 20532, 28764, 271, 586, 28873, 28710, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28731, 1879, 686, 6729, 1773, 29826, 28729, 22160, 284, 8341, 705, 28723, 550, 11245, 412, 28747, 550, 11245, 412, 1683, 28827, 16032, 271, 1342, 4295, 3191, 316, 28729, 6449, 2245, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 345, 28741, 28739, 1879, 22954, 338, 28718, 297, 12689, 345, 28741, 28739, 686, 2129, 28710, 29114, 16986, 28863, 358, 28723, 17563, 971, 28747, 17563, 971, 582, 20524, 12214, 1342, 28764, 1193, 1342, 4295, 5765, 6175, 2959, 15416, 28710, 13288, 491, 3589, 28723, 9532, 484, 28717, 3811, 18904, 17101, 29114, 22399, 971, 28725, 15612, 13335, 28728, 4371, 28711, 570, 13288, 20996, 28725, 446, 6909, 28797, 484, 28717, 3811, 18904, 17101, 29114, 28725, 264, 2513, 300, 8631, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 268, 345, 28741, 28739, 264, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Zhrnúť tento článok v jednej vete. Textový nástroj je ikona, ktorá sa podobá písmenu "T" v paneli nástrojov vľavo. Kliknite na túto ikonu. Použite rozbaľovaciu ponuku vedľa "Character" v hornej časti obrazovky na výber písma. Použite druhé rozbaľovacie menu vedľa "Character" na výber formátu (t. j. pravidelný, tučný, kurzor, atď.). Tradičné písmo ako Times New Roman alebo Garamond môže pridať formálny vzhľad loga. Bezšerifové písmo ako Helvetica môže pridať modernejší vzhľad písmu. Nepoužívajte príliš dekoratívne písma, ktoré sa ťažko čítajú. Tiež sa vyhýbajte zmiešanému písmu v návrhu loga. Ak si z internetu stiahnete bezplatné písma, mali by ste si uvedomiť licenčnú zmluvu pre tieto písma. Mnohé voľné písma sú voľné na osobné použitie, ale nie sú povolené na komerčné použitie. S vybraným textovým nástrojom kliknite kdekoľvek na grafickej doske, aby ste pridali textový kurzor. Potom napíšte svoj text. Zvážte pridanie samostatného textového objektu pre každé slovo. Obzvlášť ak plánujete používať rôzne farby alebo veľkosti pre rôzne slová vo vašom návrhu loga. Nepridávaj ešte farbu do loga. Farebná tlač nemusí byť vždy možnosťou. Návrh jednoduchého čierno-bieleho loga na začiatok zabezpečí, že vaše logo bude vyzerať dobre v jeho najjednoduchšej forme. Môžete tiež použiť nástroj Text na ceste na pridanie textu v kruhu alebo zakrivenom tvare. Dvakrát kliknite na text, aby ste ho vyzdvihli. Potom kliknite na položku znak v hornej časti obrazovky, aby sa zobrazila ponuka nastavenia znakov. Použite nasledujúce kroky na úpravu textu v ponuke znak. Veľkosť písma: Rozbaľovacia ponuka s malým "T" vedľa veľkého "T" upravuje veľkosť písma. Môžete tiež použiť nástroj výberu (ikona, ktorá pripomína čierny kurzor myši v paneli nástrojov vľavo) na zmenu veľkosti písma. Vedenie: Vedenie mení priestor medzi riadkami textu. Použite rozbaľovaciu ponuku vedľa ikony, ktorá sa podobá "A" na vrchu iného "A" zmeniť vedúce. Kerning: Kerning upravuje medzery medzi konkrétnymi znakmi. Ak chcete nastaviť kerning, vyzdvihnite znaky, ktoré chcete nastaviť, a potom použite rozbaľovaciu ponuku vedľa ikony s "A" aVyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+labels:
+Vyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+eval example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28965, 28709, 637, 8631, 25991, 3026, 28725, 13067, 10531, 28729, 350, 2591, 686, 3023, 28862, 28827, 284, 28873, 269, 13096, 28711, 28863, 6149, 3399, 28804, 550, 13881, 29826, 316, 300, 1879, 7871, 338, 28820, 6936, 1150, 28863, 7340, 6957, 8905, 3907, 12235, 28827, 28873, 424, 5765, 772, 28725, 446, 6909, 28889, 14848, 912, 12214, 4321, 28720, 751, 29365, 28723, 351, 28906, 18765, 298, 486, 29114, 28705, 28740, 387, 28705, 28750, 28734, 22116, 28827, 28723, 8294, 772, 28747, 13, 13, 27332, 12107, 28747, 13, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Čo sa používa, ak škodca zničí pšeničnú úrodu? Vzhľadom na predchádzajúcu otázku napíšte kontext, ktorý obsahuje odpoveď. Môže to byť 1 - 20 vetí. Kontext:
+
+### Response:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+labels:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+[INFO|configuration_utils.py:765] 2026-10-13 09:57:59,142 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-13 09:57:59,145 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-13 09:57:59] llamafactory.model.model_utils.kv_cache:144 >> KV cache is disabled during training.
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/unsloth/__init__.py:165: UserWarning: WARNING: Unsloth should be imported before [trl, transformers, peft] to ensure all optimizations are applied. Your code may run slower or encounter memory issues without these optimizations.
+
+Please restructure your imports with 'import unsloth' at the top of your file.
+  from ._gpu_init import *
+🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
+🦥 Unsloth Zoo will now patch everything to make training faster!
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/torch/cuda/__init__.py:734: UserWarning: Can't initialize NVML
+  warnings.warn("Can't initialize NVML")
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: WARNING `trust_remote_code` is True.
+Are you certain you want to do remote code execution?
+==((====))==  Unsloth 2026.6.1: Fast Mistral patching. Transformers: 5.5.0.
+   \\   /|    NVIDIA RTX A4500. Num GPUs = 1. Max memory: 19.698 GB. Platform: Linux.
+O^O/ \_/ \    Torch: 2.6.0+cu124. CUDA: 8.6. CUDA Toolkit: 12.4. Triton: 3.2.0
+\        /    Bfloat16 = TRUE. FA [Xformers = 0.0.29.post3. FA2 = False]
+ "-____-"     Free license: http://github.com/unslothai/unsloth
+Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading weights file model.safetensors from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/model.safetensors.index.json
+Will use dtype=torch.bfloat16 as defined in model's config object
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 2,
+  "use_cache": true
+}
+
+
+Loading weights:   0%|          | 0/291 [00:00<?, ?it/s]
+Loading weights:   2%|▏         | 5/291 [00:00<00:07, 40.52it/s]
+Loading weights:   8%|▊         | 22/291 [00:00<00:02, 106.13it/s]
+Loading weights:  12%|█▏        | 34/291 [00:00<00:02, 105.56it/s]
+Loading weights:  17%|█▋        | 49/291 [00:00<00:02, 120.42it/s]
+Loading weights:  21%|██▏       | 62/291 [00:00<00:01, 116.77it/s]
+Loading weights:  26%|██▌       | 76/291 [00:00<00:01, 120.41it/s]
+Loading weights:  31%|███       | 89/291 [00:00<00:01, 117.54it/s]
+Loading weights:  35%|███▌      | 103/291 [00:00<00:01, 122.73it/s]
+Loading weights:  40%|███▉      | 116/291 [00:01<00:01, 115.71it/s]
+Loading weights:  45%|████▍     | 130/291 [00:01<00:01, 117.19it/s]
+Loading weights:  49%|████▉     | 142/291 [00:01<00:01, 114.48it/s]
+Loading weights:  54%|█████▍    | 157/291 [00:01<00:01, 120.52it/s]
+Loading weights:  58%|█████▊    | 170/291 [00:01<00:00, 121.89it/s]
+Loading weights:  64%|██████▎   | 185/291 [00:01<00:00, 122.94it/s]
+Loading weights:  68%|██████▊   | 198/291 [00:01<00:00, 118.80it/s]
+Loading weights:  73%|███████▎  | 212/291 [00:01<00:00, 121.00it/s]
+Loading weights:  77%|███████▋  | 225/291 [00:01<00:00, 117.91it/s]
+Loading weights:  81%|████████▏ | 237/291 [00:02<00:00, 117.99it/s]
+Loading weights:  86%|████████▌ | 249/291 [00:02<00:00, 108.78it/s]
+Loading weights:  91%|█████████ | 265/291 [00:02<00:00, 121.25it/s]
+Loading weights:  96%|█████████▌| 278/291 [00:02<00:00, 117.18it/s]
+Loading weights: 100%|█████████▉| 290/291 [00:02<00:00, 115.86it/s]
+Loading weights: 100%|██████████| 291/291 [00:02<00:00, 116.62it/s]
+loading configuration file generation_config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/generation_config.json
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2
+}
+
+Could not locate the custom_generate/generate.py inside slovak-nlp/mistral-sk-7b.
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: slovak-nlp/mistral-sk-7b had a bad pad_token (</s>). Using pad_token = <unk>.
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:68: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_fwd
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:85: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_bwd
+[INFO|2026-10-13 09:58:16] llamafactory.model.model_utils.checkpointing:144 >> Gradient checkpointing enabled.
+[INFO|2026-10-13 09:58:16] llamafactory.model.adapter:144 >> Upcasting trainable params to float32.
+[INFO|2026-10-13 09:58:16] llamafactory.model.adapter:144 >> Fine-tuning method: LoRA
+Unsloth 2026.6.1 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers.
+[INFO|2026-10-13 09:58:19] llamafactory.model.loader:144 >> trainable params: 41,943,040 || all params: 7,283,675,136 || trainable%: 0.5758
+The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'pad_token_id': 2}.
+==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 1
+   \\   /|    Num examples = 29,000 | Num Epochs = 1 | Total steps = 1,813
+O^O/ \_/ \    Batch size per device = 1 | Gradient accumulation steps = 16
+\        /    Data Parallel GPUs = 1 | Total batch size (1 x 16 x 1) = 16
+ "-____-"     Trainable parameters = 41,943,040 of 7,283,675,136 (0.58% trained)
+
+  0%|          | 0/1813 [00:00<?, ?it/s]`use_return_dict` is deprecated! Use `return_dict` instead!
+Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
+Unsloth: Will smartly offload gradients to save VRAM!
+
+  0%|          | 1/1813 [00:07<3:51:16,  7.66s/it]
+  0%|          | 2/1813 [00:15<3:50:50,  7.65s/it]
+  0%|          | 3/1813 [00:21<3:31:52,  7.02s/it]
+  0%|          | 4/1813 [00:26<3:02:36,  6.06s/it]
+  0%|          | 5/1813 [00:33<3:18:28,  6.59s/it]
+  0%|          | 6/1813 [00:42<3:38:23,  7.25s/it]
+  0%|          | 7/1813 [00:49<3:39:41,  7.30s/it]
+  0%|          | 8/1813 [00:55<3:27:02,  6.88s/it]
+  0%|          | 9/1813 [01:03<3:37:43,  7.24s/it]
+  1%|          | 10/1813 [01:09<3:21:19,  6.70s/it]
+                                                   
+{'loss': '1.78', 'grad_norm': '3.368', 'learning_rate': '1.121e-05', 'epoch': '0.005517', 'num_input_tokens_seen': 65928, 'train_runtime': '69.13', 'train_tokens_per_second': '953.7'}
+
+  1%|          | 10/1813 [01:09<3:21:19,  6.70s/it]
+  1%|          | 11/1813 [01:15<3:16:40,  6.55s/it]
+  1%|          | 12/1813 [01:23<3:27:52,  6.93s/it]
+  1%|          | 13/1813 [01:27<3:02:33,  6.09s/it]
+  1%|          | 14/1813 [01:33<3:03:16,  6.11s/it]
+  1%|          | 15/1813 [01:38<2:56:15,  5.88s/it]
+  1%|          | 16/1813 [01:45<2:59:08,  5.98s/it]
+  1%|          | 17/1813 [01:51<3:02:40,  6.10s/it]
+  1%|          | 18/1813 [01:57<3:02:36,  6.10s/it]
+  1%|          | 19/1813 [02:04<3:07:50,  6.28s/it]
+  1%|          | 20/1813 [02:10<3:05:26,  6.21s/it]
+                                                   
+{'loss': '1.461', 'grad_norm': '1.289', 'learning_rate': '2.367e-05', 'epoch': '0.01103', 'num_input_tokens_seen': 131352, 'train_runtime': '130.2', 'train_tokens_per_second': '1009'}
+
+  1%|          | 20/1813 [02:10<3:05:26,  6.21s/it]
+  1%|          | 21/1813 [02:16<3:02:06,  6.10s/it]
+  1%|          | 22/1813 [02:21<2:53:30,  5.81s/it]
+  1%|▏         | 23/1813 [02:26<2:50:39,  5.72s/it]
+  1%|▏         | 24/1813 [02:32<2:52:20,  5.78s/it]
+  1%|▏         | 25/1813 [02:39<3:05:00,  6.21s/it]
+  1%|▏         | 26/1813 [02:47<3:15:35,  6.57s/it]
+  1%|▏         | 27/1813 [02:53<3:11:12,  6.42s/it]
+  2%|▏         | 28/1813 [03:00<3:15:58,  6.59s/it]
+  2%|▏         | 29/1813 [03:08<3:28:26,  7.01s/it]
+  2%|▏         | 30/1813 [03:13<3:08:02,  6.33s/it]
+                                                   
+{'loss': '1.335', 'grad_norm': '1.347', 'learning_rate': '3.613e-05', 'epoch': '0.01655', 'num_input_tokens_seen': 197376, 'train_runtime': '193', 'train_tokens_per_second': '1022'}
+
+  2%|▏         | 30/1813 [03:13<3:08:02,  6.33s/it]
+  2%|▏         | 31/1813 [03:19<3:08:08,  6.33s/it]
+  2%|▏         | 32/1813 [03:24<3:00:40,  6.09s/it]
+  2%|▏         | 33/1813 [03:31<3:01:25,  6.12s/it]
+  2%|▏         | 34/1813 [03:38<3:15:09,  6.58s/it]
+  2%|▏         | 35/1813 [03:45<3:15:31,  6.60s/it]
+  2%|▏         | 36/1813 [03:51<3:15:10,  6.59s/it]
+  2%|▏         | 37/1813 [03:57<3:03:08,  6.19s/it]
+  2%|▏         | 38/1813 [04:02<2:55:27,  5.93s/it]
+  2%|▏         | 39/1813 [04:08<2:52:39,  5.84s/it]
+  2%|▏         | 40/1813 [04:13<2:51:22,  5.80s/it]
+                                                   
+{'loss': '1.215', 'grad_norm': '1.199', 'learning_rate': '4.86e-05', 'epoch': '0.02207', 'num_input_tokens_seen': 261168, 'train_runtime': '253.9', 'train_tokens_per_second': '1029'}
+
+  2%|▏         | 40/1813 [04:13<2:51:22,  5.80s/it]
+  2%|▏         | 41/1813 [04:19<2:45:49,  5.61s/it]
+  2%|▏         | 42/1813 [04:24<2:47:54,  5.69s/it]
+  2%|▏         | 43/1813 [04:32<3:04:27,  6.25s/it]
+  2%|▏         | 44/1813 [04:38<3:06:24,  6.32s/it]
+  2%|▏         | 45/1813 [04:46<3:15:50,  6.65s/it]
+  3%|▎         | 46/1813 [04:52<3:10:40,  6.47s/it]
+  3%|▎         | 47/1813 [04:58<3:03:33,  6.24s/it]
+  3%|▎         | 48/1813 [05:05<3:09:10,  6.43s/it]
+  3%|▎         | 49/1813 [05:11<3:07:54,  6.39s/it]
+  3%|▎         | 50/1813 [05:17<3:06:29,  6.35s/it]
+                                                   
+{'loss': '1.281', 'grad_norm': '1.491', 'learning_rate': '6.106e-05', 'epoch': '0.02759', 'num_input_tokens_seen': 330104, 'train_runtime': '317.6', 'train_tokens_per_second': '1039'}
+
+  3%|▎         | 50/1813 [05:17<3:06:29,  6.35s/it]
+  3%|▎         | 51/1813 [05:25<3:16:31,  6.69s/it]
+  3%|▎         | 52/1813 [05:34<3:39:34,  7.48s/it]
+  3%|▎         | 53/1813 [05:40<3:27:34,  7.08s/it]
+  3%|▎         | 54/1813 [05:48<3:32:34,  7.25s/it]
+  3%|▎         | 55/1813 [05:54<3:25:45,  7.02s/it]
+  3%|▎         | 56/1813 [06:01<3:22:46,  6.92s/it]
+  3%|▎         | 57/1813 [06:06<3:07:21,  6.40s/it]
+  3%|▎         | 58/1813 [06:14<3:24:11,  6.98s/it]
+  3%|▎         | 59/1813 [06:21<3:24:27,  6.99s/it]
+  3%|▎         | 60/1813 [06:27<3:08:59,  6.47s/it]
+                                                   
+{'loss': '1.174', 'grad_norm': '1.42', 'learning_rate': '7.352e-05', 'epoch': '0.0331', 'num_input_tokens_seen': 407792, 'train_runtime': '387.1', 'train_tokens_per_second': '1053'}
+
+  3%|▎         | 60/1813 [06:27<3:08:59,  6.47s/it]
+  3%|▎         | 61/1813 [06:32<2:57:02,  6.06s/it]
+  3%|▎         | 62/1813 [06:37<2:53:56,  5.96s/it]
+  3%|▎         | 63/1813 [06:43<2:54:08,  5.97s/it]
+  4%|▎         | 64/1813 [06:49<2:53:37,  5.96s/it]
+  4%|▎         | 65/1813 [06:57<3:04:22,  6.33s/it]
+  4%|▎         | 66/1813 [07:03<3:05:17,  6.36s/it]
+  4%|▎         | 67/1813 [07:09<3:00:40,  6.21s/it]
+  4%|▍         | 68/1813 [07:15<3:02:07,  6.26s/it]
+  4%|▍         | 69/1813 [07:23<3:10:29,  6.55s/it]
+  4%|▍         | 70/1813 [07:30<3:19:10,  6.86s/it]
+                                                   
+{'loss': '1.069', 'grad_norm': '1.099', 'learning_rate': '8.598e-05', 'epoch': '0.03862', 'num_input_tokens_seen': 474368, 'train_runtime': '450.6', 'train_tokens_per_second': '1053'}
+
+  4%|▍         | 70/1813 [07:30<3:19:10,  6.86s/it]
+  4%|▍         | 71/1813 [07:36<3:08:59,  6.51s/it]
+  4%|▍         | 72/1813 [07:45<3:28:42,  7.19s/it]
+  4%|▍         | 73/1813 [07:50<3:14:12,  6.70s/it]
+  4%|▍         | 74/1813 [07:56<3:10:09,  6.56s/it]
+  4%|▍         | 75/1813 [08:03<3:10:12,  6.57s/it]
+  4%|▍         | 76/1813 [08:09<3:05:09,  6.40s/it]
+  4%|▍         | 77/1813 [08:16<3:09:45,  6.56s/it]
+  4%|▍         | 78/1813 [08:21<2:58:40,  6.18s/it]
+  4%|▍         | 79/1813 [08:27<2:52:32,  5.97s/it]
+  4%|▍         | 80/1813 [08:33<2:53:50,  6.02s/it]
+                                                   
+{'loss': '1.092', 'grad_norm': '1.223', 'learning_rate': '9.844e-05', 'epoch': '0.04414', 'num_input_tokens_seen': 541616, 'train_runtime': '513.3', 'train_tokens_per_second': '1055'}
+
+  4%|▍         | 80/1813 [08:33<2:53:50,  6.02s/it]
+  4%|▍         | 81/1813 [08:39<2:53:30,  6.01s/it]
+  5%|▍         | 82/1813 [08:46<3:03:03,  6.35s/it]
+  5%|▍         | 83/1813 [08:53<3:09:30,  6.57s/it]
+  5%|▍         | 84/1813 [09:00<3:13:39,  6.72s/it]
+  5%|▍         | 85/1813 [09:06<3:09:29,  6.58s/it]
+  5%|▍         | 86/1813 [09:13<3:07:46,  6.52s/it]
+  5%|▍         | 87/1813 [09:18<2:55:35,  6.10s/it]
+  5%|▍         | 88/1813 [09:24<2:56:11,  6.13s/it]
+  5%|▍         | 89/1813 [09:30<2:55:13,  6.10s/it]
+  5%|▍         | 90/1813 [09:37<2:58:39,  6.22s/it]
+                                                   
+{'loss': '1.025', 'grad_norm': '0.8986', 'learning_rate': '0.0001109', 'epoch': '0.04966', 'num_input_tokens_seen': 608792, 'train_runtime': '577', 'train_tokens_per_second': '1055'}
+
+  5%|▍         | 90/1813 [09:37<2:58:39,  6.22s/it]
+  5%|▌         | 91/1813 [09:42<2:55:29,  6.11s/it]
+  5%|▌         | 92/1813 [09:47<2:45:47,  5.78s/it]
+  5%|▌         | 93/1813 [09:53<2:46:41,  5.82s/it]
+  5%|▌         | 94/1813 [10:00<2:52:22,  6.02s/it]
+  5%|▌         | 95/1813 [10:06<2:57:21,  6.19s/it]
+  5%|▌         | 96/1813 [10:12<2:55:22,  6.13s/it]
+  5%|▌         | 97/1813 [10:19<2:58:30,  6.24s/it]
+  5%|▌         | 98/1813 [10:24<2:48:33,  5.90s/it]
+  5%|▌         | 99/1813 [10:30<2:45:34,  5.80s/it]
+  6%|▌         | 100/1813 [10:36<2:49:55,  5.95s/it]
+                                                    
+{'loss': '1.064', 'grad_norm': '0.8584', 'learning_rate': '0.0001234', 'epoch': '0.05517', 'num_input_tokens_seen': 670936, 'train_runtime': '636.3', 'train_tokens_per_second': '1054'}
+
+  6%|▌         | 100/1813 [10:36<2:49:55,  5.95s/it]
+  6%|▌         | 101/1813 [10:43<2:56:51,  6.20s/it]
+  6%|▌         | 102/1813 [10:49<3:01:54,  6.38s/it]
+  6%|▌         | 103/1813 [10:56<2:59:18,  6.29s/it]
+  6%|▌         | 104/1813 [11:02<2:58:39,  6.27s/it]
+  6%|▌         | 105/1813 [11:07<2:51:24,  6.02s/it]
+  6%|▌         | 106/1813 [11:13<2:49:19,  5.95s/it]
+  6%|▌         | 107/1813 [11:19<2:51:53,  6.05s/it]
+  6%|▌         | 108/1813 [11:26<2:56:28,  6.21s/it]
+  6%|▌         | 109/1813 [11:32<2:59:45,  6.33s/it]
+  6%|▌         | 110/1813 [11:41<3:17:49,  6.97s/it]
+                                                    
+{'loss': '1.098', 'grad_norm': '0.8312', 'learning_rate': '0.0001358', 'epoch': '0.06069', 'num_input_tokens_seen': 738632, 'train_runtime': '701.4', 'train_tokens_per_second': '1053'}
+
+  6%|▌         | 110/1813 [11:41<3:17:49,  6.97s/it]
+  6%|▌         | 111/1813 [11:49<3:24:04,  7.19s/it]
+  6%|▌         | 112/1813 [11:55<3:19:50,  7.05s/it]
+  6%|▌         | 113/1813 [12:02<3:14:47,  6.87s/it]
+  6%|▋         | 114/1813 [12:09<3:19:32,  7.05s/it]
+  6%|▋         | 115/1813 [12:15<3:09:39,  6.70s/it]
+  6%|▋         | 116/1813 [12:22<3:11:15,  6.76s/it]
+  6%|▋         | 117/1813 [12:28<3:01:57,  6.44s/it]
+  7%|▋         | 118/1813 [12:34<2:58:40,  6.32s/it]
+  7%|▋         | 119/1813 [12:40<2:59:24,  6.35s/it]
+  7%|▋         | 120/1813 [12:47<3:00:27,  6.40s/it]
+                                                    
+{'loss': '1.104', 'grad_norm': '1.158', 'learning_rate': '0.0001483', 'epoch': '0.06621', 'num_input_tokens_seen': 808824, 'train_runtime': '767.2', 'train_tokens_per_second': '1054'}
+
+  7%|▋         | 120/1813 [12:47<3:00:27,  6.40s/it]
+  7%|▋         | 121/1813 [12:51<2:44:06,  5.82s/it]
+  7%|▋         | 122/1813 [12:56<2:39:24,  5.66s/it]
+  7%|▋         | 123/1813 [13:03<2:47:29,  5.95s/it]
+  7%|▋         | 124/1813 [13:09<2:43:48,  5.82s/it]
+  7%|▋         | 125/1813 [13:15<2:51:23,  6.09s/it]
+  7%|▋         | 126/1813 [13:22<2:52:49,  6.15s/it]
+  7%|▋         | 127/1813 [13:26<2:39:27,  5.67s/it]
+  7%|▋         | 128/1813 [13:31<2:31:57,  5.41s/it]
+  7%|▋         | 129/1813 [13:39<2:54:20,  6.21s/it]
+  7%|▋         | 130/1813 [13:46<2:57:42,  6.34s/it]
+                                                    
+{'loss': '1.046', 'grad_norm': '0.7445', 'learning_rate': '0.0001607', 'epoch': '0.07172', 'num_input_tokens_seen': 869720, 'train_runtime': '826.2', 'train_tokens_per_second': '1053'}
+
+  7%|▋         | 130/1813 [13:46<2:57:42,  6.34s/it]
+  7%|▋         | 131/1813 [13:52<2:55:27,  6.26s/it]
+  7%|▋         | 132/1813 [13:57<2:49:35,  6.05s/it]
+  7%|▋         | 133/1813 [14:03<2:47:32,  5.98s/it]
+  7%|▋         | 134/1813 [14:08<2:41:21,  5.77s/it]
+  7%|▋         | 135/1813 [14:14<2:37:09,  5.62s/it]
+  8%|▊         | 136/1813 [14:19<2:37:43,  5.64s/it]
+  8%|▊         | 137/1813 [14:25<2:41:24,  5.78s/it]
+  8%|▊         | 138/1813 [14:33<2:52:48,  6.19s/it]
+  8%|▊         | 139/1813 [14:39<2:56:30,  6.33s/it]
+  8%|▊         | 140/1813 [14:46<3:03:05,  6.57s/it]
+                                                    
+{'loss': '1.044', 'grad_norm': '1.078', 'learning_rate': '0.0001732', 'epoch': '0.07724', 'num_input_tokens_seen': 932000, 'train_runtime': '886.9', 'train_tokens_per_second': '1051'}
+
+  8%|▊         | 140/1813 [14:46<3:03:05,  6.57s/it]
+  8%|▊         | 141/1813 [14:52<2:56:23,  6.33s/it]
+  8%|▊         | 142/1813 [15:00<3:10:18,  6.83s/it]
+  8%|▊         | 143/1813 [15:06<2:59:51,  6.46s/it]
+  8%|▊         | 144/1813 [15:11<2:51:09,  6.15s/it]
+  8%|▊         | 145/1813 [15:18<2:59:36,  6.46s/it]
+  8%|▊         | 146/1813 [15:25<2:59:01,  6.44s/it]
+  8%|▊         | 147/1813 [15:30<2:52:06,  6.20s/it]
+  8%|▊         | 148/1813 [15:37<2:51:19,  6.17s/it]
+  8%|▊         | 149/1813 [15:42<2:48:37,  6.08s/it]
+  8%|▊         | 150/1813 [15:49<2:52:07,  6.21s/it]
+                                                    
+{'loss': '1.099', 'grad_norm': '0.9282', 'learning_rate': '0.0001857', 'epoch': '0.08276', 'num_input_tokens_seen': 997296, 'train_runtime': '949.4', 'train_tokens_per_second': '1050'}
+
+  8%|▊         | 150/1813 [15:49<2:52:07,  6.21s/it]
+  8%|▊         | 151/1813 [15:55<2:47:21,  6.04s/it]
+  8%|▊         | 152/1813 [16:00<2:44:40,  5.95s/it]
+  8%|▊         | 153/1813 [16:06<2:45:05,  5.97s/it]
+  8%|▊         | 154/1813 [16:13<2:49:24,  6.13s/it]
+  9%|▊         | 155/1813 [16:20<2:56:54,  6.40s/it]
+  9%|▊         | 156/1813 [16:26<2:51:44,  6.22s/it]
+  9%|▊         | 157/1813 [16:32<2:49:28,  6.14s/it]
+  9%|▊         | 158/1813 [16:37<2:39:47,  5.79s/it]
+  9%|▉         | 159/1813 [16:42<2:40:21,  5.82s/it]
+  9%|▉         | 160/1813 [16:48<2:38:29,  5.75s/it]
+                                                    
+{'loss': '1.026', 'grad_norm': '1.537', 'learning_rate': '0.0001869', 'epoch': '0.08828', 'num_input_tokens_seen': 1060096, 'train_runtime': '1009', 'train_tokens_per_second': '1051'}
+
+  9%|▉         | 160/1813 [16:48<2:38:29,  5.75s/it]
+  9%|▉         | 161/1813 [16:54<2:38:20,  5.75s/it]
+  9%|▉         | 162/1813 [16:58<2:27:03,  5.34s/it]
+  9%|▉         | 163/1813 [17:05<2:39:17,  5.79s/it]
+  9%|▉         | 164/1813 [17:12<2:51:16,  6.23s/it]
+  9%|▉         | 165/1813 [17:19<2:51:07,  6.23s/it]
+  9%|▉         | 166/1813 [17:26<3:03:54,  6.70s/it]
+  9%|▉         | 167/1813 [17:33<3:03:09,  6.68s/it]
+  9%|▉         | 168/1813 [17:40<3:06:46,  6.81s/it]
+  9%|▉         | 169/1813 [17:46<2:57:09,  6.47s/it]
+  9%|▉         | 170/1813 [17:52<2:58:42,  6.53s/it]
+                                                    
+{'loss': '1.051', 'grad_norm': '0.6274', 'learning_rate': '0.0001868', 'epoch': '0.09379', 'num_input_tokens_seen': 1130368, 'train_runtime': '1073', 'train_tokens_per_second': '1054'}
+
+  9%|▉         | 170/1813 [17:52<2:58:42,  6.53s/it]
+  9%|▉         | 171/1813 [17:58<2:48:54,  6.17s/it]
+  9%|▉         | 172/1813 [18:03<2:42:11,  5.93s/it]
+ 10%|▉         | 173/1813 [18:09<2:45:58,  6.07s/it]
+ 10%|▉         | 174/1813 [18:15<2:43:47,  6.00s/it]
+ 10%|▉         | 175/1813 [18:22<2:47:53,  6.15s/it]
+ 10%|▉         | 176/1813 [18:27<2:41:22,  5.91s/it]
+ 10%|▉         | 177/1813 [18:35<2:53:28,  6.36s/it]
+ 10%|▉         | 178/1813 [18:39<2:41:08,  5.91s/it]
+ 10%|▉         | 179/1813 [18:45<2:40:43,  5.90s/it]
+ 10%|▉         | 180/1813 [18:51<2:41:48,  5.95s/it]
+                                                    
+{'loss': '1.037', 'grad_norm': '0.9569', 'learning_rate': '0.0001868', 'epoch': '0.09931', 'num_input_tokens_seen': 1193192, 'train_runtime': '1132', 'train_tokens_per_second': '1054'}
+
+ 10%|▉         | 180/1813 [18:51<2:41:48,  5.95s/it]
+ 10%|▉         | 181/1813 [18:57<2:38:07,  5.81s/it]
+ 10%|█         | 182/1813 [19:02<2:34:28,  5.68s/it]
+ 10%|█         | 183/1813 [19:08<2:33:12,  5.64s/it]
+ 10%|█         | 184/1813 [19:13<2:31:41,  5.59s/it]
+ 10%|█         | 185/1813 [19:21<2:51:26,  6.32s/it]
+ 10%|█         | 186/1813 [19:28<2:51:16,  6.32s/it]
+ 10%|█         | 187/1813 [19:34<2:49:13,  6.24s/it]
+ 10%|█         | 188/1813 [19:39<2:40:49,  5.94s/it]
+ 10%|█         | 189/1813 [19:46<2:46:23,  6.15s/it]
+ 10%|█         | 190/1813 [19:52<2:50:22,  6.30s/it]
+                                                    
+{'loss': '0.9761', 'grad_norm': '0.949', 'learning_rate': '0.0001867', 'epoch': '0.1048', 'num_input_tokens_seen': 1259336, 'train_runtime': '1193', 'train_tokens_per_second': '1056'}
+
+ 10%|█         | 190/1813 [19:52<2:50:22,  6.30s/it]
+ 11%|█         | 191/1813 [19:58<2:42:36,  6.02s/it]
+ 11%|█         | 192/1813 [20:04<2:46:33,  6.17s/it]
+ 11%|█         | 193/1813 [20:11<2:49:13,  6.27s/it]
+ 11%|█         | 194/1813 [20:16<2:46:13,  6.16s/it]
+ 11%|█         | 195/1813 [20:24<2:57:54,  6.60s/it]
+ 11%|█         | 196/1813 [20:30<2:48:48,  6.26s/it]
+ 11%|█         | 197/1813 [20:37<2:55:14,  6.51s/it]
+ 11%|█         | 198/1813 [20:44<3:02:05,  6.77s/it]
+ 11%|█         | 199/1813 [20:51<3:01:34,  6.75s/it]
+ 11%|█         | 200/1813 [20:57<3:00:53,  6.73s/it]
+                                                    
+{'loss': '1.086', 'grad_norm': '1.19', 'learning_rate': '0.0001865', 'epoch': '0.1103', 'num_input_tokens_seen': 1331224, 'train_runtime': '1258', 'train_tokens_per_second': '1058'}
+
+ 11%|█         | 200/1813 [20:57<3:00:53,  6.73s/it]
+ 11%|█         | 201/1813 [21:03<2:52:37,  6.43s/it]
+ 11%|█         | 202/1813 [21:08<2:43:18,  6.08s/it]
+ 11%|█         | 203/1813 [21:13<2:35:09,  5.78s/it]
+ 11%|█▏        | 204/1813 [21:20<2:44:03,  6.12s/it]
+ 11%|█▏        | 205/1813 [21:26<2:36:51,  5.85s/it]
+ 11%|█▏        | 206/1813 [21:31<2:36:25,  5.84s/it]
+ 11%|█▏        | 207/1813 [21:37<2:35:48,  5.82s/it]
+ 11%|█▏        | 208/1813 [21:45<2:52:26,  6.45s/it]
+ 12%|█▏        | 209/1813 [21:50<2:41:55,  6.06s/it]
+ 12%|█▏        | 210/1813 [21:56<2:42:55,  6.10s/it]
+                                                    
+{'loss': '0.9669', 'grad_norm': '1.058', 'learning_rate': '0.0001863', 'epoch': '0.1159', 'num_input_tokens_seen': 1392472, 'train_runtime': '1317', 'train_tokens_per_second': '1057'}
+
+ 12%|█▏        | 210/1813 [21:56<2:42:55,  6.10s/it]
+ 12%|█▏        | 211/1813 [22:05<3:03:38,  6.88s/it]
+ 12%|█▏        | 212/1813 [22:11<2:55:48,  6.59s/it]
+ 12%|█▏        | 213/1813 [22:17<2:46:38,  6.25s/it]
+ 12%|█▏        | 214/1813 [22:22<2:37:23,  5.91s/it]
+ 12%|█▏        | 215/1813 [22:27<2:33:34,  5.77s/it]
+ 12%|█▏        | 216/1813 [22:32<2:24:20,  5.42s/it]
+ 12%|█▏        | 217/1813 [22:37<2:24:06,  5.42s/it]
+ 12%|█▏        | 218/1813 [22:44<2:31:55,  5.71s/it]
+ 12%|█▏        | 219/1813 [22:50<2:35:18,  5.85s/it]
+ 12%|█▏        | 220/1813 [22:56<2:37:51,  5.95s/it]
+                                                    
+{'loss': '1.038', 'grad_norm': '0.8456', 'learning_rate': '0.0001861', 'epoch': '0.1214', 'num_input_tokens_seen': 1454040, 'train_runtime': '1376', 'train_tokens_per_second': '1056'}
+
+ 12%|█▏        | 220/1813 [22:56<2:37:51,  5.95s/it]
+ 12%|█▏        | 221/1813 [23:02<2:42:15,  6.12s/it]
+ 12%|█▏        | 222/1813 [23:07<2:34:17,  5.82s/it]
+ 12%|█▏        | 223/1813 [23:15<2:49:35,  6.40s/it]
+ 12%|█▏        | 224/1813 [23:21<2:40:43,  6.07s/it]
+ 12%|█▏        | 225/1813 [23:28<2:48:40,  6.37s/it]
+ 12%|█▏        | 226/1813 [23:34<2:52:14,  6.51s/it]
+ 13%|█▎        | 227/1813 [23:40<2:45:44,  6.27s/it]
+ 13%|█▎        | 228/1813 [23:47<2:46:45,  6.31s/it]
+ 13%|█▎        | 229/1813 [23:53<2:51:01,  6.48s/it]
+ 13%|█▎        | 230/1813 [23:58<2:38:19,  6.00s/it]
+                                                    
+{'loss': '1.007', 'grad_norm': '1.365', 'learning_rate': '0.0001859', 'epoch': '0.1269', 'num_input_tokens_seen': 1522056, 'train_runtime': '1439', 'train_tokens_per_second': '1058'}
+
+ 13%|█▎        | 230/1813 [23:58<2:38:19,  6.00s/it]
+ 13%|█▎        | 231/1813 [24:03<2:28:49,  5.64s/it]
+ 13%|█▎        | 232/1813 [24:10<2:42:04,  6.15s/it]
+ 13%|█▎        | 233/1813 [24:15<2:32:33,  5.79s/it]
+ 13%|█▎        | 234/1813 [24:21<2:27:21,  5.60s/it]
+ 13%|█▎        | 235/1813 [24:27<2:36:37,  5.96s/it]
+ 13%|█▎        | 236/1813 [24:35<2:48:20,  6.41s/it]
+ 13%|█▎        | 237/1813 [24:42<2:51:24,  6.53s/it]
+ 13%|█▎        | 238/1813 [24:47<2:45:30,  6.30s/it]
+ 13%|█▎        | 239/1813 [24:55<2:56:07,  6.71s/it]
+ 13%|█▎        | 240/1813 [25:01<2:52:19,  6.57s/it]
+                                                    
+{'loss': '1.004', 'grad_norm': '0.8374', 'learning_rate': '0.0001856', 'epoch': '0.1324', 'num_input_tokens_seen': 1590200, 'train_runtime': '1502', 'train_tokens_per_second': '1059'}
+
+ 13%|█▎        | 240/1813 [25:01<2:52:19,  6.57s/it]
+ 13%|█▎        | 241/1813 [25:08<2:50:55,  6.52s/it]
+ 13%|█▎        | 242/1813 [25:14<2:48:25,  6.43s/it]
+ 13%|█▎        | 243/1813 [25:19<2:40:29,  6.13s/it]
+ 13%|█▎        | 244/1813 [25:26<2:47:04,  6.39s/it]
+ 14%|█▎        | 245/1813 [25:33<2:51:42,  6.57s/it]
+ 14%|█▎        | 246/1813 [25:39<2:44:55,  6.32s/it]
+ 14%|█▎        | 247/1813 [25:46<2:50:04,  6.52s/it]
+ 14%|█▎        | 248/1813 [25:54<2:59:50,  6.89s/it]
+ 14%|█▎        | 249/1813 [25:59<2:47:03,  6.41s/it]
+ 14%|█▍        | 250/1813 [26:05<2:40:04,  6.14s/it]
+                                                    
+{'loss': '1.084', 'grad_norm': '0.9052', 'learning_rate': '0.0001853', 'epoch': '0.1379', 'num_input_tokens_seen': 1658648, 'train_runtime': '1565', 'train_tokens_per_second': '1060'}
+
+ 14%|█▍        | 250/1813 [26:05<2:40:04,  6.14s/it]
+ 14%|█▍        | 251/1813 [26:09<2:27:22,  5.66s/it]
+ 14%|█▍        | 252/1813 [26:15<2:24:36,  5.56s/it]
+ 14%|█▍        | 253/1813 [26:21<2:29:22,  5.74s/it]
+ 14%|█▍        | 254/1813 [26:28<2:41:32,  6.22s/it]
+ 14%|█▍        | 255/1813 [26:35<2:44:41,  6.34s/it]
+ 14%|█▍        | 256/1813 [26:40<2:36:55,  6.05s/it]
+ 14%|█▍        | 257/1813 [26:47<2:41:49,  6.24s/it]
+ 14%|█▍        | 258/1813 [26:54<2:46:14,  6.41s/it]
+ 14%|█▍        | 259/1813 [26:58<2:31:25,  5.85s/it]
+ 14%|█▍        | 260/1813 [27:04<2:32:46,  5.90s/it]
+                                                    
+{'loss': '0.8729', 'grad_norm': '1.076', 'learning_rate': '0.0001849', 'epoch': '0.1434', 'num_input_tokens_seen': 1722792, 'train_runtime': '1625', 'train_tokens_per_second': '1060'}
+
+ 14%|█▍        | 260/1813 [27:04<2:32:46,  5.90s/it]
+ 14%|█▍        | 261/1813 [27:11<2:42:08,  6.27s/it]
+ 14%|█▍        | 262/1813 [27:18<2:44:37,  6.37s/it]
+ 15%|█▍        | 263/1813 [27:24<2:45:06,  6.39s/it]
+ 15%|█▍        | 264/1813 [27:30<2:43:10,  6.32s/it]
+ 15%|█▍        | 265/1813 [27:37<2:42:23,  6.29s/it]
+ 15%|█▍        | 266/1813 [27:41<2:30:36,  5.84s/it]
+ 15%|█▍        | 267/1813 [27:47<2:27:14,  5.71s/it]
+ 15%|█▍        | 268/1813 [27:52<2:24:08,  5.60s/it]
+ 15%|█▍        | 269/1813 [27:59<2:30:03,  5.83s/it]
+ 15%|█▍        | 270/1813 [28:05<2:31:27,  5.89s/it]
+                                                    
+{'loss': '0.9312', 'grad_norm': '0.9894', 'learning_rate': '0.0001846', 'epoch': '0.149', 'num_input_tokens_seen': 1786808, 'train_runtime': '1685', 'train_tokens_per_second': '1060'}
+
+ 15%|█▍        | 270/1813 [28:05<2:31:27,  5.89s/it]
+ 15%|█▍        | 271/1813 [28:13<2:48:11,  6.54s/it]
+ 15%|█▌        | 272/1813 [28:18<2:39:36,  6.21s/it]
+ 15%|█▌        | 273/1813 [28:26<2:55:09,  6.82s/it]
+ 15%|█▌        | 274/1813 [28:33<2:53:45,  6.77s/it]
+ 15%|█▌        | 275/1813 [28:38<2:40:05,  6.25s/it]
+ 15%|█▌        | 276/1813 [28:43<2:30:29,  5.87s/it]
+ 15%|█▌        | 277/1813 [28:49<2:32:29,  5.96s/it]
+ 15%|█▌        | 278/1813 [28:55<2:33:42,  6.01s/it]
+ 15%|█▌        | 279/1813 [29:03<2:44:50,  6.45s/it]
+ 15%|█▌        | 280/1813 [29:09<2:44:30,  6.44s/it]
+                                                    
+{'loss': '1.006', 'grad_norm': '1.018', 'learning_rate': '0.0001841', 'epoch': '0.1545', 'num_input_tokens_seen': 1858112, 'train_runtime': '1750', 'train_tokens_per_second': '1062'}
+
+ 15%|█▌        | 280/1813 [29:09<2:44:30,  6.44s/it]
+ 15%|█▌        | 281/1813 [29:15<2:38:17,  6.20s/it]
+ 16%|█▌        | 282/1813 [29:20<2:31:30,  5.94s/it]
+ 16%|█▌        | 283/1813 [29:25<2:25:11,  5.69s/it]
+ 16%|█▌        | 284/1813 [29:31<2:23:48,  5.64s/it]
+ 16%|█▌        | 285/1813 [29:37<2:31:17,  5.94s/it]
+ 16%|█▌        | 286/1813 [29:43<2:31:09,  5.94s/it]
+ 16%|█▌        | 287/1813 [29:49<2:32:32,  6.00s/it]
+ 16%|█▌        | 288/1813 [29:55<2:27:29,  5.80s/it]
+ 16%|█▌        | 289/1813 [30:01<2:31:39,  5.97s/it]
+ 16%|█▌        | 290/1813 [30:08<2:35:38,  6.13s/it]
+                                                    
+{'loss': '0.9673', 'grad_norm': '0.8781', 'learning_rate': '0.0001837', 'epoch': '0.16', 'num_input_tokens_seen': 1920600, 'train_runtime': '1808', 'train_tokens_per_second': '1062'}
+
+ 16%|█▌        | 290/1813 [30:08<2:35:38,  6.13s/it]
+ 16%|█▌        | 291/1813 [30:15<2:41:37,  6.37s/it]
+ 16%|█▌        | 292/1813 [30:22<2:47:49,  6.62s/it]
+ 16%|█▌        | 293/1813 [30:27<2:40:22,  6.33s/it]
+ 16%|█▌        | 294/1813 [30:35<2:46:21,  6.57s/it]
+ 16%|█▋        | 295/1813 [30:42<2:51:45,  6.79s/it]
+ 16%|█▋        | 296/1813 [30:47<2:42:14,  6.42s/it]
+ 16%|█▋        | 297/1813 [30:53<2:39:11,  6.30s/it]
+ 16%|█▋        | 298/1813 [30:58<2:26:39,  5.81s/it]
+ 16%|█▋        | 299/1813 [31:04<2:26:42,  5.81s/it]
+ 17%|█▋        | 300/1813 [31:10<2:25:38,  5.78s/it]
+                                                    
+{'loss': '1.041', 'grad_norm': '4.404', 'learning_rate': '0.0001832', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312, 'train_runtime': '1870', 'train_tokens_per_second': '1063'}
+
+ 17%|█▋        | 300/1813 [31:10<2:25:38,  5.78s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.74it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.23it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.32it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.09it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.01it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.11it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.59it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.76it/s]
+
+  4%|▍         | 42/1000 [00:06<01:44,  9.12it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.92it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.30it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.76it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+
+  7%|▋         | 71/1000 [00:09<02:04,  7.45it/s]
+
+  7%|▋         | 72/1000 [00:09<02:27,  6.31it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.09it/s]
+
+  8%|▊         | 75/1000 [00:10<02:18,  6.69it/s]
+
+  8%|▊         | 76/1000 [00:10<02:21,  6.52it/s]
+
+  8%|▊         | 77/1000 [00:10<02:09,  7.14it/s]
+
+  8%|▊         | 78/1000 [00:10<02:34,  5.95it/s]
+
+  8%|▊         | 80/1000 [00:10<01:55,  7.97it/s]
+
+  8%|▊         | 82/1000 [00:10<01:36,  9.53it/s]
+
+  8%|▊         | 84/1000 [00:11<01:53,  8.10it/s]
+
+  8%|▊         | 85/1000 [00:11<02:15,  6.74it/s]
+
+  9%|▊         | 86/1000 [00:11<02:35,  5.88it/s]
+
+  9%|▊         | 87/1000 [00:12<02:51,  5.32it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  6.99it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.16it/s]
+
+  9%|▉         | 93/1000 [00:12<02:37,  5.77it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.51it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.38it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.65it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.32it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.45it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.21it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.05it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.55it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.83it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.23it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.09it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.44it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.81it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.14it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:29,  9.65it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:27,  9.81it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  8.95it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:37,  8.82it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25, 10.00it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.40it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.47it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.90it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.02it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.36it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.63it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.96it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.60it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.77it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.77it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.12it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.13it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.21it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.72it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:38,  8.38it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.78it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:36,  8.54it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.49it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.74it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.71it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.13it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.64it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.20it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.68it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.78it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.64it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.96it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.53it/s]
+
+ 21%|██        | 210/1000 [00:27<01:32,  8.50it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.69it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.75it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.16it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:29,  8.75it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:04,  6.27it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:04,  6.27it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.53it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.02it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.07it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.81it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.36it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.03it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:19,  9.56it/s]
+
+ 24%|██▎       | 236/1000 [00:30<01:20,  9.51it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:23,  9.19it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:27,  8.74it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:34,  8.06it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:12,  5.73it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:24,  5.25it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:55,  6.52it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.19it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:50,  6.79it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:07,  5.91it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:08,  5.83it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.01it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.72it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:42,  7.28it/s]
+
+ 26%|██▌       | 256/1000 [00:34<01:37,  7.63it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:25,  8.70it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.66it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.48it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:55,  6.41it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.82it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.63it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.72it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.99it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.11it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.77it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.15it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.78it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.80it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.22it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.29it/s]
+
+ 32%|███▏      | 315/1000 [00:42<01:06, 10.25it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.58it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.45it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.77it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.80it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.36it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.83it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.02it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.87it/s]
+
+ 34%|███▍      | 344/1000 [00:46<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.23it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.42it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:33,  6.93it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:27,  7.42it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:28,  7.27it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:12,  8.87it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.53it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:08,  9.38it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.51it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.69it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.06it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.21it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:51<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.78it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.92it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.22it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:07,  9.15it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:26,  7.11it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:20,  7.62it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.36it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.78it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.45it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.74it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.24it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.51it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:56<01:23,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.27it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.46it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.52it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.70it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:36,  5.94it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:12,  7.85it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.56it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.04it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.82it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.50it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.09it/s]
+
+ 44%|████▍     | 441/1000 [00:59<00:59,  9.32it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.11it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.52it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.27it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.82it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.32it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.50it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.32it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.23it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.42it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.44it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.94it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.56it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.81it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.57it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.97it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.55it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.12it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.04it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:58,  8.04it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.65it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.67it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.34it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.96it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.86it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.07it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.39it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.46it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.84it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.59it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.76it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.19it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.40it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.86it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:45,  8.48it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:40,  9.51it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:36, 10.62it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.54it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.70it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:42,  9.05it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  7.99it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.18it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.63it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.18it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.43it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.96it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.71it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.58it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.49it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.34it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.65it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.96it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.92it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.89it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.82it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.85it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.27it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.35it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.23it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.04it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.64it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.51it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.72it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.88it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.38it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.26it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.14it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.62it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.34it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.62it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.46it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.69it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.24it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.69it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.78it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.13it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.40it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.80it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.91it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.38it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  9.00it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.74it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.98it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.21it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.84it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.98it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.22it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.24it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.04it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.31it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.81it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.76it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.46it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.14it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.48it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.19it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.64it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.73it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.27it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.30it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.79it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.64it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.89it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.88it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:11,  9.91it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.55it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.54it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:14,  7.54it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.75it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.51it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.68it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.84it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.05it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.16it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.62it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.84it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.06it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.15it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.35it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.73it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.85it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.42it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.41it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.41it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.83it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.43it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.04it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.21it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.81it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.42it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.05it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.35it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                    
+
+                                                   
+{'eval_loss': '0.8437', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.382', 'eval_steps_per_second': '7.382', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312}
+
+ 17%|█▋        | 300/1813 [33:25<2:25:38,  5.78s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 17%|█▋        | 301/1813 [33:31<19:27:18, 46.32s/it]
+ 17%|█▋        | 302/1813 [33:35<14:13:21, 33.89s/it]
+ 17%|█▋        | 303/1813 [33:42<10:49:58, 25.83s/it]
+ 17%|█▋        | 304/1813 [33:50<8:31:37, 20.34s/it] 
+ 17%|█▋        | 305/1813 [33:56<6:44:41, 16.10s/it]
+ 17%|█▋        | 306/1813 [34:02<5:28:32, 13.08s/it]
+ 17%|█▋        | 307/1813 [34:08<4:32:21, 10.85s/it]
+ 17%|█▋        | 308/1813 [34:14<3:53:22,  9.30s/it]
+ 17%|█▋        | 309/1813 [34:20<3:31:48,  8.45s/it]
+ 17%|█▋        | 310/1813 [34:25<3:04:49,  7.38s/it]
+                                                    
+{'loss': '0.9646', 'grad_norm': '1.161', 'learning_rate': '0.0001827', 'epoch': '0.171', 'num_input_tokens_seen': 2050672, 'train_runtime': '2065', 'train_tokens_per_second': '992.8'}
+
+ 17%|█▋        | 310/1813 [34:25<3:04:49,  7.38s/it]
+ 17%|█▋        | 311/1813 [34:32<3:01:59,  7.27s/it]
+ 17%|█▋        | 312/1813 [34:39<3:01:55,  7.27s/it]
+ 17%|█▋        | 313/1813 [34:46<2:57:31,  7.10s/it]
+ 17%|█▋        | 314/1813 [34:51<2:45:00,  6.60s/it]
+ 17%|█▋        | 315/1813 [34:58<2:46:44,  6.68s/it]
+ 17%|█▋        | 316/1813 [35:05<2:43:29,  6.55s/it]
+ 17%|█▋        | 317/1813 [35:10<2:37:29,  6.32s/it]
+ 18%|█▊        | 318/1813 [35:16<2:29:56,  6.02s/it]
+ 18%|█▊        | 319/1813 [35:20<2:20:06,  5.63s/it]
+ 18%|█▊        | 320/1813 [35:26<2:21:06,  5.67s/it]
+                                                    
+{'loss': '1.048', 'grad_norm': '0.618', 'learning_rate': '0.0001822', 'epoch': '0.1766', 'num_input_tokens_seen': 2118000, 'train_runtime': '2127', 'train_tokens_per_second': '996'}
+
+ 18%|█▊        | 320/1813 [35:26<2:21:06,  5.67s/it]
+ 18%|█▊        | 321/1813 [35:32<2:22:03,  5.71s/it]
+ 18%|█▊        | 322/1813 [35:38<2:22:16,  5.73s/it]
+ 18%|█▊        | 323/1813 [35:44<2:26:03,  5.88s/it]
+ 18%|█▊        | 324/1813 [35:51<2:36:03,  6.29s/it]
+ 18%|█▊        | 325/1813 [35:58<2:42:32,  6.55s/it]
+ 18%|█▊        | 326/1813 [36:04<2:39:11,  6.42s/it]
+ 18%|█▊        | 327/1813 [36:10<2:34:06,  6.22s/it]
+ 18%|█▊        | 328/1813 [36:17<2:36:01,  6.30s/it]
+ 18%|█▊        | 329/1813 [36:23<2:33:10,  6.19s/it]
+ 18%|█▊        | 330/1813 [36:28<2:30:25,  6.09s/it]
+                                                    
+{'loss': '0.9573', 'grad_norm': '1.023', 'learning_rate': '0.0001816', 'epoch': '0.1821', 'num_input_tokens_seen': 2184640, 'train_runtime': '2189', 'train_tokens_per_second': '998'}
+
+ 18%|█▊        | 330/1813 [36:28<2:30:25,  6.09s/it]
+ 18%|█▊        | 331/1813 [36:35<2:31:03,  6.12s/it]
+ 18%|█▊        | 332/1813 [36:40<2:22:34,  5.78s/it]
+ 18%|█▊        | 333/1813 [36:46<2:23:47,  5.83s/it]
+ 18%|█▊        | 334/1813 [36:51<2:21:49,  5.75s/it]
+ 18%|█▊        | 335/1813 [36:59<2:36:04,  6.34s/it]
+ 19%|█▊        | 336/1813 [37:06<2:41:29,  6.56s/it]
+ 19%|█▊        | 337/1813 [37:12<2:35:48,  6.33s/it]
+ 19%|█▊        | 338/1813 [37:18<2:35:43,  6.33s/it]
+ 19%|█▊        | 339/1813 [37:24<2:31:42,  6.18s/it]
+ 19%|█▉        | 340/1813 [37:31<2:39:06,  6.48s/it]
+                                                    
+{'loss': '0.9417', 'grad_norm': '1.173', 'learning_rate': '0.000181', 'epoch': '0.1876', 'num_input_tokens_seen': 2251392, 'train_runtime': '2252', 'train_tokens_per_second': '999.9'}
+
+ 19%|█▉        | 340/1813 [37:31<2:39:06,  6.48s/it]
+ 19%|█▉        | 341/1813 [37:37<2:38:28,  6.46s/it]
+ 19%|█▉        | 342/1813 [37:45<2:45:54,  6.77s/it]
+ 19%|█▉        | 343/1813 [37:51<2:38:51,  6.48s/it]
+ 19%|█▉        | 344/1813 [37:56<2:33:11,  6.26s/it]
+ 19%|█▉        | 345/1813 [38:04<2:40:07,  6.54s/it]
+ 19%|█▉        | 346/1813 [38:10<2:37:56,  6.46s/it]
+ 19%|█▉        | 347/1813 [38:15<2:26:47,  6.01s/it]
+ 19%|█▉        | 348/1813 [38:21<2:30:19,  6.16s/it]
+ 19%|█▉        | 349/1813 [38:30<2:46:47,  6.84s/it]
+ 19%|█▉        | 350/1813 [38:38<2:52:46,  7.09s/it]
+                                                    
+{'loss': '1.054', 'grad_norm': '0.9134', 'learning_rate': '0.0001804', 'epoch': '0.1931', 'num_input_tokens_seen': 2324712, 'train_runtime': '2318', 'train_tokens_per_second': '1003'}
+
+ 19%|█▉        | 350/1813 [38:38<2:52:46,  7.09s/it]
+ 19%|█▉        | 351/1813 [38:44<2:51:29,  7.04s/it]
+ 19%|█▉        | 352/1813 [38:50<2:43:03,  6.70s/it]
+ 19%|█▉        | 353/1813 [38:56<2:34:25,  6.35s/it]
+ 20%|█▉        | 354/1813 [39:01<2:28:30,  6.11s/it]
+ 20%|█▉        | 355/1813 [39:07<2:27:21,  6.06s/it]
+ 20%|█▉        | 356/1813 [39:14<2:33:40,  6.33s/it]
+ 20%|█▉        | 357/1813 [39:20<2:28:35,  6.12s/it]
+ 20%|█▉        | 358/1813 [39:26<2:29:09,  6.15s/it]
+ 20%|█▉        | 359/1813 [39:32<2:25:17,  6.00s/it]
+ 20%|█▉        | 360/1813 [39:39<2:31:29,  6.26s/it]
+                                                    
+{'loss': '1.006', 'grad_norm': '0.8931', 'learning_rate': '0.0001797', 'epoch': '0.1986', 'num_input_tokens_seen': 2390776, 'train_runtime': '2379', 'train_tokens_per_second': '1005'}
+
+ 20%|█▉        | 360/1813 [39:39<2:31:29,  6.26s/it]
+ 20%|█▉        | 361/1813 [39:44<2:22:51,  5.90s/it]
+ 20%|█▉        | 362/1813 [39:49<2:20:58,  5.83s/it]
+ 20%|██        | 363/1813 [39:56<2:28:29,  6.14s/it]
+ 20%|██        | 364/1813 [40:03<2:31:39,  6.28s/it]
+ 20%|██        | 365/1813 [40:09<2:28:44,  6.16s/it]
+ 20%|██        | 366/1813 [40:16<2:34:34,  6.41s/it]
+ 20%|██        | 367/1813 [40:22<2:31:09,  6.27s/it]
+ 20%|██        | 368/1813 [40:27<2:24:37,  6.01s/it]
+ 20%|██        | 369/1813 [40:32<2:14:40,  5.60s/it]
+ 20%|██        | 370/1813 [40:38<2:16:40,  5.68s/it]
+                                                    
+{'loss': '0.996', 'grad_norm': '1.11', 'learning_rate': '0.000179', 'epoch': '0.2041', 'num_input_tokens_seen': 2454808, 'train_runtime': '2438', 'train_tokens_per_second': '1007'}
+
+ 20%|██        | 370/1813 [40:38<2:16:40,  5.68s/it]
+ 20%|██        | 371/1813 [40:43<2:12:08,  5.50s/it]
+ 21%|██        | 372/1813 [40:51<2:31:21,  6.30s/it]
+ 21%|██        | 373/1813 [40:57<2:29:46,  6.24s/it]
+ 21%|██        | 374/1813 [41:04<2:38:02,  6.59s/it]
+ 21%|██        | 375/1813 [41:11<2:39:27,  6.65s/it]
+ 21%|██        | 376/1813 [41:16<2:29:19,  6.24s/it]
+ 21%|██        | 377/1813 [41:22<2:27:43,  6.17s/it]
+ 21%|██        | 378/1813 [41:28<2:25:55,  6.10s/it]
+ 21%|██        | 379/1813 [41:33<2:18:18,  5.79s/it]
+ 21%|██        | 380/1813 [41:39<2:13:00,  5.57s/it]
+                                                    
+{'loss': '0.9639', 'grad_norm': '1.194', 'learning_rate': '0.0001783', 'epoch': '0.2097', 'num_input_tokens_seen': 2520472, 'train_runtime': '2499', 'train_tokens_per_second': '1009'}
+
+ 21%|██        | 380/1813 [41:39<2:13:00,  5.57s/it]
+ 21%|██        | 381/1813 [41:45<2:22:45,  5.98s/it]
+ 21%|██        | 382/1813 [41:52<2:28:37,  6.23s/it]
+ 21%|██        | 383/1813 [41:58<2:27:23,  6.18s/it]
+ 21%|██        | 384/1813 [42:04<2:26:03,  6.13s/it]
+ 21%|██        | 385/1813 [42:11<2:26:00,  6.13s/it]
+ 21%|██▏       | 386/1813 [42:15<2:14:55,  5.67s/it]
+ 21%|██▏       | 387/1813 [42:21<2:18:35,  5.83s/it]
+ 21%|██▏       | 388/1813 [42:28<2:28:13,  6.24s/it]
+ 21%|██▏       | 389/1813 [42:36<2:39:17,  6.71s/it]
+ 22%|██▏       | 390/1813 [42:43<2:39:41,  6.73s/it]
+                                                    
+{'loss': '0.9971', 'grad_norm': '0.8472', 'learning_rate': '0.0001775', 'epoch': '0.2152', 'num_input_tokens_seen': 2591512, 'train_runtime': '2564', 'train_tokens_per_second': '1011'}
+
+ 22%|██▏       | 390/1813 [42:43<2:39:41,  6.73s/it]
+ 22%|██▏       | 391/1813 [42:49<2:36:08,  6.59s/it]
+ 22%|██▏       | 392/1813 [42:56<2:35:39,  6.57s/it]
+ 22%|██▏       | 393/1813 [43:03<2:37:02,  6.64s/it]
+ 22%|██▏       | 394/1813 [43:08<2:26:19,  6.19s/it]
+ 22%|██▏       | 395/1813 [43:13<2:21:54,  6.00s/it]
+ 22%|██▏       | 396/1813 [43:20<2:25:14,  6.15s/it]
+ 22%|██▏       | 397/1813 [43:25<2:17:43,  5.84s/it]
+ 22%|██▏       | 398/1813 [43:31<2:20:15,  5.95s/it]
+ 22%|██▏       | 399/1813 [43:39<2:31:59,  6.45s/it]
+ 22%|██▏       | 400/1813 [43:47<2:44:24,  6.98s/it]
+                                                    
+{'loss': '0.9765', 'grad_norm': '0.9735', 'learning_rate': '0.0001768', 'epoch': '0.2207', 'num_input_tokens_seen': 2660336, 'train_runtime': '2628', 'train_tokens_per_second': '1012'}
+
+ 22%|██▏       | 400/1813 [43:47<2:44:24,  6.98s/it]
+ 22%|██▏       | 401/1813 [43:54<2:45:55,  7.05s/it]
+ 22%|██▏       | 402/1813 [43:59<2:30:57,  6.42s/it]
+ 22%|██▏       | 403/1813 [44:05<2:28:15,  6.31s/it]
+ 22%|██▏       | 404/1813 [44:12<2:32:23,  6.49s/it]
+ 22%|██▏       | 405/1813 [44:19<2:36:29,  6.67s/it]
+ 22%|██▏       | 406/1813 [44:24<2:23:17,  6.11s/it]
+ 22%|██▏       | 407/1813 [44:28<2:10:41,  5.58s/it]
+ 23%|██▎       | 408/1813 [44:34<2:11:46,  5.63s/it]
+ 23%|██▎       | 409/1813 [44:40<2:13:39,  5.71s/it]
+ 23%|██▎       | 410/1813 [44:45<2:10:00,  5.56s/it]
+                                                    
+{'loss': '1.01', 'grad_norm': '2.55', 'learning_rate': '0.0001759', 'epoch': '0.2262', 'num_input_tokens_seen': 2722056, 'train_runtime': '2686', 'train_tokens_per_second': '1014'}
+
+ 23%|██▎       | 410/1813 [44:45<2:10:00,  5.56s/it]
+ 23%|██▎       | 411/1813 [44:51<2:13:39,  5.72s/it]
+ 23%|██▎       | 412/1813 [44:58<2:18:21,  5.93s/it]
+ 23%|██▎       | 413/1813 [45:05<2:26:24,  6.27s/it]
+ 23%|██▎       | 414/1813 [45:10<2:18:55,  5.96s/it]
+ 23%|██▎       | 415/1813 [45:16<2:21:27,  6.07s/it]
+ 23%|██▎       | 416/1813 [45:23<2:25:21,  6.24s/it]
+ 23%|██▎       | 417/1813 [45:31<2:34:22,  6.63s/it]
+ 23%|██▎       | 418/1813 [45:36<2:27:43,  6.35s/it]
+ 23%|██▎       | 419/1813 [45:43<2:29:03,  6.42s/it]
+ 23%|██▎       | 420/1813 [45:49<2:24:49,  6.24s/it]
+                                                    
+{'loss': '1.001', 'grad_norm': '0.7771', 'learning_rate': '0.0001751', 'epoch': '0.2317', 'num_input_tokens_seen': 2790960, 'train_runtime': '2749', 'train_tokens_per_second': '1015'}
+
+ 23%|██▎       | 420/1813 [45:49<2:24:49,  6.24s/it]
+ 23%|██▎       | 421/1813 [45:55<2:24:57,  6.25s/it]
+ 23%|██▎       | 422/1813 [46:02<2:31:22,  6.53s/it]
+ 23%|██▎       | 423/1813 [46:08<2:26:27,  6.32s/it]
+ 23%|██▎       | 424/1813 [46:14<2:23:53,  6.22s/it]
+ 23%|██▎       | 425/1813 [46:19<2:18:36,  5.99s/it]
+ 23%|██▎       | 426/1813 [46:27<2:27:05,  6.36s/it]
+ 24%|██▎       | 427/1813 [46:32<2:23:39,  6.22s/it]
+ 24%|██▎       | 428/1813 [46:39<2:22:30,  6.17s/it]
+ 24%|██▎       | 429/1813 [46:44<2:15:51,  5.89s/it]
+ 24%|██▎       | 430/1813 [46:51<2:26:37,  6.36s/it]
+                                                    
+{'loss': '0.9917', 'grad_norm': '0.7437', 'learning_rate': '0.0001742', 'epoch': '0.2372', 'num_input_tokens_seen': 2860096, 'train_runtime': '2812', 'train_tokens_per_second': '1017'}
+
+ 24%|██▎       | 430/1813 [46:51<2:26:37,  6.36s/it]
+ 24%|██▍       | 431/1813 [46:57<2:22:55,  6.21s/it]
+ 24%|██▍       | 432/1813 [47:03<2:19:12,  6.05s/it]
+ 24%|██▍       | 433/1813 [47:08<2:14:09,  5.83s/it]
+ 24%|██▍       | 434/1813 [47:15<2:18:28,  6.03s/it]
+ 24%|██▍       | 435/1813 [47:20<2:17:25,  5.98s/it]
+ 24%|██▍       | 436/1813 [47:27<2:20:03,  6.10s/it]
+ 24%|██▍       | 437/1813 [47:33<2:17:14,  5.98s/it]
+ 24%|██▍       | 438/1813 [47:39<2:19:18,  6.08s/it]
+ 24%|██▍       | 439/1813 [47:45<2:19:28,  6.09s/it]
+ 24%|██▍       | 440/1813 [47:51<2:21:16,  6.17s/it]
+                                                    
+{'loss': '0.9001', 'grad_norm': '1.189', 'learning_rate': '0.0001733', 'epoch': '0.2428', 'num_input_tokens_seen': 2924856, 'train_runtime': '2872', 'train_tokens_per_second': '1018'}
+
+ 24%|██▍       | 440/1813 [47:51<2:21:16,  6.17s/it]
+ 24%|██▍       | 441/1813 [47:57<2:19:40,  6.11s/it]
+ 24%|██▍       | 442/1813 [48:04<2:20:25,  6.15s/it]
+ 24%|██▍       | 443/1813 [48:10<2:24:19,  6.32s/it]
+ 24%|██▍       | 444/1813 [48:16<2:18:41,  6.08s/it]
+ 25%|██▍       | 445/1813 [48:23<2:26:45,  6.44s/it]
+ 25%|██▍       | 446/1813 [48:29<2:26:19,  6.42s/it]
+ 25%|██▍       | 447/1813 [48:36<2:23:51,  6.32s/it]
+ 25%|██▍       | 448/1813 [48:42<2:22:37,  6.27s/it]
+ 25%|██▍       | 449/1813 [48:47<2:15:16,  5.95s/it]
+ 25%|██▍       | 450/1813 [48:52<2:11:00,  5.77s/it]
+                                                    
+{'loss': '1.021', 'grad_norm': '1.076', 'learning_rate': '0.0001724', 'epoch': '0.2483', 'num_input_tokens_seen': 2990040, 'train_runtime': '2933', 'train_tokens_per_second': '1020'}
+
+ 25%|██▍       | 450/1813 [48:52<2:11:00,  5.77s/it]
+ 25%|██▍       | 451/1813 [49:00<2:22:08,  6.26s/it]
+ 25%|██▍       | 452/1813 [49:05<2:19:13,  6.14s/it]
+ 25%|██▍       | 453/1813 [49:11<2:12:13,  5.83s/it]
+ 25%|██▌       | 454/1813 [49:18<2:19:43,  6.17s/it]
+ 25%|██▌       | 455/1813 [49:24<2:22:22,  6.29s/it]
+ 25%|██▌       | 456/1813 [49:29<2:11:11,  5.80s/it]
+ 25%|██▌       | 457/1813 [49:35<2:15:32,  6.00s/it]
+ 25%|██▌       | 458/1813 [49:43<2:29:54,  6.64s/it]
+ 25%|██▌       | 459/1813 [49:50<2:32:03,  6.74s/it]
+ 25%|██▌       | 460/1813 [49:57<2:29:42,  6.64s/it]
+                                                    
+{'loss': '1.09', 'grad_norm': '0.8333', 'learning_rate': '0.0001714', 'epoch': '0.2538', 'num_input_tokens_seen': 3058848, 'train_runtime': '2997', 'train_tokens_per_second': '1021'}
+
+ 25%|██▌       | 460/1813 [49:57<2:29:42,  6.64s/it]
+ 25%|██▌       | 461/1813 [50:03<2:26:47,  6.51s/it]
+ 25%|██▌       | 462/1813 [50:11<2:34:01,  6.84s/it]
+ 26%|██▌       | 463/1813 [50:17<2:29:24,  6.64s/it]
+ 26%|██▌       | 464/1813 [50:22<2:21:14,  6.28s/it]
+ 26%|██▌       | 465/1813 [50:28<2:18:09,  6.15s/it]
+ 26%|██▌       | 466/1813 [50:35<2:21:54,  6.32s/it]
+ 26%|██▌       | 467/1813 [50:41<2:19:47,  6.23s/it]
+ 26%|██▌       | 468/1813 [50:48<2:23:12,  6.39s/it]
+ 26%|██▌       | 469/1813 [50:54<2:25:33,  6.50s/it]
+ 26%|██▌       | 470/1813 [51:01<2:26:19,  6.54s/it]
+                                                    
+{'loss': '0.957', 'grad_norm': '0.7684', 'learning_rate': '0.0001704', 'epoch': '0.2593', 'num_input_tokens_seen': 3126680, 'train_runtime': '3061', 'train_tokens_per_second': '1021'}
+
+ 26%|██▌       | 470/1813 [51:01<2:26:19,  6.54s/it]
+ 26%|██▌       | 471/1813 [51:07<2:20:44,  6.29s/it]
+ 26%|██▌       | 472/1813 [51:13<2:24:11,  6.45s/it]
+ 26%|██▌       | 473/1813 [51:20<2:26:01,  6.54s/it]
+ 26%|██▌       | 474/1813 [51:26<2:22:05,  6.37s/it]
+ 26%|██▌       | 475/1813 [51:31<2:13:53,  6.00s/it]
+ 26%|██▋       | 476/1813 [51:37<2:13:11,  5.98s/it]
+ 26%|██▋       | 477/1813 [51:44<2:17:43,  6.19s/it]
+ 26%|██▋       | 478/1813 [51:51<2:23:56,  6.47s/it]
+ 26%|██▋       | 479/1813 [51:57<2:19:43,  6.28s/it]
+ 26%|██▋       | 480/1813 [52:03<2:20:46,  6.34s/it]
+                                                    
+{'loss': '1.061', 'grad_norm': '1.037', 'learning_rate': '0.0001694', 'epoch': '0.2648', 'num_input_tokens_seen': 3193704, 'train_runtime': '3124', 'train_tokens_per_second': '1022'}
+
+ 26%|██▋       | 480/1813 [52:03<2:20:46,  6.34s/it]
+ 27%|██▋       | 481/1813 [52:10<2:20:23,  6.32s/it]
+ 27%|██▋       | 482/1813 [52:16<2:23:10,  6.45s/it]
+ 27%|██▋       | 483/1813 [52:24<2:31:05,  6.82s/it]
+ 27%|██▋       | 484/1813 [52:30<2:24:09,  6.51s/it]
+ 27%|██▋       | 485/1813 [52:38<2:32:08,  6.87s/it]
+ 27%|██▋       | 486/1813 [52:44<2:28:15,  6.70s/it]
+ 27%|██▋       | 487/1813 [52:51<2:29:36,  6.77s/it]
+ 27%|██▋       | 488/1813 [52:57<2:23:08,  6.48s/it]
+ 27%|██▋       | 489/1813 [53:04<2:28:28,  6.73s/it]
+ 27%|██▋       | 490/1813 [53:10<2:25:03,  6.58s/it]
+                                                    
+{'loss': '1.017', 'grad_norm': '1.114', 'learning_rate': '0.0001684', 'epoch': '0.2703', 'num_input_tokens_seen': 3268064, 'train_runtime': '3191', 'train_tokens_per_second': '1024'}
+
+ 27%|██▋       | 490/1813 [53:10<2:25:03,  6.58s/it]
+ 27%|██▋       | 491/1813 [53:17<2:25:52,  6.62s/it]
+ 27%|██▋       | 492/1813 [53:24<2:31:49,  6.90s/it]
+ 27%|██▋       | 493/1813 [53:30<2:21:31,  6.43s/it]
+ 27%|██▋       | 494/1813 [53:35<2:15:47,  6.18s/it]
+ 27%|██▋       | 495/1813 [53:42<2:21:25,  6.44s/it]
+ 27%|██▋       | 496/1813 [53:49<2:20:46,  6.41s/it]
+ 27%|██▋       | 497/1813 [53:54<2:15:43,  6.19s/it]
+ 27%|██▋       | 498/1813 [54:02<2:22:16,  6.49s/it]
+ 28%|██▊       | 499/1813 [54:08<2:19:34,  6.37s/it]
+ 28%|██▊       | 500/1813 [54:17<2:37:47,  7.21s/it]
+                                                    
+{'loss': '1.088', 'grad_norm': '0.7856', 'learning_rate': '0.0001673', 'epoch': '0.2759', 'num_input_tokens_seen': 3341536, 'train_runtime': '3257', 'train_tokens_per_second': '1026'}
+
+ 28%|██▊       | 500/1813 [54:17<2:37:47,  7.21s/it]
+ 28%|██▊       | 501/1813 [54:23<2:31:30,  6.93s/it]
+ 28%|██▊       | 502/1813 [54:29<2:23:50,  6.58s/it]
+ 28%|██▊       | 503/1813 [54:34<2:13:42,  6.12s/it]
+ 28%|██▊       | 504/1813 [54:40<2:14:15,  6.15s/it]
+ 28%|██▊       | 505/1813 [54:47<2:20:45,  6.46s/it]
+ 28%|██▊       | 506/1813 [54:53<2:17:50,  6.33s/it]
+ 28%|██▊       | 507/1813 [55:01<2:23:45,  6.60s/it]
+ 28%|██▊       | 508/1813 [55:06<2:12:58,  6.11s/it]
+ 28%|██▊       | 509/1813 [55:11<2:06:32,  5.82s/it]
+ 28%|██▊       | 510/1813 [55:16<2:02:32,  5.64s/it]
+                                                    
+{'loss': '0.9253', 'grad_norm': '1.214', 'learning_rate': '0.0001662', 'epoch': '0.2814', 'num_input_tokens_seen': 3402712, 'train_runtime': '3316', 'train_tokens_per_second': '1026'}
+
+ 28%|██▊       | 510/1813 [55:16<2:02:32,  5.64s/it]
+ 28%|██▊       | 511/1813 [55:21<2:00:41,  5.56s/it]
+ 28%|██▊       | 512/1813 [55:28<2:10:29,  6.02s/it]
+ 28%|██▊       | 513/1813 [55:36<2:19:12,  6.42s/it]
+ 28%|██▊       | 514/1813 [55:43<2:21:29,  6.54s/it]
+ 28%|██▊       | 515/1813 [55:48<2:12:29,  6.12s/it]
+ 28%|██▊       | 516/1813 [55:54<2:10:48,  6.05s/it]
+ 29%|██▊       | 517/1813 [56:00<2:14:16,  6.22s/it]
+ 29%|██▊       | 518/1813 [56:07<2:14:33,  6.23s/it]
+ 29%|██▊       | 519/1813 [56:13<2:14:57,  6.26s/it]
+ 29%|██▊       | 520/1813 [56:20<2:18:30,  6.43s/it]
+                                                    
+{'loss': '0.9761', 'grad_norm': '0.9834', 'learning_rate': '0.0001651', 'epoch': '0.2869', 'num_input_tokens_seen': 3469624, 'train_runtime': '3380', 'train_tokens_per_second': '1026'}
+
+ 29%|██▊       | 520/1813 [56:20<2:18:30,  6.43s/it]
+ 29%|██▊       | 521/1813 [56:24<2:05:18,  5.82s/it]
+ 29%|██▉       | 522/1813 [56:30<2:06:58,  5.90s/it]
+ 29%|██▉       | 523/1813 [56:37<2:10:03,  6.05s/it]
+ 29%|██▉       | 524/1813 [56:43<2:13:05,  6.20s/it]
+ 29%|██▉       | 525/1813 [56:48<2:07:16,  5.93s/it]
+ 29%|██▉       | 526/1813 [56:55<2:13:18,  6.21s/it]
+ 29%|██▉       | 527/1813 [57:02<2:17:22,  6.41s/it]
+ 29%|██▉       | 528/1813 [57:09<2:22:10,  6.64s/it]
+ 29%|██▉       | 529/1813 [57:16<2:25:32,  6.80s/it]
+ 29%|██▉       | 530/1813 [57:23<2:21:20,  6.61s/it]
+                                                    
+{'loss': '0.9427', 'grad_norm': '1.131', 'learning_rate': '0.000164', 'epoch': '0.2924', 'num_input_tokens_seen': 3534904, 'train_runtime': '3443', 'train_tokens_per_second': '1027'}
+
+ 29%|██▉       | 530/1813 [57:23<2:21:20,  6.61s/it]
+ 29%|██▉       | 531/1813 [57:31<2:31:37,  7.10s/it]
+ 29%|██▉       | 532/1813 [57:37<2:26:05,  6.84s/it]
+ 29%|██▉       | 533/1813 [57:44<2:28:14,  6.95s/it]
+ 29%|██▉       | 534/1813 [57:52<2:29:50,  7.03s/it]
+ 30%|██▉       | 535/1813 [57:56<2:13:50,  6.28s/it]
+ 30%|██▉       | 536/1813 [58:02<2:09:09,  6.07s/it]
+ 30%|██▉       | 537/1813 [58:08<2:13:39,  6.28s/it]
+ 30%|██▉       | 538/1813 [58:14<2:10:20,  6.13s/it]
+ 30%|██▉       | 539/1813 [58:21<2:11:52,  6.21s/it]
+ 30%|██▉       | 540/1813 [58:27<2:12:22,  6.24s/it]
+                                                    
+{'loss': '0.9719', 'grad_norm': '0.9388', 'learning_rate': '0.0001628', 'epoch': '0.2979', 'num_input_tokens_seen': 3603488, 'train_runtime': '3507', 'train_tokens_per_second': '1027'}
+
+ 30%|██▉       | 540/1813 [58:27<2:12:22,  6.24s/it]
+ 30%|██▉       | 541/1813 [58:32<2:07:22,  6.01s/it]
+ 30%|██▉       | 542/1813 [58:38<2:05:38,  5.93s/it]
+ 30%|██▉       | 543/1813 [58:45<2:14:35,  6.36s/it]
+ 30%|███       | 544/1813 [58:54<2:27:17,  6.96s/it]
+ 30%|███       | 545/1813 [59:01<2:25:15,  6.87s/it]
+ 30%|███       | 546/1813 [59:07<2:23:22,  6.79s/it]
+ 30%|███       | 547/1813 [59:15<2:29:39,  7.09s/it]
+ 30%|███       | 548/1813 [59:23<2:33:22,  7.27s/it]
+ 30%|███       | 549/1813 [59:29<2:24:31,  6.86s/it]
+ 30%|███       | 550/1813 [59:34<2:18:46,  6.59s/it]
+                                                    
+{'loss': '0.9065', 'grad_norm': '1.039', 'learning_rate': '0.0001616', 'epoch': '0.3034', 'num_input_tokens_seen': 3676576, 'train_runtime': '3575', 'train_tokens_per_second': '1028'}
+
+ 30%|███       | 550/1813 [59:34<2:18:46,  6.59s/it]
+ 30%|███       | 551/1813 [59:41<2:16:00,  6.47s/it]
+ 30%|███       | 552/1813 [59:46<2:09:18,  6.15s/it]
+ 31%|███       | 553/1813 [59:51<2:00:56,  5.76s/it]
+ 31%|███       | 554/1813 [59:58<2:06:51,  6.05s/it]
+ 31%|███       | 555/1813 [1:00:03<1:59:39,  5.71s/it]
+ 31%|███       | 556/1813 [1:00:09<2:01:56,  5.82s/it]
+ 31%|███       | 557/1813 [1:00:16<2:08:42,  6.15s/it]
+ 31%|███       | 558/1813 [1:00:21<2:06:22,  6.04s/it]
+ 31%|███       | 559/1813 [1:00:26<1:59:53,  5.74s/it]
+ 31%|███       | 560/1813 [1:00:33<2:03:13,  5.90s/it]
+                                                      
+{'loss': '0.979', 'grad_norm': '0.7216', 'learning_rate': '0.0001604', 'epoch': '0.309', 'num_input_tokens_seen': 3738056, 'train_runtime': '3633', 'train_tokens_per_second': '1029'}
+
+ 31%|███       | 560/1813 [1:00:33<2:03:13,  5.90s/it]
+ 31%|███       | 561/1813 [1:00:38<2:02:30,  5.87s/it]
+ 31%|███       | 562/1813 [1:00:45<2:06:57,  6.09s/it]
+ 31%|███       | 563/1813 [1:00:52<2:11:51,  6.33s/it]
+ 31%|███       | 564/1813 [1:00:58<2:09:11,  6.21s/it]
+ 31%|███       | 565/1813 [1:01:03<2:04:14,  5.97s/it]
+ 31%|███       | 566/1813 [1:01:10<2:06:55,  6.11s/it]
+ 31%|███▏      | 567/1813 [1:01:15<2:02:25,  5.89s/it]
+ 31%|███▏      | 568/1813 [1:01:22<2:07:04,  6.12s/it]
+ 31%|███▏      | 569/1813 [1:01:29<2:15:15,  6.52s/it]
+ 31%|███▏      | 570/1813 [1:01:37<2:20:05,  6.76s/it]
+                                                      
+{'loss': '0.9734', 'grad_norm': '0.7898', 'learning_rate': '0.0001591', 'epoch': '0.3145', 'num_input_tokens_seen': 3808208, 'train_runtime': '3697', 'train_tokens_per_second': '1030'}
+
+ 31%|███▏      | 570/1813 [1:01:37<2:20:05,  6.76s/it]
+ 31%|███▏      | 571/1813 [1:01:43<2:19:58,  6.76s/it]
+ 32%|███▏      | 572/1813 [1:01:49<2:15:22,  6.55s/it]
+ 32%|███▏      | 573/1813 [1:01:55<2:06:44,  6.13s/it]
+ 32%|███▏      | 574/1813 [1:02:00<2:03:40,  5.99s/it]
+ 32%|███▏      | 575/1813 [1:02:07<2:07:05,  6.16s/it]
+ 32%|███▏      | 576/1813 [1:02:11<1:56:37,  5.66s/it]
+ 32%|███▏      | 577/1813 [1:02:18<2:02:17,  5.94s/it]
+ 32%|███▏      | 578/1813 [1:02:24<2:02:43,  5.96s/it]
+ 32%|███▏      | 579/1813 [1:02:30<2:03:23,  6.00s/it]
+ 32%|███▏      | 580/1813 [1:02:36<2:04:10,  6.04s/it]
+                                                      
+{'loss': '0.981', 'grad_norm': '0.889', 'learning_rate': '0.0001579', 'epoch': '0.32', 'num_input_tokens_seen': 3872328, 'train_runtime': '3757', 'train_tokens_per_second': '1031'}
+
+ 32%|███▏      | 580/1813 [1:02:36<2:04:10,  6.04s/it]
+ 32%|███▏      | 581/1813 [1:02:42<2:01:35,  5.92s/it]
+ 32%|███▏      | 582/1813 [1:02:47<1:58:58,  5.80s/it]
+ 32%|███▏      | 583/1813 [1:02:53<1:59:19,  5.82s/it]
+ 32%|███▏      | 584/1813 [1:03:00<2:03:26,  6.03s/it]
+ 32%|███▏      | 585/1813 [1:03:06<2:04:24,  6.08s/it]
+ 32%|███▏      | 586/1813 [1:03:12<2:05:52,  6.15s/it]
+ 32%|███▏      | 587/1813 [1:03:18<2:05:14,  6.13s/it]
+ 32%|███▏      | 588/1813 [1:03:24<2:06:17,  6.19s/it]
+ 32%|███▏      | 589/1813 [1:03:31<2:06:38,  6.21s/it]
+ 33%|███▎      | 590/1813 [1:03:37<2:05:47,  6.17s/it]
+                                                      
+{'loss': '0.8943', 'grad_norm': '1.067', 'learning_rate': '0.0001566', 'epoch': '0.3255', 'num_input_tokens_seen': 3936024, 'train_runtime': '3817', 'train_tokens_per_second': '1031'}
+
+ 33%|███▎      | 590/1813 [1:03:37<2:05:47,  6.17s/it]
+ 33%|███▎      | 591/1813 [1:03:42<2:00:52,  5.94s/it]
+ 33%|███▎      | 592/1813 [1:03:48<2:00:54,  5.94s/it]
+ 33%|███▎      | 593/1813 [1:03:55<2:05:11,  6.16s/it]
+ 33%|███▎      | 594/1813 [1:04:01<2:05:32,  6.18s/it]
+ 33%|███▎      | 595/1813 [1:04:07<2:02:52,  6.05s/it]
+ 33%|███▎      | 596/1813 [1:04:13<2:02:03,  6.02s/it]
+ 33%|███▎      | 597/1813 [1:04:18<1:59:49,  5.91s/it]
+ 33%|███▎      | 598/1813 [1:04:24<1:59:22,  5.90s/it]
+ 33%|███▎      | 599/1813 [1:04:31<2:01:55,  6.03s/it]
+ 33%|███▎      | 600/1813 [1:04:38<2:08:01,  6.33s/it]
+                                                      
+{'loss': '0.9286', 'grad_norm': '0.9217', 'learning_rate': '0.0001553', 'epoch': '0.331', 'num_input_tokens_seen': 3999176, 'train_runtime': '3878', 'train_tokens_per_second': '1031'}
+
+ 33%|███▎      | 600/1813 [1:04:38<2:08:01,  6.33s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.78it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.06it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.99it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.82it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.37it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.95it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.98it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.11it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.75it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.95it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.76it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.31it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.74it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.24it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.11it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.83it/s]
+
+  8%|▊         | 82/1000 [00:10<01:40,  9.18it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.01it/s]
+
+  8%|▊         | 85/1000 [00:11<02:15,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.94it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:12,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.38it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.64it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.29it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.41it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.16it/s]
+
+ 11%|█         | 112/1000 [00:14<01:14, 11.99it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.48it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.78it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.22it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.40it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.71it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.13it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.78it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.90it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.90it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.08it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.06it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.42it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.98it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.17it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.77it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.83it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.44it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.52it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.71it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:42,  7.91it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.07it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.60it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.47it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.36it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.74it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.18it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.65it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.77it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.63it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.93it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.68it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.16it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:29,  8.72it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:04,  6.27it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:04,  6.28it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.54it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.03it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.08it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.35it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:36,  7.99it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.19it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.93it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.01it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.74it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.52it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:44,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.75it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:20,  8.63it/s]
+
+ 30%|███       | 304/1000 [00:40<01:27,  8.00it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.08it/s]
+
+ 31%|███       | 306/1000 [00:41<01:40,  6.91it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.76it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.77it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.16it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.22it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:07, 10.21it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.56it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.88it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.16it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.37it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.03it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.87it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.40it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.90it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.63it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.34it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.94it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.56it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.41it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:53,  5.58it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.54it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.71it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.21it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.47it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.18it/s]
+
+ 38%|███▊      | 379/1000 [00:51<01:16,  8.17it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.19it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.56it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.06it/s]
+
+ 39%|███▊      | 387/1000 [00:52<01:21,  7.55it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.29it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.84it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.48it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  8.99it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.51it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.83it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.67it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.91it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.51it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.27it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.46it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.68it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.54it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.03it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.80it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.53it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.09it/s]
+
+ 44%|████▍     | 441/1000 [00:59<00:59,  9.32it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.26it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.74it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.19it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<01:00,  8.88it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:51, 10.48it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:11,  7.51it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:20,  6.65it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:29,  5.95it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:51,  4.79it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.27it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.82it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.17it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.35it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:25,  6.17it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.17it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.29it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.08it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.50it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.05it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.42it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.45it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.95it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.10it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.07it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.58it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.39it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.72it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.68it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.35it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.05it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.49it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.96it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.84it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.04it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.82it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.24it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.61it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.76it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.38it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.72it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.79it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.62it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.74it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.08it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:39,  9.40it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.94it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.91it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.65it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.96it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.91it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.87it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.82it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.25it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.21it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.71it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.38it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.07it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:27, 10.11it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.58it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.88it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.50it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.74it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.35it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.60it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.44it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.69it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.71it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.66it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.54it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.79it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.63it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.14it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.98it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:26,  7.96it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.84it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.98it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.22it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.25it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.06it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.29it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.13it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.19it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.03it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.81it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.12it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.15it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.46it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.04it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.19it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.51it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.63it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.68it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.30it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.26it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.27it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.79it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.64it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.89it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.88it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.97it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.79it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.62it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.84it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.69it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.00it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:12,  7.42it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.63it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.31it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.02it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.33it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.59it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.57it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.93it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.09it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.64it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.16it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.29it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.82it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.03it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.13it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.41it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.74it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.84it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.42it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.00it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.17it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.79it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.80it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.05it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.8095', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.384', 'eval_steps_per_second': '7.384', 'epoch': '0.331', 'num_input_tokens_seen': 3999176}
+
+ 33%|███▎      | 600/1813 [1:06:53<2:08:01,  6.33s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 33%|███▎      | 601/1813 [1:06:58<15:38:43, 46.47s/it]
+ 33%|███▎      | 602/1813 [1:07:03<11:27:25, 34.06s/it]
+ 33%|███▎      | 603/1813 [1:07:09<8:35:32, 25.56s/it] 
+ 33%|███▎      | 604/1813 [1:07:15<6:38:51, 19.79s/it]
+ 33%|███▎      | 605/1813 [1:07:23<5:27:29, 16.27s/it]
+ 33%|███▎      | 606/1813 [1:07:30<4:33:42, 13.61s/it]
+ 33%|███▎      | 607/1813 [1:07:37<3:48:17, 11.36s/it]
+ 34%|███▎      | 608/1813 [1:07:43<3:17:37,  9.84s/it]
+ 34%|███▎      | 609/1813 [1:07:49<2:56:38,  8.80s/it]
+ 34%|███▎      | 610/1813 [1:07:56<2:42:51,  8.12s/it]
+                                                      
+{'loss': '0.9755', 'grad_norm': '0.7377', 'learning_rate': '0.0001539', 'epoch': '0.3366', 'num_input_tokens_seen': 4066376, 'train_runtime': '4076', 'train_tokens_per_second': '997.6'}
+
+ 34%|███▎      | 610/1813 [1:07:56<2:42:51,  8.12s/it]
+ 34%|███▎      | 611/1813 [1:08:01<2:27:34,  7.37s/it]
+ 34%|███▍      | 612/1813 [1:08:06<2:09:57,  6.49s/it]
+ 34%|███▍      | 613/1813 [1:08:12<2:05:58,  6.30s/it]
+ 34%|███▍      | 614/1813 [1:08:18<2:03:52,  6.20s/it]
+ 34%|███▍      | 615/1813 [1:08:23<2:00:08,  6.02s/it]
+ 34%|███▍      | 616/1813 [1:08:28<1:54:49,  5.76s/it]
+ 34%|███▍      | 617/1813 [1:08:34<1:52:00,  5.62s/it]
+ 34%|███▍      | 618/1813 [1:08:41<2:00:33,  6.05s/it]
+ 34%|███▍      | 619/1813 [1:08:46<1:54:52,  5.77s/it]
+ 34%|███▍      | 620/1813 [1:08:52<1:59:18,  6.00s/it]
+                                                      
+{'loss': '0.8739', 'grad_norm': '0.9884', 'learning_rate': '0.0001526', 'epoch': '0.3421', 'num_input_tokens_seen': 4123976, 'train_runtime': '4133', 'train_tokens_per_second': '997.9'}
+
+ 34%|███▍      | 620/1813 [1:08:52<1:59:18,  6.00s/it]
+ 34%|███▍      | 621/1813 [1:08:59<2:04:14,  6.25s/it]
+ 34%|███▍      | 622/1813 [1:09:07<2:12:04,  6.65s/it]
+ 34%|███▍      | 623/1813 [1:09:12<2:01:37,  6.13s/it]
+ 34%|███▍      | 624/1813 [1:09:18<2:03:10,  6.22s/it]
+ 34%|███▍      | 625/1813 [1:09:23<1:55:53,  5.85s/it]
+ 35%|███▍      | 626/1813 [1:09:28<1:48:08,  5.47s/it]
+ 35%|███▍      | 627/1813 [1:09:34<1:52:13,  5.68s/it]
+ 35%|███▍      | 628/1813 [1:09:40<1:54:24,  5.79s/it]
+ 35%|███▍      | 629/1813 [1:09:48<2:09:07,  6.54s/it]
+ 35%|███▍      | 630/1813 [1:09:55<2:10:09,  6.60s/it]
+                                                      
+{'loss': '0.9482', 'grad_norm': '0.8105', 'learning_rate': '0.0001512', 'epoch': '0.3476', 'num_input_tokens_seen': 4191992, 'train_runtime': '4195', 'train_tokens_per_second': '999.2'}
+
+ 35%|███▍      | 630/1813 [1:09:55<2:10:09,  6.60s/it]
+ 35%|███▍      | 631/1813 [1:10:02<2:09:47,  6.59s/it]
+ 35%|███▍      | 632/1813 [1:10:07<2:06:09,  6.41s/it]
+ 35%|███▍      | 633/1813 [1:10:13<2:01:45,  6.19s/it]
+ 35%|███▍      | 634/1813 [1:10:20<2:07:52,  6.51s/it]
+ 35%|███▌      | 635/1813 [1:10:27<2:08:30,  6.55s/it]
+ 35%|███▌      | 636/1813 [1:10:32<2:00:23,  6.14s/it]
+ 35%|███▌      | 637/1813 [1:10:39<2:06:05,  6.43s/it]
+ 35%|███▌      | 638/1813 [1:10:45<2:01:11,  6.19s/it]
+ 35%|███▌      | 639/1813 [1:10:50<1:55:18,  5.89s/it]
+ 35%|███▌      | 640/1813 [1:10:56<1:56:22,  5.95s/it]
+                                                      
+{'loss': '0.9549', 'grad_norm': '1.072', 'learning_rate': '0.0001498', 'epoch': '0.3531', 'num_input_tokens_seen': 4257760, 'train_runtime': '4257', 'train_tokens_per_second': '1000'}
+
+ 35%|███▌      | 640/1813 [1:10:56<1:56:22,  5.95s/it]
+ 35%|███▌      | 641/1813 [1:11:02<1:52:42,  5.77s/it]
+ 35%|███▌      | 642/1813 [1:11:09<2:00:51,  6.19s/it]
+ 35%|███▌      | 643/1813 [1:11:15<2:03:25,  6.33s/it]
+ 36%|███▌      | 644/1813 [1:11:21<1:59:31,  6.13s/it]
+ 36%|███▌      | 645/1813 [1:11:27<1:59:26,  6.14s/it]
+ 36%|███▌      | 646/1813 [1:11:33<1:57:52,  6.06s/it]
+ 36%|███▌      | 647/1813 [1:11:40<2:00:25,  6.20s/it]
+ 36%|███▌      | 648/1813 [1:11:47<2:05:48,  6.48s/it]
+ 36%|███▌      | 649/1813 [1:11:52<2:00:53,  6.23s/it]
+ 36%|███▌      | 650/1813 [1:11:58<1:53:53,  5.88s/it]
+                                                      
+{'loss': '0.9422', 'grad_norm': '0.9532', 'learning_rate': '0.0001484', 'epoch': '0.3586', 'num_input_tokens_seen': 4322136, 'train_runtime': '4318', 'train_tokens_per_second': '1001'}
+
+ 36%|███▌      | 650/1813 [1:11:58<1:53:53,  5.88s/it]
+ 36%|███▌      | 651/1813 [1:12:04<1:57:14,  6.05s/it]
+ 36%|███▌      | 652/1813 [1:12:11<2:01:13,  6.26s/it]
+ 36%|███▌      | 653/1813 [1:12:16<1:55:56,  6.00s/it]
+ 36%|███▌      | 654/1813 [1:12:22<1:53:20,  5.87s/it]
+ 36%|███▌      | 655/1813 [1:12:28<1:57:10,  6.07s/it]
+ 36%|███▌      | 656/1813 [1:12:34<1:54:46,  5.95s/it]
+ 36%|███▌      | 657/1813 [1:12:39<1:50:49,  5.75s/it]
+ 36%|███▋      | 658/1813 [1:12:46<1:55:57,  6.02s/it]
+ 36%|███▋      | 659/1813 [1:12:52<1:58:27,  6.16s/it]
+ 36%|███▋      | 660/1813 [1:12:58<1:55:19,  6.00s/it]
+                                                      
+{'loss': '0.901', 'grad_norm': '1.051', 'learning_rate': '0.0001469', 'epoch': '0.3641', 'num_input_tokens_seen': 4385688, 'train_runtime': '4378', 'train_tokens_per_second': '1002'}
+
+ 36%|███▋      | 660/1813 [1:12:58<1:55:19,  6.00s/it]
+ 36%|███▋      | 661/1813 [1:13:04<1:56:18,  6.06s/it]
+ 37%|███▋      | 662/1813 [1:13:10<1:56:39,  6.08s/it]
+ 37%|███▋      | 663/1813 [1:13:17<2:01:16,  6.33s/it]
+ 37%|███▋      | 664/1813 [1:13:24<2:01:53,  6.37s/it]
+ 37%|███▋      | 665/1813 [1:13:29<1:54:49,  6.00s/it]
+ 37%|███▋      | 666/1813 [1:13:34<1:52:17,  5.87s/it]
+ 37%|███▋      | 667/1813 [1:13:41<1:58:02,  6.18s/it]
+ 37%|███▋      | 668/1813 [1:13:48<1:59:57,  6.29s/it]
+ 37%|███▋      | 669/1813 [1:13:54<1:58:38,  6.22s/it]
+ 37%|███▋      | 670/1813 [1:14:00<1:55:20,  6.06s/it]
+                                                      
+{'loss': '0.978', 'grad_norm': '1.091', 'learning_rate': '0.0001455', 'epoch': '0.3697', 'num_input_tokens_seen': 4449456, 'train_runtime': '4440', 'train_tokens_per_second': '1002'}
+
+ 37%|███▋      | 670/1813 [1:14:00<1:55:20,  6.06s/it]
+ 37%|███▋      | 671/1813 [1:14:06<1:56:49,  6.14s/it]
+ 37%|███▋      | 672/1813 [1:14:13<2:00:21,  6.33s/it]
+ 37%|███▋      | 673/1813 [1:14:19<2:00:09,  6.32s/it]
+ 37%|███▋      | 674/1813 [1:14:26<2:02:52,  6.47s/it]
+ 37%|███▋      | 675/1813 [1:14:32<1:59:25,  6.30s/it]
+ 37%|███▋      | 676/1813 [1:14:37<1:56:29,  6.15s/it]
+ 37%|███▋      | 677/1813 [1:14:46<2:07:51,  6.75s/it]
+ 37%|███▋      | 678/1813 [1:14:51<1:59:20,  6.31s/it]
+ 37%|███▋      | 679/1813 [1:14:56<1:53:29,  6.00s/it]
+ 38%|███▊      | 680/1813 [1:15:02<1:52:13,  5.94s/it]
+                                                      
+{'loss': '0.9276', 'grad_norm': '0.9476', 'learning_rate': '0.000144', 'epoch': '0.3752', 'num_input_tokens_seen': 4514984, 'train_runtime': '4502', 'train_tokens_per_second': '1003'}
+
+ 38%|███▊      | 680/1813 [1:15:02<1:52:13,  5.94s/it]
+ 38%|███▊      | 681/1813 [1:15:09<1:55:49,  6.14s/it]
+ 38%|███▊      | 682/1813 [1:15:14<1:50:31,  5.86s/it]
+ 38%|███▊      | 683/1813 [1:15:20<1:50:40,  5.88s/it]
+ 38%|███▊      | 684/1813 [1:15:26<1:53:30,  6.03s/it]
+ 38%|███▊      | 685/1813 [1:15:34<2:03:00,  6.54s/it]
+ 38%|███▊      | 686/1813 [1:15:40<2:01:35,  6.47s/it]
+ 38%|███▊      | 687/1813 [1:15:48<2:08:45,  6.86s/it]
+ 38%|███▊      | 688/1813 [1:15:53<1:57:23,  6.26s/it]
+ 38%|███▊      | 689/1813 [1:16:01<2:10:47,  6.98s/it]
+ 38%|███▊      | 690/1813 [1:16:09<2:11:31,  7.03s/it]
+                                                      
+{'loss': '0.981', 'grad_norm': '1.064', 'learning_rate': '0.0001425', 'epoch': '0.3807', 'num_input_tokens_seen': 4587840, 'train_runtime': '4569', 'train_tokens_per_second': '1004'}
+
+ 38%|███▊      | 690/1813 [1:16:09<2:11:31,  7.03s/it]
+ 38%|███▊      | 691/1813 [1:16:14<2:03:37,  6.61s/it]
+ 38%|███▊      | 692/1813 [1:16:20<2:00:45,  6.46s/it]
+ 38%|███▊      | 693/1813 [1:16:26<1:55:57,  6.21s/it]
+ 38%|███▊      | 694/1813 [1:16:32<1:56:18,  6.24s/it]
+ 38%|███▊      | 695/1813 [1:16:38<1:54:34,  6.15s/it]
+ 38%|███▊      | 696/1813 [1:16:46<2:04:35,  6.69s/it]
+ 38%|███▊      | 697/1813 [1:16:51<1:54:07,  6.14s/it]
+ 38%|███▊      | 698/1813 [1:16:56<1:50:32,  5.95s/it]
+ 39%|███▊      | 699/1813 [1:17:03<1:56:19,  6.27s/it]
+ 39%|███▊      | 700/1813 [1:17:10<1:57:56,  6.36s/it]
+                                                      
+{'loss': '1.007', 'grad_norm': '1.238', 'learning_rate': '0.000141', 'epoch': '0.3862', 'num_input_tokens_seen': 4653312, 'train_runtime': '4631', 'train_tokens_per_second': '1005'}
+
+ 39%|███▊      | 700/1813 [1:17:10<1:57:56,  6.36s/it]
+ 39%|███▊      | 701/1813 [1:17:17<2:03:46,  6.68s/it]
+ 39%|███▊      | 702/1813 [1:17:24<2:02:14,  6.60s/it]
+ 39%|███▉      | 703/1813 [1:17:32<2:10:08,  7.03s/it]
+ 39%|███▉      | 704/1813 [1:17:38<2:03:50,  6.70s/it]
+ 39%|███▉      | 705/1813 [1:17:45<2:06:13,  6.84s/it]
+ 39%|███▉      | 706/1813 [1:17:52<2:04:08,  6.73s/it]
+ 39%|███▉      | 707/1813 [1:18:01<2:20:11,  7.61s/it]
+ 39%|███▉      | 708/1813 [1:18:09<2:19:08,  7.55s/it]
+ 39%|███▉      | 709/1813 [1:18:15<2:14:37,  7.32s/it]
+ 39%|███▉      | 710/1813 [1:18:21<2:06:55,  6.90s/it]
+                                                      
+{'loss': '0.9663', 'grad_norm': '0.7786', 'learning_rate': '0.0001395', 'epoch': '0.3917', 'num_input_tokens_seen': 4732080, 'train_runtime': '4702', 'train_tokens_per_second': '1006'}
+
+ 39%|███▉      | 710/1813 [1:18:21<2:06:55,  6.90s/it]
+ 39%|███▉      | 711/1813 [1:18:28<2:06:54,  6.91s/it]
+ 39%|███▉      | 712/1813 [1:18:35<2:06:01,  6.87s/it]
+ 39%|███▉      | 713/1813 [1:18:41<2:00:04,  6.55s/it]
+ 39%|███▉      | 714/1813 [1:18:47<1:58:37,  6.48s/it]
+ 39%|███▉      | 715/1813 [1:18:54<1:59:34,  6.53s/it]
+ 39%|███▉      | 716/1813 [1:19:01<2:02:06,  6.68s/it]
+ 40%|███▉      | 717/1813 [1:19:07<1:59:41,  6.55s/it]
+ 40%|███▉      | 718/1813 [1:19:14<1:59:30,  6.55s/it]
+ 40%|███▉      | 719/1813 [1:19:19<1:52:54,  6.19s/it]
+ 40%|███▉      | 720/1813 [1:19:25<1:53:29,  6.23s/it]
+                                                      
+{'loss': '0.984', 'grad_norm': '0.9324', 'learning_rate': '0.0001379', 'epoch': '0.3972', 'num_input_tokens_seen': 4799552, 'train_runtime': '4766', 'train_tokens_per_second': '1007'}
+
+ 40%|███▉      | 720/1813 [1:19:25<1:53:29,  6.23s/it]
+ 40%|███▉      | 721/1813 [1:19:32<1:56:04,  6.38s/it]
+ 40%|███▉      | 722/1813 [1:19:37<1:50:15,  6.06s/it]
+ 40%|███▉      | 723/1813 [1:19:43<1:45:48,  5.82s/it]
+ 40%|███▉      | 724/1813 [1:19:50<1:56:57,  6.44s/it]
+ 40%|███▉      | 725/1813 [1:19:59<2:07:27,  7.03s/it]
+ 40%|████      | 726/1813 [1:20:05<2:04:15,  6.86s/it]
+ 40%|████      | 727/1813 [1:20:11<1:58:52,  6.57s/it]
+ 40%|████      | 728/1813 [1:20:16<1:50:28,  6.11s/it]
+ 40%|████      | 729/1813 [1:20:22<1:48:09,  5.99s/it]
+ 40%|████      | 730/1813 [1:20:28<1:45:49,  5.86s/it]
+                                                      
+{'loss': '0.9556', 'grad_norm': '1.03', 'learning_rate': '0.0001364', 'epoch': '0.4028', 'num_input_tokens_seen': 4866864, 'train_runtime': '4828', 'train_tokens_per_second': '1008'}
+
+ 40%|████      | 730/1813 [1:20:28<1:45:49,  5.86s/it]
+ 40%|████      | 731/1813 [1:20:35<1:52:34,  6.24s/it]
+ 40%|████      | 732/1813 [1:20:41<1:52:16,  6.23s/it]
+ 40%|████      | 733/1813 [1:20:48<1:59:00,  6.61s/it]
+ 40%|████      | 734/1813 [1:20:56<2:01:44,  6.77s/it]
+ 41%|████      | 735/1813 [1:21:02<2:01:53,  6.78s/it]
+ 41%|████      | 736/1813 [1:21:08<1:57:06,  6.52s/it]
+ 41%|████      | 737/1813 [1:21:14<1:53:06,  6.31s/it]
+ 41%|████      | 738/1813 [1:21:19<1:46:20,  5.94s/it]
+ 41%|████      | 739/1813 [1:21:25<1:45:38,  5.90s/it]
+ 41%|████      | 740/1813 [1:21:31<1:49:04,  6.10s/it]
+                                                      
+{'loss': '0.884', 'grad_norm': '0.7964', 'learning_rate': '0.0001348', 'epoch': '0.4083', 'num_input_tokens_seen': 4936352, 'train_runtime': '4892', 'train_tokens_per_second': '1009'}
+
+ 41%|████      | 740/1813 [1:21:31<1:49:04,  6.10s/it]
+ 41%|████      | 741/1813 [1:21:38<1:50:38,  6.19s/it]
+ 41%|████      | 742/1813 [1:21:45<1:57:36,  6.59s/it]
+ 41%|████      | 743/1813 [1:21:53<2:02:32,  6.87s/it]
+ 41%|████      | 744/1813 [1:22:00<2:02:45,  6.89s/it]
+ 41%|████      | 745/1813 [1:22:07<2:03:47,  6.95s/it]
+ 41%|████      | 746/1813 [1:22:13<1:59:55,  6.74s/it]
+ 41%|████      | 747/1813 [1:22:21<2:06:25,  7.12s/it]
+ 41%|████▏     | 748/1813 [1:22:27<1:58:00,  6.65s/it]
+ 41%|████▏     | 749/1813 [1:22:33<1:57:17,  6.61s/it]
+ 41%|████▏     | 750/1813 [1:22:39<1:53:42,  6.42s/it]
+                                                      
+{'loss': '0.9705', 'grad_norm': '0.827', 'learning_rate': '0.0001332', 'epoch': '0.4138', 'num_input_tokens_seen': 5011064, 'train_runtime': '4960', 'train_tokens_per_second': '1010'}
+
+ 41%|████▏     | 750/1813 [1:22:39<1:53:42,  6.42s/it]
+ 41%|████▏     | 751/1813 [1:22:46<1:52:32,  6.36s/it]
+ 41%|████▏     | 752/1813 [1:22:52<1:50:35,  6.25s/it]
+ 42%|████▏     | 753/1813 [1:22:58<1:52:30,  6.37s/it]
+ 42%|████▏     | 754/1813 [1:23:07<2:03:43,  7.01s/it]
+ 42%|████▏     | 755/1813 [1:23:12<1:52:14,  6.37s/it]
+ 42%|████▏     | 756/1813 [1:23:17<1:47:21,  6.09s/it]
+ 42%|████▏     | 757/1813 [1:23:23<1:46:25,  6.05s/it]
+ 42%|████▏     | 758/1813 [1:23:30<1:49:13,  6.21s/it]
+ 42%|████▏     | 759/1813 [1:23:36<1:53:00,  6.43s/it]
+ 42%|████▏     | 760/1813 [1:23:43<1:51:34,  6.36s/it]
+                                                      
+{'loss': '0.9994', 'grad_norm': '1.166', 'learning_rate': '0.0001316', 'epoch': '0.4193', 'num_input_tokens_seen': 5078960, 'train_runtime': '5023', 'train_tokens_per_second': '1011'}
+
+ 42%|████▏     | 760/1813 [1:23:43<1:51:34,  6.36s/it]
+ 42%|████▏     | 761/1813 [1:23:49<1:53:43,  6.49s/it]
+ 42%|████▏     | 762/1813 [1:23:56<1:54:39,  6.55s/it]
+ 42%|████▏     | 763/1813 [1:24:03<1:58:16,  6.76s/it]
+ 42%|████▏     | 764/1813 [1:24:10<1:59:20,  6.83s/it]
+ 42%|████▏     | 765/1813 [1:24:18<2:02:38,  7.02s/it]
+ 42%|████▏     | 766/1813 [1:24:24<2:00:03,  6.88s/it]
+ 42%|████▏     | 767/1813 [1:24:30<1:53:32,  6.51s/it]
+ 42%|████▏     | 768/1813 [1:24:35<1:43:01,  5.92s/it]
+ 42%|████▏     | 769/1813 [1:24:41<1:48:17,  6.22s/it]
+ 42%|████▏     | 770/1813 [1:24:48<1:48:35,  6.25s/it]
+                                                      
+{'loss': '0.9156', 'grad_norm': '0.9971', 'learning_rate': '0.00013', 'epoch': '0.4248', 'num_input_tokens_seen': 5148504, 'train_runtime': '5088', 'train_tokens_per_second': '1012'}
+
+ 42%|████▏     | 770/1813 [1:24:48<1:48:35,  6.25s/it]
+ 43%|████▎     | 771/1813 [1:24:54<1:50:41,  6.37s/it]
+ 43%|████▎     | 772/1813 [1:25:02<1:55:08,  6.64s/it]
+ 43%|████▎     | 773/1813 [1:25:08<1:52:10,  6.47s/it]
+ 43%|████▎     | 774/1813 [1:25:13<1:46:02,  6.12s/it]
+ 43%|████▎     | 775/1813 [1:25:19<1:46:27,  6.15s/it]
+ 43%|████▎     | 776/1813 [1:25:26<1:49:28,  6.33s/it]
+ 43%|████▎     | 777/1813 [1:25:34<1:56:32,  6.75s/it]
+ 43%|████▎     | 778/1813 [1:25:39<1:48:36,  6.30s/it]
+ 43%|████▎     | 779/1813 [1:25:44<1:43:50,  6.03s/it]
+ 43%|████▎     | 780/1813 [1:25:51<1:47:38,  6.25s/it]
+                                                      
+{'loss': '0.885', 'grad_norm': '0.9551', 'learning_rate': '0.0001283', 'epoch': '0.4303', 'num_input_tokens_seen': 5215784, 'train_runtime': '5152', 'train_tokens_per_second': '1012'}
+
+ 43%|████▎     | 780/1813 [1:25:51<1:47:38,  6.25s/it]
+ 43%|████▎     | 781/1813 [1:25:58<1:50:54,  6.45s/it]
+ 43%|████▎     | 782/1813 [1:26:06<2:00:04,  6.99s/it]
+ 43%|████▎     | 783/1813 [1:26:14<2:01:02,  7.05s/it]
+ 43%|████▎     | 784/1813 [1:26:20<1:55:33,  6.74s/it]
+ 43%|████▎     | 785/1813 [1:26:27<1:57:06,  6.84s/it]
+ 43%|████▎     | 786/1813 [1:26:34<1:58:39,  6.93s/it]
+ 43%|████▎     | 787/1813 [1:26:39<1:50:46,  6.48s/it]
+ 43%|████▎     | 788/1813 [1:26:45<1:45:23,  6.17s/it]
+ 44%|████▎     | 789/1813 [1:26:50<1:39:19,  5.82s/it]
+ 44%|████▎     | 790/1813 [1:26:55<1:38:19,  5.77s/it]
+                                                      
+{'loss': '0.9901', 'grad_norm': '0.9279', 'learning_rate': '0.0001267', 'epoch': '0.4359', 'num_input_tokens_seen': 5285160, 'train_runtime': '5216', 'train_tokens_per_second': '1013'}
+
+ 44%|████▎     | 790/1813 [1:26:55<1:38:19,  5.77s/it]
+ 44%|████▎     | 791/1813 [1:27:02<1:40:50,  5.92s/it]
+ 44%|████▎     | 792/1813 [1:27:09<1:47:51,  6.34s/it]
+ 44%|████▎     | 793/1813 [1:27:16<1:52:41,  6.63s/it]
+ 44%|████▍     | 794/1813 [1:27:23<1:52:54,  6.65s/it]
+ 44%|████▍     | 795/1813 [1:27:29<1:49:35,  6.46s/it]
+ 44%|████▍     | 796/1813 [1:27:36<1:51:28,  6.58s/it]
+ 44%|████▍     | 797/1813 [1:27:43<1:56:24,  6.87s/it]
+ 44%|████▍     | 798/1813 [1:27:50<1:53:34,  6.71s/it]
+ 44%|████▍     | 799/1813 [1:27:56<1:49:26,  6.48s/it]
+ 44%|████▍     | 800/1813 [1:28:01<1:46:07,  6.29s/it]
+                                                      
+{'loss': '0.9413', 'grad_norm': '0.8946', 'learning_rate': '0.000125', 'epoch': '0.4414', 'num_input_tokens_seen': 5357720, 'train_runtime': '5282', 'train_tokens_per_second': '1014'}
+
+ 44%|████▍     | 800/1813 [1:28:01<1:46:07,  6.29s/it]
+ 44%|████▍     | 801/1813 [1:28:07<1:41:43,  6.03s/it]
+ 44%|████▍     | 802/1813 [1:28:13<1:42:06,  6.06s/it]
+ 44%|████▍     | 803/1813 [1:28:21<1:51:50,  6.64s/it]
+ 44%|████▍     | 804/1813 [1:28:27<1:49:30,  6.51s/it]
+ 44%|████▍     | 805/1813 [1:28:33<1:46:49,  6.36s/it]
+ 44%|████▍     | 806/1813 [1:28:39<1:45:55,  6.31s/it]
+ 45%|████▍     | 807/1813 [1:28:45<1:43:03,  6.15s/it]
+ 45%|████▍     | 808/1813 [1:28:53<1:51:11,  6.64s/it]
+ 45%|████▍     | 809/1813 [1:29:00<1:52:10,  6.70s/it]
+ 45%|████▍     | 810/1813 [1:29:08<1:58:01,  7.06s/it]
+                                                      
+{'loss': '0.9497', 'grad_norm': '0.7668', 'learning_rate': '0.0001234', 'epoch': '0.4469', 'num_input_tokens_seen': 5430288, 'train_runtime': '5348', 'train_tokens_per_second': '1015'}
+
+ 45%|████▍     | 810/1813 [1:29:08<1:58:01,  7.06s/it]
+ 45%|████▍     | 811/1813 [1:29:14<1:56:22,  6.97s/it]
+ 45%|████▍     | 812/1813 [1:29:20<1:50:23,  6.62s/it]
+ 45%|████▍     | 813/1813 [1:29:26<1:45:44,  6.34s/it]
+ 45%|████▍     | 814/1813 [1:29:32<1:44:58,  6.30s/it]
+ 45%|████▍     | 815/1813 [1:29:37<1:38:07,  5.90s/it]
+ 45%|████▌     | 816/1813 [1:29:44<1:42:58,  6.20s/it]
+ 45%|████▌     | 817/1813 [1:29:50<1:41:10,  6.10s/it]
+ 45%|████▌     | 818/1813 [1:29:57<1:46:56,  6.45s/it]
+ 45%|████▌     | 819/1813 [1:30:04<1:47:05,  6.46s/it]
+ 45%|████▌     | 820/1813 [1:30:11<1:51:55,  6.76s/it]
+                                                      
+{'loss': '0.8841', 'grad_norm': '0.9096', 'learning_rate': '0.0001217', 'epoch': '0.4524', 'num_input_tokens_seen': 5497552, 'train_runtime': '5412', 'train_tokens_per_second': '1016'}
+
+ 45%|████▌     | 820/1813 [1:30:11<1:51:55,  6.76s/it]
+ 45%|████▌     | 821/1813 [1:30:18<1:50:27,  6.68s/it]
+ 45%|████▌     | 822/1813 [1:30:23<1:42:48,  6.22s/it]
+ 45%|████▌     | 823/1813 [1:30:28<1:35:37,  5.80s/it]
+ 45%|████▌     | 824/1813 [1:30:34<1:37:47,  5.93s/it]
+ 46%|████▌     | 825/1813 [1:30:40<1:39:22,  6.03s/it]
+ 46%|████▌     | 826/1813 [1:30:47<1:44:02,  6.33s/it]
+ 46%|████▌     | 827/1813 [1:30:53<1:42:16,  6.22s/it]
+ 46%|████▌     | 828/1813 [1:30:59<1:41:55,  6.21s/it]
+ 46%|████▌     | 829/1813 [1:31:05<1:39:49,  6.09s/it]
+ 46%|████▌     | 830/1813 [1:31:13<1:47:32,  6.56s/it]
+                                                      
+{'loss': '0.7937', 'grad_norm': '0.8988', 'learning_rate': '0.00012', 'epoch': '0.4579', 'num_input_tokens_seen': 5563144, 'train_runtime': '5473', 'train_tokens_per_second': '1016'}
+
+ 46%|████▌     | 830/1813 [1:31:13<1:47:32,  6.56s/it]
+ 46%|████▌     | 831/1813 [1:31:19<1:45:27,  6.44s/it]
+ 46%|████▌     | 832/1813 [1:31:26<1:49:44,  6.71s/it]
+ 46%|████▌     | 833/1813 [1:31:33<1:48:24,  6.64s/it]
+ 46%|████▌     | 834/1813 [1:31:39<1:48:59,  6.68s/it]
+ 46%|████▌     | 835/1813 [1:31:46<1:46:20,  6.52s/it]
+ 46%|████▌     | 836/1813 [1:31:53<1:48:14,  6.65s/it]
+ 46%|████▌     | 837/1813 [1:32:01<1:54:21,  7.03s/it]
+ 46%|████▌     | 838/1813 [1:32:06<1:48:49,  6.70s/it]
+ 46%|████▋     | 839/1813 [1:32:13<1:46:42,  6.57s/it]
+ 46%|████▋     | 840/1813 [1:32:18<1:42:15,  6.31s/it]
+                                                      
+{'loss': '0.9689', 'grad_norm': '1.129', 'learning_rate': '0.0001183', 'epoch': '0.4634', 'num_input_tokens_seen': 5634104, 'train_runtime': '5539', 'train_tokens_per_second': '1017'}
+
+ 46%|████▋     | 840/1813 [1:32:18<1:42:15,  6.31s/it]
+ 46%|████▋     | 841/1813 [1:32:24<1:39:30,  6.14s/it]
+ 46%|████▋     | 842/1813 [1:32:31<1:41:37,  6.28s/it]
+ 46%|████▋     | 843/1813 [1:32:37<1:39:57,  6.18s/it]
+ 47%|████▋     | 844/1813 [1:32:42<1:36:18,  5.96s/it]
+ 47%|████▋     | 845/1813 [1:32:48<1:37:22,  6.04s/it]
+ 47%|████▋     | 846/1813 [1:32:55<1:39:31,  6.18s/it]
+ 47%|████▋     | 847/1813 [1:33:02<1:43:13,  6.41s/it]
+ 47%|████▋     | 848/1813 [1:33:09<1:46:55,  6.65s/it]
+ 47%|████▋     | 849/1813 [1:33:14<1:39:38,  6.20s/it]
+ 47%|████▋     | 850/1813 [1:33:23<1:49:46,  6.84s/it]
+                                                      
+{'loss': '0.924', 'grad_norm': '0.7741', 'learning_rate': '0.0001166', 'epoch': '0.469', 'num_input_tokens_seen': 5704744, 'train_runtime': '5603', 'train_tokens_per_second': '1018'}
+
+ 47%|████▋     | 850/1813 [1:33:23<1:49:46,  6.84s/it]
+ 47%|████▋     | 851/1813 [1:33:29<1:49:04,  6.80s/it]
+ 47%|████▋     | 852/1813 [1:33:34<1:38:43,  6.16s/it]
+ 47%|████▋     | 853/1813 [1:33:40<1:38:37,  6.16s/it]
+ 47%|████▋     | 854/1813 [1:33:47<1:41:56,  6.38s/it]
+ 47%|████▋     | 855/1813 [1:33:53<1:40:54,  6.32s/it]
+ 47%|████▋     | 856/1813 [1:33:59<1:40:30,  6.30s/it]
+ 47%|████▋     | 857/1813 [1:34:05<1:36:01,  6.03s/it]
+ 47%|████▋     | 858/1813 [1:34:09<1:26:55,  5.46s/it]
+ 47%|████▋     | 859/1813 [1:34:15<1:30:22,  5.68s/it]
+ 47%|████▋     | 860/1813 [1:34:22<1:33:54,  5.91s/it]
+                                                      
+{'loss': '0.9442', 'grad_norm': '1.175', 'learning_rate': '0.0001149', 'epoch': '0.4745', 'num_input_tokens_seen': 5766424, 'train_runtime': '5662', 'train_tokens_per_second': '1018'}
+
+ 47%|████▋     | 860/1813 [1:34:22<1:33:54,  5.91s/it]
+ 47%|████▋     | 861/1813 [1:34:27<1:30:22,  5.70s/it]
+ 48%|████▊     | 862/1813 [1:34:34<1:35:12,  6.01s/it]
+ 48%|████▊     | 863/1813 [1:34:40<1:35:47,  6.05s/it]
+ 48%|████▊     | 864/1813 [1:34:44<1:26:46,  5.49s/it]
+ 48%|████▊     | 865/1813 [1:34:51<1:33:19,  5.91s/it]
+ 48%|████▊     | 866/1813 [1:34:57<1:34:08,  5.96s/it]
+ 48%|████▊     | 867/1813 [1:35:02<1:32:41,  5.88s/it]
+ 48%|████▊     | 868/1813 [1:35:08<1:31:21,  5.80s/it]
+ 48%|████▊     | 869/1813 [1:35:15<1:38:27,  6.26s/it]
+ 48%|████▊     | 870/1813 [1:35:23<1:44:59,  6.68s/it]
+                                                      
+{'loss': '0.902', 'grad_norm': '0.8346', 'learning_rate': '0.0001132', 'epoch': '0.48', 'num_input_tokens_seen': 5831352, 'train_runtime': '5724', 'train_tokens_per_second': '1019'}
+
+ 48%|████▊     | 870/1813 [1:35:23<1:44:59,  6.68s/it]
+ 48%|████▊     | 871/1813 [1:35:28<1:38:19,  6.26s/it]
+ 48%|████▊     | 872/1813 [1:35:36<1:42:41,  6.55s/it]
+ 48%|████▊     | 873/1813 [1:35:42<1:43:50,  6.63s/it]
+ 48%|████▊     | 874/1813 [1:35:50<1:47:25,  6.86s/it]
+ 48%|████▊     | 875/1813 [1:35:58<1:51:18,  7.12s/it]
+ 48%|████▊     | 876/1813 [1:36:04<1:50:14,  7.06s/it]
+ 48%|████▊     | 877/1813 [1:36:13<1:54:47,  7.36s/it]
+ 48%|████▊     | 878/1813 [1:36:19<1:50:41,  7.10s/it]
+ 48%|████▊     | 879/1813 [1:36:27<1:53:20,  7.28s/it]
+ 49%|████▊     | 880/1813 [1:36:32<1:41:50,  6.55s/it]
+                                                      
+{'loss': '0.9373', 'grad_norm': '1.363', 'learning_rate': '0.0001114', 'epoch': '0.4855', 'num_input_tokens_seen': 5907208, 'train_runtime': '5792', 'train_tokens_per_second': '1020'}
+
+ 49%|████▊     | 880/1813 [1:36:32<1:41:50,  6.55s/it]
+ 49%|████▊     | 881/1813 [1:36:37<1:37:00,  6.25s/it]
+ 49%|████▊     | 882/1813 [1:36:43<1:36:52,  6.24s/it]
+ 49%|████▊     | 883/1813 [1:36:50<1:39:26,  6.42s/it]
+ 49%|████▉     | 884/1813 [1:36:56<1:38:03,  6.33s/it]
+ 49%|████▉     | 885/1813 [1:37:02<1:37:01,  6.27s/it]
+ 49%|████▉     | 886/1813 [1:37:09<1:37:01,  6.28s/it]
+ 49%|████▉     | 887/1813 [1:37:17<1:44:15,  6.76s/it]
+ 49%|████▉     | 888/1813 [1:37:23<1:43:44,  6.73s/it]
+ 49%|████▉     | 889/1813 [1:37:31<1:45:59,  6.88s/it]
+ 49%|████▉     | 890/1813 [1:37:37<1:43:31,  6.73s/it]
+                                                      
+{'loss': '0.9018', 'grad_norm': '0.8232', 'learning_rate': '0.0001097', 'epoch': '0.491', 'num_input_tokens_seen': 5978048, 'train_runtime': '5857', 'train_tokens_per_second': '1021'}
+
+ 49%|████▉     | 890/1813 [1:37:37<1:43:31,  6.73s/it]
+ 49%|████▉     | 891/1813 [1:37:43<1:40:21,  6.53s/it]
+ 49%|████▉     | 892/1813 [1:37:50<1:44:54,  6.83s/it]
+ 49%|████▉     | 893/1813 [1:37:57<1:42:06,  6.66s/it]
+ 49%|████▉     | 894/1813 [1:38:02<1:37:38,  6.37s/it]
+ 49%|████▉     | 895/1813 [1:38:10<1:41:42,  6.65s/it]
+ 49%|████▉     | 896/1813 [1:38:15<1:35:24,  6.24s/it]
+ 49%|████▉     | 897/1813 [1:38:21<1:32:59,  6.09s/it]
+ 50%|████▉     | 898/1813 [1:38:28<1:37:25,  6.39s/it]
+ 50%|████▉     | 899/1813 [1:38:35<1:41:17,  6.65s/it]
+ 50%|████▉     | 900/1813 [1:38:43<1:45:44,  6.95s/it]
+                                                      
+{'loss': '0.9858', 'grad_norm': '0.9776', 'learning_rate': '0.000108', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856, 'train_runtime': '5923', 'train_tokens_per_second': '1022'}
+
+ 50%|████▉     | 900/1813 [1:38:43<1:45:44,  6.95s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.75it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.21it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.30it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.46it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.06it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.99it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.82it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.04it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.35it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.35it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.95it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.98it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.09it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:06<02:04,  7.73it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.07it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.29it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.02it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.89it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.92it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.74it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.30it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.76it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.70it/s]
+
+  7%|▋         | 74/1000 [00:09<01:56,  7.98it/s]
+
+  8%|▊         | 75/1000 [00:10<02:16,  6.79it/s]
+
+  8%|▊         | 76/1000 [00:10<02:19,  6.62it/s]
+
+  8%|▊         | 78/1000 [00:10<02:26,  6.29it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.82it/s]
+
+  8%|▊         | 82/1000 [00:10<01:40,  9.16it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.00it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:34,  5.93it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.37it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  6.99it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:37,  5.77it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.73it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.30it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.42it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.19it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.04it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:12, 12.27it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.65it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:07, 13.09it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.88it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:49,  8.05it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.71it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.13it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:42,  8.50it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:26,  9.96it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:30,  9.54it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.85it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.92it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  8.99it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.85it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25, 10.01it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.41it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.90it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:19, 10.71it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:22, 10.20it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:08,  6.59it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:46,  7.91it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:39,  8.46it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:48,  7.74it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:52,  7.45it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:11,  6.38it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.64it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.17it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.76it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.82it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.14it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.14it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.44it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.71it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.89it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.10it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.20it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.66it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.77it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.62it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.91it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.50it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.47it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.64it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.72it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.13it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.49it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:29,  8.77it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.38it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.03it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.90it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.43it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.96it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+
+ 24%|██▍       | 242/1000 [00:32<02:06,  6.01it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:08,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.73it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.52it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.74it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.99it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.13it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.77it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:41<01:25,  8.13it/s]
+
+ 31%|███       | 306/1000 [00:41<01:40,  6.94it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.78it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.80it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.21it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.27it/s]
+
+ 32%|███▏      | 315/1000 [00:42<01:06, 10.23it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.57it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:43<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.40it/s]
+
+ 33%|███▎      | 328/1000 [00:44<01:33,  7.16it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.37it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.86it/s]
+
+ 34%|███▍      | 344/1000 [00:46<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:47<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.63it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.57it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.42it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.53it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.71it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.21it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:51<01:30,  6.91it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.75it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.89it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.19it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.19it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.64it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.37it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.80it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.48it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.76it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:54<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:56<01:22,  7.07it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.26it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.92it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.94it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.55it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.03it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.79it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.50it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.24it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.49it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.27it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.83it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.21it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.37it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:25,  6.18it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.18it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.29it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.08it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.51it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.05it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.97it/s]
+
+ 50%|████▉     | 496/1000 [01:06<01:00,  8.39it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:08<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.95it/s]
+
+ 51%|█████     | 512/1000 [01:09<00:48,  9.97it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.52it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.10it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.06it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.38it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.58it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.71it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.51it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.91it/s]
+
+ 55%|█████▌    | 551/1000 [01:15<00:59,  7.49it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.25it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.17it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.94it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.76it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.01it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.84it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.04it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:06,  6.51it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:07,  6.42it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.42it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:08,  6.29it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.82it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.24it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.55it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.59it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.74it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.68it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.46it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.74it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.81it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.62it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.75it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.10it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.04it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.42it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:52,  6.98it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.89it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.13it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.94it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.56it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.09it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.57it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.51it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.91it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.10it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  5.99it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.66it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:56,  5.86it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.61it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.24it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.64it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.98it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.90it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.82it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.02it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.55it/s]
+
+ 70%|███████   | 702/1000 [01:36<00:53,  5.59it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.19it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.40it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.00it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.61it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.49it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.68it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.84it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.36it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.16it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.18it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.25it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:27, 10.08it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.56it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.31it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.50it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.88it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.67it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:32,  7.36it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:33,  7.13it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.55it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.59it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:36,  6.55it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.44it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.76it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.88it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.55it/s]
+
+ 78%|███████▊  | 780/1000 [01:46<00:31,  6.92it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.35it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.14it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.52it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.35it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.97it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.21it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.96it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.21it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.23it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.30it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:53<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.46it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.78it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.56it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.46it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.21it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.75it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.61it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.66it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.31it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.26it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.25it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.62it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.59it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.77it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.85it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.71it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.87it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.07it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.32it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:12,  7.45it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.64it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.32it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.04it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.33it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.59it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.11it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.62it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.73it/s]
+
+ 95%|█████████▌| 950/1000 [02:08<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.04it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.44it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.03it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.79it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.54it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.07it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.69it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.61it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.7758', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.38', 'eval_steps_per_second': '7.38', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856}
+
+ 50%|████▉     | 900/1813 [1:40:58<1:45:44,  6.95s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.61it/s]
+
+                                                   
+ 50%|████▉     | 901/1813 [1:41:05<12:01:13, 47.45s/it]
+ 50%|████▉     | 902/1813 [1:41:11<8:51:48, 35.03s/it] 
+ 50%|████▉     | 903/1813 [1:41:17<6:39:58, 26.37s/it]
+ 50%|████▉     | 904/1813 [1:41:23<5:06:58, 20.26s/it]
+ 50%|████▉     | 905/1813 [1:41:31<4:10:35, 16.56s/it]
+ 50%|████▉     | 906/1813 [1:41:36<3:19:43, 13.21s/it]
+ 50%|█████     | 907/1813 [1:41:41<2:39:43, 10.58s/it]
+ 50%|█████     | 908/1813 [1:41:46<2:16:06,  9.02s/it]
+ 50%|█████     | 909/1813 [1:41:53<2:05:04,  8.30s/it]
+ 50%|█████     | 910/1813 [1:41:59<1:54:50,  7.63s/it]
+                                                      
+{'loss': '0.912', 'grad_norm': '1.084', 'learning_rate': '0.0001062', 'epoch': '0.5021', 'num_input_tokens_seen': 6115160, 'train_runtime': '6119', 'train_tokens_per_second': '999.3'}
+
+ 50%|█████     | 910/1813 [1:41:59<1:54:50,  7.63s/it]
+ 50%|█████     | 911/1813 [1:42:04<1:45:27,  7.02s/it]
+ 50%|█████     | 912/1813 [1:42:10<1:39:28,  6.62s/it]
+ 50%|█████     | 913/1813 [1:42:15<1:34:03,  6.27s/it]
+ 50%|█████     | 914/1813 [1:42:20<1:28:08,  5.88s/it]
+ 50%|█████     | 915/1813 [1:42:28<1:36:52,  6.47s/it]
+ 51%|█████     | 916/1813 [1:42:36<1:40:22,  6.71s/it]
+ 51%|█████     | 917/1813 [1:42:43<1:41:24,  6.79s/it]
+ 51%|█████     | 918/1813 [1:42:48<1:35:37,  6.41s/it]
+ 51%|█████     | 919/1813 [1:42:54<1:34:38,  6.35s/it]
+ 51%|█████     | 920/1813 [1:43:00<1:30:33,  6.08s/it]
+                                                      
+{'loss': '0.9394', 'grad_norm': '0.8053', 'learning_rate': '0.0001045', 'epoch': '0.5076', 'num_input_tokens_seen': 6178904, 'train_runtime': '6180', 'train_tokens_per_second': '999.8'}
+
+ 51%|█████     | 920/1813 [1:43:00<1:30:33,  6.08s/it]
+ 51%|█████     | 921/1813 [1:43:05<1:28:13,  5.93s/it]
+ 51%|█████     | 922/1813 [1:43:11<1:26:21,  5.82s/it]
+ 51%|█████     | 923/1813 [1:43:17<1:26:28,  5.83s/it]
+ 51%|█████     | 924/1813 [1:43:24<1:31:04,  6.15s/it]
+ 51%|█████     | 925/1813 [1:43:30<1:29:57,  6.08s/it]
+ 51%|█████     | 926/1813 [1:43:36<1:32:07,  6.23s/it]
+ 51%|█████     | 927/1813 [1:43:42<1:30:51,  6.15s/it]
+ 51%|█████     | 928/1813 [1:43:48<1:28:07,  5.97s/it]
+ 51%|█████     | 929/1813 [1:43:54<1:27:58,  5.97s/it]
+ 51%|█████▏    | 930/1813 [1:44:01<1:31:51,  6.24s/it]
+                                                      
+{'loss': '0.9005', 'grad_norm': '0.9171', 'learning_rate': '0.0001027', 'epoch': '0.5131', 'num_input_tokens_seen': 6244360, 'train_runtime': '6241', 'train_tokens_per_second': '1001'}
+
+ 51%|█████▏    | 930/1813 [1:44:01<1:31:51,  6.24s/it]
+ 51%|█████▏    | 931/1813 [1:44:07<1:34:25,  6.42s/it]
+ 51%|█████▏    | 932/1813 [1:44:13<1:29:54,  6.12s/it]
+ 51%|█████▏    | 933/1813 [1:44:19<1:28:28,  6.03s/it]
+ 52%|█████▏    | 934/1813 [1:44:25<1:30:47,  6.20s/it]
+ 52%|█████▏    | 935/1813 [1:44:34<1:42:17,  6.99s/it]
+ 52%|█████▏    | 936/1813 [1:44:40<1:39:37,  6.82s/it]
+ 52%|█████▏    | 937/1813 [1:44:47<1:36:15,  6.59s/it]
+ 52%|█████▏    | 938/1813 [1:44:54<1:38:09,  6.73s/it]
+ 52%|█████▏    | 939/1813 [1:44:59<1:34:06,  6.46s/it]
+ 52%|█████▏    | 940/1813 [1:45:06<1:36:30,  6.63s/it]
+                                                      
+{'loss': '0.8898', 'grad_norm': '1.111', 'learning_rate': '0.0001009', 'epoch': '0.5186', 'num_input_tokens_seen': 6318968, 'train_runtime': '6307', 'train_tokens_per_second': '1002'}
+
+ 52%|█████▏    | 940/1813 [1:45:06<1:36:30,  6.63s/it]
+ 52%|█████▏    | 941/1813 [1:45:12<1:29:39,  6.17s/it]
+ 52%|█████▏    | 942/1813 [1:45:17<1:26:39,  5.97s/it]
+ 52%|█████▏    | 943/1813 [1:45:23<1:24:40,  5.84s/it]
+ 52%|█████▏    | 944/1813 [1:45:28<1:24:47,  5.85s/it]
+ 52%|█████▏    | 945/1813 [1:45:34<1:22:18,  5.69s/it]
+ 52%|█████▏    | 946/1813 [1:45:40<1:24:17,  5.83s/it]
+ 52%|█████▏    | 947/1813 [1:45:46<1:25:17,  5.91s/it]
+ 52%|█████▏    | 948/1813 [1:45:52<1:26:42,  6.01s/it]
+ 52%|█████▏    | 949/1813 [1:45:57<1:22:31,  5.73s/it]
+ 52%|█████▏    | 950/1813 [1:46:04<1:25:28,  5.94s/it]
+                                                      
+{'loss': '0.9472', 'grad_norm': '1.198', 'learning_rate': '9.919e-05', 'epoch': '0.5241', 'num_input_tokens_seen': 6379704, 'train_runtime': '6364', 'train_tokens_per_second': '1002'}
+
+ 52%|█████▏    | 950/1813 [1:46:04<1:25:28,  5.94s/it]
+ 52%|█████▏    | 951/1813 [1:46:09<1:22:53,  5.77s/it]
+ 53%|█████▎    | 952/1813 [1:46:15<1:22:14,  5.73s/it]
+ 53%|█████▎    | 953/1813 [1:46:21<1:22:59,  5.79s/it]
+ 53%|█████▎    | 954/1813 [1:46:26<1:22:15,  5.75s/it]
+ 53%|█████▎    | 955/1813 [1:46:32<1:21:51,  5.72s/it]
+ 53%|█████▎    | 956/1813 [1:46:38<1:21:07,  5.68s/it]
+ 53%|█████▎    | 957/1813 [1:46:43<1:20:25,  5.64s/it]
+ 53%|█████▎    | 958/1813 [1:46:48<1:15:27,  5.29s/it]
+ 53%|█████▎    | 959/1813 [1:46:53<1:17:40,  5.46s/it]
+ 53%|█████▎    | 960/1813 [1:47:02<1:29:42,  6.31s/it]
+                                                      
+{'loss': '0.951', 'grad_norm': '0.7629', 'learning_rate': '9.742e-05', 'epoch': '0.5297', 'num_input_tokens_seen': 6441656, 'train_runtime': '6422', 'train_tokens_per_second': '1003'}
+
+ 53%|█████▎    | 960/1813 [1:47:02<1:29:42,  6.31s/it]
+ 53%|█████▎    | 961/1813 [1:47:08<1:31:10,  6.42s/it]
+ 53%|█████▎    | 962/1813 [1:47:16<1:37:24,  6.87s/it]
+ 53%|█████▎    | 963/1813 [1:47:23<1:37:02,  6.85s/it]
+ 53%|█████▎    | 964/1813 [1:47:29<1:32:19,  6.53s/it]
+ 53%|█████▎    | 965/1813 [1:47:35<1:29:26,  6.33s/it]
+ 53%|█████▎    | 966/1813 [1:47:42<1:34:19,  6.68s/it]
+ 53%|█████▎    | 967/1813 [1:47:50<1:37:14,  6.90s/it]
+ 53%|█████▎    | 968/1813 [1:47:55<1:31:13,  6.48s/it]
+ 53%|█████▎    | 969/1813 [1:48:01<1:28:51,  6.32s/it]
+ 54%|█████▎    | 970/1813 [1:48:06<1:21:47,  5.82s/it]
+                                                      
+{'loss': '0.9726', 'grad_norm': '1.076', 'learning_rate': '9.566e-05', 'epoch': '0.5352', 'num_input_tokens_seen': 6512320, 'train_runtime': '6486', 'train_tokens_per_second': '1004'}
+
+ 54%|█████▎    | 970/1813 [1:48:06<1:21:47,  5.82s/it]
+ 54%|█████▎    | 971/1813 [1:48:12<1:25:19,  6.08s/it]
+ 54%|█████▎    | 972/1813 [1:48:19<1:26:26,  6.17s/it]
+ 54%|█████▎    | 973/1813 [1:48:25<1:25:09,  6.08s/it]
+ 54%|█████▎    | 974/1813 [1:48:33<1:34:42,  6.77s/it]
+ 54%|█████▍    | 975/1813 [1:48:40<1:32:57,  6.66s/it]
+ 54%|█████▍    | 976/1813 [1:48:45<1:28:19,  6.33s/it]
+ 54%|█████▍    | 977/1813 [1:48:53<1:34:00,  6.75s/it]
+ 54%|█████▍    | 978/1813 [1:48:59<1:31:45,  6.59s/it]
+ 54%|█████▍    | 979/1813 [1:49:06<1:33:32,  6.73s/it]
+ 54%|█████▍    | 980/1813 [1:49:12<1:32:06,  6.63s/it]
+                                                      
+{'loss': '0.9342', 'grad_norm': '1.505', 'learning_rate': '9.389e-05', 'epoch': '0.5407', 'num_input_tokens_seen': 6586672, 'train_runtime': '6553', 'train_tokens_per_second': '1005'}
+
+ 54%|█████▍    | 980/1813 [1:49:12<1:32:06,  6.63s/it]
+ 54%|█████▍    | 981/1813 [1:49:19<1:31:04,  6.57s/it]
+ 54%|█████▍    | 982/1813 [1:49:24<1:25:10,  6.15s/it]
+ 54%|█████▍    | 983/1813 [1:49:29<1:21:16,  5.87s/it]
+ 54%|█████▍    | 984/1813 [1:49:35<1:19:01,  5.72s/it]
+ 54%|█████▍    | 985/1813 [1:49:41<1:22:33,  5.98s/it]
+ 54%|█████▍    | 986/1813 [1:49:49<1:28:29,  6.42s/it]
+ 54%|█████▍    | 987/1813 [1:49:55<1:26:28,  6.28s/it]
+ 54%|█████▍    | 988/1813 [1:50:00<1:22:59,  6.04s/it]
+ 55%|█████▍    | 989/1813 [1:50:05<1:18:30,  5.72s/it]
+ 55%|█████▍    | 990/1813 [1:50:13<1:27:29,  6.38s/it]
+                                                      
+{'loss': '0.8826', 'grad_norm': '1.03', 'learning_rate': '9.213e-05', 'epoch': '0.5462', 'num_input_tokens_seen': 6653096, 'train_runtime': '6614', 'train_tokens_per_second': '1006'}
+
+ 55%|█████▍    | 990/1813 [1:50:13<1:27:29,  6.38s/it]
+ 55%|█████▍    | 991/1813 [1:50:18<1:22:13,  6.00s/it]
+ 55%|█████▍    | 992/1813 [1:50:25<1:27:19,  6.38s/it]
+ 55%|█████▍    | 993/1813 [1:50:31<1:25:26,  6.25s/it]
+ 55%|█████▍    | 994/1813 [1:50:40<1:34:14,  6.90s/it]
+ 55%|█████▍    | 995/1813 [1:50:46<1:30:12,  6.62s/it]
+ 55%|█████▍    | 996/1813 [1:50:52<1:29:03,  6.54s/it]
+ 55%|█████▍    | 997/1813 [1:50:58<1:26:08,  6.33s/it]
+ 55%|█████▌    | 998/1813 [1:51:03<1:21:45,  6.02s/it]
+ 55%|█████▌    | 999/1813 [1:51:09<1:20:40,  5.95s/it]
+ 55%|█████▌    | 1000/1813 [1:51:14<1:17:31,  5.72s/it]
+                                                       
+{'loss': '0.9987', 'grad_norm': '1.053', 'learning_rate': '9.036e-05', 'epoch': '0.5517', 'num_input_tokens_seen': 6716784, 'train_runtime': '6675', 'train_tokens_per_second': '1006'}
+
+ 55%|█████▌    | 1000/1813 [1:51:14<1:17:31,  5.72s/it]
+ 55%|█████▌    | 1001/1813 [1:51:21<1:20:30,  5.95s/it]
+ 55%|█████▌    | 1002/1813 [1:51:27<1:21:22,  6.02s/it]
+ 55%|█████▌    | 1003/1813 [1:51:34<1:24:52,  6.29s/it]
+ 55%|█████▌    | 1004/1813 [1:51:39<1:21:42,  6.06s/it]
+ 55%|█████▌    | 1005/1813 [1:51:47<1:26:10,  6.40s/it]
+ 55%|█████▌    | 1006/1813 [1:51:53<1:27:28,  6.50s/it]
+ 56%|█████▌    | 1007/1813 [1:51:59<1:24:23,  6.28s/it]
+ 56%|█████▌    | 1008/1813 [1:52:05<1:22:04,  6.12s/it]
+ 56%|█████▌    | 1009/1813 [1:52:11<1:21:31,  6.08s/it]
+ 56%|█████▌    | 1010/1813 [1:52:17<1:20:17,  6.00s/it]
+                                                       
+{'loss': '0.9297', 'grad_norm': '1.148', 'learning_rate': '8.86e-05', 'epoch': '0.5572', 'num_input_tokens_seen': 6783768, 'train_runtime': '6737', 'train_tokens_per_second': '1007'}
+
+ 56%|█████▌    | 1010/1813 [1:52:17<1:20:17,  6.00s/it]
+ 56%|█████▌    | 1011/1813 [1:52:23<1:23:15,  6.23s/it]
+ 56%|█████▌    | 1012/1813 [1:52:28<1:17:12,  5.78s/it]
+ 56%|█████▌    | 1013/1813 [1:52:32<1:11:19,  5.35s/it]
+ 56%|█████▌    | 1014/1813 [1:52:41<1:23:03,  6.24s/it]
+ 56%|█████▌    | 1015/1813 [1:52:46<1:18:41,  5.92s/it]
+ 56%|█████▌    | 1016/1813 [1:52:51<1:15:47,  5.71s/it]
+ 56%|█████▌    | 1017/1813 [1:52:58<1:20:02,  6.03s/it]
+ 56%|█████▌    | 1018/1813 [1:53:04<1:21:10,  6.13s/it]
+ 56%|█████▌    | 1019/1813 [1:53:09<1:17:03,  5.82s/it]
+ 56%|█████▋    | 1020/1813 [1:53:14<1:13:47,  5.58s/it]
+                                                       
+{'loss': '0.8492', 'grad_norm': '1.182', 'learning_rate': '8.684e-05', 'epoch': '0.5628', 'num_input_tokens_seen': 6844992, 'train_runtime': '6795', 'train_tokens_per_second': '1007'}
+
+ 56%|█████▋    | 1020/1813 [1:53:14<1:13:47,  5.58s/it]
+ 56%|█████▋    | 1021/1813 [1:53:21<1:18:53,  5.98s/it]
+ 56%|█████▋    | 1022/1813 [1:53:28<1:20:15,  6.09s/it]
+ 56%|█████▋    | 1023/1813 [1:53:35<1:26:28,  6.57s/it]
+ 56%|█████▋    | 1024/1813 [1:53:42<1:25:53,  6.53s/it]
+ 57%|█████▋    | 1025/1813 [1:53:51<1:36:20,  7.34s/it]
+ 57%|█████▋    | 1026/1813 [1:53:56<1:28:55,  6.78s/it]
+ 57%|█████▋    | 1027/1813 [1:54:03<1:26:13,  6.58s/it]
+ 57%|█████▋    | 1028/1813 [1:54:10<1:29:06,  6.81s/it]
+ 57%|█████▋    | 1029/1813 [1:54:16<1:27:13,  6.68s/it]
+ 57%|█████▋    | 1030/1813 [1:54:23<1:27:02,  6.67s/it]
+                                                       
+{'loss': '0.9907', 'grad_norm': '0.8044', 'learning_rate': '8.508e-05', 'epoch': '0.5683', 'num_input_tokens_seen': 6920856, 'train_runtime': '6863', 'train_tokens_per_second': '1008'}
+
+ 57%|█████▋    | 1030/1813 [1:54:23<1:27:02,  6.67s/it]
+ 57%|█████▋    | 1031/1813 [1:54:28<1:22:16,  6.31s/it]
+ 57%|█████▋    | 1032/1813 [1:54:35<1:23:24,  6.41s/it]
+ 57%|█████▋    | 1033/1813 [1:54:41<1:22:22,  6.34s/it]
+ 57%|█████▋    | 1034/1813 [1:54:47<1:20:35,  6.21s/it]
+ 57%|█████▋    | 1035/1813 [1:54:54<1:21:19,  6.27s/it]
+ 57%|█████▋    | 1036/1813 [1:55:00<1:23:03,  6.41s/it]
+ 57%|█████▋    | 1037/1813 [1:55:06<1:21:26,  6.30s/it]
+ 57%|█████▋    | 1038/1813 [1:55:13<1:21:30,  6.31s/it]
+ 57%|█████▋    | 1039/1813 [1:55:20<1:24:20,  6.54s/it]
+ 57%|█████▋    | 1040/1813 [1:55:27<1:26:11,  6.69s/it]
+                                                       
+{'loss': '0.9106', 'grad_norm': '0.848', 'learning_rate': '8.332e-05', 'epoch': '0.5738', 'num_input_tokens_seen': 6988568, 'train_runtime': '6927', 'train_tokens_per_second': '1009'}
+
+ 57%|█████▋    | 1040/1813 [1:55:27<1:26:11,  6.69s/it]
+ 57%|█████▋    | 1041/1813 [1:55:33<1:23:58,  6.53s/it]
+ 57%|█████▋    | 1042/1813 [1:55:39<1:22:57,  6.46s/it]
+ 58%|█████▊    | 1043/1813 [1:55:45<1:19:51,  6.22s/it]
+ 58%|█████▊    | 1044/1813 [1:55:50<1:16:19,  5.96s/it]
+ 58%|█████▊    | 1045/1813 [1:55:56<1:16:08,  5.95s/it]
+ 58%|█████▊    | 1046/1813 [1:56:02<1:15:30,  5.91s/it]
+ 58%|█████▊    | 1047/1813 [1:56:08<1:16:57,  6.03s/it]
+ 58%|█████▊    | 1048/1813 [1:56:15<1:20:46,  6.34s/it]
+ 58%|█████▊    | 1049/1813 [1:56:21<1:18:12,  6.14s/it]
+ 58%|█████▊    | 1050/1813 [1:56:27<1:17:44,  6.11s/it]
+                                                       
+{'loss': '0.8998', 'grad_norm': '1.143', 'learning_rate': '8.157e-05', 'epoch': '0.5793', 'num_input_tokens_seen': 7051808, 'train_runtime': '6988', 'train_tokens_per_second': '1009'}
+
+ 58%|█████▊    | 1050/1813 [1:56:27<1:17:44,  6.11s/it]
+ 58%|█████▊    | 1051/1813 [1:56:32<1:14:37,  5.88s/it]
+ 58%|█████▊    | 1052/1813 [1:56:38<1:14:42,  5.89s/it]
+ 58%|█████▊    | 1053/1813 [1:56:44<1:15:09,  5.93s/it]
+ 58%|█████▊    | 1054/1813 [1:56:50<1:14:59,  5.93s/it]
+ 58%|█████▊    | 1055/1813 [1:56:56<1:13:15,  5.80s/it]
+ 58%|█████▊    | 1056/1813 [1:57:02<1:14:03,  5.87s/it]
+ 58%|█████▊    | 1057/1813 [1:57:08<1:16:57,  6.11s/it]
+ 58%|█████▊    | 1058/1813 [1:57:14<1:13:10,  5.82s/it]
+ 58%|█████▊    | 1059/1813 [1:57:18<1:09:25,  5.53s/it]
+ 58%|█████▊    | 1060/1813 [1:57:25<1:15:06,  5.99s/it]
+                                                       
+{'loss': '0.9022', 'grad_norm': '0.7365', 'learning_rate': '7.982e-05', 'epoch': '0.5848', 'num_input_tokens_seen': 7110624, 'train_runtime': '7046', 'train_tokens_per_second': '1009'}
+
+ 58%|█████▊    | 1060/1813 [1:57:25<1:15:06,  5.99s/it]
+ 59%|█████▊    | 1061/1813 [1:57:31<1:12:07,  5.75s/it]
+ 59%|█████▊    | 1062/1813 [1:57:36<1:09:52,  5.58s/it]
+ 59%|█████▊    | 1063/1813 [1:57:43<1:14:33,  5.96s/it]
+ 59%|█████▊    | 1064/1813 [1:57:50<1:20:13,  6.43s/it]
+ 59%|█████▊    | 1065/1813 [1:57:57<1:21:12,  6.51s/it]
+ 59%|█████▉    | 1066/1813 [1:58:04<1:24:36,  6.80s/it]
+ 59%|█████▉    | 1067/1813 [1:58:11<1:23:47,  6.74s/it]
+ 59%|█████▉    | 1068/1813 [1:58:16<1:18:51,  6.35s/it]
+ 59%|█████▉    | 1069/1813 [1:58:25<1:25:09,  6.87s/it]
+ 59%|█████▉    | 1070/1813 [1:58:30<1:21:03,  6.55s/it]
+                                                       
+{'loss': '0.9061', 'grad_norm': '1.076', 'learning_rate': '7.808e-05', 'epoch': '0.5903', 'num_input_tokens_seen': 7179728, 'train_runtime': '7111', 'train_tokens_per_second': '1010'}
+
+ 59%|█████▉    | 1070/1813 [1:58:30<1:21:03,  6.55s/it]
+ 59%|█████▉    | 1071/1813 [1:58:36<1:16:59,  6.23s/it]
+ 59%|█████▉    | 1072/1813 [1:58:42<1:17:27,  6.27s/it]
+ 59%|█████▉    | 1073/1813 [1:58:48<1:15:58,  6.16s/it]
+ 59%|█████▉    | 1074/1813 [1:58:54<1:13:43,  5.99s/it]
+ 59%|█████▉    | 1075/1813 [1:59:00<1:15:50,  6.17s/it]
+ 59%|█████▉    | 1076/1813 [1:59:07<1:17:34,  6.32s/it]
+ 59%|█████▉    | 1077/1813 [1:59:12<1:11:40,  5.84s/it]
+ 59%|█████▉    | 1078/1813 [1:59:17<1:09:17,  5.66s/it]
+ 60%|█████▉    | 1079/1813 [1:59:23<1:11:59,  5.88s/it]
+ 60%|█████▉    | 1080/1813 [1:59:30<1:13:59,  6.06s/it]
+                                                       
+{'loss': '0.9067', 'grad_norm': '0.7867', 'learning_rate': '7.634e-05', 'epoch': '0.5959', 'num_input_tokens_seen': 7240984, 'train_runtime': '7170', 'train_tokens_per_second': '1010'}
+
+ 60%|█████▉    | 1080/1813 [1:59:30<1:13:59,  6.06s/it]
+ 60%|█████▉    | 1081/1813 [1:59:36<1:14:58,  6.15s/it]
+ 60%|█████▉    | 1082/1813 [1:59:42<1:13:26,  6.03s/it]
+ 60%|█████▉    | 1083/1813 [1:59:48<1:12:52,  5.99s/it]
+ 60%|█████▉    | 1084/1813 [1:59:54<1:12:06,  5.94s/it]
+ 60%|█████▉    | 1085/1813 [1:59:59<1:11:51,  5.92s/it]
+ 60%|█████▉    | 1086/1813 [2:00:03<1:04:28,  5.32s/it]
+ 60%|█████▉    | 1087/1813 [2:00:09<1:05:33,  5.42s/it]
+ 60%|██████    | 1088/1813 [2:00:15<1:06:21,  5.49s/it]
+ 60%|██████    | 1089/1813 [2:00:21<1:09:27,  5.76s/it]
+ 60%|██████    | 1090/1813 [2:00:27<1:09:44,  5.79s/it]
+                                                       
+{'loss': '0.8095', 'grad_norm': '1.012', 'learning_rate': '7.46e-05', 'epoch': '0.6014', 'num_input_tokens_seen': 7301824, 'train_runtime': '7227', 'train_tokens_per_second': '1010'}
+
+ 60%|██████    | 1090/1813 [2:00:27<1:09:44,  5.79s/it]
+ 60%|██████    | 1091/1813 [2:00:33<1:09:58,  5.81s/it]
+ 60%|██████    | 1092/1813 [2:00:39<1:09:58,  5.82s/it]
+ 60%|██████    | 1093/1813 [2:00:44<1:08:25,  5.70s/it]
+ 60%|██████    | 1094/1813 [2:00:50<1:10:16,  5.86s/it]
+ 60%|██████    | 1095/1813 [2:00:56<1:11:01,  5.93s/it]
+ 60%|██████    | 1096/1813 [2:01:02<1:08:53,  5.76s/it]
+ 61%|██████    | 1097/1813 [2:01:08<1:09:33,  5.83s/it]
+ 61%|██████    | 1098/1813 [2:01:15<1:14:18,  6.24s/it]
+ 61%|██████    | 1099/1813 [2:01:21<1:14:57,  6.30s/it]
+ 61%|██████    | 1100/1813 [2:01:29<1:20:57,  6.81s/it]
+                                                       
+{'loss': '0.8954', 'grad_norm': '1.044', 'learning_rate': '7.288e-05', 'epoch': '0.6069', 'num_input_tokens_seen': 7369528, 'train_runtime': '7290', 'train_tokens_per_second': '1011'}
+
+ 61%|██████    | 1100/1813 [2:01:29<1:20:57,  6.81s/it]
+ 61%|██████    | 1101/1813 [2:01:36<1:19:19,  6.68s/it]
+ 61%|██████    | 1102/1813 [2:01:43<1:20:16,  6.77s/it]
+ 61%|██████    | 1103/1813 [2:01:48<1:15:04,  6.34s/it]
+ 61%|██████    | 1104/1813 [2:01:56<1:18:56,  6.68s/it]
+ 61%|██████    | 1105/1813 [2:02:00<1:11:38,  6.07s/it]
+ 61%|██████    | 1106/1813 [2:02:06<1:09:47,  5.92s/it]
+ 61%|██████    | 1107/1813 [2:02:14<1:18:19,  6.66s/it]
+ 61%|██████    | 1108/1813 [2:02:20<1:14:44,  6.36s/it]
+ 61%|██████    | 1109/1813 [2:02:26<1:15:40,  6.45s/it]
+ 61%|██████    | 1110/1813 [2:02:34<1:20:01,  6.83s/it]
+                                                       
+{'loss': '0.9343', 'grad_norm': '0.9072', 'learning_rate': '7.116e-05', 'epoch': '0.6124', 'num_input_tokens_seen': 7441040, 'train_runtime': '7355', 'train_tokens_per_second': '1012'}
+
+ 61%|██████    | 1110/1813 [2:02:34<1:20:01,  6.83s/it]
+ 61%|██████▏   | 1111/1813 [2:02:39<1:12:31,  6.20s/it]
+ 61%|██████▏   | 1112/1813 [2:02:45<1:11:31,  6.12s/it]
+ 61%|██████▏   | 1113/1813 [2:02:51<1:12:42,  6.23s/it]
+ 61%|██████▏   | 1114/1813 [2:02:59<1:16:15,  6.55s/it]
+ 62%|██████▏   | 1115/1813 [2:03:05<1:17:01,  6.62s/it]
+ 62%|██████▏   | 1116/1813 [2:03:12<1:16:32,  6.59s/it]
+ 62%|██████▏   | 1117/1813 [2:03:18<1:13:15,  6.31s/it]
+ 62%|██████▏   | 1118/1813 [2:03:23<1:08:40,  5.93s/it]
+ 62%|██████▏   | 1119/1813 [2:03:29<1:10:28,  6.09s/it]
+ 62%|██████▏   | 1120/1813 [2:03:36<1:12:08,  6.25s/it]
+                                                       
+{'loss': '0.8723', 'grad_norm': '0.938', 'learning_rate': '6.945e-05', 'epoch': '0.6179', 'num_input_tokens_seen': 7504976, 'train_runtime': '7416', 'train_tokens_per_second': '1012'}
+
+ 62%|██████▏   | 1120/1813 [2:03:36<1:12:08,  6.25s/it]
+ 62%|██████▏   | 1121/1813 [2:03:44<1:17:39,  6.73s/it]
+ 62%|██████▏   | 1122/1813 [2:03:48<1:10:20,  6.11s/it]
+ 62%|██████▏   | 1123/1813 [2:03:56<1:15:06,  6.53s/it]
+ 62%|██████▏   | 1124/1813 [2:04:01<1:11:01,  6.18s/it]
+ 62%|██████▏   | 1125/1813 [2:04:08<1:13:00,  6.37s/it]
+ 62%|██████▏   | 1126/1813 [2:04:14<1:11:22,  6.23s/it]
+ 62%|██████▏   | 1127/1813 [2:04:19<1:08:30,  5.99s/it]
+ 62%|██████▏   | 1128/1813 [2:04:25<1:06:59,  5.87s/it]
+ 62%|██████▏   | 1129/1813 [2:04:30<1:05:28,  5.74s/it]
+ 62%|██████▏   | 1130/1813 [2:04:36<1:03:48,  5.61s/it]
+                                                       
+{'loss': '0.8549', 'grad_norm': '0.9422', 'learning_rate': '6.775e-05', 'epoch': '0.6234', 'num_input_tokens_seen': 7568576, 'train_runtime': '7476', 'train_tokens_per_second': '1012'}
+
+ 62%|██████▏   | 1130/1813 [2:04:36<1:03:48,  5.61s/it]
+ 62%|██████▏   | 1131/1813 [2:04:41<1:04:22,  5.66s/it]
+ 62%|██████▏   | 1132/1813 [2:04:48<1:05:59,  5.81s/it]
+ 62%|██████▏   | 1133/1813 [2:04:54<1:08:51,  6.08s/it]
+ 63%|██████▎   | 1134/1813 [2:05:02<1:15:03,  6.63s/it]
+ 63%|██████▎   | 1135/1813 [2:05:09<1:16:03,  6.73s/it]
+ 63%|██████▎   | 1136/1813 [2:05:16<1:16:56,  6.82s/it]
+ 63%|██████▎   | 1137/1813 [2:05:23<1:16:10,  6.76s/it]
+ 63%|██████▎   | 1138/1813 [2:05:30<1:16:55,  6.84s/it]
+ 63%|██████▎   | 1139/1813 [2:05:35<1:12:05,  6.42s/it]
+ 63%|██████▎   | 1140/1813 [2:05:41<1:08:47,  6.13s/it]
+                                                       
+{'loss': '0.9337', 'grad_norm': '0.9539', 'learning_rate': '6.606e-05', 'epoch': '0.629', 'num_input_tokens_seen': 7639168, 'train_runtime': '7541', 'train_tokens_per_second': '1013'}
+
+ 63%|██████▎   | 1140/1813 [2:05:41<1:08:47,  6.13s/it]
+ 63%|██████▎   | 1141/1813 [2:05:47<1:08:32,  6.12s/it]
+ 63%|██████▎   | 1142/1813 [2:05:53<1:07:15,  6.01s/it]
+ 63%|██████▎   | 1143/1813 [2:05:58<1:06:11,  5.93s/it]
+ 63%|██████▎   | 1144/1813 [2:06:05<1:08:42,  6.16s/it]
+ 63%|██████▎   | 1145/1813 [2:06:12<1:09:45,  6.27s/it]
+ 63%|██████▎   | 1146/1813 [2:06:17<1:06:03,  5.94s/it]
+ 63%|██████▎   | 1147/1813 [2:06:24<1:11:04,  6.40s/it]
+ 63%|██████▎   | 1148/1813 [2:06:31<1:13:19,  6.62s/it]
+ 63%|██████▎   | 1149/1813 [2:06:37<1:10:02,  6.33s/it]
+ 63%|██████▎   | 1150/1813 [2:06:45<1:14:38,  6.76s/it]
+                                                       
+{'loss': '0.8485', 'grad_norm': '0.8957', 'learning_rate': '6.437e-05', 'epoch': '0.6345', 'num_input_tokens_seen': 7707832, 'train_runtime': '7605', 'train_tokens_per_second': '1013'}
+
+ 63%|██████▎   | 1150/1813 [2:06:45<1:14:38,  6.76s/it]
+ 63%|██████▎   | 1151/1813 [2:06:49<1:05:49,  5.97s/it]
+ 64%|██████▎   | 1152/1813 [2:06:54<1:04:03,  5.81s/it]
+ 64%|██████▎   | 1153/1813 [2:07:01<1:07:49,  6.17s/it]
+ 64%|██████▎   | 1154/1813 [2:07:07<1:07:36,  6.16s/it]
+ 64%|██████▎   | 1155/1813 [2:07:14<1:09:15,  6.32s/it]
+ 64%|██████▍   | 1156/1813 [2:07:21<1:09:30,  6.35s/it]
+ 64%|██████▍   | 1157/1813 [2:07:28<1:13:44,  6.74s/it]
+ 64%|██████▍   | 1158/1813 [2:07:35<1:13:23,  6.72s/it]
+ 64%|██████▍   | 1159/1813 [2:07:42<1:14:38,  6.85s/it]
+ 64%|██████▍   | 1160/1813 [2:07:48<1:12:04,  6.62s/it]
+                                                       
+{'loss': '0.9187', 'grad_norm': '0.7358', 'learning_rate': '6.27e-05', 'epoch': '0.64', 'num_input_tokens_seen': 7775872, 'train_runtime': '7669', 'train_tokens_per_second': '1014'}
+
+ 64%|██████▍   | 1160/1813 [2:07:48<1:12:04,  6.62s/it]
+ 64%|██████▍   | 1161/1813 [2:07:55<1:14:02,  6.81s/it]
+ 64%|██████▍   | 1162/1813 [2:08:02<1:13:33,  6.78s/it]
+ 64%|██████▍   | 1163/1813 [2:08:08<1:11:36,  6.61s/it]
+ 64%|██████▍   | 1164/1813 [2:08:14<1:10:05,  6.48s/it]
+ 64%|██████▍   | 1165/1813 [2:08:21<1:09:47,  6.46s/it]
+ 64%|██████▍   | 1166/1813 [2:08:27<1:08:13,  6.33s/it]
+ 64%|██████▍   | 1167/1813 [2:08:34<1:10:52,  6.58s/it]
+ 64%|██████▍   | 1168/1813 [2:08:41<1:10:35,  6.57s/it]
+ 64%|██████▍   | 1169/1813 [2:08:47<1:09:14,  6.45s/it]
+ 65%|██████▍   | 1170/1813 [2:08:53<1:07:08,  6.26s/it]
+                                                       
+{'loss': '0.8944', 'grad_norm': '1.075', 'learning_rate': '6.104e-05', 'epoch': '0.6455', 'num_input_tokens_seen': 7846688, 'train_runtime': '7733', 'train_tokens_per_second': '1015'}
+
+ 65%|██████▍   | 1170/1813 [2:08:53<1:07:08,  6.26s/it]
+ 65%|██████▍   | 1171/1813 [2:09:00<1:10:30,  6.59s/it]
+ 65%|██████▍   | 1172/1813 [2:09:07<1:11:58,  6.74s/it]
+ 65%|██████▍   | 1173/1813 [2:09:13<1:07:50,  6.36s/it]
+ 65%|██████▍   | 1174/1813 [2:09:18<1:04:01,  6.01s/it]
+ 65%|██████▍   | 1175/1813 [2:09:24<1:03:15,  5.95s/it]
+ 65%|██████▍   | 1176/1813 [2:09:30<1:04:08,  6.04s/it]
+ 65%|██████▍   | 1177/1813 [2:09:35<1:02:48,  5.93s/it]
+ 65%|██████▍   | 1178/1813 [2:09:41<1:02:35,  5.91s/it]
+ 65%|██████▌   | 1179/1813 [2:09:47<1:02:22,  5.90s/it]
+ 65%|██████▌   | 1180/1813 [2:09:54<1:04:50,  6.15s/it]
+                                                       
+{'loss': '0.868', 'grad_norm': '1.005', 'learning_rate': '5.939e-05', 'epoch': '0.651', 'num_input_tokens_seen': 7914072, 'train_runtime': '7794', 'train_tokens_per_second': '1015'}
+
+ 65%|██████▌   | 1180/1813 [2:09:54<1:04:50,  6.15s/it]
+ 65%|██████▌   | 1181/1813 [2:10:02<1:09:50,  6.63s/it]
+ 65%|██████▌   | 1182/1813 [2:10:09<1:13:21,  6.98s/it]
+ 65%|██████▌   | 1183/1813 [2:10:16<1:13:13,  6.97s/it]
+ 65%|██████▌   | 1184/1813 [2:10:23<1:10:23,  6.71s/it]
+ 65%|██████▌   | 1185/1813 [2:10:30<1:13:55,  7.06s/it]
+ 65%|██████▌   | 1186/1813 [2:10:38<1:15:06,  7.19s/it]
+ 65%|██████▌   | 1187/1813 [2:10:45<1:14:54,  7.18s/it]
+ 66%|██████▌   | 1188/1813 [2:10:51<1:09:54,  6.71s/it]
+ 66%|██████▌   | 1189/1813 [2:10:56<1:05:22,  6.29s/it]
+ 66%|██████▌   | 1190/1813 [2:11:02<1:05:35,  6.32s/it]
+                                                       
+{'loss': '0.9343', 'grad_norm': '1.198', 'learning_rate': '5.775e-05', 'epoch': '0.6566', 'num_input_tokens_seen': 7989064, 'train_runtime': '7863', 'train_tokens_per_second': '1016'}
+
+ 66%|██████▌   | 1190/1813 [2:11:02<1:05:35,  6.32s/it]
+ 66%|██████▌   | 1191/1813 [2:11:08<1:03:36,  6.14s/it]
+ 66%|██████▌   | 1192/1813 [2:11:15<1:04:41,  6.25s/it]
+ 66%|██████▌   | 1193/1813 [2:11:21<1:04:00,  6.19s/it]
+ 66%|██████▌   | 1194/1813 [2:11:27<1:05:34,  6.36s/it]
+ 66%|██████▌   | 1195/1813 [2:11:32<1:01:04,  5.93s/it]
+ 66%|██████▌   | 1196/1813 [2:11:39<1:02:19,  6.06s/it]
+ 66%|██████▌   | 1197/1813 [2:11:45<1:03:03,  6.14s/it]
+ 66%|██████▌   | 1198/1813 [2:11:51<1:02:53,  6.14s/it]
+ 66%|██████▌   | 1199/1813 [2:11:57<1:02:52,  6.14s/it]
+ 66%|██████▌   | 1200/1813 [2:12:05<1:07:12,  6.58s/it]
+                                                       
+{'loss': '0.8579', 'grad_norm': '0.9614', 'learning_rate': '5.613e-05', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352, 'train_runtime': '7925', 'train_tokens_per_second': '1016'}
+
+ 66%|██████▌   | 1200/1813 [2:12:05<1:07:12,  6.58s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.78it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.48it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.08it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.01it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.98it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:15,  7.18it/s]
+
+  3%|▎         | 26/1000 [00:04<02:37,  6.18it/s]
+
+  3%|▎         | 28/1000 [00:04<02:43,  5.94it/s]
+
+  3%|▎         | 29/1000 [00:04<02:33,  6.32it/s]
+
+  3%|▎         | 30/1000 [00:04<02:43,  5.92it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.33it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.98it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.76it/s]
+
+  4%|▍         | 42/1000 [00:06<01:47,  8.92it/s]
+
+  4%|▍         | 44/1000 [00:06<01:33, 10.19it/s]
+
+  5%|▍         | 46/1000 [00:06<01:46,  8.98it/s]
+
+  5%|▍         | 47/1000 [00:06<02:10,  7.32it/s]
+
+  5%|▍         | 49/1000 [00:06<01:49,  8.71it/s]
+
+  5%|▌         | 51/1000 [00:07<01:36,  9.86it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.92it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.75it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.31it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:52,  8.33it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.55it/s]
+
+  7%|▋         | 69/1000 [00:09<01:53,  8.23it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.98it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.85it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.21it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.03it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.96it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:26,  6.19it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:36,  9.30it/s]
+
+ 10%|█         | 103/1000 [00:14<01:57,  7.63it/s]
+
+ 10%|█         | 104/1000 [00:14<01:56,  7.72it/s]
+
+ 11%|█         | 106/1000 [00:14<01:37,  9.19it/s]
+
+ 11%|█         | 108/1000 [00:14<01:26, 10.33it/s]
+
+ 11%|█         | 110/1000 [00:14<01:20, 11.10it/s]
+
+ 11%|█         | 112/1000 [00:14<01:14, 11.95it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:11, 12.46it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.77it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.21it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.44it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.11it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.16it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.63it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.06it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.48it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.91it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.05it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.37it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.96it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.60it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.85it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.23it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.46it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.82it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.74it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.76it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.89it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.10it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.20it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.68it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.78it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.64it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.92it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.51it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.68it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.79it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.94it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.43it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.36it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.24it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.32it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.52it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:44,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.05it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.74it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:44,  6.73it/s]
+
+ 30%|███       | 301/1000 [00:40<01:39,  7.03it/s]
+
+ 30%|███       | 303/1000 [00:40<01:20,  8.68it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.03it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.10it/s]
+
+ 31%|███       | 306/1000 [00:41<01:40,  6.92it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.77it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.78it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.26it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.24it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.58it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.59it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.05it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.83it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.04it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.89it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.73it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.59it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.43it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.74it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.10it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.23it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.77it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.90it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.15it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:07,  9.16it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.12it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.63it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.36it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.78it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.22it/s]
+
+ 41%|████      | 406/1000 [00:54<01:10,  8.47it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.07it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.28it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:27,  6.62it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.89it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.33it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.91it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:16,  7.50it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:13,  7.81it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.54it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.03it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.81it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.51it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.09it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.19it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.51it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.82it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.27it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.85it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:25,  6.23it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:08,  7.77it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.53it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:21,  6.49it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.37it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.01it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50, 10.00it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.42it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.44it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.95it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.97it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.10it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.07it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.13it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.70it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<01:00,  7.40it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:02,  7.19it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.15it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.92it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.75it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.00it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.83it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.04it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.82it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:02,  6.74it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:50,  8.33it/s]
+
+ 58%|█████▊    | 583/1000 [01:19<01:00,  6.90it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:58,  7.04it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.22it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.55it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.80it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.20it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.84it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.40it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.86it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.40it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.46it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.72it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.79it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.62it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.74it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.09it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.45it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.90it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.14it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.95it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.57it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.91it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.10it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.90it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.56it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.62it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.93it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.91it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.88it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.79it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.81it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.24it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.40it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:48,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.59it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.60it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.61it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.70it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.38it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.07it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.48it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:33,  8.18it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.11it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.59it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.31it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.50it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.88it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.52it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.62it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:28,  8.66it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.24it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.24it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.72it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.66it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.80it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.80it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.52it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.92it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.15it/s]
+
+ 78%|███████▊  | 783/1000 [01:46<00:31,  6.85it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.62it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:24,  8.56it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:22,  9.20it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.30it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.75it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  8.01it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.20it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.73it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.20it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.23it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.03it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.28it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.12it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.19it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.04it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.37it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.75it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.22it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.52it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.45it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.84it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.44it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.03it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.18it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.51it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.76it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.63it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.68it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.29it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.25it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.27it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.79it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.88it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.61it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.69it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.49it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.65it/s]
+
+ 90%|█████████ | 901/1000 [02:01<00:12,  7.66it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.88it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.38it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  6.97it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:10,  8.38it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:09,  8.83it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.58it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.56it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.92it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.15it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.13it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.32it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.66it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.81it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.39it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.18it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.80it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.81it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.39it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.45it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  6.98it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.72it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.62it/s]
+                                                       
+
+                                                   
+{'eval_loss': '0.7553', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.381', 'eval_steps_per_second': '7.381', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352}
+
+ 66%|██████▌   | 1200/1813 [2:14:20<1:07:12,  6.58s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.62it/s]
+
+                                                   
+ 66%|██████▌   | 1201/1813 [2:14:26<7:58:28, 46.91s/it]
+ 66%|██████▋   | 1202/1813 [2:14:33<5:54:52, 34.85s/it]
+ 66%|██████▋   | 1203/1813 [2:14:39<4:27:24, 26.30s/it]
+ 66%|██████▋   | 1204/1813 [2:14:46<3:26:49, 20.38s/it]
+ 66%|██████▋   | 1205/1813 [2:14:52<2:44:13, 16.21s/it]
+ 67%|██████▋   | 1206/1813 [2:14:58<2:12:59, 13.15s/it]
+ 67%|██████▋   | 1207/1813 [2:15:04<1:51:57, 11.09s/it]
+ 67%|██████▋   | 1208/1813 [2:15:12<1:41:29, 10.07s/it]
+ 67%|██████▋   | 1209/1813 [2:15:19<1:31:53,  9.13s/it]
+ 67%|██████▋   | 1210/1813 [2:15:25<1:22:32,  8.21s/it]
+                                                       
+{'loss': '0.9219', 'grad_norm': '1.072', 'learning_rate': '5.451e-05', 'epoch': '0.6676', 'num_input_tokens_seen': 8125224, 'train_runtime': '8125', 'train_tokens_per_second': '1000'}
+
+ 67%|██████▋   | 1210/1813 [2:15:25<1:22:32,  8.21s/it]
+ 67%|██████▋   | 1211/1813 [2:15:32<1:18:48,  7.86s/it]
+ 67%|██████▋   | 1212/1813 [2:15:36<1:08:19,  6.82s/it]
+ 67%|██████▋   | 1213/1813 [2:15:43<1:07:11,  6.72s/it]
+ 67%|██████▋   | 1214/1813 [2:15:51<1:12:01,  7.21s/it]
+ 67%|██████▋   | 1215/1813 [2:15:57<1:08:55,  6.92s/it]
+ 67%|██████▋   | 1216/1813 [2:16:05<1:09:13,  6.96s/it]
+ 67%|██████▋   | 1217/1813 [2:16:11<1:06:25,  6.69s/it]
+ 67%|██████▋   | 1218/1813 [2:16:17<1:05:35,  6.61s/it]
+ 67%|██████▋   | 1219/1813 [2:16:23<1:02:27,  6.31s/it]
+ 67%|██████▋   | 1220/1813 [2:16:31<1:07:38,  6.84s/it]
+                                                       
+{'loss': '0.9453', 'grad_norm': '29.03', 'learning_rate': '5.292e-05', 'epoch': '0.6731', 'num_input_tokens_seen': 8197488, 'train_runtime': '8191', 'train_tokens_per_second': '1001'}
+
+ 67%|██████▋   | 1220/1813 [2:16:31<1:07:38,  6.84s/it]
+ 67%|██████▋   | 1221/1813 [2:16:36<1:03:37,  6.45s/it]
+ 67%|██████▋   | 1222/1813 [2:16:41<59:11,  6.01s/it]  
+ 67%|██████▋   | 1223/1813 [2:16:47<57:46,  5.88s/it]
+ 68%|██████▊   | 1224/1813 [2:16:52<55:16,  5.63s/it]
+ 68%|██████▊   | 1225/1813 [2:16:58<56:55,  5.81s/it]
+ 68%|██████▊   | 1226/1813 [2:17:04<57:44,  5.90s/it]
+ 68%|██████▊   | 1227/1813 [2:17:11<59:57,  6.14s/it]
+ 68%|██████▊   | 1228/1813 [2:17:16<56:42,  5.82s/it]
+ 68%|██████▊   | 1229/1813 [2:17:22<58:17,  5.99s/it]
+ 68%|██████▊   | 1230/1813 [2:17:28<57:48,  5.95s/it]
+                                                     
+{'loss': '0.7964', 'grad_norm': '0.8726', 'learning_rate': '5.133e-05', 'epoch': '0.6786', 'num_input_tokens_seen': 8257288, 'train_runtime': '8249', 'train_tokens_per_second': '1001'}
+
+ 68%|██████▊   | 1230/1813 [2:17:28<57:48,  5.95s/it]
+ 68%|██████▊   | 1231/1813 [2:17:36<1:02:00,  6.39s/it]
+ 68%|██████▊   | 1232/1813 [2:17:43<1:04:23,  6.65s/it]
+ 68%|██████▊   | 1233/1813 [2:17:49<1:02:44,  6.49s/it]
+ 68%|██████▊   | 1234/1813 [2:17:55<1:00:55,  6.31s/it]
+ 68%|██████▊   | 1235/1813 [2:18:01<1:00:32,  6.28s/it]
+ 68%|██████▊   | 1236/1813 [2:18:10<1:06:44,  6.94s/it]
+ 68%|██████▊   | 1237/1813 [2:18:16<1:04:44,  6.74s/it]
+ 68%|██████▊   | 1238/1813 [2:18:21<1:00:43,  6.34s/it]
+ 68%|██████▊   | 1239/1813 [2:18:26<57:07,  5.97s/it]  
+ 68%|██████▊   | 1240/1813 [2:18:31<54:31,  5.71s/it]
+                                                     
+{'loss': '0.7944', 'grad_norm': '0.9163', 'learning_rate': '4.976e-05', 'epoch': '0.6841', 'num_input_tokens_seen': 8327600, 'train_runtime': '8312', 'train_tokens_per_second': '1002'}
+
+ 68%|██████▊   | 1240/1813 [2:18:31<54:31,  5.71s/it]
+ 68%|██████▊   | 1241/1813 [2:18:37<53:47,  5.64s/it]
+ 69%|██████▊   | 1242/1813 [2:18:43<54:15,  5.70s/it]
+ 69%|██████▊   | 1243/1813 [2:18:49<54:50,  5.77s/it]
+ 69%|██████▊   | 1244/1813 [2:18:56<57:39,  6.08s/it]
+ 69%|██████▊   | 1245/1813 [2:19:03<1:01:01,  6.45s/it]
+ 69%|██████▊   | 1246/1813 [2:19:09<59:14,  6.27s/it]  
+ 69%|██████▉   | 1247/1813 [2:19:16<1:01:28,  6.52s/it]
+ 69%|██████▉   | 1248/1813 [2:19:23<1:02:34,  6.64s/it]
+ 69%|██████▉   | 1249/1813 [2:19:29<1:01:54,  6.59s/it]
+ 69%|██████▉   | 1250/1813 [2:19:34<57:02,  6.08s/it]  
+                                                     
+{'loss': '0.9443', 'grad_norm': '1.258', 'learning_rate': '4.821e-05', 'epoch': '0.6897', 'num_input_tokens_seen': 8394824, 'train_runtime': '8375', 'train_tokens_per_second': '1002'}
+
+ 69%|██████▉   | 1250/1813 [2:19:34<57:02,  6.08s/it]
+ 69%|██████▉   | 1251/1813 [2:19:40<56:56,  6.08s/it]
+ 69%|██████▉   | 1252/1813 [2:19:46<55:40,  5.95s/it]
+ 69%|██████▉   | 1253/1813 [2:19:52<55:18,  5.93s/it]
+ 69%|██████▉   | 1254/1813 [2:19:58<56:12,  6.03s/it]
+ 69%|██████▉   | 1255/1813 [2:20:04<55:57,  6.02s/it]
+ 69%|██████▉   | 1256/1813 [2:20:10<55:27,  5.97s/it]
+ 69%|██████▉   | 1257/1813 [2:20:15<54:08,  5.84s/it]
+ 69%|██████▉   | 1258/1813 [2:20:22<56:37,  6.12s/it]
+ 69%|██████▉   | 1259/1813 [2:20:29<57:35,  6.24s/it]
+ 69%|██████▉   | 1260/1813 [2:20:36<59:25,  6.45s/it]
+                                                     
+{'loss': '0.9117', 'grad_norm': '1.168', 'learning_rate': '4.668e-05', 'epoch': '0.6952', 'num_input_tokens_seen': 8461096, 'train_runtime': '8436', 'train_tokens_per_second': '1003'}
+
+ 69%|██████▉   | 1260/1813 [2:20:36<59:25,  6.45s/it]
+ 70%|██████▉   | 1261/1813 [2:20:44<1:03:40,  6.92s/it]
+ 70%|██████▉   | 1262/1813 [2:20:51<1:05:00,  7.08s/it]
+ 70%|██████▉   | 1263/1813 [2:20:58<1:05:49,  7.18s/it]
+ 70%|██████▉   | 1264/1813 [2:21:04<1:02:06,  6.79s/it]
+ 70%|██████▉   | 1265/1813 [2:21:10<58:03,  6.36s/it]  
+ 70%|██████▉   | 1266/1813 [2:21:17<1:01:43,  6.77s/it]
+ 70%|██████▉   | 1267/1813 [2:21:23<57:43,  6.34s/it]  
+ 70%|██████▉   | 1268/1813 [2:21:28<54:11,  5.97s/it]
+ 70%|██████▉   | 1269/1813 [2:21:33<52:37,  5.80s/it]
+ 70%|███████   | 1270/1813 [2:21:40<53:49,  5.95s/it]
+                                                     
+{'loss': '0.893', 'grad_norm': '1.064', 'learning_rate': '4.516e-05', 'epoch': '0.7007', 'num_input_tokens_seen': 8529224, 'train_runtime': '8500', 'train_tokens_per_second': '1003'}
+
+ 70%|███████   | 1270/1813 [2:21:40<53:49,  5.95s/it]
+ 70%|███████   | 1271/1813 [2:21:44<50:42,  5.61s/it]
+ 70%|███████   | 1272/1813 [2:21:51<54:34,  6.05s/it]
+ 70%|███████   | 1273/1813 [2:21:58<56:56,  6.33s/it]
+ 70%|███████   | 1274/1813 [2:22:06<59:53,  6.67s/it]
+ 70%|███████   | 1275/1813 [2:22:13<1:00:14,  6.72s/it]
+ 70%|███████   | 1276/1813 [2:22:19<57:51,  6.47s/it]  
+ 70%|███████   | 1277/1813 [2:22:26<59:37,  6.67s/it]
+ 70%|███████   | 1278/1813 [2:22:31<56:43,  6.36s/it]
+ 71%|███████   | 1279/1813 [2:22:36<52:46,  5.93s/it]
+ 71%|███████   | 1280/1813 [2:22:43<55:39,  6.27s/it]
+                                                     
+{'loss': '0.9527', 'grad_norm': '1.357', 'learning_rate': '4.365e-05', 'epoch': '0.7062', 'num_input_tokens_seen': 8596456, 'train_runtime': '8564', 'train_tokens_per_second': '1004'}
+
+ 71%|███████   | 1280/1813 [2:22:43<55:39,  6.27s/it]
+ 71%|███████   | 1281/1813 [2:22:49<53:28,  6.03s/it]
+ 71%|███████   | 1282/1813 [2:22:55<54:12,  6.12s/it]
+ 71%|███████   | 1283/1813 [2:23:02<55:11,  6.25s/it]
+ 71%|███████   | 1284/1813 [2:23:07<53:38,  6.08s/it]
+ 71%|███████   | 1285/1813 [2:23:13<51:03,  5.80s/it]
+ 71%|███████   | 1286/1813 [2:23:20<54:26,  6.20s/it]
+ 71%|███████   | 1287/1813 [2:23:26<54:24,  6.21s/it]
+ 71%|███████   | 1288/1813 [2:23:31<51:52,  5.93s/it]
+ 71%|███████   | 1289/1813 [2:23:38<53:54,  6.17s/it]
+ 71%|███████   | 1290/1813 [2:23:45<56:28,  6.48s/it]
+                                                     
+{'loss': '0.97', 'grad_norm': '0.7718', 'learning_rate': '4.217e-05', 'epoch': '0.7117', 'num_input_tokens_seen': 8662472, 'train_runtime': '8626', 'train_tokens_per_second': '1004'}
+
+ 71%|███████   | 1290/1813 [2:23:45<56:28,  6.48s/it]
+ 71%|███████   | 1291/1813 [2:23:51<55:20,  6.36s/it]
+ 71%|███████▏  | 1292/1813 [2:23:58<56:25,  6.50s/it]
+ 71%|███████▏  | 1293/1813 [2:24:05<57:33,  6.64s/it]
+ 71%|███████▏  | 1294/1813 [2:24:11<56:56,  6.58s/it]
+ 71%|███████▏  | 1295/1813 [2:24:18<55:23,  6.42s/it]
+ 71%|███████▏  | 1296/1813 [2:24:23<53:12,  6.18s/it]
+ 72%|███████▏  | 1297/1813 [2:24:29<51:16,  5.96s/it]
+ 72%|███████▏  | 1298/1813 [2:24:35<53:24,  6.22s/it]
+ 72%|███████▏  | 1299/1813 [2:24:43<55:29,  6.48s/it]
+ 72%|███████▏  | 1300/1813 [2:24:49<54:47,  6.41s/it]
+                                                     
+{'loss': '0.8623', 'grad_norm': '1.206', 'learning_rate': '4.07e-05', 'epoch': '0.7172', 'num_input_tokens_seen': 8730600, 'train_runtime': '8689', 'train_tokens_per_second': '1005'}
+
+ 72%|███████▏  | 1300/1813 [2:24:49<54:47,  6.41s/it]
+ 72%|███████▏  | 1301/1813 [2:24:55<53:12,  6.24s/it]
+ 72%|███████▏  | 1302/1813 [2:25:01<54:17,  6.38s/it]
+ 72%|███████▏  | 1303/1813 [2:25:09<57:26,  6.76s/it]
+ 72%|███████▏  | 1304/1813 [2:25:14<54:10,  6.39s/it]
+ 72%|███████▏  | 1305/1813 [2:25:22<56:25,  6.67s/it]
+ 72%|███████▏  | 1306/1813 [2:25:28<54:03,  6.40s/it]
+ 72%|███████▏  | 1307/1813 [2:25:34<54:09,  6.42s/it]
+ 72%|███████▏  | 1308/1813 [2:25:40<54:03,  6.42s/it]
+ 72%|███████▏  | 1309/1813 [2:25:47<53:43,  6.40s/it]
+ 72%|███████▏  | 1310/1813 [2:25:54<54:58,  6.56s/it]
+                                                     
+{'loss': '0.9521', 'grad_norm': '0.9934', 'learning_rate': '3.925e-05', 'epoch': '0.7228', 'num_input_tokens_seen': 8802000, 'train_runtime': '8754', 'train_tokens_per_second': '1005'}
+
+ 72%|███████▏  | 1310/1813 [2:25:54<54:58,  6.56s/it]
+ 72%|███████▏  | 1311/1813 [2:26:00<53:51,  6.44s/it]
+ 72%|███████▏  | 1312/1813 [2:26:06<51:45,  6.20s/it]
+ 72%|███████▏  | 1313/1813 [2:26:11<51:00,  6.12s/it]
+ 72%|███████▏  | 1314/1813 [2:26:18<51:12,  6.16s/it]
+ 73%|███████▎  | 1315/1813 [2:26:23<49:45,  5.99s/it]
+ 73%|███████▎  | 1316/1813 [2:26:29<49:06,  5.93s/it]
+ 73%|███████▎  | 1317/1813 [2:26:35<49:07,  5.94s/it]
+ 73%|███████▎  | 1318/1813 [2:26:40<47:47,  5.79s/it]
+ 73%|███████▎  | 1319/1813 [2:26:47<49:24,  6.00s/it]
+ 73%|███████▎  | 1320/1813 [2:26:54<50:55,  6.20s/it]
+                                                     
+{'loss': '0.9258', 'grad_norm': '0.8659', 'learning_rate': '3.782e-05', 'epoch': '0.7283', 'num_input_tokens_seen': 8866824, 'train_runtime': '8814', 'train_tokens_per_second': '1006'}
+
+ 73%|███████▎  | 1320/1813 [2:26:54<50:55,  6.20s/it]
+ 73%|███████▎  | 1321/1813 [2:27:00<52:24,  6.39s/it]
+ 73%|███████▎  | 1322/1813 [2:27:06<51:13,  6.26s/it]
+ 73%|███████▎  | 1323/1813 [2:27:13<52:04,  6.38s/it]
+ 73%|███████▎  | 1324/1813 [2:27:19<50:32,  6.20s/it]
+ 73%|███████▎  | 1325/1813 [2:27:25<49:16,  6.06s/it]
+ 73%|███████▎  | 1326/1813 [2:27:31<49:35,  6.11s/it]
+ 73%|███████▎  | 1327/1813 [2:27:39<53:16,  6.58s/it]
+ 73%|███████▎  | 1328/1813 [2:27:45<54:07,  6.70s/it]
+ 73%|███████▎  | 1329/1813 [2:27:52<52:56,  6.56s/it]
+ 73%|███████▎  | 1330/1813 [2:27:57<50:44,  6.30s/it]
+                                                     
+{'loss': '0.8752', 'grad_norm': '1.136', 'learning_rate': '3.642e-05', 'epoch': '0.7338', 'num_input_tokens_seen': 8935584, 'train_runtime': '8878', 'train_tokens_per_second': '1006'}
+
+ 73%|███████▎  | 1330/1813 [2:27:57<50:44,  6.30s/it]
+ 73%|███████▎  | 1331/1813 [2:28:03<49:34,  6.17s/it]
+ 73%|███████▎  | 1332/1813 [2:28:09<48:22,  6.03s/it]
+ 74%|███████▎  | 1333/1813 [2:28:15<47:18,  5.91s/it]
+ 74%|███████▎  | 1334/1813 [2:28:20<45:55,  5.75s/it]
+ 74%|███████▎  | 1335/1813 [2:28:26<45:14,  5.68s/it]
+ 74%|███████▎  | 1336/1813 [2:28:32<46:16,  5.82s/it]
+ 74%|███████▎  | 1337/1813 [2:28:39<50:02,  6.31s/it]
+ 74%|███████▍  | 1338/1813 [2:28:46<52:08,  6.59s/it]
+ 74%|███████▍  | 1339/1813 [2:28:51<48:19,  6.12s/it]
+ 74%|███████▍  | 1340/1813 [2:28:58<49:50,  6.32s/it]
+                                                     
+{'loss': '0.9722', 'grad_norm': '0.9533', 'learning_rate': '3.503e-05', 'epoch': '0.7393', 'num_input_tokens_seen': 9000416, 'train_runtime': '8939', 'train_tokens_per_second': '1007'}
+
+ 74%|███████▍  | 1340/1813 [2:28:58<49:50,  6.32s/it]
+ 74%|███████▍  | 1341/1813 [2:29:05<50:12,  6.38s/it]
+ 74%|███████▍  | 1342/1813 [2:29:10<48:14,  6.15s/it]
+ 74%|███████▍  | 1343/1813 [2:29:16<47:09,  6.02s/it]
+ 74%|███████▍  | 1344/1813 [2:29:22<46:38,  5.97s/it]
+ 74%|███████▍  | 1345/1813 [2:29:30<50:31,  6.48s/it]
+ 74%|███████▍  | 1346/1813 [2:29:36<50:41,  6.51s/it]
+ 74%|███████▍  | 1347/1813 [2:29:42<48:41,  6.27s/it]
+ 74%|███████▍  | 1348/1813 [2:29:48<48:29,  6.26s/it]
+ 74%|███████▍  | 1349/1813 [2:29:55<49:06,  6.35s/it]
+ 74%|███████▍  | 1350/1813 [2:30:00<47:22,  6.14s/it]
+                                                     
+{'loss': '0.9168', 'grad_norm': '1.093', 'learning_rate': '3.366e-05', 'epoch': '0.7448', 'num_input_tokens_seen': 9066864, 'train_runtime': '9001', 'train_tokens_per_second': '1007'}
+
+ 74%|███████▍  | 1350/1813 [2:30:00<47:22,  6.14s/it]
+ 75%|███████▍  | 1351/1813 [2:30:08<51:15,  6.66s/it]
+ 75%|███████▍  | 1352/1813 [2:30:14<50:12,  6.54s/it]
+ 75%|███████▍  | 1353/1813 [2:30:21<50:57,  6.65s/it]
+ 75%|███████▍  | 1354/1813 [2:30:28<51:25,  6.72s/it]
+ 75%|███████▍  | 1355/1813 [2:30:37<55:49,  7.31s/it]
+ 75%|███████▍  | 1356/1813 [2:30:43<52:26,  6.88s/it]
+ 75%|███████▍  | 1357/1813 [2:30:49<51:00,  6.71s/it]
+ 75%|███████▍  | 1358/1813 [2:30:56<50:45,  6.69s/it]
+ 75%|███████▍  | 1359/1813 [2:31:02<48:55,  6.47s/it]
+ 75%|███████▌  | 1360/1813 [2:31:08<47:42,  6.32s/it]
+                                                     
+{'loss': '0.8724', 'grad_norm': '1.199', 'learning_rate': '3.231e-05', 'epoch': '0.7503', 'num_input_tokens_seen': 9140520, 'train_runtime': '9068', 'train_tokens_per_second': '1008'}
+
+ 75%|███████▌  | 1360/1813 [2:31:08<47:42,  6.32s/it]
+ 75%|███████▌  | 1361/1813 [2:31:14<48:21,  6.42s/it]
+ 75%|███████▌  | 1362/1813 [2:31:21<47:58,  6.38s/it]
+ 75%|███████▌  | 1363/1813 [2:31:28<50:06,  6.68s/it]
+ 75%|███████▌  | 1364/1813 [2:31:34<48:22,  6.46s/it]
+ 75%|███████▌  | 1365/1813 [2:31:41<50:30,  6.77s/it]
+ 75%|███████▌  | 1366/1813 [2:31:47<48:38,  6.53s/it]
+ 75%|███████▌  | 1367/1813 [2:31:54<48:36,  6.54s/it]
+ 75%|███████▌  | 1368/1813 [2:31:59<45:08,  6.09s/it]
+ 76%|███████▌  | 1369/1813 [2:32:07<49:31,  6.69s/it]
+ 76%|███████▌  | 1370/1813 [2:32:13<48:12,  6.53s/it]
+                                                     
+{'loss': '0.8564', 'grad_norm': '0.954', 'learning_rate': '3.099e-05', 'epoch': '0.7559', 'num_input_tokens_seen': 9213048, 'train_runtime': '9134', 'train_tokens_per_second': '1009'}
+
+ 76%|███████▌  | 1370/1813 [2:32:13<48:12,  6.53s/it]
+ 76%|███████▌  | 1371/1813 [2:32:21<51:21,  6.97s/it]
+ 76%|███████▌  | 1372/1813 [2:32:26<47:20,  6.44s/it]
+ 76%|███████▌  | 1373/1813 [2:32:32<46:22,  6.32s/it]
+ 76%|███████▌  | 1374/1813 [2:32:38<45:18,  6.19s/it]
+ 76%|███████▌  | 1375/1813 [2:32:44<44:03,  6.04s/it]
+ 76%|███████▌  | 1376/1813 [2:32:49<42:27,  5.83s/it]
+ 76%|███████▌  | 1377/1813 [2:32:56<43:08,  5.94s/it]
+ 76%|███████▌  | 1378/1813 [2:33:01<42:06,  5.81s/it]
+ 76%|███████▌  | 1379/1813 [2:33:07<41:57,  5.80s/it]
+ 76%|███████▌  | 1380/1813 [2:33:14<44:16,  6.13s/it]
+                                                     
+{'loss': '0.8474', 'grad_norm': '1.132', 'learning_rate': '2.969e-05', 'epoch': '0.7614', 'num_input_tokens_seen': 9279088, 'train_runtime': '9194', 'train_tokens_per_second': '1009'}
+
+ 76%|███████▌  | 1380/1813 [2:33:14<44:16,  6.13s/it]
+ 76%|███████▌  | 1381/1813 [2:33:21<45:52,  6.37s/it]
+ 76%|███████▌  | 1382/1813 [2:33:26<43:32,  6.06s/it]
+ 76%|███████▋  | 1383/1813 [2:33:32<43:00,  6.00s/it]
+ 76%|███████▋  | 1384/1813 [2:33:38<42:09,  5.90s/it]
+ 76%|███████▋  | 1385/1813 [2:33:44<43:46,  6.14s/it]
+ 76%|███████▋  | 1386/1813 [2:33:51<44:15,  6.22s/it]
+ 77%|███████▋  | 1387/1813 [2:33:58<45:53,  6.46s/it]
+ 77%|███████▋  | 1388/1813 [2:34:03<43:25,  6.13s/it]
+ 77%|███████▋  | 1389/1813 [2:34:07<39:25,  5.58s/it]
+ 77%|███████▋  | 1390/1813 [2:34:14<41:44,  5.92s/it]
+                                                     
+{'loss': '0.9373', 'grad_norm': '0.9963', 'learning_rate': '2.841e-05', 'epoch': '0.7669', 'num_input_tokens_seen': 9344880, 'train_runtime': '9255', 'train_tokens_per_second': '1010'}
+
+ 77%|███████▋  | 1390/1813 [2:34:14<41:44,  5.92s/it]
+ 77%|███████▋  | 1391/1813 [2:34:19<40:08,  5.71s/it]
+ 77%|███████▋  | 1392/1813 [2:34:26<42:15,  6.02s/it]
+ 77%|███████▋  | 1393/1813 [2:34:32<41:02,  5.86s/it]
+ 77%|███████▋  | 1394/1813 [2:34:38<42:32,  6.09s/it]
+ 77%|███████▋  | 1395/1813 [2:34:44<42:46,  6.14s/it]
+ 77%|███████▋  | 1396/1813 [2:34:50<42:23,  6.10s/it]
+ 77%|███████▋  | 1397/1813 [2:34:57<43:42,  6.30s/it]
+ 77%|███████▋  | 1398/1813 [2:35:02<40:23,  5.84s/it]
+ 77%|███████▋  | 1399/1813 [2:35:07<38:12,  5.54s/it]
+ 77%|███████▋  | 1400/1813 [2:35:13<40:28,  5.88s/it]
+                                                     
+{'loss': '0.8791', 'grad_norm': '0.8832', 'learning_rate': '2.715e-05', 'epoch': '0.7724', 'num_input_tokens_seen': 9408280, 'train_runtime': '9314', 'train_tokens_per_second': '1010'}
+
+ 77%|███████▋  | 1400/1813 [2:35:13<40:28,  5.88s/it]
+ 77%|███████▋  | 1401/1813 [2:35:19<39:17,  5.72s/it]
+ 77%|███████▋  | 1402/1813 [2:35:25<39:50,  5.82s/it]
+ 77%|███████▋  | 1403/1813 [2:35:31<41:01,  6.00s/it]
+ 77%|███████▋  | 1404/1813 [2:35:38<41:45,  6.13s/it]
+ 77%|███████▋  | 1405/1813 [2:35:43<39:16,  5.77s/it]
+ 78%|███████▊  | 1406/1813 [2:35:48<39:14,  5.79s/it]
+ 78%|███████▊  | 1407/1813 [2:35:55<40:55,  6.05s/it]
+ 78%|███████▊  | 1408/1813 [2:36:00<39:23,  5.83s/it]
+ 78%|███████▊  | 1409/1813 [2:36:05<37:16,  5.53s/it]
+ 78%|███████▊  | 1410/1813 [2:36:11<37:25,  5.57s/it]
+                                                     
+{'loss': '0.924', 'grad_norm': '1.033', 'learning_rate': '2.592e-05', 'epoch': '0.7779', 'num_input_tokens_seen': 9467408, 'train_runtime': '9371', 'train_tokens_per_second': '1010'}
+
+ 78%|███████▊  | 1410/1813 [2:36:11<37:25,  5.57s/it]
+ 78%|███████▊  | 1411/1813 [2:36:16<37:07,  5.54s/it]
+ 78%|███████▊  | 1412/1813 [2:36:23<38:54,  5.82s/it]
+ 78%|███████▊  | 1413/1813 [2:36:28<38:20,  5.75s/it]
+ 78%|███████▊  | 1414/1813 [2:36:35<39:07,  5.88s/it]
+ 78%|███████▊  | 1415/1813 [2:36:42<41:19,  6.23s/it]
+ 78%|███████▊  | 1416/1813 [2:36:48<41:01,  6.20s/it]
+ 78%|███████▊  | 1417/1813 [2:36:53<39:12,  5.94s/it]
+ 78%|███████▊  | 1418/1813 [2:36:59<38:47,  5.89s/it]
+ 78%|███████▊  | 1419/1813 [2:37:08<44:01,  6.70s/it]
+ 78%|███████▊  | 1420/1813 [2:37:12<39:27,  6.02s/it]
+                                                     
+{'loss': '0.875', 'grad_norm': '1.11', 'learning_rate': '2.471e-05', 'epoch': '0.7834', 'num_input_tokens_seen': 9532720, 'train_runtime': '9432', 'train_tokens_per_second': '1011'}
+
+ 78%|███████▊  | 1420/1813 [2:37:12<39:27,  6.02s/it]
+ 78%|███████▊  | 1421/1813 [2:37:19<40:52,  6.26s/it]
+ 78%|███████▊  | 1422/1813 [2:37:24<39:09,  6.01s/it]
+ 78%|███████▊  | 1423/1813 [2:37:30<38:04,  5.86s/it]
+ 79%|███████▊  | 1424/1813 [2:37:37<40:43,  6.28s/it]
+ 79%|███████▊  | 1425/1813 [2:37:42<37:20,  5.77s/it]
+ 79%|███████▊  | 1426/1813 [2:37:50<42:36,  6.61s/it]
+ 79%|███████▊  | 1427/1813 [2:37:55<40:06,  6.23s/it]
+ 79%|███████▉  | 1428/1813 [2:38:02<40:35,  6.33s/it]
+ 79%|███████▉  | 1429/1813 [2:38:08<40:08,  6.27s/it]
+ 79%|███████▉  | 1430/1813 [2:38:15<41:07,  6.44s/it]
+                                                     
+{'loss': '0.8403', 'grad_norm': '0.9655', 'learning_rate': '2.353e-05', 'epoch': '0.789', 'num_input_tokens_seen': 9602112, 'train_runtime': '9496', 'train_tokens_per_second': '1011'}
+
+ 79%|███████▉  | 1430/1813 [2:38:15<41:07,  6.44s/it]
+ 79%|███████▉  | 1431/1813 [2:38:21<40:08,  6.30s/it]
+ 79%|███████▉  | 1432/1813 [2:38:27<38:59,  6.14s/it]
+ 79%|███████▉  | 1433/1813 [2:38:33<38:55,  6.15s/it]
+ 79%|███████▉  | 1434/1813 [2:38:39<39:00,  6.18s/it]
+ 79%|███████▉  | 1435/1813 [2:38:45<37:36,  5.97s/it]
+ 79%|███████▉  | 1436/1813 [2:38:51<39:03,  6.22s/it]
+ 79%|███████▉  | 1437/1813 [2:38:57<37:56,  6.06s/it]
+ 79%|███████▉  | 1438/1813 [2:39:04<40:05,  6.41s/it]
+ 79%|███████▉  | 1439/1813 [2:39:10<38:06,  6.11s/it]
+ 79%|███████▉  | 1440/1813 [2:39:15<35:53,  5.77s/it]
+                                                     
+{'loss': '0.8569', 'grad_norm': '0.9602', 'learning_rate': '2.237e-05', 'epoch': '0.7945', 'num_input_tokens_seen': 9667112, 'train_runtime': '9555', 'train_tokens_per_second': '1012'}
+
+ 79%|███████▉  | 1440/1813 [2:39:15<35:53,  5.77s/it]
+ 79%|███████▉  | 1441/1813 [2:39:21<36:53,  5.95s/it]
+ 80%|███████▉  | 1442/1813 [2:39:26<35:24,  5.73s/it]
+ 80%|███████▉  | 1443/1813 [2:39:31<33:54,  5.50s/it]
+ 80%|███████▉  | 1444/1813 [2:39:38<36:06,  5.87s/it]
+ 80%|███████▉  | 1445/1813 [2:39:44<35:31,  5.79s/it]
+ 80%|███████▉  | 1446/1813 [2:39:49<34:45,  5.68s/it]
+ 80%|███████▉  | 1447/1813 [2:39:57<39:13,  6.43s/it]
+ 80%|███████▉  | 1448/1813 [2:40:04<40:31,  6.66s/it]
+ 80%|███████▉  | 1449/1813 [2:40:11<40:13,  6.63s/it]
+ 80%|███████▉  | 1450/1813 [2:40:18<40:02,  6.62s/it]
+                                                     
+{'loss': '0.9396', 'grad_norm': '1.029', 'learning_rate': '2.124e-05', 'epoch': '0.8', 'num_input_tokens_seen': 9735792, 'train_runtime': '9618', 'train_tokens_per_second': '1012'}
+
+ 80%|███████▉  | 1450/1813 [2:40:18<40:02,  6.62s/it]
+ 80%|████████  | 1451/1813 [2:40:25<40:34,  6.73s/it]
+ 80%|████████  | 1452/1813 [2:40:29<36:46,  6.11s/it]
+ 80%|████████  | 1453/1813 [2:40:38<40:45,  6.79s/it]
+ 80%|████████  | 1454/1813 [2:40:43<38:33,  6.44s/it]
+ 80%|████████  | 1455/1813 [2:40:49<38:03,  6.38s/it]
+ 80%|████████  | 1456/1813 [2:40:57<39:15,  6.60s/it]
+ 80%|████████  | 1457/1813 [2:41:00<34:04,  5.74s/it]
+ 80%|████████  | 1458/1813 [2:41:06<33:57,  5.74s/it]
+ 80%|████████  | 1459/1813 [2:41:13<36:23,  6.17s/it]
+ 81%|████████  | 1460/1813 [2:41:20<38:04,  6.47s/it]
+                                                     
+{'loss': '0.8914', 'grad_norm': '1.027', 'learning_rate': '2.013e-05', 'epoch': '0.8055', 'num_input_tokens_seen': 9802968, 'train_runtime': '9681', 'train_tokens_per_second': '1013'}
+
+ 81%|████████  | 1460/1813 [2:41:20<38:04,  6.47s/it]
+ 81%|████████  | 1461/1813 [2:41:26<36:52,  6.29s/it]
+ 81%|████████  | 1462/1813 [2:41:32<35:03,  5.99s/it]
+ 81%|████████  | 1463/1813 [2:41:39<37:23,  6.41s/it]
+ 81%|████████  | 1464/1813 [2:41:46<37:36,  6.47s/it]
+ 81%|████████  | 1465/1813 [2:41:54<40:34,  7.00s/it]
+ 81%|████████  | 1466/1813 [2:41:59<37:42,  6.52s/it]
+ 81%|████████  | 1467/1813 [2:42:07<39:00,  6.76s/it]
+ 81%|████████  | 1468/1813 [2:42:13<39:00,  6.78s/it]
+ 81%|████████  | 1469/1813 [2:42:22<42:20,  7.39s/it]
+ 81%|████████  | 1470/1813 [2:42:29<41:09,  7.20s/it]
+                                                     
+{'loss': '0.9034', 'grad_norm': '0.9785', 'learning_rate': '1.905e-05', 'epoch': '0.811', 'num_input_tokens_seen': 9879232, 'train_runtime': '9749', 'train_tokens_per_second': '1013'}
+
+ 81%|████████  | 1470/1813 [2:42:29<41:09,  7.20s/it]
+ 81%|████████  | 1471/1813 [2:42:35<39:47,  6.98s/it]
+ 81%|████████  | 1472/1813 [2:42:42<39:23,  6.93s/it]
+ 81%|████████  | 1473/1813 [2:42:48<37:22,  6.59s/it]
+ 81%|████████▏ | 1474/1813 [2:42:55<37:13,  6.59s/it]
+ 81%|████████▏ | 1475/1813 [2:43:01<36:24,  6.46s/it]
+ 81%|████████▏ | 1476/1813 [2:43:07<35:54,  6.39s/it]
+ 81%|████████▏ | 1477/1813 [2:43:13<35:37,  6.36s/it]
+ 82%|████████▏ | 1478/1813 [2:43:19<34:03,  6.10s/it]
+ 82%|████████▏ | 1479/1813 [2:43:25<33:55,  6.09s/it]
+ 82%|████████▏ | 1480/1813 [2:43:32<35:25,  6.38s/it]
+                                                     
+{'loss': '0.8747', 'grad_norm': '0.7843', 'learning_rate': '1.799e-05', 'epoch': '0.8166', 'num_input_tokens_seen': 9948032, 'train_runtime': '9812', 'train_tokens_per_second': '1014'}
+
+ 82%|████████▏ | 1480/1813 [2:43:32<35:25,  6.38s/it]
+ 82%|████████▏ | 1481/1813 [2:43:39<35:52,  6.48s/it]
+ 82%|████████▏ | 1482/1813 [2:43:44<34:27,  6.25s/it]
+ 82%|████████▏ | 1483/1813 [2:43:51<35:12,  6.40s/it]
+ 82%|████████▏ | 1484/1813 [2:43:58<35:06,  6.40s/it]
+ 82%|████████▏ | 1485/1813 [2:44:04<35:35,  6.51s/it]
+ 82%|████████▏ | 1486/1813 [2:44:09<33:21,  6.12s/it]
+ 82%|████████▏ | 1487/1813 [2:44:17<36:04,  6.64s/it]
+ 82%|████████▏ | 1488/1813 [2:44:24<35:22,  6.53s/it]
+ 82%|████████▏ | 1489/1813 [2:44:31<36:38,  6.79s/it]
+ 82%|████████▏ | 1490/1813 [2:44:38<36:29,  6.78s/it]
+                                                     
+{'loss': '0.8929', 'grad_norm': '1.053', 'learning_rate': '1.697e-05', 'epoch': '0.8221', 'num_input_tokens_seen': 10021400, 'train_runtime': '9878', 'train_tokens_per_second': '1014'}
+
+ 82%|████████▏ | 1490/1813 [2:44:38<36:29,  6.78s/it]
+ 82%|████████▏ | 1491/1813 [2:44:44<35:02,  6.53s/it]
+ 82%|████████▏ | 1492/1813 [2:44:52<37:47,  7.06s/it]
+ 82%|████████▏ | 1493/1813 [2:44:58<36:12,  6.79s/it]
+ 82%|████████▏ | 1494/1813 [2:45:04<34:45,  6.54s/it]
+ 82%|████████▏ | 1495/1813 [2:45:12<36:57,  6.97s/it]
+ 83%|████████▎ | 1496/1813 [2:45:17<34:13,  6.48s/it]
+ 83%|████████▎ | 1497/1813 [2:45:23<31:59,  6.07s/it]
+ 83%|████████▎ | 1498/1813 [2:45:29<32:21,  6.16s/it]
+ 83%|████████▎ | 1499/1813 [2:45:34<29:58,  5.73s/it]
+ 83%|████████▎ | 1500/1813 [2:45:41<32:54,  6.31s/it]
+                                                     
+{'loss': '0.8516', 'grad_norm': '0.8604', 'learning_rate': '1.596e-05', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904, 'train_runtime': '9942', 'train_tokens_per_second': '1015'}
+
+ 83%|████████▎ | 1500/1813 [2:45:41<32:54,  6.31s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.76it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.21it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.32it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.09it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.01it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.83it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.68it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.11it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.76it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.12it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.35it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.37it/s]
+
+  5%|▍         | 49/1000 [00:06<01:49,  8.65it/s]
+
+  5%|▌         | 51/1000 [00:07<01:36,  9.84it/s]
+
+  5%|▌         | 53/1000 [00:07<01:27, 10.88it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.70it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.27it/s]
+
+  6%|▌         | 59/1000 [00:08<01:55,  8.17it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.30it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.72it/s]
+
+  6%|▋         | 65/1000 [00:08<01:52,  8.34it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.56it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.11it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.86it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.19it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.77it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.34it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.60it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.30it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.44it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.21it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.05it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.53it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.82it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.20it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.09it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.87it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.94it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.02it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.88it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.06it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.42it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.48it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.91it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.03it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.36it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.96it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.59it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.85it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.23it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.56it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:22,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.73it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.87it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.10it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.76it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.20it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.67it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.76it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.64it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.93it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.66it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.16it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.51it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.79it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.56it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.36it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.03it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.93it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.98it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.36it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.48it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.52it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.05it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.72it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.17it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.83it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.11it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.75it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.07it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.13it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.94it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.78it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.80it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.28it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.58it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.05it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.86it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.59it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.21it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.42it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.63it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.58it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.41it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.17it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:58,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.59it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.51it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.69it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.06it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.20it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:54,  5.47it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.17it/s]
+
+ 38%|███▊      | 379/1000 [00:51<01:16,  8.14it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.16it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.54it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.06it/s]
+
+ 39%|███▊      | 387/1000 [00:51<01:21,  7.56it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.30it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.83it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.48it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  8.99it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.53it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.84it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.50it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.05it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.26it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.44it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.49it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.68it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.82it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.50it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.00it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.78it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.50it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.49it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.79it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.24it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.81it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.30it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.31it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.79it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.55it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.42it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.97it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.10it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.13it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.69it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.35it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.91it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.87it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.43it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.33it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.82it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.59it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.40it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.82it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.62it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.11it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.04it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.22it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.44it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.73it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.08it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.96it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.19it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.91it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.57it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.97it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.92it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.88it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.85it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.15it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.64it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.51it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.69it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.37it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.07it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.14it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.12it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.61it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.30it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.88it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.76it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.29it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.35it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.60it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.44it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.25it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.40it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.70it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.46it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.77it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.61it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  8.99it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.16it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.96it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.19it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.20it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.31it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.19it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.03it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.00it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.44it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.15it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.76it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.24it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.51it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.64it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.72it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.25it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.27it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.62it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.95it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.37it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.74it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.83it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.69it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.85it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.05it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.32it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.76it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.66it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.13it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.31it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.06it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.37it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.69it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.81it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.40it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.39it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.39it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.39it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.30it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:04,  7.97it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.39it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  7.98it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.16it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.77it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.80it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.05it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.37it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.97it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7438', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.387', 'eval_steps_per_second': '7.387', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904}
+
+ 83%|████████▎ | 1500/1813 [2:47:57<32:54,  6.31s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+
+                                                   
+ 83%|████████▎ | 1501/1813 [2:48:03<4:04:07, 46.95s/it]
+ 83%|████████▎ | 1502/1813 [2:48:07<2:57:10, 34.18s/it]
+ 83%|████████▎ | 1503/1813 [2:48:14<2:13:15, 25.79s/it]
+ 83%|████████▎ | 1504/1813 [2:48:19<1:40:43, 19.56s/it]
+ 83%|████████▎ | 1505/1813 [2:48:24<1:18:25, 15.28s/it]
+ 83%|████████▎ | 1506/1813 [2:48:31<1:05:19, 12.77s/it]
+ 83%|████████▎ | 1507/1813 [2:48:36<53:05, 10.41s/it]  
+ 83%|████████▎ | 1508/1813 [2:48:43<47:35,  9.36s/it]
+ 83%|████████▎ | 1509/1813 [2:48:48<41:52,  8.27s/it]
+ 83%|████████▎ | 1510/1813 [2:48:54<38:00,  7.53s/it]
+                                                     
+{'loss': '0.8227', 'grad_norm': '1.188', 'learning_rate': '1.499e-05', 'epoch': '0.8331', 'num_input_tokens_seen': 10151928, 'train_runtime': '1.013e+04', 'train_tokens_per_second': '1002'}
+
+ 83%|████████▎ | 1510/1813 [2:48:54<38:00,  7.53s/it]
+ 83%|████████▎ | 1511/1813 [2:49:00<35:58,  7.15s/it]
+ 83%|████████▎ | 1512/1813 [2:49:06<33:26,  6.67s/it]
+ 83%|████████▎ | 1513/1813 [2:49:11<31:28,  6.29s/it]
+ 84%|████████▎ | 1514/1813 [2:49:18<31:54,  6.40s/it]
+ 84%|████████▎ | 1515/1813 [2:49:26<33:20,  6.71s/it]
+ 84%|████████▎ | 1516/1813 [2:49:32<33:29,  6.76s/it]
+ 84%|████████▎ | 1517/1813 [2:49:39<33:02,  6.70s/it]
+ 84%|████████▎ | 1518/1813 [2:49:47<34:09,  6.95s/it]
+ 84%|████████▍ | 1519/1813 [2:49:54<34:17,  7.00s/it]
+ 84%|████████▍ | 1520/1813 [2:50:00<32:59,  6.75s/it]
+                                                     
+{'loss': '0.8583', 'grad_norm': '1.127', 'learning_rate': '1.405e-05', 'epoch': '0.8386', 'num_input_tokens_seen': 10225024, 'train_runtime': '1.02e+04', 'train_tokens_per_second': '1002'}
+
+ 84%|████████▍ | 1520/1813 [2:50:00<32:59,  6.75s/it]
+ 84%|████████▍ | 1521/1813 [2:50:07<33:01,  6.79s/it]
+ 84%|████████▍ | 1522/1813 [2:50:13<32:05,  6.62s/it]
+ 84%|████████▍ | 1523/1813 [2:50:20<32:58,  6.82s/it]
+ 84%|████████▍ | 1524/1813 [2:50:27<32:09,  6.68s/it]
+ 84%|████████▍ | 1525/1813 [2:50:31<29:17,  6.10s/it]
+ 84%|████████▍ | 1526/1813 [2:50:38<29:40,  6.21s/it]
+ 84%|████████▍ | 1527/1813 [2:50:44<30:06,  6.32s/it]
+ 84%|████████▍ | 1528/1813 [2:50:51<29:59,  6.32s/it]
+ 84%|████████▍ | 1529/1813 [2:50:58<30:47,  6.51s/it]
+ 84%|████████▍ | 1530/1813 [2:51:03<29:23,  6.23s/it]
+                                                     
+{'loss': '0.9076', 'grad_norm': '0.9086', 'learning_rate': '1.313e-05', 'epoch': '0.8441', 'num_input_tokens_seen': 10294368, 'train_runtime': '1.026e+04', 'train_tokens_per_second': '1003'}
+
+ 84%|████████▍ | 1530/1813 [2:51:03<29:23,  6.23s/it]
+ 84%|████████▍ | 1531/1813 [2:51:08<27:43,  5.90s/it]
+ 85%|████████▍ | 1532/1813 [2:51:15<29:06,  6.22s/it]
+ 85%|████████▍ | 1533/1813 [2:51:20<27:18,  5.85s/it]
+ 85%|████████▍ | 1534/1813 [2:51:28<30:18,  6.52s/it]
+ 85%|████████▍ | 1535/1813 [2:51:34<28:36,  6.18s/it]
+ 85%|████████▍ | 1536/1813 [2:51:41<30:20,  6.57s/it]
+ 85%|████████▍ | 1537/1813 [2:51:48<30:27,  6.62s/it]
+ 85%|████████▍ | 1538/1813 [2:51:53<27:53,  6.09s/it]
+ 85%|████████▍ | 1539/1813 [2:51:59<27:33,  6.04s/it]
+ 85%|████████▍ | 1540/1813 [2:52:04<26:58,  5.93s/it]
+                                                     
+{'loss': '0.9004', 'grad_norm': '1.266', 'learning_rate': '1.224e-05', 'epoch': '0.8497', 'num_input_tokens_seen': 10359416, 'train_runtime': '1.032e+04', 'train_tokens_per_second': '1003'}
+
+ 85%|████████▍ | 1540/1813 [2:52:04<26:58,  5.93s/it]
+ 85%|████████▍ | 1541/1813 [2:52:11<28:07,  6.20s/it]
+ 85%|████████▌ | 1542/1813 [2:52:17<28:03,  6.21s/it]
+ 85%|████████▌ | 1543/1813 [2:52:22<26:08,  5.81s/it]
+ 85%|████████▌ | 1544/1813 [2:52:27<24:47,  5.53s/it]
+ 85%|████████▌ | 1545/1813 [2:52:35<27:42,  6.20s/it]
+ 85%|████████▌ | 1546/1813 [2:52:41<27:15,  6.12s/it]
+ 85%|████████▌ | 1547/1813 [2:52:46<25:53,  5.84s/it]
+ 85%|████████▌ | 1548/1813 [2:52:54<28:50,  6.53s/it]
+ 85%|████████▌ | 1549/1813 [2:52:59<27:04,  6.15s/it]
+ 85%|████████▌ | 1550/1813 [2:53:07<28:29,  6.50s/it]
+                                                     
+{'loss': '0.9695', 'grad_norm': '0.9476', 'learning_rate': '1.138e-05', 'epoch': '0.8552', 'num_input_tokens_seen': 10427368, 'train_runtime': '1.039e+04', 'train_tokens_per_second': '1004'}
+
+ 85%|████████▌ | 1550/1813 [2:53:07<28:29,  6.50s/it]
+ 86%|████████▌ | 1551/1813 [2:53:14<29:52,  6.84s/it]
+ 86%|████████▌ | 1552/1813 [2:53:20<28:35,  6.57s/it]
+ 86%|████████▌ | 1553/1813 [2:53:26<27:25,  6.33s/it]
+ 86%|████████▌ | 1554/1813 [2:53:33<27:40,  6.41s/it]
+ 86%|████████▌ | 1555/1813 [2:53:39<27:39,  6.43s/it]
+ 86%|████████▌ | 1556/1813 [2:53:45<27:05,  6.32s/it]
+ 86%|████████▌ | 1557/1813 [2:53:53<28:41,  6.72s/it]
+ 86%|████████▌ | 1558/1813 [2:53:59<27:30,  6.47s/it]
+ 86%|████████▌ | 1559/1813 [2:54:05<27:07,  6.41s/it]
+ 86%|████████▌ | 1560/1813 [2:54:10<25:24,  6.02s/it]
+                                                     
+{'loss': '0.9766', 'grad_norm': '0.8517', 'learning_rate': '1.055e-05', 'epoch': '0.8607', 'num_input_tokens_seen': 10497472, 'train_runtime': '1.045e+04', 'train_tokens_per_second': '1004'}
+
+ 86%|████████▌ | 1560/1813 [2:54:10<25:24,  6.02s/it]
+ 86%|████████▌ | 1561/1813 [2:54:17<26:24,  6.29s/it]
+ 86%|████████▌ | 1562/1813 [2:54:24<26:28,  6.33s/it]
+ 86%|████████▌ | 1563/1813 [2:54:30<26:09,  6.28s/it]
+ 86%|████████▋ | 1564/1813 [2:54:36<26:01,  6.27s/it]
+ 86%|████████▋ | 1565/1813 [2:54:40<23:33,  5.70s/it]
+ 86%|████████▋ | 1566/1813 [2:54:46<23:42,  5.76s/it]
+ 86%|████████▋ | 1567/1813 [2:54:52<23:54,  5.83s/it]
+ 86%|████████▋ | 1568/1813 [2:54:57<22:25,  5.49s/it]
+ 87%|████████▋ | 1569/1813 [2:55:04<24:42,  6.08s/it]
+ 87%|████████▋ | 1570/1813 [2:55:12<26:04,  6.44s/it]
+                                                     
+{'loss': '0.7988', 'grad_norm': '0.9651', 'learning_rate': '9.753e-06', 'epoch': '0.8662', 'num_input_tokens_seen': 10564336, 'train_runtime': '1.051e+04', 'train_tokens_per_second': '1005'}
+
+ 87%|████████▋ | 1570/1813 [2:55:12<26:04,  6.44s/it]
+ 87%|████████▋ | 1571/1813 [2:55:17<24:26,  6.06s/it]
+ 87%|████████▋ | 1572/1813 [2:55:22<23:50,  5.94s/it]
+ 87%|████████▋ | 1573/1813 [2:55:28<23:40,  5.92s/it]
+ 87%|████████▋ | 1574/1813 [2:55:35<24:06,  6.05s/it]
+ 87%|████████▋ | 1575/1813 [2:55:40<23:24,  5.90s/it]
+ 87%|████████▋ | 1576/1813 [2:55:48<24:59,  6.33s/it]
+ 87%|████████▋ | 1577/1813 [2:55:54<25:23,  6.46s/it]
+ 87%|████████▋ | 1578/1813 [2:56:01<24:59,  6.38s/it]
+ 87%|████████▋ | 1579/1813 [2:56:07<24:34,  6.30s/it]
+ 87%|████████▋ | 1580/1813 [2:56:12<22:46,  5.87s/it]
+                                                     
+{'loss': '0.9316', 'grad_norm': '1.139', 'learning_rate': '8.983e-06', 'epoch': '0.8717', 'num_input_tokens_seen': 10628400, 'train_runtime': '1.057e+04', 'train_tokens_per_second': '1005'}
+
+ 87%|████████▋ | 1580/1813 [2:56:12<22:46,  5.87s/it]
+ 87%|████████▋ | 1581/1813 [2:56:17<22:32,  5.83s/it]
+ 87%|████████▋ | 1582/1813 [2:56:21<20:32,  5.34s/it]
+ 87%|████████▋ | 1583/1813 [2:56:28<21:27,  5.60s/it]
+ 87%|████████▋ | 1584/1813 [2:56:34<22:08,  5.80s/it]
+ 87%|████████▋ | 1585/1813 [2:56:41<23:37,  6.22s/it]
+ 87%|████████▋ | 1586/1813 [2:56:48<24:47,  6.55s/it]
+ 88%|████████▊ | 1587/1813 [2:56:56<25:42,  6.82s/it]
+ 88%|████████▊ | 1588/1813 [2:57:02<24:15,  6.47s/it]
+ 88%|████████▊ | 1589/1813 [2:57:07<22:28,  6.02s/it]
+ 88%|████████▊ | 1590/1813 [2:57:13<22:59,  6.18s/it]
+                                                     
+{'loss': '0.8679', 'grad_norm': '1.076', 'learning_rate': '8.243e-06', 'epoch': '0.8772', 'num_input_tokens_seen': 10695288, 'train_runtime': '1.063e+04', 'train_tokens_per_second': '1006'}
+
+ 88%|████████▊ | 1590/1813 [2:57:13<22:59,  6.18s/it]
+ 88%|████████▊ | 1591/1813 [2:57:19<22:42,  6.14s/it]
+ 88%|████████▊ | 1592/1813 [2:57:25<22:50,  6.20s/it]
+ 88%|████████▊ | 1593/1813 [2:57:32<23:11,  6.32s/it]
+ 88%|████████▊ | 1594/1813 [2:57:39<23:25,  6.42s/it]
+ 88%|████████▊ | 1595/1813 [2:57:46<24:15,  6.68s/it]
+ 88%|████████▊ | 1596/1813 [2:57:55<26:24,  7.30s/it]
+ 88%|████████▊ | 1597/1813 [2:58:01<25:36,  7.11s/it]
+ 88%|████████▊ | 1598/1813 [2:58:08<24:45,  6.91s/it]
+ 88%|████████▊ | 1599/1813 [2:58:13<23:07,  6.48s/it]
+ 88%|████████▊ | 1600/1813 [2:58:18<20:59,  5.92s/it]
+                                                     
+{'loss': '0.8756', 'grad_norm': '0.9339', 'learning_rate': '7.533e-06', 'epoch': '0.8828', 'num_input_tokens_seen': 10767512, 'train_runtime': '1.07e+04', 'train_tokens_per_second': '1006'}
+
+ 88%|████████▊ | 1600/1813 [2:58:18<20:59,  5.92s/it]
+ 88%|████████▊ | 1601/1813 [2:58:26<23:01,  6.51s/it]
+ 88%|████████▊ | 1602/1813 [2:58:32<22:18,  6.34s/it]
+ 88%|████████▊ | 1603/1813 [2:58:39<22:38,  6.47s/it]
+ 88%|████████▊ | 1604/1813 [2:58:45<22:57,  6.59s/it]
+ 89%|████████▊ | 1605/1813 [2:58:54<24:43,  7.13s/it]
+ 89%|████████▊ | 1606/1813 [2:58:59<22:47,  6.61s/it]
+ 89%|████████▊ | 1607/1813 [2:59:06<22:51,  6.66s/it]
+ 89%|████████▊ | 1608/1813 [2:59:13<23:30,  6.88s/it]
+ 89%|████████▊ | 1609/1813 [2:59:19<21:49,  6.42s/it]
+ 89%|████████▉ | 1610/1813 [2:59:25<21:08,  6.25s/it]
+                                                     
+{'loss': '0.8957', 'grad_norm': '1.436', 'learning_rate': '6.854e-06', 'epoch': '0.8883', 'num_input_tokens_seen': 10840800, 'train_runtime': '1.077e+04', 'train_tokens_per_second': '1007'}
+
+ 89%|████████▉ | 1610/1813 [2:59:25<21:08,  6.25s/it]
+ 89%|████████▉ | 1611/1813 [2:59:29<19:40,  5.84s/it]
+ 89%|████████▉ | 1612/1813 [2:59:35<19:29,  5.82s/it]
+ 89%|████████▉ | 1613/1813 [2:59:40<18:16,  5.48s/it]
+ 89%|████████▉ | 1614/1813 [2:59:46<18:38,  5.62s/it]
+ 89%|████████▉ | 1615/1813 [2:59:53<19:34,  5.93s/it]
+ 89%|████████▉ | 1616/1813 [2:59:59<20:01,  6.10s/it]
+ 89%|████████▉ | 1617/1813 [3:00:06<20:23,  6.24s/it]
+ 89%|████████▉ | 1618/1813 [3:00:11<19:07,  5.88s/it]
+ 89%|████████▉ | 1619/1813 [3:00:16<18:44,  5.80s/it]
+ 89%|████████▉ | 1620/1813 [3:00:21<17:42,  5.51s/it]
+                                                     
+{'loss': '0.9036', 'grad_norm': '1.087', 'learning_rate': '6.206e-06', 'epoch': '0.8938', 'num_input_tokens_seen': 10899640, 'train_runtime': '1.082e+04', 'train_tokens_per_second': '1007'}
+
+ 89%|████████▉ | 1620/1813 [3:00:21<17:42,  5.51s/it]
+ 89%|████████▉ | 1621/1813 [3:00:26<17:25,  5.44s/it]
+ 89%|████████▉ | 1622/1813 [3:00:34<19:24,  6.10s/it]
+ 90%|████████▉ | 1623/1813 [3:00:41<20:10,  6.37s/it]
+ 90%|████████▉ | 1624/1813 [3:00:47<19:46,  6.28s/it]
+ 90%|████████▉ | 1625/1813 [3:00:54<20:45,  6.62s/it]
+ 90%|████████▉ | 1626/1813 [3:01:01<20:14,  6.50s/it]
+ 90%|████████▉ | 1627/1813 [3:01:07<19:34,  6.32s/it]
+ 90%|████████▉ | 1628/1813 [3:01:12<18:49,  6.10s/it]
+ 90%|████████▉ | 1629/1813 [3:01:19<19:46,  6.45s/it]
+ 90%|████████▉ | 1630/1813 [3:01:25<18:48,  6.17s/it]
+                                                     
+{'loss': '0.8973', 'grad_norm': '1.454', 'learning_rate': '5.589e-06', 'epoch': '0.8993', 'num_input_tokens_seen': 10968528, 'train_runtime': '1.089e+04', 'train_tokens_per_second': '1008'}
+
+ 90%|████████▉ | 1630/1813 [3:01:25<18:48,  6.17s/it]
+ 90%|████████▉ | 1631/1813 [3:01:30<18:00,  5.94s/it]
+ 90%|█████████ | 1632/1813 [3:01:35<16:51,  5.59s/it]
+ 90%|█████████ | 1633/1813 [3:01:43<18:22,  6.12s/it]
+ 90%|█████████ | 1634/1813 [3:01:47<17:09,  5.75s/it]
+ 90%|█████████ | 1635/1813 [3:01:54<17:55,  6.04s/it]
+ 90%|█████████ | 1636/1813 [3:02:00<17:17,  5.86s/it]
+ 90%|█████████ | 1637/1813 [3:02:06<17:34,  5.99s/it]
+ 90%|█████████ | 1638/1813 [3:02:11<16:46,  5.75s/it]
+ 90%|█████████ | 1639/1813 [3:02:18<17:37,  6.08s/it]
+ 90%|█████████ | 1640/1813 [3:02:23<16:43,  5.80s/it]
+                                                     
+{'loss': '0.9044', 'grad_norm': '0.857', 'learning_rate': '5.003e-06', 'epoch': '0.9048', 'num_input_tokens_seen': 11030048, 'train_runtime': '1.094e+04', 'train_tokens_per_second': '1008'}
+
+ 90%|█████████ | 1640/1813 [3:02:23<16:43,  5.80s/it]
+ 91%|█████████ | 1641/1813 [3:02:29<16:37,  5.80s/it]
+ 91%|█████████ | 1642/1813 [3:02:35<17:16,  6.06s/it]
+ 91%|█████████ | 1643/1813 [3:02:42<17:12,  6.07s/it]
+ 91%|█████████ | 1644/1813 [3:02:48<17:44,  6.30s/it]
+ 91%|█████████ | 1645/1813 [3:02:58<20:16,  7.24s/it]
+ 91%|█████████ | 1646/1813 [3:03:05<19:44,  7.09s/it]
+ 91%|█████████ | 1647/1813 [3:03:11<18:48,  6.80s/it]
+ 91%|█████████ | 1648/1813 [3:03:17<18:34,  6.75s/it]
+ 91%|█████████ | 1649/1813 [3:03:23<17:11,  6.29s/it]
+ 91%|█████████ | 1650/1813 [3:03:28<16:12,  5.97s/it]
+                                                     
+{'loss': '0.8597', 'grad_norm': '0.956', 'learning_rate': '4.449e-06', 'epoch': '0.9103', 'num_input_tokens_seen': 11098768, 'train_runtime': '1.101e+04', 'train_tokens_per_second': '1008'}
+
+ 91%|█████████ | 1650/1813 [3:03:28<16:12,  5.97s/it]
+ 91%|█████████ | 1651/1813 [3:03:34<16:25,  6.09s/it]
+ 91%|█████████ | 1652/1813 [3:03:41<16:38,  6.20s/it]
+ 91%|█████████ | 1653/1813 [3:03:47<17:01,  6.39s/it]
+ 91%|█████████ | 1654/1813 [3:03:54<16:44,  6.32s/it]
+ 91%|█████████▏| 1655/1813 [3:04:00<16:48,  6.38s/it]
+ 91%|█████████▏| 1656/1813 [3:04:06<16:16,  6.22s/it]
+ 91%|█████████▏| 1657/1813 [3:04:14<17:20,  6.67s/it]
+ 91%|█████████▏| 1658/1813 [3:04:20<17:08,  6.64s/it]
+ 92%|█████████▏| 1659/1813 [3:04:27<17:21,  6.77s/it]
+ 92%|█████████▏| 1660/1813 [3:04:34<16:54,  6.63s/it]
+                                                     
+{'loss': '0.9257', 'grad_norm': '0.9872', 'learning_rate': '3.927e-06', 'epoch': '0.9159', 'num_input_tokens_seen': 11168848, 'train_runtime': '1.107e+04', 'train_tokens_per_second': '1009'}
+
+ 92%|█████████▏| 1660/1813 [3:04:34<16:54,  6.63s/it]
+ 92%|█████████▏| 1661/1813 [3:04:41<17:09,  6.77s/it]
+ 92%|█████████▏| 1662/1813 [3:04:47<16:50,  6.70s/it]
+ 92%|█████████▏| 1663/1813 [3:04:54<16:54,  6.76s/it]
+ 92%|█████████▏| 1664/1813 [3:05:01<17:00,  6.85s/it]
+ 92%|█████████▏| 1665/1813 [3:05:09<17:31,  7.11s/it]
+ 92%|█████████▏| 1666/1813 [3:05:13<15:31,  6.33s/it]
+ 92%|█████████▏| 1667/1813 [3:05:20<15:43,  6.46s/it]
+ 92%|█████████▏| 1668/1813 [3:05:27<15:30,  6.42s/it]
+ 92%|█████████▏| 1669/1813 [3:05:36<17:55,  7.47s/it]
+ 92%|█████████▏| 1670/1813 [3:05:42<16:11,  6.79s/it]
+                                                     
+{'loss': '0.8768', 'grad_norm': '1.171', 'learning_rate': '3.437e-06', 'epoch': '0.9214', 'num_input_tokens_seen': 11244288, 'train_runtime': '1.114e+04', 'train_tokens_per_second': '1009'}
+
+ 92%|█████████▏| 1670/1813 [3:05:42<16:11,  6.79s/it]
+ 92%|█████████▏| 1671/1813 [3:05:47<14:51,  6.28s/it]
+ 92%|█████████▏| 1672/1813 [3:05:53<14:25,  6.14s/it]
+ 92%|█████████▏| 1673/1813 [3:05:59<14:12,  6.09s/it]
+ 92%|█████████▏| 1674/1813 [3:06:05<14:40,  6.33s/it]
+ 92%|█████████▏| 1675/1813 [3:06:13<15:07,  6.58s/it]
+ 92%|█████████▏| 1676/1813 [3:06:20<15:33,  6.81s/it]
+ 92%|█████████▏| 1677/1813 [3:06:27<15:19,  6.76s/it]
+ 93%|█████████▎| 1678/1813 [3:06:33<14:55,  6.64s/it]
+ 93%|█████████▎| 1679/1813 [3:06:39<14:15,  6.38s/it]
+ 93%|█████████▎| 1680/1813 [3:06:47<15:40,  7.07s/it]
+                                                     
+{'loss': '0.9494', 'grad_norm': '0.8919', 'learning_rate': '2.978e-06', 'epoch': '0.9269', 'num_input_tokens_seen': 11317056, 'train_runtime': '1.121e+04', 'train_tokens_per_second': '1010'}
+
+ 93%|█████████▎| 1680/1813 [3:06:47<15:40,  7.07s/it]
+ 93%|█████████▎| 1681/1813 [3:06:52<14:01,  6.38s/it]
+ 93%|█████████▎| 1682/1813 [3:06:57<13:05,  6.00s/it]
+ 93%|█████████▎| 1683/1813 [3:07:02<12:04,  5.58s/it]
+ 93%|█████████▎| 1684/1813 [3:07:09<12:46,  5.94s/it]
+ 93%|█████████▎| 1685/1813 [3:07:15<13:13,  6.20s/it]
+ 93%|█████████▎| 1686/1813 [3:07:23<13:41,  6.47s/it]
+ 93%|█████████▎| 1687/1813 [3:07:29<13:40,  6.51s/it]
+ 93%|█████████▎| 1688/1813 [3:07:36<13:46,  6.61s/it]
+ 93%|█████████▎| 1689/1813 [3:07:41<12:28,  6.04s/it]
+ 93%|█████████▎| 1690/1813 [3:07:47<12:46,  6.23s/it]
+                                                     
+{'loss': '0.8721', 'grad_norm': '0.8994', 'learning_rate': '2.552e-06', 'epoch': '0.9324', 'num_input_tokens_seen': 11380560, 'train_runtime': '1.127e+04', 'train_tokens_per_second': '1010'}
+
+ 93%|█████████▎| 1690/1813 [3:07:47<12:46,  6.23s/it]
+ 93%|█████████▎| 1691/1813 [3:07:54<12:43,  6.26s/it]
+ 93%|█████████▎| 1692/1813 [3:08:01<13:30,  6.70s/it]
+ 93%|█████████▎| 1693/1813 [3:08:08<13:07,  6.56s/it]
+ 93%|█████████▎| 1694/1813 [3:08:16<14:07,  7.12s/it]
+ 93%|█████████▎| 1695/1813 [3:08:22<13:26,  6.83s/it]
+ 94%|█████████▎| 1696/1813 [3:08:28<12:45,  6.55s/it]
+ 94%|█████████▎| 1697/1813 [3:08:35<12:40,  6.56s/it]
+ 94%|█████████▎| 1698/1813 [3:08:40<11:58,  6.25s/it]
+ 94%|█████████▎| 1699/1813 [3:08:47<11:58,  6.30s/it]
+ 94%|█████████▍| 1700/1813 [3:08:53<12:01,  6.38s/it]
+                                                     
+{'loss': '0.8525', 'grad_norm': '1.077', 'learning_rate': '2.159e-06', 'epoch': '0.9379', 'num_input_tokens_seen': 11450712, 'train_runtime': '1.133e+04', 'train_tokens_per_second': '1010'}
+
+ 94%|█████████▍| 1700/1813 [3:08:53<12:01,  6.38s/it]
+ 94%|█████████▍| 1701/1813 [3:08:58<11:03,  5.93s/it]
+ 94%|█████████▍| 1702/1813 [3:09:03<10:34,  5.72s/it]
+ 94%|█████████▍| 1703/1813 [3:09:09<10:33,  5.76s/it]
+ 94%|█████████▍| 1704/1813 [3:09:17<11:31,  6.35s/it]
+ 94%|█████████▍| 1705/1813 [3:09:24<11:35,  6.44s/it]
+ 94%|█████████▍| 1706/1813 [3:09:30<11:29,  6.45s/it]
+ 94%|█████████▍| 1707/1813 [3:09:36<10:56,  6.19s/it]
+ 94%|█████████▍| 1708/1813 [3:09:41<10:12,  5.83s/it]
+ 94%|█████████▍| 1709/1813 [3:09:47<10:23,  6.00s/it]
+ 94%|█████████▍| 1710/1813 [3:09:53<10:15,  5.98s/it]
+                                                     
+{'loss': '0.9381', 'grad_norm': '1.253', 'learning_rate': '1.798e-06', 'epoch': '0.9434', 'num_input_tokens_seen': 11511984, 'train_runtime': '1.139e+04', 'train_tokens_per_second': '1010'}
+
+ 94%|█████████▍| 1710/1813 [3:09:53<10:15,  5.98s/it]
+ 94%|█████████▍| 1711/1813 [3:09:58<09:55,  5.84s/it]
+ 94%|█████████▍| 1712/1813 [3:10:04<09:55,  5.89s/it]
+ 94%|█████████▍| 1713/1813 [3:10:10<09:49,  5.89s/it]
+ 95%|█████████▍| 1714/1813 [3:10:16<09:41,  5.87s/it]
+ 95%|█████████▍| 1715/1813 [3:10:22<09:29,  5.81s/it]
+ 95%|█████████▍| 1716/1813 [3:10:27<09:06,  5.63s/it]
+ 95%|█████████▍| 1717/1813 [3:10:32<08:51,  5.54s/it]
+ 95%|█████████▍| 1718/1813 [3:10:39<09:10,  5.79s/it]
+ 95%|█████████▍| 1719/1813 [3:10:45<09:20,  5.97s/it]
+ 95%|█████████▍| 1720/1813 [3:10:51<09:18,  6.01s/it]
+                                                     
+{'loss': '0.8572', 'grad_norm': '0.9037', 'learning_rate': '1.47e-06', 'epoch': '0.949', 'num_input_tokens_seen': 11571368, 'train_runtime': '1.145e+04', 'train_tokens_per_second': '1010'}
+
+ 95%|█████████▍| 1720/1813 [3:10:51<09:18,  6.01s/it]
+ 95%|█████████▍| 1721/1813 [3:10:59<09:52,  6.44s/it]
+ 95%|█████████▍| 1722/1813 [3:11:05<09:42,  6.40s/it]
+ 95%|█████████▌| 1723/1813 [3:11:11<09:33,  6.37s/it]
+ 95%|█████████▌| 1724/1813 [3:11:17<09:21,  6.30s/it]
+ 95%|█████████▌| 1725/1813 [3:11:25<09:43,  6.63s/it]
+ 95%|█████████▌| 1726/1813 [3:11:32<09:58,  6.88s/it]
+ 95%|█████████▌| 1727/1813 [3:11:38<09:26,  6.59s/it]
+ 95%|█████████▌| 1728/1813 [3:11:43<08:42,  6.15s/it]
+ 95%|█████████▌| 1729/1813 [3:11:49<08:20,  5.96s/it]
+ 95%|█████████▌| 1730/1813 [3:11:56<08:34,  6.19s/it]
+                                                     
+{'loss': '0.9203', 'grad_norm': '0.8309', 'learning_rate': '1.174e-06', 'epoch': '0.9545', 'num_input_tokens_seen': 11640392, 'train_runtime': '1.152e+04', 'train_tokens_per_second': '1011'}
+
+ 95%|█████████▌| 1730/1813 [3:11:56<08:34,  6.19s/it]
+ 95%|█████████▌| 1731/1813 [3:12:02<08:29,  6.21s/it]
+ 96%|█████████▌| 1732/1813 [3:12:09<08:35,  6.37s/it]
+ 96%|█████████▌| 1733/1813 [3:12:14<07:57,  5.97s/it]
+ 96%|█████████▌| 1734/1813 [3:12:19<07:48,  5.93s/it]
+ 96%|█████████▌| 1735/1813 [3:12:26<07:53,  6.07s/it]
+ 96%|█████████▌| 1736/1813 [3:12:33<08:04,  6.29s/it]
+ 96%|█████████▌| 1737/1813 [3:12:39<07:54,  6.24s/it]
+ 96%|█████████▌| 1738/1813 [3:12:44<07:17,  5.83s/it]
+ 96%|█████████▌| 1739/1813 [3:12:52<07:58,  6.46s/it]
+ 96%|█████████▌| 1740/1813 [3:12:58<07:52,  6.47s/it]
+                                                     
+{'loss': '0.909', 'grad_norm': '1.041', 'learning_rate': '9.117e-07', 'epoch': '0.96', 'num_input_tokens_seen': 11709120, 'train_runtime': '1.158e+04', 'train_tokens_per_second': '1011'}
+
+ 96%|█████████▌| 1740/1813 [3:12:58<07:52,  6.47s/it]
+ 96%|█████████▌| 1741/1813 [3:13:04<07:30,  6.26s/it]
+ 96%|█████████▌| 1742/1813 [3:13:11<07:34,  6.40s/it]
+ 96%|█████████▌| 1743/1813 [3:13:18<07:40,  6.58s/it]
+ 96%|█████████▌| 1744/1813 [3:13:25<07:43,  6.72s/it]
+ 96%|█████████▌| 1745/1813 [3:13:31<07:20,  6.48s/it]
+ 96%|█████████▋| 1746/1813 [3:13:38<07:31,  6.73s/it]
+ 96%|█████████▋| 1747/1813 [3:13:43<06:45,  6.15s/it]
+ 96%|█████████▋| 1748/1813 [3:13:50<07:03,  6.51s/it]
+ 96%|█████████▋| 1749/1813 [3:13:55<06:34,  6.17s/it]
+ 97%|█████████▋| 1750/1813 [3:14:01<06:12,  5.92s/it]
+                                                     
+{'loss': '0.8577', 'grad_norm': '0.8912', 'learning_rate': '6.822e-07', 'epoch': '0.9655', 'num_input_tokens_seen': 11777768, 'train_runtime': '1.164e+04', 'train_tokens_per_second': '1012'}
+
+ 97%|█████████▋| 1750/1813 [3:14:01<06:12,  5.92s/it]
+ 97%|█████████▋| 1751/1813 [3:14:07<06:13,  6.02s/it]
+ 97%|█████████▋| 1752/1813 [3:14:13<06:09,  6.05s/it]
+ 97%|█████████▋| 1753/1813 [3:14:21<06:30,  6.51s/it]
+ 97%|█████████▋| 1754/1813 [3:14:28<06:37,  6.73s/it]
+ 97%|█████████▋| 1755/1813 [3:14:34<06:25,  6.64s/it]
+ 97%|█████████▋| 1756/1813 [3:14:40<05:55,  6.24s/it]
+ 97%|█████████▋| 1757/1813 [3:14:45<05:38,  6.04s/it]
+ 97%|█████████▋| 1758/1813 [3:14:51<05:24,  5.91s/it]
+ 97%|█████████▋| 1759/1813 [3:14:57<05:21,  5.96s/it]
+ 97%|█████████▋| 1760/1813 [3:15:02<05:06,  5.79s/it]
+                                                     
+{'loss': '0.8701', 'grad_norm': '1.061', 'learning_rate': '4.858e-07', 'epoch': '0.971', 'num_input_tokens_seen': 11843192, 'train_runtime': '1.17e+04', 'train_tokens_per_second': '1012'}
+
+ 97%|█████████▋| 1760/1813 [3:15:02<05:06,  5.79s/it]
+ 97%|█████████▋| 1761/1813 [3:15:08<04:54,  5.66s/it]
+ 97%|█████████▋| 1762/1813 [3:15:12<04:32,  5.34s/it]
+ 97%|█████████▋| 1763/1813 [3:15:20<05:00,  6.02s/it]
+ 97%|█████████▋| 1764/1813 [3:15:25<04:41,  5.74s/it]
+ 97%|█████████▋| 1765/1813 [3:15:30<04:28,  5.60s/it]
+ 97%|█████████▋| 1766/1813 [3:15:36<04:20,  5.54s/it]
+ 97%|█████████▋| 1767/1813 [3:15:41<04:14,  5.53s/it]
+ 98%|█████████▊| 1768/1813 [3:15:47<04:12,  5.62s/it]
+ 98%|█████████▊| 1769/1813 [3:15:53<04:09,  5.66s/it]
+ 98%|█████████▊| 1770/1813 [3:16:00<04:18,  6.01s/it]
+                                                     
+{'loss': '0.8834', 'grad_norm': '0.915', 'learning_rate': '3.226e-07', 'epoch': '0.9766', 'num_input_tokens_seen': 11903728, 'train_runtime': '1.176e+04', 'train_tokens_per_second': '1012'}
+
+ 98%|█████████▊| 1770/1813 [3:16:00<04:18,  6.01s/it]
+ 98%|█████████▊| 1771/1813 [3:16:06<04:16,  6.12s/it]
+ 98%|█████████▊| 1772/1813 [3:16:12<04:14,  6.21s/it]
+ 98%|█████████▊| 1773/1813 [3:16:19<04:18,  6.47s/it]
+ 98%|█████████▊| 1774/1813 [3:16:24<03:51,  5.93s/it]
+ 98%|█████████▊| 1775/1813 [3:16:32<04:02,  6.38s/it]
+ 98%|█████████▊| 1776/1813 [3:16:39<04:06,  6.66s/it]
+ 98%|█████████▊| 1777/1813 [3:16:46<04:08,  6.90s/it]
+ 98%|█████████▊| 1778/1813 [3:16:52<03:47,  6.49s/it]
+ 98%|█████████▊| 1779/1813 [3:16:57<03:31,  6.21s/it]
+ 98%|█████████▊| 1780/1813 [3:17:03<03:21,  6.10s/it]
+                                                     
+{'loss': '0.8601', 'grad_norm': '0.8732', 'learning_rate': '1.927e-07', 'epoch': '0.9821', 'num_input_tokens_seen': 11972864, 'train_runtime': '1.182e+04', 'train_tokens_per_second': '1013'}
+
+ 98%|█████████▊| 1780/1813 [3:17:03<03:21,  6.10s/it]
+ 98%|█████████▊| 1781/1813 [3:17:09<03:15,  6.11s/it]
+ 98%|█████████▊| 1782/1813 [3:17:15<03:07,  6.06s/it]
+ 98%|█████████▊| 1783/1813 [3:17:21<03:00,  6.01s/it]
+ 98%|█████████▊| 1784/1813 [3:17:28<03:00,  6.23s/it]
+ 98%|█████████▊| 1785/1813 [3:17:34<02:54,  6.22s/it]
+ 99%|█████████▊| 1786/1813 [3:17:40<02:47,  6.20s/it]
+ 99%|█████████▊| 1787/1813 [3:17:46<02:36,  6.02s/it]
+ 99%|█████████▊| 1788/1813 [3:17:53<02:35,  6.24s/it]
+ 99%|█████████▊| 1789/1813 [3:18:01<02:46,  6.92s/it]
+ 99%|█████████▊| 1790/1813 [3:18:08<02:41,  7.00s/it]
+                                                     
+{'loss': '0.8854', 'grad_norm': '0.8062', 'learning_rate': '9.603e-08', 'epoch': '0.9876', 'num_input_tokens_seen': 12043496, 'train_runtime': '1.189e+04', 'train_tokens_per_second': '1013'}
+
+ 99%|█████████▊| 1790/1813 [3:18:08<02:41,  7.00s/it]
+ 99%|█████████▉| 1791/1813 [3:18:14<02:24,  6.56s/it]
+ 99%|█████████▉| 1792/1813 [3:18:20<02:15,  6.47s/it]
+ 99%|█████████▉| 1793/1813 [3:18:27<02:11,  6.57s/it]
+ 99%|█████████▉| 1794/1813 [3:18:34<02:07,  6.69s/it]
+ 99%|█████████▉| 1795/1813 [3:18:41<02:00,  6.70s/it]
+ 99%|█████████▉| 1796/1813 [3:18:47<01:53,  6.70s/it]
+ 99%|█████████▉| 1797/1813 [3:18:54<01:46,  6.64s/it]
+ 99%|█████████▉| 1798/1813 [3:19:03<01:52,  7.47s/it]
+ 99%|█████████▉| 1799/1813 [3:19:10<01:40,  7.15s/it]
+ 99%|█████████▉| 1800/1813 [3:19:14<01:22,  6.34s/it]
+                                                     
+{'loss': '0.8533', 'grad_norm': '0.9493', 'learning_rate': '3.268e-08', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448, 'train_runtime': '1.195e+04', 'train_tokens_per_second': '1013'}
+
+ 99%|█████████▉| 1800/1813 [3:19:14<01:22,  6.34s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.76it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.22it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.32it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:06,  7.79it/s]
+
+  1%|          | 12/1000 [00:01<02:04,  7.92it/s]
+
+  1%|▏         | 13/1000 [00:01<02:30,  6.56it/s]
+
+  1%|▏         | 14/1000 [00:01<02:31,  6.50it/s]
+
+  2%|▏         | 15/1000 [00:02<02:54,  5.65it/s]
+
+  2%|▏         | 16/1000 [00:02<02:59,  5.47it/s]
+
+  2%|▏         | 18/1000 [00:02<02:46,  5.90it/s]
+
+  2%|▏         | 19/1000 [00:02<03:15,  5.02it/s]
+
+  2%|▏         | 20/1000 [00:03<03:37,  4.50it/s]
+
+  2%|▏         | 22/1000 [00:03<02:37,  6.20it/s]
+
+  2%|▏         | 23/1000 [00:03<02:56,  5.54it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.33it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:06<02:04,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.10it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.32it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.35it/s]
+
+  5%|▍         | 49/1000 [00:07<01:53,  8.36it/s]
+
+  5%|▌         | 51/1000 [00:07<01:38,  9.62it/s]
+
+  5%|▌         | 53/1000 [00:07<01:28, 10.71it/s]
+
+  6%|▌         | 55/1000 [00:07<01:29, 10.59it/s]
+
+  6%|▌         | 57/1000 [00:07<01:42,  9.21it/s]
+
+  6%|▌         | 58/1000 [00:07<01:43,  9.09it/s]
+
+  6%|▌         | 59/1000 [00:08<01:59,  7.85it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.32it/s]
+
+  6%|▋         | 63/1000 [00:08<01:35,  9.81it/s]
+
+  6%|▋         | 65/1000 [00:08<01:53,  8.24it/s]
+
+  7%|▋         | 67/1000 [00:09<02:05,  7.45it/s]
+
+  7%|▋         | 69/1000 [00:09<01:53,  8.20it/s]
+
+  7%|▋         | 70/1000 [00:09<02:14,  6.92it/s]
+
+  7%|▋         | 72/1000 [00:09<02:19,  6.66it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.09it/s]
+
+  8%|▊         | 75/1000 [00:10<02:15,  6.83it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.66it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:59,  7.67it/s]
+
+  8%|▊         | 82/1000 [00:11<01:46,  8.63it/s]
+
+  8%|▊         | 83/1000 [00:11<02:08,  7.15it/s]
+
+  8%|▊         | 85/1000 [00:11<02:13,  6.84it/s]
+
+  9%|▊         | 86/1000 [00:11<02:32,  6.00it/s]
+
+  9%|▊         | 87/1000 [00:12<02:48,  5.41it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.10it/s]
+
+  9%|▉         | 92/1000 [00:12<02:26,  6.19it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:56,  7.72it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.31it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.44it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.14it/s]
+
+ 11%|█         | 112/1000 [00:14<01:14, 11.98it/s]
+
+ 11%|█▏        | 114/1000 [00:15<01:11, 12.47it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.76it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.19it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.90it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:50,  7.96it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.37it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.69it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:23,  6.10it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:31,  5.77it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.27it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.07it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.68it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.11it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.62it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.95it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.88it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.07it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.91it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.03it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.40it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:21<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:22<02:24,  5.77it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.83it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.14it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.43it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.79it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.56it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.49it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.71it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.73it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.89it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:26<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:28,  9.10it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.61it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.74it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.61it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.91it/s]
+
+ 21%|██        | 208/1000 [00:27<01:34,  8.40it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.41it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.61it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.70it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:26,  9.11it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.47it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:29,  8.76it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:29<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.54it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.95it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+
+ 24%|██▍       | 242/1000 [00:32<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.84it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:08,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.73it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:25,  8.72it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.67it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.04it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:30,  8.00it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.77it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:49,  6.60it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.00it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.83it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.42it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.73it/s]
+
+ 29%|██▉       | 290/1000 [00:39<01:31,  7.78it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.84it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.13it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.09it/s]
+
+ 30%|███       | 305/1000 [00:41<01:25,  8.16it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:41,  6.80it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.80it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.26it/s]
+
+ 32%|███▏      | 315/1000 [00:42<01:06, 10.24it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.57it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:43<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.73it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.40it/s]
+
+ 33%|███▎      | 328/1000 [00:44<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.77it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.57it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:45<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.80it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.01it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.87it/s]
+
+ 34%|███▍      | 344/1000 [00:46<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.42it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:47<01:48,  5.97it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.58it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.42it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.59it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 378/1000 [00:51<01:30,  6.91it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.76it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.20it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.19it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.65it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.37it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.79it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:54<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.23it/s]
+
+ 41%|████      | 406/1000 [00:54<01:10,  8.48it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.37it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.42it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.82it/s]
+
+ 41%|████      | 412/1000 [00:55<01:50,  5.34it/s]
+
+ 41%|████▏     | 414/1000 [00:56<01:23,  7.04it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.80it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.55it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.03it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.78it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.51it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.09it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:01,  9.09it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:09,  8.00it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:17,  7.19it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.67it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:14,  7.44it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.00it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.57it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.17it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.49it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.27it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.84it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.02it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.58it/s]
+
+ 47%|████▋     | 474/1000 [01:04<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.37it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.48it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.14it/s]
+
+ 48%|████▊     | 480/1000 [01:05<01:19,  6.57it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.04it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▉     | 488/1000 [01:06<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.23it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.98it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<01:00,  8.25it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:16,  6.56it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:25,  5.87it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:07,  7.38it/s]
+
+ 50%|█████     | 503/1000 [01:08<01:14,  6.70it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:16,  6.51it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.76it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:07,  7.31it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:55,  8.85it/s]
+
+ 51%|█████     | 512/1000 [01:09<00:51,  9.56it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:07,  7.15it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:05,  7.42it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.02it/s]
+
+ 52%|█████▏    | 518/1000 [01:10<01:01,  7.78it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:17,  6.23it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<01:00,  7.88it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:12,  6.61it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.76it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:26,  5.53it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.56it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.39it/s]
+
+ 53%|█████▎    | 531/1000 [01:12<00:57,  8.11it/s]
+
+ 53%|█████▎    | 533/1000 [01:12<00:48,  9.71it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.32it/s]
+
+ 54%|█████▎    | 537/1000 [01:13<01:22,  5.63it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:20,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:06,  6.92it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.05it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:15<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:15<00:59,  7.50it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.25it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+
+ 56%|█████▌    | 557/1000 [01:16<01:03,  7.02it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.85it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.04it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.99it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.76it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:49,  8.44it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:55,  7.54it/s]
+
+ 58%|█████▊    | 585/1000 [01:20<01:00,  6.88it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.13it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.48it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.74it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:21<00:59,  6.79it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.86it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:23<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.81it/s]
+
+ 62%|██████▏   | 616/1000 [01:24<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.75it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.08it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.02it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.63it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.18it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:26<00:47,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.41it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:27<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.95it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.58it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:28<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:29<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:44,  7.71it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.11it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:30<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:32<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.71it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.98it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.94it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.89it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.82it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.27it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:36<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.49it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.71it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.38it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.48it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.26it/s]
+
+ 72%|███████▎  | 725/1000 [01:39<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.14it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.63it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.53it/s]
+
+ 74%|███████▎  | 735/1000 [01:40<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.78it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:41<00:30,  8.37it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.62it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.46it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.24it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:43<00:31,  7.62it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.33it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.72it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.67it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.62it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.78it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.57it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.14it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.77it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.60it/s]
+
+ 78%|███████▊  | 780/1000 [01:46<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.38it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+
+ 79%|███████▉  | 788/1000 [01:47<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.98it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.20it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:24,  7.84it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.18it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.20it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.22it/s]
+
+ 81%|████████▏ | 813/1000 [01:50<00:20,  9.07it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.31it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:51<00:22,  8.06it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.83it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:53<00:26,  6.21it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.52it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.46it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.47it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.00it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.36it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:27,  5.72it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.22it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.52it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.85it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.45it/s]
+
+ 86%|████████▌ | 855/1000 [01:56<00:14, 10.03it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.78it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.64it/s]
+
+ 86%|████████▋ | 865/1000 [01:57<00:11, 11.35it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.17it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.14it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.16it/s]
+
+ 87%|████████▋ | 873/1000 [01:58<00:16,  7.73it/s]
+
+ 87%|████████▋ | 874/1000 [01:58<00:17,  7.05it/s]
+
+ 88%|████████▊ | 876/1000 [01:58<00:14,  8.52it/s]
+
+ 88%|████████▊ | 878/1000 [01:58<00:15,  7.64it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.94it/s]
+
+ 88%|████████▊ | 881/1000 [01:59<00:15,  7.79it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  8.11it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.37it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.58it/s]
+
+ 89%|████████▉ | 890/1000 [02:00<00:11,  9.88it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:09, 10.88it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.63it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.87it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.88it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.09it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.59it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.78it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.61it/s]
+
+ 92%|█████████▏| 919/1000 [02:04<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.93it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:05<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:06<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.84it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.71it/s]
+
+ 95%|█████████▌| 950/1000 [02:08<00:05,  8.82it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.40it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:09<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.74it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.00it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.45it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.42it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.30it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:04,  7.98it/s]
+
+ 97%|█████████▋| 970/1000 [02:11<00:03,  9.39it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.21it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.79it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.61it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.37it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.03it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.41it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.91it/s]
+
+ 99%|█████████▊| 986/1000 [02:13<00:01,  8.36it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.99it/s]
+
+ 99%|█████████▉| 991/1000 [02:14<00:01,  6.14it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7413', 'eval_runtime': '135.6', 'eval_samples_per_second': '7.374', 'eval_steps_per_second': '7.374', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448}
+
+ 99%|█████████▉| 1800/1813 [3:21:30<01:22,  6.34s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_004_lr_1.869e-04/checkpoint-1800
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+ 99%|█████████▉| 1801/1813 [3:21:39<09:33, 47.77s/it]
+ 99%|█████████▉| 1802/1813 [3:21:44<06:25, 35.02s/it]
+ 99%|█████████▉| 1803/1813 [3:21:50<04:22, 26.27s/it]
+100%|█████████▉| 1804/1813 [3:21:57<03:05, 20.66s/it]
+100%|█████████▉| 1805/1813 [3:22:03<02:10, 16.25s/it]
+100%|█████████▉| 1806/1813 [3:22:10<01:33, 13.29s/it]
+100%|█████████▉| 1807/1813 [3:22:17<01:08, 11.39s/it]
+100%|█████████▉| 1808/1813 [3:22:24<00:51, 10.21s/it]
+100%|█████████▉| 1809/1813 [3:22:30<00:36,  9.07s/it]
+100%|█████████▉| 1810/1813 [3:22:36<00:24,  8.00s/it]
+                                                     
+{'loss': '0.8544', 'grad_norm': '0.8773', 'learning_rate': '2.668e-09', 'epoch': '0.9986', 'num_input_tokens_seen': 12187432, 'train_runtime': '1.216e+04', 'train_tokens_per_second': '1003'}
+
+100%|█████████▉| 1810/1813 [3:22:36<00:24,  8.00s/it]
+100%|█████████▉| 1811/1813 [3:22:41<00:14,  7.22s/it]
+100%|█████████▉| 1812/1813 [3:22:47<00:06,  6.62s/it]
+100%|██████████| 1813/1813 [3:22:50<00:00,  5.52s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.80it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.34it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.06it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  7.00it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.83it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.59it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.37it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.11it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.37it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.75it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.94it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.75it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.30it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.21it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.35it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.76it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.21it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.62it/s]
+
+ 10%|█         | 103/1000 [00:14<01:56,  7.73it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.31it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.43it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.19it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.02it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.49it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.81it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.24it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.44it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.11it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.63it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.89it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25, 10.04it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.42it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.92it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.05it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.39it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.68it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.85it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.15it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.47it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.74it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.71it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.78it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.72it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.88it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.62it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.20it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.67it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.77it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.65it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.93it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.69it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.73it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.54it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.08it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.81it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.36it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.01it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.20it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.92it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.47it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.98it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:05,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.70it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.77it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.52it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.75it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.78it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.20it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.17it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.45it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.81it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.10it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.75it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.07it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.12it/s]
+
+ 31%|███       | 306/1000 [00:41<01:40,  6.93it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.77it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.79it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.21it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.27it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:07, 10.22it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.55it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.88it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.69it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.18it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.86it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.59it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.21it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.89it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.94it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:25,  7.62it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.54it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.59it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.42it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:58,  5.36it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.52it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.70it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.21it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:30,  6.91it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.74it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.89it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.19it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:07,  9.16it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.13it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.64it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.36it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.78it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:25,  6.86it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:28,  6.63it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:11,  8.09it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:09,  8.33it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:18,  7.42it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.64it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.91it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.35it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.94it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:16,  7.51it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:13,  7.80it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.51it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.00it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.77it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.53it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.07it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.29it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.71it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.46it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.19it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:17,  7.07it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.39it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:10,  7.75it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.21it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:50, 10.77it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.59it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.73it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.30it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.38it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.05it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.31it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:53,  9.38it/s]
+
+ 50%|████▉     | 496/1000 [01:06<01:01,  8.15it/s]
+
+ 50%|████▉     | 498/1000 [01:07<01:00,  8.28it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:16,  6.58it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:25,  5.86it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.48it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.76it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.78it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.40it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.93it/s]
+
+ 51%|█████     | 512/1000 [01:09<00:49,  9.95it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.09it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.05it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:12<00:48,  9.65it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.67it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.35it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:20,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.49it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:15<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.86it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.36it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.55it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.59it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.79it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:02,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:03,  6.31it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.64it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:10,  5.65it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.05it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:50,  7.82it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.46it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.80it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.60it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.73it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.08it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.20it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.45it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.63it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.84it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.42it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.15it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.95it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.60it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.57it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.51it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.91it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.11it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.47it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.59it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.24it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.51it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.57it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.83it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.86it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:32,  9.82it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.84it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.14it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.82it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.25it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.41it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:48,  6.32it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.93it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:36<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.70it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.37it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.07it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.14it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.29it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:27, 10.11it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.59it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.87it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.52it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.62it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.69it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.39it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.70it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.44it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.76it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.61it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.54it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  9.00it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.20it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.63it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.82it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.14it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.94it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.18it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.20it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.04it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.30it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.13it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.19it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.99it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.00it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 833/1000 [01:53<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.14it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.76it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.86it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.05it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.63it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.72it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.31it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.27it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.28it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.64it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.34it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.75it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.74it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.84it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.87it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.08it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.59it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.78it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.38it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.61it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.18it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.67it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.03it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.45it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.80it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.60it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.16it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.82it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.13it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.84it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.39it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.48it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.43it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.32it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:04,  7.99it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.38it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.18it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.77it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.80it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7413', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.382', 'eval_steps_per_second': '7.382', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:25:05<00:00,  5.52s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_004_lr_1.869e-04/checkpoint-1813
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+
+Training completed. Do not forget to share your model on huggingface.co/models =)
+
+
+
+                                                     
+{'train_runtime': '1.231e+04', 'train_samples_per_second': '2.356', 'train_steps_per_second': '0.147', 'train_loss': '0.9521', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:25:06<00:00,  5.52s/it]
+100%|██████████| 1813/1813 [3:25:06<00:00,  6.79s/it]
+Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_004_lr_1.869e-04
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+***** train metrics *****
+  epoch                    =         1.0
+  num_input_tokens_seen    =    12201816
+  total_flos               = 487685654GF
+  train_loss               =      0.9521
+  train_runtime            =  3:25:06.56
+  train_samples_per_second =       2.356
+  train_steps_per_second   =       0.147
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_004_lr_1.869e-04/training_loss.png
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_004_lr_1.869e-04/training_eval_loss.png
+[WARNING|2026-10-13 13:23:27] llamafactory.extras.ploting:149 >> No metric eval_accuracy to plot.
+
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+  0%|          | 3/1000 [00:00<01:02, 15.83it/s]
+  0%|          | 5/1000 [00:00<02:17,  7.26it/s]
+  1%|          | 7/1000 [00:00<01:58,  8.37it/s]
+  1%|          | 9/1000 [00:01<02:32,  6.49it/s]
+  1%|          | 11/1000 [00:01<02:04,  7.92it/s]
+  1%|          | 12/1000 [00:01<02:03,  8.03it/s]
+  1%|▏         | 13/1000 [00:01<02:28,  6.63it/s]
+  1%|▏         | 14/1000 [00:01<02:30,  6.54it/s]
+  2%|▏         | 15/1000 [00:02<02:53,  5.67it/s]
+  2%|▏         | 16/1000 [00:02<02:59,  5.48it/s]
+  2%|▏         | 18/1000 [00:02<02:45,  5.92it/s]
+  2%|▏         | 19/1000 [00:02<03:14,  5.03it/s]
+  2%|▏         | 20/1000 [00:03<03:37,  4.51it/s]
+  2%|▏         | 22/1000 [00:03<02:37,  6.20it/s]
+  2%|▏         | 23/1000 [00:03<02:56,  5.55it/s]
+  2%|▎         | 25/1000 [00:03<02:12,  7.33it/s]
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+  3%|▎         | 28/1000 [00:04<02:42,  5.98it/s]
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+  3%|▎         | 32/1000 [00:05<02:41,  6.00it/s]
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+  4%|▎         | 37/1000 [00:05<01:45,  9.11it/s]
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+  4%|▍         | 40/1000 [00:06<02:03,  7.76it/s]
+  4%|▍         | 42/1000 [00:06<01:44,  9.13it/s]
+  4%|▍         | 44/1000 [00:06<01:32, 10.35it/s]
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+  5%|▌         | 53/1000 [00:07<01:26, 10.94it/s]
+  6%|▌         | 55/1000 [00:07<01:27, 10.75it/s]
+  6%|▌         | 57/1000 [00:07<01:41,  9.30it/s]
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+  6%|▌         | 61/1000 [00:08<01:40,  9.35it/s]
+  6%|▋         | 63/1000 [00:08<01:35,  9.77it/s]
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+  8%|▊         | 82/1000 [00:10<01:39,  9.20it/s]
+  8%|▊         | 84/1000 [00:11<01:54,  8.03it/s]
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+  9%|▊         | 87/1000 [00:11<02:49,  5.38it/s]
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+  9%|▉         | 91/1000 [00:12<02:29,  6.09it/s]
+  9%|▉         | 92/1000 [00:12<02:27,  6.18it/s]
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+ 10%|▉         | 97/1000 [00:13<02:12,  6.79it/s]
+ 10%|▉         | 99/1000 [00:13<01:47,  8.38it/s]
+ 10%|█         | 101/1000 [00:13<01:33,  9.64it/s]
+ 10%|█         | 103/1000 [00:14<01:55,  7.76it/s]
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+ 11%|█         | 106/1000 [00:14<01:36,  9.29it/s]
+ 11%|█         | 108/1000 [00:14<01:25, 10.42it/s]
+ 11%|█         | 110/1000 [00:14<01:19, 11.18it/s]
+ 11%|█         | 112/1000 [00:14<01:14, 11.91it/s]
+ 11%|█▏        | 114/1000 [00:14<01:11, 12.42it/s]
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.73it/s]
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.20it/s]
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.90it/s]
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.44it/s]
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.72it/s]
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.65it/s]
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.90it/s]
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.98it/s]
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.05it/s]
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.90it/s]
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.08it/s]
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.51it/s]
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.04it/s]
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.38it/s]
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.42it/s]
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.18it/s]
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.84it/s]
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.19it/s]
+ 17%|█▋        | 174/1000 [00:22<02:03,  6.71it/s]
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.43it/s]
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.49it/s]
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.73it/s]
+ 18%|█▊        | 182/1000 [00:24<02:22,  5.72it/s]
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.76it/s]
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.37it/s]
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.09it/s]
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.74it/s]
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+ 20%|██        | 200/1000 [00:26<01:14, 10.67it/s]
+ 20%|██        | 202/1000 [00:26<01:30,  8.78it/s]
+ 20%|██        | 204/1000 [00:26<01:32,  8.64it/s]
+ 21%|██        | 206/1000 [00:26<01:20,  9.90it/s]
+ 21%|██        | 208/1000 [00:27<01:33,  8.50it/s]
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+ 21%|██        | 212/1000 [00:27<01:21,  9.68it/s]
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.73it/s]
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.15it/s]
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.79it/s]
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.03it/s]
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.22it/s]
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.98it/s]
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.48it/s]
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.99it/s]
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.01it/s]
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.95it/s]
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.86it/s]
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.03it/s]
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.38it/s]
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.03it/s]
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.06it/s]
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.74it/s]
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.78it/s]
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.18it/s]
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+ 30%|███       | 301/1000 [00:40<01:38,  7.13it/s]
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+ 30%|███       | 304/1000 [00:40<01:26,  8.07it/s]
+ 30%|███       | 305/1000 [00:40<01:25,  8.13it/s]
+ 31%|███       | 306/1000 [00:41<01:40,  6.94it/s]
+ 31%|███       | 307/1000 [00:41<01:42,  6.77it/s]
+ 31%|███       | 309/1000 [00:41<01:18,  8.79it/s]
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.26it/s]
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.59it/s]
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.59it/s]
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.05it/s]
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.59it/s]
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.44it/s]
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.92it/s]
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.54it/s]
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.72it/s]
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.93it/s]
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.78it/s]
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.20it/s]
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.18it/s]
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.66it/s]
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.39it/s]
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.79it/s]
+ 40%|███▉      | 395/1000 [00:52<01:21,  7.46it/s]
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.24it/s]
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+ 41%|████      | 406/1000 [00:54<01:09,  8.50it/s]
+ 41%|████      | 408/1000 [00:54<01:32,  6.40it/s]
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+ 41%|████      | 411/1000 [00:55<01:40,  5.83it/s]
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.06it/s]
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+ 42%|████▏     | 418/1000 [00:56<01:12,  8.01it/s]
+ 42%|████▏     | 420/1000 [00:56<01:10,  8.27it/s]
+ 42%|████▏     | 421/1000 [00:56<01:18,  7.39it/s]
+ 42%|████▏     | 423/1000 [00:57<01:27,  6.63it/s]
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.88it/s]
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.33it/s]
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.91it/s]
+ 43%|████▎     | 428/1000 [00:57<01:16,  7.52it/s]
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.83it/s]
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.54it/s]
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.01it/s]
+ 44%|████▎     | 435/1000 [00:58<00:48, 11.77it/s]
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.50it/s]
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.10it/s]
+ 44%|████▍     | 441/1000 [00:59<00:59,  9.33it/s]
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.19it/s]
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.51it/s]
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.82it/s]
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.82it/s]
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.40it/s]
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.32it/s]
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.00it/s]
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.23it/s]
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.98it/s]
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.44it/s]
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+ 51%|█████     | 510/1000 [01:08<00:54,  8.97it/s]
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.38it/s]
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.01it/s]
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.57it/s]
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.39it/s]
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.72it/s]
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.07it/s]
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.71it/s]
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.85it/s]
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.04it/s]
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.74it/s]
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.30it/s]
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.24it/s]
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.96it/s]
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.55it/s]
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.14it/s]
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.52it/s]
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.74it/s]
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+ 61%|██████    | 608/1000 [01:22<00:49,  7.84it/s]
+ 61%|██████    | 610/1000 [01:23<00:46,  8.36it/s]
+ 61%|██████    | 612/1000 [01:23<00:41,  9.35it/s]
+ 61%|██████▏   | 614/1000 [01:23<00:36, 10.46it/s]
+ 62%|██████▏   | 616/1000 [01:23<00:51,  7.51it/s]
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.66it/s]
+ 62%|██████▏   | 619/1000 [01:24<00:42,  9.00it/s]
+ 62%|██████▏   | 621/1000 [01:24<00:47,  7.97it/s]
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.17it/s]
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.43it/s]
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.62it/s]
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.18it/s]
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.85it/s]
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.43it/s]
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.73it/s]
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.95it/s]
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.10it/s]
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.54it/s]
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.31it/s]
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.69it/s]
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.90it/s]
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.05it/s]
+ 66%|██████▌   | 659/1000 [01:29<00:57,  5.98it/s]
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.54it/s]
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.47it/s]
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.69it/s]
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.96it/s]
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.91it/s]
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.87it/s]
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.83it/s]
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+ 70%|███████   | 704/1000 [01:36<00:40,  7.22it/s]
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+ 71%|███████   | 710/1000 [01:37<00:44,  6.49it/s]
+ 71%|███████   | 712/1000 [01:37<00:37,  7.69it/s]
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.36it/s]
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.12it/s]
+ 73%|███████▎  | 729/1000 [01:39<00:24, 10.95it/s]
+ 73%|███████▎  | 731/1000 [01:39<00:33,  8.11it/s]
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.34it/s]
+ 74%|███████▎  | 735/1000 [01:39<00:27,  9.75it/s]
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.42it/s]
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.69it/s]
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.27it/s]
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.32it/s]
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.58it/s]
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.42it/s]
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.63it/s]
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.67it/s]
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.16it/s]
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.38it/s]
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.58it/s]
+ 76%|███████▌  | 759/1000 [01:43<00:33,  7.30it/s]
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.70it/s]
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.46it/s]
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.78it/s]
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.54it/s]
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.99it/s]
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+ 79%|███████▉  | 793/1000 [01:47<00:26,  7.95it/s]
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.11it/s]
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.82it/s]
+ 80%|███████▉  | 798/1000 [01:48<00:33,  5.98it/s]
+ 80%|████████  | 800/1000 [01:48<00:29,  6.69it/s]
+ 80%|████████  | 802/1000 [01:48<00:25,  7.63it/s]
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+ 81%|████████  | 809/1000 [01:49<00:18, 10.21it/s]
+ 81%|████████  | 811/1000 [01:49<00:17, 10.87it/s]
+ 81%|████████▏ | 813/1000 [01:49<00:20,  8.92it/s]
+ 81%|████████▏ | 814/1000 [01:50<00:21,  8.77it/s]
+ 82%|████████▏ | 816/1000 [01:50<00:18,  9.85it/s]
+ 82%|████████▏ | 818/1000 [01:50<00:21,  8.31it/s]
+ 82%|████████▏ | 820/1000 [01:50<00:23,  7.52it/s]
+ 82%|████████▏ | 822/1000 [01:50<00:20,  8.68it/s]
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.71it/s]
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.53it/s]
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.83it/s]
+ 83%|████████▎ | 826/1000 [01:51<00:31,  5.61it/s]
+ 83%|████████▎ | 828/1000 [01:52<00:33,  5.12it/s]
+ 83%|████████▎ | 829/1000 [01:52<00:30,  5.65it/s]
+ 83%|████████▎ | 831/1000 [01:52<00:24,  6.97it/s]
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.70it/s]
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.20it/s]
+ 83%|████████▎ | 834/1000 [01:53<00:27,  5.99it/s]
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.52it/s]
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.62it/s]
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.15it/s]
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.24it/s]
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.54it/s]
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.05it/s]
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.53it/s]
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.71it/s]
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.55it/s]
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.29it/s]
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.13it/s]
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.11it/s]
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.15it/s]
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.74it/s]
+ 87%|████████▋ | 874/1000 [01:58<00:17,  7.05it/s]
+ 88%|████████▊ | 876/1000 [01:58<00:14,  8.52it/s]
+ 88%|████████▊ | 878/1000 [01:58<00:15,  7.63it/s]
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.95it/s]
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.80it/s]
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.53it/s]
+ 88%|████████▊ | 884/1000 [01:59<00:14,  8.10it/s]
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.37it/s]
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.58it/s]
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.90it/s]
+ 89%|████████▉ | 892/1000 [02:00<00:09, 10.88it/s]
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.62it/s]
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.86it/s]
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.83it/s]
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.03it/s]
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.04it/s]
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.30it/s]
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.57it/s]
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.75it/s]
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.38it/s]
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.07it/s]
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.16it/s]
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.09it/s]
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.64it/s]
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.81it/s]
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.84it/s]
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.13it/s]
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.38it/s]
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.71it/s]
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.83it/s]
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.42it/s]
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.41it/s]
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.18it/s]
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.78it/s]
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.81it/s]
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.43it/s]
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.37it/s]
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+100%|█████████▉| 999/1000 [02:15<00:00,  6.73it/s]
+100%|██████████| 1000/1000 [02:15<00:00,  6.65it/s]
+100%|██████████| 1000/1000 [02:15<00:00,  7.39it/s]
+***** eval metrics *****
+  epoch                   =        1.0
+  eval_loss               =     0.7413
+  eval_runtime            = 0:02:15.45
+  eval_samples_per_second =      7.382
+  eval_steps_per_second   =      7.382
+  num_input_tokens_seen   =   12201816
+Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
diff --git a/optuna_30k_experiments/logs/trial_006_lr_6.257e-05.log b/optuna_30k_experiments/logs/trial_006_lr_6.257e-05.log
new file mode 100644
index 0000000..fde1a35
--- /dev/null
+++ b/optuna_30k_experiments/logs/trial_006_lr_6.257e-05.log
@@ -0,0 +1,12457 @@
+[INFO|2026-10-15 18:39:46] llamafactory.hparams.parser:651 >> Process rank: 0, world size: 1, device: cuda:0, distributed training: False, compute dtype: torch.bfloat16
+[INFO|configuration_utils.py:765] 2026-10-15 18:39:47,175 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-15 18:39:47,179 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
+[INFO|configuration_utils.py:765] 2026-10-15 18:39:48,887 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-15 18:39:48,890 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|configuration_utils.py:765] 2026-10-15 18:39:49,037 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-15 18:39:49,040 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-15 18:39:49] llamafactory.data.loader:144 >> Loading dataset aya_slovak_hpo_30k.jsonl...
+training example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28828, 2176, 28711, 28863, 29114, 10268, 28709, 6612, 28714, 2892, 493, 363, 6294, 10719, 363, 3811, 28723, 7379, 626, 28889, 307, 14637, 311, 28768, 2218, 22885, 3748, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 284, 8341, 6729, 345, 28738, 28739, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28723, 524, 4634, 28711, 570, 1879, 261, 28863, 532, 613, 14483, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 345, 15962, 28739, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 1879, 15173, 537, 284, 8341, 705, 28723, 367, 280, 28879, 570, 4415, 28716, 28797, 9335, 3175, 29826, 626, 323, 412, 8514, 16986, 29826, 28708, 345, 15962, 28739, 1879, 15173, 537, 1221, 4566, 28718, 325, 28707, 28723, 461, 28723, 724, 494, 8891, 24160, 28725, 8582, 28862, 24160, 28725, 20532, 28764, 271, 28725, 438, 29365, 15745, 1319, 12035, 28862, 8269, 284, 28827, 2762, 28709, 264, 4798, 8359, 1450, 8325, 12121, 1798, 420, 762, 1089, 290, 28906, 18765, 724, 2331, 29114, 1221, 4304, 1780, 363, 13881, 29826, 316, 2085, 28708, 28723, 22627, 28873, 263, 335, 20208, 284, 28827, 2762, 28709, 264, 4798, 5424, 10056, 1286, 290, 28906, 18765, 724, 2331, 29114, 11323, 10719, 12490, 363, 13881, 29826, 316, 284, 8341, 2390, 28723, 22127, 280, 28879, 21022, 1150, 424, 724, 28827, 1144, 28873, 340, 17066, 270, 21022, 485, 284, 8341, 705, 28725, 446, 6909, 28797, 637, 28705, 29114, 28708, 28879, 4798, 6612, 28827, 1100, 28768, 28863, 28723, 320, 412, 28879, 637, 15612, 28716, 28889, 28726, 1150, 424, 19523, 412, 28873, 276, 28797, 2390, 284, 8341, 2390, 363, 23407, 11277, 9538, 2085, 28708, 28723, 9532, 2144, 686, 7865, 28718, 341, 14475, 1687, 28706, 18037, 452, 270, 8269, 284, 8341, 705, 28725, 290, 4827, 486, 2710, 2144, 332, 1800, 16344, 29114, 6588, 269, 28862, 28711, 28863, 686, 1653, 5388, 28718, 710, 261, 1296, 28709, 284, 8341, 705, 28723, 351, 28711, 1371, 28797, 1776, 29826, 8269, 284, 8341, 705, 268, 28863, 1776, 29826, 8269, 1879, 3497, 598, 8269, 8631, 28879, 279, 412, 28725, 12121, 7988, 268, 28863, 24270, 12115, 28797, 1879, 5626, 263, 28862, 8269, 8631, 28879, 279, 412, 28723, 318, 15612, 1473, 276, 15690, 2245, 626, 15690, 307, 14637, 311, 28768, 300, 15469, 28711, 570, 446, 450, 4798, 29826, 333, 28729, 1879, 319, 3976, 294, 405, 28768, 4882, 405, 28725, 534, 28724, 2710, 724, 313, 4827, 2245, 626, 28889, 20532, 28764, 271, 28723, 10650, 300, 12235, 28827, 28873, 424, 268, 27172, 2245, 28723, 1054, 15504, 28879, 424, 724, 313, 8768, 4545, 504, 270, 8269, 2124, 2245, 626, 12689, 4584, 10906, 28718, 710, 18782, 28879, 28715, 28797, 1135, 10356, 28723, 2645, 28764, 12275, 28820, 28873, 29114, 13067, 549, 2892, 5872, 3811, 8631, 25991, 3026, 29114, 408, 28906, 28764, 485, 2082, 1403, 12121, 1798, 1773, 29826, 28729, 22160, 710, 408, 28906, 28764, 485, 1135, 10874, 1776, 4536, 28873, 300, 23407, 11277, 9538, 2085, 28708, 28723, 22127, 3361, 13470, 1150, 317, 28873, 424, 2082, 7330, 511, 2085, 28708, 28723, 401, 492, 13591, 28820, 261, 2220, 28862, 10766, 381, 28827, 486, 29114, 363, 28879, 4689, 3418, 28879, 21775, 29114, 280, 28723, 418, 28820, 11277, 28716, 6294, 26177, 903, 12689, 6612, 20122, 28733, 12868, 291, 2124, 2085, 28708, 1879, 6670, 28862, 9348, 493, 686, 375, 5620, 386, 28862, 28827, 28725, 23374, 4536, 25306, 16388, 957, 450, 15612, 28764, 2126, 29114, 511, 2152, 363, 25769, 14110, 17413, 26177, 903, 28873, 15808, 19396, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 7379, 1879, 277, 7939, 1879, 724, 313, 8768, 2245, 28718, 363, 446, 551, 9538, 12121, 1798, 24473, 21694, 300, 261, 5869, 28723, 384, 28728, 491, 28712, 4566, 15469, 28711, 570, 1879, 2245, 28725, 534, 28724, 2710, 7463, 15612, 13335, 28728, 4371, 1144, 28723, 10650, 300, 15469, 28711, 570, 1879, 1160, 28709, 28879, 3907, 13288, 491, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 28725, 534, 28724, 637, 686, 598, 5237, 5945, 17984, 28718, 2117, 18904, 3146, 515, 13288, 25208, 28723, 367, 280, 28879, 570, 12472, 1006, 5872, 28863, 358, 446, 311, 4845, 1879, 6149, 783, 494, 28718, 2245, 28718, 363, 17984, 28718, 405, 13288, 491, 28723, 13343, 29826, 28729, 385, 29114, 284, 8341, 705, 28747, 399, 3711, 3175, 29826, 626, 12990, 17984, 28718, 2117, 268, 6125, 15690, 345, 28738, 28739, 16986, 29826, 28708, 1773, 29826, 10641, 2124, 345, 28738, 28739, 582, 20524, 12214, 1773, 29826, 28729, 385, 29114, 284, 8341, 705, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 15173, 537, 28718, 325, 849, 3748, 28725, 446, 6909, 28820, 3061, 28720, 300, 28827, 1520, 6612, 749, 1780, 20532, 28764, 271, 586, 28873, 28710, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28731, 1879, 686, 6729, 1773, 29826, 28729, 22160, 284, 8341, 705, 28723, 550, 11245, 412, 28747, 550, 11245, 412, 1683, 28827, 16032, 271, 1342, 4295, 3191, 316, 28729, 6449, 2245, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 345, 28741, 28739, 1879, 22954, 338, 28718, 297, 12689, 345, 28741, 28739, 686, 2129, 28710, 29114, 16986, 28863, 358, 28723, 17563, 971, 28747, 17563, 971, 582, 20524, 12214, 1342, 28764, 1193, 1342, 4295, 5765, 6175, 2959, 15416, 28710, 13288, 491, 3589, 28723, 9532, 484, 28717, 3811, 18904, 17101, 29114, 22399, 971, 28725, 15612, 13335, 28728, 4371, 28711, 570, 13288, 20996, 28725, 446, 6909, 28797, 484, 28717, 3811, 18904, 17101, 29114, 28725, 264, 2513, 300, 8631, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 268, 345, 28741, 28739, 264, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Zhrnúť tento článok v jednej vete. Textový nástroj je ikona, ktorá sa podobá písmenu "T" v paneli nástrojov vľavo. Kliknite na túto ikonu. Použite rozbaľovaciu ponuku vedľa "Character" v hornej časti obrazovky na výber písma. Použite druhé rozbaľovacie menu vedľa "Character" na výber formátu (t. j. pravidelný, tučný, kurzor, atď.). Tradičné písmo ako Times New Roman alebo Garamond môže pridať formálny vzhľad loga. Bezšerifové písmo ako Helvetica môže pridať modernejší vzhľad písmu. Nepoužívajte príliš dekoratívne písma, ktoré sa ťažko čítajú. Tiež sa vyhýbajte zmiešanému písmu v návrhu loga. Ak si z internetu stiahnete bezplatné písma, mali by ste si uvedomiť licenčnú zmluvu pre tieto písma. Mnohé voľné písma sú voľné na osobné použitie, ale nie sú povolené na komerčné použitie. S vybraným textovým nástrojom kliknite kdekoľvek na grafickej doske, aby ste pridali textový kurzor. Potom napíšte svoj text. Zvážte pridanie samostatného textového objektu pre každé slovo. Obzvlášť ak plánujete používať rôzne farby alebo veľkosti pre rôzne slová vo vašom návrhu loga. Nepridávaj ešte farbu do loga. Farebná tlač nemusí byť vždy možnosťou. Návrh jednoduchého čierno-bieleho loga na začiatok zabezpečí, že vaše logo bude vyzerať dobre v jeho najjednoduchšej forme. Môžete tiež použiť nástroj Text na ceste na pridanie textu v kruhu alebo zakrivenom tvare. Dvakrát kliknite na text, aby ste ho vyzdvihli. Potom kliknite na položku znak v hornej časti obrazovky, aby sa zobrazila ponuka nastavenia znakov. Použite nasledujúce kroky na úpravu textu v ponuke znak. Veľkosť písma: Rozbaľovacia ponuka s malým "T" vedľa veľkého "T" upravuje veľkosť písma. Môžete tiež použiť nástroj výberu (ikona, ktorá pripomína čierny kurzor myši v paneli nástrojov vľavo) na zmenu veľkosti písma. Vedenie: Vedenie mení priestor medzi riadkami textu. Použite rozbaľovaciu ponuku vedľa ikony, ktorá sa podobá "A" na vrchu iného "A" zmeniť vedúce. Kerning: Kerning upravuje medzery medzi konkrétnymi znakmi. Ak chcete nastaviť kerning, vyzdvihnite znaky, ktoré chcete nastaviť, a potom použite rozbaľovaciu ponuku vedľa ikony s "A" aVyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+labels:
+Vyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+eval example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28965, 28709, 637, 8631, 25991, 3026, 28725, 13067, 10531, 28729, 350, 2591, 686, 3023, 28862, 28827, 284, 28873, 269, 13096, 28711, 28863, 6149, 3399, 28804, 550, 13881, 29826, 316, 300, 1879, 7871, 338, 28820, 6936, 1150, 28863, 7340, 6957, 8905, 3907, 12235, 28827, 28873, 424, 5765, 772, 28725, 446, 6909, 28889, 14848, 912, 12214, 4321, 28720, 751, 29365, 28723, 351, 28906, 18765, 298, 486, 29114, 28705, 28740, 387, 28705, 28750, 28734, 22116, 28827, 28723, 8294, 772, 28747, 13, 13, 27332, 12107, 28747, 13, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Čo sa používa, ak škodca zničí pšeničnú úrodu? Vzhľadom na predchádzajúcu otázku napíšte kontext, ktorý obsahuje odpoveď. Môže to byť 1 - 20 vetí. Kontext:
+
+### Response:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+labels:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+[INFO|configuration_utils.py:765] 2026-10-15 18:39:50,573 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-15 18:39:50,577 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-15 18:39:50] llamafactory.model.model_utils.kv_cache:144 >> KV cache is disabled during training.
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/unsloth/__init__.py:165: UserWarning: WARNING: Unsloth should be imported before [trl, transformers, peft] to ensure all optimizations are applied. Your code may run slower or encounter memory issues without these optimizations.
+
+Please restructure your imports with 'import unsloth' at the top of your file.
+  from ._gpu_init import *
+🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
+🦥 Unsloth Zoo will now patch everything to make training faster!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: WARNING `trust_remote_code` is True.
+Are you certain you want to do remote code execution?
+==((====))==  Unsloth 2026.6.1: Fast Mistral patching. Transformers: 5.5.0.
+   \\   /|    NVIDIA RTX A4500. Num GPUs = 1. Max memory: 19.573 GB. Platform: Linux.
+O^O/ \_/ \    Torch: 2.6.0+cu124. CUDA: 8.6. CUDA Toolkit: 12.4. Triton: 3.2.0
+\        /    Bfloat16 = TRUE. FA [Xformers = 0.0.29.post3. FA2 = False]
+ "-____-"     Free license: http://github.com/unslothai/unsloth
+Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading weights file model.safetensors from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/model.safetensors.index.json
+Will use dtype=torch.bfloat16 as defined in model's config object
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 2,
+  "use_cache": true
+}
+
+
+Loading weights:   0%|          | 0/291 [00:00<?, ?it/s]
+Loading weights:   0%|          | 1/291 [00:00<01:15,  3.87it/s]
+Loading weights:   1%|          | 2/291 [00:00<01:04,  4.46it/s]
+Loading weights:   1%|▏         | 4/291 [00:00<00:33,  8.60it/s]
+Loading weights:   2%|▏         | 6/291 [00:00<00:30,  9.34it/s]
+Loading weights:   4%|▍         | 13/291 [00:00<00:13, 20.52it/s]
+Loading weights:   5%|▌         | 16/291 [00:01<00:15, 18.21it/s]
+Loading weights:   8%|▊         | 22/291 [00:01<00:11, 22.98it/s]
+Loading weights:   9%|▊         | 25/291 [00:01<00:12, 20.96it/s]
+Loading weights:  11%|█         | 31/291 [00:01<00:10, 25.72it/s]
+Loading weights:  12%|█▏        | 34/291 [00:01<00:11, 21.85it/s]
+Loading weights:  14%|█▎        | 40/291 [00:02<00:09, 25.46it/s]
+Loading weights:  15%|█▍        | 43/291 [00:02<00:11, 22.36it/s]
+Loading weights:  17%|█▋        | 49/291 [00:02<00:09, 26.14it/s]
+Loading weights:  18%|█▊        | 52/291 [00:02<00:10, 22.65it/s]
+Loading weights:  20%|█▉        | 58/291 [00:02<00:08, 26.33it/s]
+Loading weights:  21%|██        | 61/291 [00:02<00:10, 22.89it/s]
+Loading weights:  23%|██▎       | 67/291 [00:03<00:08, 26.43it/s]
+Loading weights:  24%|██▍       | 70/291 [00:03<00:09, 23.16it/s]
+Loading weights:  26%|██▌       | 76/291 [00:03<00:08, 26.67it/s]
+Loading weights:  27%|██▋       | 79/291 [00:03<00:09, 22.96it/s]
+Loading weights:  29%|██▉       | 85/291 [00:03<00:07, 26.67it/s]
+Loading weights:  30%|███       | 88/291 [00:04<00:08, 23.12it/s]
+Loading weights:  32%|███▏      | 94/291 [00:04<00:07, 26.90it/s]
+Loading weights:  33%|███▎      | 97/291 [00:04<00:08, 23.58it/s]
+Loading weights:  35%|███▌      | 103/291 [00:04<00:06, 26.97it/s]
+Loading weights:  36%|███▋      | 106/291 [00:04<00:08, 23.09it/s]
+Loading weights:  38%|███▊      | 112/291 [00:04<00:06, 26.53it/s]
+Loading weights:  40%|███▉      | 115/291 [00:05<00:07, 22.82it/s]
+Loading weights:  42%|████▏     | 121/291 [00:05<00:06, 26.57it/s]
+Loading weights:  43%|████▎     | 124/291 [00:05<00:07, 23.33it/s]
+Loading weights:  45%|████▍     | 130/291 [00:05<00:05, 27.61it/s]
+Loading weights:  46%|████▌     | 133/291 [00:05<00:06, 23.62it/s]
+Loading weights:  48%|████▊     | 139/291 [00:05<00:05, 26.95it/s]
+Loading weights:  49%|████▉     | 142/291 [00:06<00:06, 23.07it/s]
+Loading weights:  51%|█████     | 148/291 [00:06<00:05, 26.33it/s]
+Loading weights:  52%|█████▏    | 151/291 [00:06<00:06, 22.71it/s]
+Loading weights:  54%|█████▍    | 157/291 [00:06<00:05, 26.75it/s]
+Loading weights:  55%|█████▍    | 160/291 [00:06<00:05, 24.01it/s]
+Loading weights:  57%|█████▋    | 166/291 [00:07<00:04, 27.37it/s]
+Loading weights:  58%|█████▊    | 169/291 [00:07<00:05, 23.11it/s]
+Loading weights:  60%|██████    | 175/291 [00:07<00:04, 26.07it/s]
+Loading weights:  61%|██████    | 178/291 [00:07<00:05, 22.01it/s]
+Loading weights:  63%|██████▎   | 184/291 [00:07<00:04, 25.33it/s]
+Loading weights:  64%|██████▍   | 187/291 [00:08<00:04, 22.00it/s]
+Loading weights:  66%|██████▋   | 193/291 [00:08<00:03, 25.10it/s]
+Loading weights:  67%|██████▋   | 196/291 [00:08<00:04, 21.67it/s]
+Loading weights:  69%|██████▉   | 202/291 [00:08<00:03, 24.94it/s]
+Loading weights:  70%|███████   | 205/291 [00:08<00:03, 22.00it/s]
+Loading weights:  73%|███████▎  | 211/291 [00:09<00:03, 25.14it/s]
+Loading weights:  74%|███████▎  | 214/291 [00:09<00:03, 21.26it/s]
+Loading weights:  76%|███████▌  | 220/291 [00:09<00:02, 24.37it/s]
+Loading weights:  77%|███████▋  | 223/291 [00:09<00:03, 20.95it/s]
+Loading weights:  79%|███████▊  | 229/291 [00:09<00:02, 24.18it/s]
+Loading weights:  80%|███████▉  | 232/291 [00:10<00:02, 20.80it/s]
+Loading weights:  82%|████████▏ | 238/291 [00:10<00:02, 23.95it/s]
+Loading weights:  83%|████████▎ | 241/291 [00:10<00:02, 20.41it/s]
+Loading weights:  85%|████████▍ | 247/291 [00:10<00:01, 23.76it/s]
+Loading weights:  86%|████████▌ | 250/291 [00:10<00:02, 20.35it/s]
+Loading weights:  88%|████████▊ | 256/291 [00:11<00:01, 23.41it/s]
+Loading weights:  89%|████████▉ | 259/291 [00:11<00:01, 20.17it/s]
+Loading weights:  91%|█████████ | 265/291 [00:11<00:01, 23.27it/s]
+Loading weights:  92%|█████████▏| 268/291 [00:11<00:01, 20.51it/s]
+Loading weights:  94%|█████████▍| 274/291 [00:11<00:00, 24.01it/s]
+Loading weights:  95%|█████████▌| 277/291 [00:12<00:00, 22.14it/s]
+Loading weights:  97%|█████████▋| 283/291 [00:12<00:00, 27.03it/s]
+Loading weights:  98%|█████████▊| 286/291 [00:12<00:00, 23.58it/s]
+Loading weights: 100%|██████████| 291/291 [00:12<00:00, 23.41it/s]
+loading configuration file generation_config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/generation_config.json
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2
+}
+
+Could not locate the custom_generate/generate.py inside slovak-nlp/mistral-sk-7b.
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: slovak-nlp/mistral-sk-7b had a bad pad_token (</s>). Using pad_token = <unk>.
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:68: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_fwd
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:85: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_bwd
+[INFO|2026-10-15 18:40:18] llamafactory.model.model_utils.checkpointing:144 >> Gradient checkpointing enabled.
+[INFO|2026-10-15 18:40:18] llamafactory.model.adapter:144 >> Upcasting trainable params to float32.
+[INFO|2026-10-15 18:40:18] llamafactory.model.adapter:144 >> Fine-tuning method: LoRA
+Unsloth 2026.6.1 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers.
+[INFO|2026-10-15 18:40:21] llamafactory.model.loader:144 >> trainable params: 41,943,040 || all params: 7,283,675,136 || trainable%: 0.5758
+The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'pad_token_id': 2}.
+==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 1
+   \\   /|    Num examples = 29,000 | Num Epochs = 1 | Total steps = 1,813
+O^O/ \_/ \    Batch size per device = 1 | Gradient accumulation steps = 16
+\        /    Data Parallel GPUs = 1 | Total batch size (1 x 16 x 1) = 16
+ "-____-"     Trainable parameters = 41,943,040 of 7,283,675,136 (0.58% trained)
+
+  0%|          | 0/1813 [00:00<?, ?it/s]`use_return_dict` is deprecated! Use `return_dict` instead!
+Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
+Unsloth: Will smartly offload gradients to save VRAM!
+
+  0%|          | 1/1813 [00:10<5:14:19, 10.41s/it]
+  0%|          | 2/1813 [00:16<4:03:40,  8.07s/it]
+  0%|          | 3/1813 [00:22<3:30:11,  6.97s/it]
+  0%|          | 4/1813 [00:26<2:58:09,  5.91s/it]
+  0%|          | 5/1813 [00:33<3:03:14,  6.08s/it]
+  0%|          | 6/1813 [00:40<3:13:55,  6.44s/it]
+  0%|          | 7/1813 [00:46<3:12:38,  6.40s/it]
+  0%|          | 8/1813 [00:51<3:02:40,  6.07s/it]
+  0%|          | 9/1813 [00:59<3:15:59,  6.52s/it]
+  1%|          | 10/1813 [01:04<3:06:21,  6.20s/it]
+                                                   
+{'loss': '1.801', 'grad_norm': '2.053', 'learning_rate': '3.754e-06', 'epoch': '0.005517', 'num_input_tokens_seen': 65928, 'train_runtime': '64.99', 'train_tokens_per_second': '1014'}
+
+  1%|          | 10/1813 [01:04<3:06:21,  6.20s/it]
+  1%|          | 11/1813 [01:11<3:04:55,  6.16s/it]
+  1%|          | 12/1813 [01:18<3:19:20,  6.64s/it]
+  1%|          | 13/1813 [01:23<2:59:49,  5.99s/it]
+  1%|          | 14/1813 [01:29<3:01:48,  6.06s/it]
+  1%|          | 15/1813 [01:34<2:54:49,  5.83s/it]
+  1%|          | 16/1813 [01:40<2:57:09,  5.92s/it]
+  1%|          | 17/1813 [01:47<3:00:23,  6.03s/it]
+  1%|          | 18/1813 [01:53<3:00:48,  6.04s/it]
+  1%|          | 19/1813 [02:00<3:07:55,  6.29s/it]
+  1%|          | 20/1813 [02:06<3:06:01,  6.22s/it]
+                                                   
+{'loss': '1.599', 'grad_norm': '1.428', 'learning_rate': '7.926e-06', 'epoch': '0.01103', 'num_input_tokens_seen': 131352, 'train_runtime': '126.2', 'train_tokens_per_second': '1041'}
+
+  1%|          | 20/1813 [02:06<3:06:01,  6.22s/it]
+  1%|          | 21/1813 [02:11<3:01:38,  6.08s/it]
+  1%|          | 22/1813 [02:16<2:49:48,  5.69s/it]
+  1%|▏         | 23/1813 [02:21<2:44:41,  5.52s/it]
+  1%|▏         | 24/1813 [02:27<2:45:38,  5.56s/it]
+  1%|▏         | 25/1813 [02:34<2:57:45,  5.96s/it]
+  1%|▏         | 26/1813 [02:41<3:08:01,  6.31s/it]
+  1%|▏         | 27/1813 [02:47<3:02:58,  6.15s/it]
+  2%|▏         | 28/1813 [02:54<3:08:29,  6.34s/it]
+  2%|▏         | 29/1813 [03:01<3:21:53,  6.79s/it]
+  2%|▏         | 30/1813 [03:06<3:04:40,  6.21s/it]
+                                                   
+{'loss': '1.502', 'grad_norm': '1.59', 'learning_rate': '1.21e-05', 'epoch': '0.01655', 'num_input_tokens_seen': 197376, 'train_runtime': '186.8', 'train_tokens_per_second': '1056'}
+
+  2%|▏         | 30/1813 [03:06<3:04:40,  6.21s/it]
+  2%|▏         | 31/1813 [03:13<3:05:14,  6.24s/it]
+  2%|▏         | 32/1813 [03:18<2:56:29,  5.95s/it]
+  2%|▏         | 33/1813 [03:24<2:56:05,  5.94s/it]
+  2%|▏         | 34/1813 [03:31<3:09:41,  6.40s/it]
+  2%|▏         | 35/1813 [03:38<3:10:30,  6.43s/it]
+  2%|▏         | 36/1813 [03:44<3:12:18,  6.49s/it]
+  2%|▏         | 37/1813 [03:50<3:01:44,  6.14s/it]
+  2%|▏         | 38/1813 [03:55<2:53:28,  5.86s/it]
+  2%|▏         | 39/1813 [04:01<2:53:38,  5.87s/it]
+  2%|▏         | 40/1813 [04:07<2:53:56,  5.89s/it]
+                                                   
+{'loss': '1.375', 'grad_norm': '1.015', 'learning_rate': '1.627e-05', 'epoch': '0.02207', 'num_input_tokens_seen': 261168, 'train_runtime': '247.3', 'train_tokens_per_second': '1056'}
+
+  2%|▏         | 40/1813 [04:07<2:53:56,  5.89s/it]
+  2%|▏         | 41/1813 [04:12<2:48:23,  5.70s/it]
+  2%|▏         | 42/1813 [04:18<2:49:44,  5.75s/it]
+  2%|▏         | 43/1813 [04:25<3:05:26,  6.29s/it]
+  2%|▏         | 44/1813 [04:32<3:07:20,  6.35s/it]
+  2%|▏         | 45/1813 [04:39<3:16:16,  6.66s/it]
+  3%|▎         | 46/1813 [04:45<3:08:53,  6.41s/it]
+  3%|▎         | 47/1813 [04:51<3:01:07,  6.15s/it]
+  3%|▎         | 48/1813 [04:58<3:06:51,  6.35s/it]
+  3%|▎         | 49/1813 [05:04<3:07:47,  6.39s/it]
+  3%|▎         | 50/1813 [05:11<3:09:29,  6.45s/it]
+                                                   
+{'loss': '1.381', 'grad_norm': '1.306', 'learning_rate': '2.044e-05', 'epoch': '0.02759', 'num_input_tokens_seen': 330104, 'train_runtime': '311.1', 'train_tokens_per_second': '1061'}
+
+  3%|▎         | 50/1813 [05:11<3:09:29,  6.45s/it]
+  3%|▎         | 51/1813 [05:18<3:19:04,  6.78s/it]
+  3%|▎         | 52/1813 [05:27<3:41:40,  7.55s/it]
+  3%|▎         | 53/1813 [05:33<3:27:55,  7.09s/it]
+  3%|▎         | 54/1813 [05:41<3:30:36,  7.18s/it]
+  3%|▎         | 55/1813 [05:47<3:22:59,  6.93s/it]
+  3%|▎         | 56/1813 [05:54<3:22:24,  6.91s/it]
+  3%|▎         | 57/1813 [05:59<3:05:35,  6.34s/it]
+  3%|▎         | 58/1813 [06:07<3:22:48,  6.93s/it]
+  3%|▎         | 59/1813 [06:14<3:23:30,  6.96s/it]
+  3%|▎         | 60/1813 [06:20<3:11:06,  6.54s/it]
+                                                   
+{'loss': '1.257', 'grad_norm': '1.443', 'learning_rate': '2.461e-05', 'epoch': '0.0331', 'num_input_tokens_seen': 407792, 'train_runtime': '380.5', 'train_tokens_per_second': '1072'}
+
+  3%|▎         | 60/1813 [06:20<3:11:06,  6.54s/it]
+  3%|▎         | 61/1813 [06:25<2:58:50,  6.12s/it]
+  3%|▎         | 62/1813 [06:31<2:55:59,  6.03s/it]
+  3%|▎         | 63/1813 [06:37<2:55:10,  6.01s/it]
+  4%|▎         | 64/1813 [06:43<2:52:36,  5.92s/it]
+  4%|▎         | 65/1813 [06:50<3:03:50,  6.31s/it]
+  4%|▎         | 66/1813 [06:56<3:06:07,  6.39s/it]
+  4%|▎         | 67/1813 [07:02<3:01:32,  6.24s/it]
+  4%|▍         | 68/1813 [07:09<3:01:38,  6.25s/it]
+  4%|▍         | 69/1813 [07:16<3:10:20,  6.55s/it]
+  4%|▍         | 70/1813 [07:23<3:17:54,  6.81s/it]
+                                                   
+{'loss': '1.133', 'grad_norm': '1.247', 'learning_rate': '2.878e-05', 'epoch': '0.03862', 'num_input_tokens_seen': 474368, 'train_runtime': '443.8', 'train_tokens_per_second': '1069'}
+
+  4%|▍         | 70/1813 [07:23<3:17:54,  6.81s/it]
+  4%|▍         | 71/1813 [07:29<3:06:51,  6.44s/it]
+  4%|▍         | 72/1813 [07:38<3:26:27,  7.12s/it]
+  4%|▍         | 73/1813 [07:43<3:13:16,  6.66s/it]
+  4%|▍         | 74/1813 [07:50<3:10:27,  6.57s/it]
+  4%|▍         | 75/1813 [07:56<3:12:37,  6.65s/it]
+  4%|▍         | 76/1813 [08:02<3:08:14,  6.50s/it]
+  4%|▍         | 77/1813 [08:09<3:12:02,  6.64s/it]
+  4%|▍         | 78/1813 [08:15<3:01:21,  6.27s/it]
+  4%|▍         | 79/1813 [08:20<2:54:30,  6.04s/it]
+  4%|▍         | 80/1813 [08:26<2:54:55,  6.06s/it]
+                                                   
+{'loss': '1.16', 'grad_norm': '1.337', 'learning_rate': '3.296e-05', 'epoch': '0.04414', 'num_input_tokens_seen': 541616, 'train_runtime': '507', 'train_tokens_per_second': '1068'}
+
+  4%|▍         | 80/1813 [08:26<2:54:55,  6.06s/it]
+  4%|▍         | 81/1813 [08:32<2:53:23,  6.01s/it]
+  5%|▍         | 82/1813 [08:39<3:01:14,  6.28s/it]
+  5%|▍         | 83/1813 [08:46<3:06:24,  6.47s/it]
+  5%|▍         | 84/1813 [08:53<3:09:44,  6.58s/it]
+  5%|▍         | 85/1813 [08:59<3:04:29,  6.41s/it]
+  5%|▍         | 86/1813 [09:06<3:05:11,  6.43s/it]
+  5%|▍         | 87/1813 [09:11<2:54:16,  6.06s/it]
+  5%|▍         | 88/1813 [09:17<2:55:58,  6.12s/it]
+  5%|▍         | 89/1813 [09:23<2:56:00,  6.13s/it]
+  5%|▍         | 90/1813 [09:30<2:59:26,  6.25s/it]
+                                                   
+{'loss': '1.081', 'grad_norm': '1.121', 'learning_rate': '3.713e-05', 'epoch': '0.04966', 'num_input_tokens_seen': 608792, 'train_runtime': '570.1', 'train_tokens_per_second': '1068'}
+
+  5%|▍         | 90/1813 [09:30<2:59:26,  6.25s/it]
+  5%|▌         | 91/1813 [09:36<2:56:29,  6.15s/it]
+  5%|▌         | 92/1813 [09:41<2:47:41,  5.85s/it]
+  5%|▌         | 93/1813 [09:47<2:49:50,  5.92s/it]
+  5%|▌         | 94/1813 [09:53<2:55:04,  6.11s/it]
+  5%|▌         | 95/1813 [10:00<2:59:18,  6.26s/it]
+  5%|▌         | 96/1813 [10:06<2:57:43,  6.21s/it]
+  5%|▌         | 97/1813 [10:13<3:01:16,  6.34s/it]
+  5%|▌         | 98/1813 [10:18<2:52:48,  6.05s/it]
+  5%|▌         | 99/1813 [10:24<2:51:09,  5.99s/it]
+  6%|▌         | 100/1813 [10:30<2:55:32,  6.15s/it]
+                                                    
+{'loss': '1.136', 'grad_norm': '1.102', 'learning_rate': '4.13e-05', 'epoch': '0.05517', 'num_input_tokens_seen': 670936, 'train_runtime': '630.9', 'train_tokens_per_second': '1063'}
+
+  6%|▌         | 100/1813 [10:30<2:55:32,  6.15s/it]
+  6%|▌         | 101/1813 [10:37<3:02:03,  6.38s/it]
+  6%|▌         | 102/1813 [10:44<3:05:03,  6.49s/it]
+  6%|▌         | 103/1813 [10:50<3:02:42,  6.41s/it]
+  6%|▌         | 104/1813 [10:57<3:02:14,  6.40s/it]
+  6%|▌         | 105/1813 [11:02<2:55:08,  6.15s/it]
+  6%|▌         | 106/1813 [11:08<2:54:39,  6.14s/it]
+  6%|▌         | 107/1813 [11:15<2:57:33,  6.24s/it]
+  6%|▌         | 108/1813 [11:21<2:59:13,  6.31s/it]
+  6%|▌         | 109/1813 [11:27<2:57:19,  6.24s/it]
+  6%|▌         | 110/1813 [11:34<3:01:16,  6.39s/it]
+                                                    
+{'loss': '1.132', 'grad_norm': '1.014', 'learning_rate': '4.547e-05', 'epoch': '0.06069', 'num_input_tokens_seen': 738632, 'train_runtime': '694.6', 'train_tokens_per_second': '1063'}
+
+  6%|▌         | 110/1813 [11:34<3:01:16,  6.39s/it]
+  6%|▌         | 111/1813 [11:41<3:03:58,  6.49s/it]
+  6%|▌         | 112/1813 [11:48<3:05:34,  6.55s/it]
+  6%|▌         | 113/1813 [11:54<3:02:05,  6.43s/it]
+  6%|▋         | 114/1813 [12:01<3:10:36,  6.73s/it]
+  6%|▋         | 115/1813 [12:07<3:02:37,  6.45s/it]
+  6%|▋         | 116/1813 [12:14<3:05:45,  6.57s/it]
+  6%|▋         | 117/1813 [12:19<2:56:02,  6.23s/it]
+  7%|▋         | 118/1813 [12:25<2:52:25,  6.10s/it]
+  7%|▋         | 119/1813 [12:31<2:54:09,  6.17s/it]
+  7%|▋         | 120/1813 [12:38<2:55:08,  6.21s/it]
+                                                    
+{'loss': '1.147', 'grad_norm': '1.58', 'learning_rate': '4.964e-05', 'epoch': '0.06621', 'num_input_tokens_seen': 808824, 'train_runtime': '758.1', 'train_tokens_per_second': '1067'}
+
+  7%|▋         | 120/1813 [12:38<2:55:08,  6.21s/it]
+  7%|▋         | 121/1813 [12:42<2:38:49,  5.63s/it]
+  7%|▋         | 122/1813 [12:47<2:34:26,  5.48s/it]
+  7%|▋         | 123/1813 [12:54<2:44:20,  5.83s/it]
+  7%|▋         | 124/1813 [12:59<2:42:45,  5.78s/it]
+  7%|▋         | 125/1813 [13:06<2:51:04,  6.08s/it]
+  7%|▋         | 126/1813 [13:12<2:52:47,  6.15s/it]
+  7%|▋         | 127/1813 [13:17<2:39:22,  5.67s/it]
+  7%|▋         | 128/1813 [13:22<2:29:49,  5.34s/it]
+  7%|▋         | 129/1813 [13:30<2:52:24,  6.14s/it]
+  7%|▋         | 130/1813 [13:36<2:57:32,  6.33s/it]
+                                                    
+{'loss': '1.083', 'grad_norm': '0.9444', 'learning_rate': '5.381e-05', 'epoch': '0.07172', 'num_input_tokens_seen': 869720, 'train_runtime': '816.9', 'train_tokens_per_second': '1065'}
+
+  7%|▋         | 130/1813 [13:36<2:57:32,  6.33s/it]
+  7%|▋         | 131/1813 [13:42<2:53:14,  6.18s/it]
+  7%|▋         | 132/1813 [13:47<2:44:47,  5.88s/it]
+  7%|▋         | 133/1813 [13:53<2:43:29,  5.84s/it]
+  7%|▋         | 134/1813 [13:58<2:37:53,  5.64s/it]
+  7%|▋         | 135/1813 [14:04<2:34:40,  5.53s/it]
+  8%|▊         | 136/1813 [14:09<2:34:57,  5.54s/it]
+  8%|▊         | 137/1813 [14:15<2:37:48,  5.65s/it]
+  8%|▊         | 138/1813 [14:22<2:49:42,  6.08s/it]
+  8%|▊         | 139/1813 [14:29<2:54:29,  6.25s/it]
+  8%|▊         | 140/1813 [14:36<3:01:20,  6.50s/it]
+                                                    
+{'loss': '1.072', 'grad_norm': '1.354', 'learning_rate': '5.799e-05', 'epoch': '0.07724', 'num_input_tokens_seen': 932000, 'train_runtime': '876.4', 'train_tokens_per_second': '1063'}
+
+  8%|▊         | 140/1813 [14:36<3:01:20,  6.50s/it]
+  8%|▊         | 141/1813 [14:42<2:54:45,  6.27s/it]
+  8%|▊         | 142/1813 [14:50<3:09:19,  6.80s/it]
+  8%|▊         | 143/1813 [14:55<2:57:52,  6.39s/it]
+  8%|▊         | 144/1813 [15:00<2:47:11,  6.01s/it]
+  8%|▊         | 145/1813 [15:07<2:55:31,  6.31s/it]
+  8%|▊         | 146/1813 [15:13<2:53:54,  6.26s/it]
+  8%|▊         | 147/1813 [15:19<2:48:15,  6.06s/it]
+  8%|▊         | 148/1813 [15:25<2:48:48,  6.08s/it]
+  8%|▊         | 149/1813 [15:31<2:49:03,  6.10s/it]
+  8%|▊         | 150/1813 [15:37<2:50:04,  6.14s/it]
+                                                    
+{'loss': '1.124', 'grad_norm': '1.193', 'learning_rate': '6.216e-05', 'epoch': '0.08276', 'num_input_tokens_seen': 997296, 'train_runtime': '937.9', 'train_tokens_per_second': '1063'}
+
+  8%|▊         | 150/1813 [15:37<2:50:04,  6.14s/it]
+  8%|▊         | 151/1813 [15:43<2:44:31,  5.94s/it]
+  8%|▊         | 152/1813 [15:48<2:41:18,  5.83s/it]
+  8%|▊         | 153/1813 [15:54<2:41:21,  5.83s/it]
+  8%|▊         | 154/1813 [16:01<2:47:04,  6.04s/it]
+  9%|▊         | 155/1813 [16:08<2:56:26,  6.39s/it]
+  9%|▊         | 156/1813 [16:14<2:52:53,  6.26s/it]
+  9%|▊         | 157/1813 [16:20<2:52:13,  6.24s/it]
+  9%|▊         | 158/1813 [16:25<2:42:58,  5.91s/it]
+  9%|▉         | 159/1813 [16:31<2:43:21,  5.93s/it]
+  9%|▉         | 160/1813 [16:37<2:42:35,  5.90s/it]
+                                                    
+{'loss': '1.051', 'grad_norm': '1.745', 'learning_rate': '6.257e-05', 'epoch': '0.08828', 'num_input_tokens_seen': 1060096, 'train_runtime': '997.6', 'train_tokens_per_second': '1063'}
+
+  9%|▉         | 160/1813 [16:37<2:42:35,  5.90s/it]
+  9%|▉         | 161/1813 [16:43<2:41:50,  5.88s/it]
+  9%|▉         | 162/1813 [16:47<2:30:11,  5.46s/it]
+  9%|▉         | 163/1813 [16:54<2:39:54,  5.81s/it]
+  9%|▉         | 164/1813 [17:02<2:52:59,  6.29s/it]
+  9%|▉         | 165/1813 [17:08<2:52:56,  6.30s/it]
+  9%|▉         | 166/1813 [17:16<3:06:08,  6.78s/it]
+  9%|▉         | 167/1813 [17:22<3:04:46,  6.74s/it]
+  9%|▉         | 168/1813 [17:30<3:08:09,  6.86s/it]
+  9%|▉         | 169/1813 [17:35<2:58:08,  6.50s/it]
+  9%|▉         | 170/1813 [17:42<2:59:14,  6.55s/it]
+                                                    
+{'loss': '1.07', 'grad_norm': '0.7644', 'learning_rate': '6.255e-05', 'epoch': '0.09379', 'num_input_tokens_seen': 1130368, 'train_runtime': '1062', 'train_tokens_per_second': '1064'}
+
+  9%|▉         | 170/1813 [17:42<2:59:14,  6.55s/it]
+  9%|▉         | 171/1813 [17:47<2:49:09,  6.18s/it]
+  9%|▉         | 172/1813 [17:53<2:42:21,  5.94s/it]
+ 10%|▉         | 173/1813 [17:59<2:46:42,  6.10s/it]
+ 10%|▉         | 174/1813 [18:05<2:43:38,  5.99s/it]
+ 10%|▉         | 175/1813 [18:11<2:45:19,  6.06s/it]
+ 10%|▉         | 176/1813 [18:16<2:37:31,  5.77s/it]
+ 10%|▉         | 177/1813 [18:23<2:50:41,  6.26s/it]
+ 10%|▉         | 178/1813 [18:28<2:39:11,  5.84s/it]
+ 10%|▉         | 179/1813 [18:34<2:39:17,  5.85s/it]
+ 10%|▉         | 180/1813 [18:40<2:42:16,  5.96s/it]
+                                                    
+{'loss': '1.056', 'grad_norm': '1.351', 'learning_rate': '6.253e-05', 'epoch': '0.09931', 'num_input_tokens_seen': 1193192, 'train_runtime': '1121', 'train_tokens_per_second': '1064'}
+
+ 10%|▉         | 180/1813 [18:40<2:42:16,  5.96s/it]
+ 10%|▉         | 181/1813 [18:46<2:41:36,  5.94s/it]
+ 10%|█         | 182/1813 [18:52<2:39:41,  5.87s/it]
+ 10%|█         | 183/1813 [18:58<2:38:56,  5.85s/it]
+ 10%|█         | 184/1813 [19:04<2:38:33,  5.84s/it]
+ 10%|█         | 185/1813 [19:12<2:57:14,  6.53s/it]
+ 10%|█         | 186/1813 [19:18<2:55:53,  6.49s/it]
+ 10%|█         | 187/1813 [19:24<2:52:22,  6.36s/it]
+ 10%|█         | 188/1813 [19:29<2:42:56,  6.02s/it]
+ 10%|█         | 189/1813 [19:36<2:47:36,  6.19s/it]
+ 10%|█         | 190/1813 [19:43<2:51:14,  6.33s/it]
+                                                    
+{'loss': '1', 'grad_norm': '1.221', 'learning_rate': '6.249e-05', 'epoch': '0.1048', 'num_input_tokens_seen': 1259336, 'train_runtime': '1183', 'train_tokens_per_second': '1064'}
+
+ 10%|█         | 190/1813 [19:43<2:51:14,  6.33s/it]
+ 11%|█         | 191/1813 [19:48<2:43:11,  6.04s/it]
+ 11%|█         | 192/1813 [19:55<2:46:56,  6.18s/it]
+ 11%|█         | 193/1813 [20:01<2:50:34,  6.32s/it]
+ 11%|█         | 194/1813 [20:07<2:47:18,  6.20s/it]
+ 11%|█         | 195/1813 [20:15<2:58:24,  6.62s/it]
+ 11%|█         | 196/1813 [20:20<2:48:59,  6.27s/it]
+ 11%|█         | 197/1813 [20:27<2:56:05,  6.54s/it]
+ 11%|█         | 198/1813 [20:35<3:02:57,  6.80s/it]
+ 11%|█         | 199/1813 [20:42<3:02:37,  6.79s/it]
+ 11%|█         | 200/1813 [20:48<3:01:47,  6.76s/it]
+                                                    
+{'loss': '1.105', 'grad_norm': '1.371', 'learning_rate': '6.244e-05', 'epoch': '0.1103', 'num_input_tokens_seen': 1331224, 'train_runtime': '1249', 'train_tokens_per_second': '1066'}
+
+ 11%|█         | 200/1813 [20:48<3:01:47,  6.76s/it]
+ 11%|█         | 201/1813 [20:54<2:53:22,  6.45s/it]
+ 11%|█         | 202/1813 [20:59<2:44:06,  6.11s/it]
+ 11%|█         | 203/1813 [21:04<2:35:43,  5.80s/it]
+ 11%|█▏        | 204/1813 [21:11<2:45:32,  6.17s/it]
+ 11%|█▏        | 205/1813 [21:17<2:41:02,  6.01s/it]
+ 11%|█▏        | 206/1813 [21:23<2:41:57,  6.05s/it]
+ 11%|█▏        | 207/1813 [21:29<2:40:17,  5.99s/it]
+ 11%|█▏        | 208/1813 [21:37<2:54:11,  6.51s/it]
+ 12%|█▏        | 209/1813 [21:42<2:43:17,  6.11s/it]
+ 12%|█▏        | 210/1813 [21:48<2:44:03,  6.14s/it]
+                                                    
+{'loss': '0.9826', 'grad_norm': '1.437', 'learning_rate': '6.238e-05', 'epoch': '0.1159', 'num_input_tokens_seen': 1392472, 'train_runtime': '1309', 'train_tokens_per_second': '1064'}
+
+ 12%|█▏        | 210/1813 [21:48<2:44:03,  6.14s/it]
+ 12%|█▏        | 211/1813 [21:57<3:04:28,  6.91s/it]
+ 12%|█▏        | 212/1813 [22:03<2:55:14,  6.57s/it]
+ 12%|█▏        | 213/1813 [22:08<2:46:48,  6.26s/it]
+ 12%|█▏        | 214/1813 [22:13<2:38:54,  5.96s/it]
+ 12%|█▏        | 215/1813 [22:19<2:36:54,  5.89s/it]
+ 12%|█▏        | 216/1813 [22:24<2:28:18,  5.57s/it]
+ 12%|█▏        | 217/1813 [22:29<2:27:00,  5.53s/it]
+ 12%|█▏        | 218/1813 [22:35<2:31:34,  5.70s/it]
+ 12%|█▏        | 219/1813 [22:41<2:32:55,  5.76s/it]
+ 12%|█▏        | 220/1813 [22:47<2:35:41,  5.86s/it]
+                                                    
+{'loss': '1.056', 'grad_norm': '0.9344', 'learning_rate': '6.231e-05', 'epoch': '0.1214', 'num_input_tokens_seen': 1454040, 'train_runtime': '1368', 'train_tokens_per_second': '1063'}
+
+ 12%|█▏        | 220/1813 [22:47<2:35:41,  5.86s/it]
+ 12%|█▏        | 221/1813 [22:54<2:40:45,  6.06s/it]
+ 12%|█▏        | 222/1813 [22:59<2:35:07,  5.85s/it]
+ 12%|█▏        | 223/1813 [23:07<2:50:35,  6.44s/it]
+ 12%|█▏        | 224/1813 [23:12<2:41:39,  6.10s/it]
+ 12%|█▏        | 225/1813 [23:20<2:50:19,  6.44s/it]
+ 12%|█▏        | 226/1813 [23:27<2:53:23,  6.56s/it]
+ 13%|█▎        | 227/1813 [23:32<2:46:28,  6.30s/it]
+ 13%|█▎        | 228/1813 [23:39<2:47:10,  6.33s/it]
+ 13%|█▎        | 229/1813 [23:46<2:52:34,  6.54s/it]
+ 13%|█▎        | 230/1813 [23:51<2:41:56,  6.14s/it]
+                                                    
+{'loss': '1.025', 'grad_norm': '1.546', 'learning_rate': '6.223e-05', 'epoch': '0.1269', 'num_input_tokens_seen': 1522056, 'train_runtime': '1431', 'train_tokens_per_second': '1063'}
+
+ 13%|█▎        | 230/1813 [23:51<2:41:56,  6.14s/it]
+ 13%|█▎        | 231/1813 [23:56<2:33:53,  5.84s/it]
+ 13%|█▎        | 232/1813 [24:04<2:47:10,  6.34s/it]
+ 13%|█▎        | 233/1813 [24:09<2:38:36,  6.02s/it]
+ 13%|█▎        | 234/1813 [24:14<2:33:22,  5.83s/it]
+ 13%|█▎        | 235/1813 [24:21<2:41:59,  6.16s/it]
+ 13%|█▎        | 236/1813 [24:29<2:51:55,  6.54s/it]
+ 13%|█▎        | 237/1813 [24:35<2:53:22,  6.60s/it]
+ 13%|█▎        | 238/1813 [24:41<2:45:41,  6.31s/it]
+ 13%|█▎        | 239/1813 [24:49<2:55:52,  6.70s/it]
+ 13%|█▎        | 240/1813 [24:55<2:50:34,  6.51s/it]
+                                                    
+{'loss': '1.014', 'grad_norm': '0.9715', 'learning_rate': '6.213e-05', 'epoch': '0.1324', 'num_input_tokens_seen': 1590200, 'train_runtime': '1495', 'train_tokens_per_second': '1064'}
+
+ 13%|█▎        | 240/1813 [24:55<2:50:34,  6.51s/it]
+ 13%|█▎        | 241/1813 [25:01<2:48:24,  6.43s/it]
+ 13%|█▎        | 242/1813 [25:07<2:45:35,  6.32s/it]
+ 13%|█▎        | 243/1813 [25:12<2:39:52,  6.11s/it]
+ 13%|█▎        | 244/1813 [25:20<2:48:20,  6.44s/it]
+ 14%|█▎        | 245/1813 [25:27<2:54:37,  6.68s/it]
+ 14%|█▎        | 246/1813 [25:32<2:44:39,  6.30s/it]
+ 14%|█▎        | 247/1813 [25:39<2:48:55,  6.47s/it]
+ 14%|█▎        | 248/1813 [25:47<2:58:25,  6.84s/it]
+ 14%|█▎        | 249/1813 [25:52<2:46:10,  6.37s/it]
+ 14%|█▍        | 250/1813 [25:58<2:39:17,  6.12s/it]
+                                                    
+{'loss': '1.102', 'grad_norm': '1.139', 'learning_rate': '6.203e-05', 'epoch': '0.1379', 'num_input_tokens_seen': 1658648, 'train_runtime': '1558', 'train_tokens_per_second': '1064'}
+
+ 14%|█▍        | 250/1813 [25:58<2:39:17,  6.12s/it]
+ 14%|█▍        | 251/1813 [26:02<2:27:30,  5.67s/it]
+ 14%|█▍        | 252/1813 [26:08<2:24:35,  5.56s/it]
+ 14%|█▍        | 253/1813 [26:14<2:30:51,  5.80s/it]
+ 14%|█▍        | 254/1813 [26:21<2:43:21,  6.29s/it]
+ 14%|█▍        | 255/1813 [26:28<2:47:22,  6.45s/it]
+ 14%|█▍        | 256/1813 [26:34<2:38:45,  6.12s/it]
+ 14%|█▍        | 257/1813 [26:40<2:42:56,  6.28s/it]
+ 14%|█▍        | 258/1813 [26:47<2:47:15,  6.45s/it]
+ 14%|█▍        | 259/1813 [26:52<2:34:48,  5.98s/it]
+ 14%|█▍        | 260/1813 [26:58<2:36:36,  6.05s/it]
+                                                    
+{'loss': '0.9103', 'grad_norm': '1.616', 'learning_rate': '6.191e-05', 'epoch': '0.1434', 'num_input_tokens_seen': 1722792, 'train_runtime': '1619', 'train_tokens_per_second': '1064'}
+
+ 14%|█▍        | 260/1813 [26:58<2:36:36,  6.05s/it]
+ 14%|█▍        | 261/1813 [27:06<2:46:09,  6.42s/it]
+ 14%|█▍        | 262/1813 [27:12<2:48:58,  6.54s/it]
+ 15%|█▍        | 263/1813 [27:19<2:50:13,  6.59s/it]
+ 15%|█▍        | 264/1813 [27:25<2:46:36,  6.45s/it]
+ 15%|█▍        | 265/1813 [27:31<2:42:26,  6.30s/it]
+ 15%|█▍        | 266/1813 [27:36<2:29:29,  5.80s/it]
+ 15%|█▍        | 267/1813 [27:41<2:27:40,  5.73s/it]
+ 15%|█▍        | 268/1813 [27:47<2:25:01,  5.63s/it]
+ 15%|█▍        | 269/1813 [27:53<2:31:03,  5.87s/it]
+ 15%|█▍        | 270/1813 [27:59<2:33:19,  5.96s/it]
+                                                    
+{'loss': '0.9379', 'grad_norm': '1.103', 'learning_rate': '6.179e-05', 'epoch': '0.149', 'num_input_tokens_seen': 1786808, 'train_runtime': '1680', 'train_tokens_per_second': '1064'}
+
+ 15%|█▍        | 270/1813 [27:59<2:33:19,  5.96s/it]
+ 15%|█▍        | 271/1813 [28:07<2:48:44,  6.57s/it]
+ 15%|█▌        | 272/1813 [28:13<2:40:09,  6.24s/it]
+ 15%|█▌        | 273/1813 [28:21<2:56:13,  6.87s/it]
+ 15%|█▌        | 274/1813 [28:28<2:55:42,  6.85s/it]
+ 15%|█▌        | 275/1813 [28:33<2:43:59,  6.40s/it]
+ 15%|█▌        | 276/1813 [28:39<2:36:03,  6.09s/it]
+ 15%|█▌        | 277/1813 [28:45<2:37:55,  6.17s/it]
+ 15%|█▌        | 278/1813 [28:51<2:38:48,  6.21s/it]
+ 15%|█▌        | 279/1813 [28:59<2:50:00,  6.65s/it]
+ 15%|█▌        | 280/1813 [29:05<2:48:29,  6.59s/it]
+                                                    
+{'loss': '1.012', 'grad_norm': '1.059', 'learning_rate': '6.165e-05', 'epoch': '0.1545', 'num_input_tokens_seen': 1858112, 'train_runtime': '1746', 'train_tokens_per_second': '1064'}
+
+ 15%|█▌        | 280/1813 [29:05<2:48:29,  6.59s/it]
+ 15%|█▌        | 281/1813 [29:11<2:41:06,  6.31s/it]
+ 16%|█▌        | 282/1813 [29:17<2:35:05,  6.08s/it]
+ 16%|█▌        | 283/1813 [29:22<2:29:59,  5.88s/it]
+ 16%|█▌        | 284/1813 [29:28<2:29:31,  5.87s/it]
+ 16%|█▌        | 285/1813 [29:35<2:36:26,  6.14s/it]
+ 16%|█▌        | 286/1813 [29:41<2:34:47,  6.08s/it]
+ 16%|█▌        | 287/1813 [29:47<2:35:54,  6.13s/it]
+ 16%|█▌        | 288/1813 [29:52<2:32:10,  5.99s/it]
+ 16%|█▌        | 289/1813 [29:59<2:36:04,  6.14s/it]
+ 16%|█▌        | 290/1813 [30:06<2:38:49,  6.26s/it]
+                                                    
+{'loss': '0.9835', 'grad_norm': '1.006', 'learning_rate': '6.15e-05', 'epoch': '0.16', 'num_input_tokens_seen': 1920600, 'train_runtime': '1806', 'train_tokens_per_second': '1063'}
+
+ 16%|█▌        | 290/1813 [30:06<2:38:49,  6.26s/it]
+ 16%|█▌        | 291/1813 [30:12<2:43:43,  6.45s/it]
+ 16%|█▌        | 292/1813 [30:20<2:49:20,  6.68s/it]
+ 16%|█▌        | 293/1813 [30:25<2:41:27,  6.37s/it]
+ 16%|█▌        | 294/1813 [30:32<2:47:12,  6.60s/it]
+ 16%|█▋        | 295/1813 [30:40<2:52:18,  6.81s/it]
+ 16%|█▋        | 296/1813 [30:45<2:42:38,  6.43s/it]
+ 16%|█▋        | 297/1813 [30:51<2:39:20,  6.31s/it]
+ 16%|█▋        | 298/1813 [30:56<2:27:11,  5.83s/it]
+ 16%|█▋        | 299/1813 [31:02<2:29:13,  5.91s/it]
+ 17%|█▋        | 300/1813 [31:08<2:29:20,  5.92s/it]
+                                                    
+{'loss': '1.053', 'grad_norm': '1.038', 'learning_rate': '6.134e-05', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312, 'train_runtime': '1869', 'train_tokens_per_second': '1064'}
+
+ 17%|█▋        | 300/1813 [31:08<2:29:20,  5.92s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 3/1000 [00:00<01:02, 15.86it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.25it/s]
+
+  1%|          | 7/1000 [00:00<01:58,  8.41it/s]
+
+  1%|          | 9/1000 [00:01<02:32,  6.49it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.09it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.02it/s]
+
+  1%|▏         | 14/1000 [00:01<02:23,  6.86it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.98it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.06it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.16it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.61it/s]
+
+  2%|▏         | 22/1000 [00:03<02:38,  6.18it/s]
+
+  2%|▏         | 23/1000 [00:03<02:55,  5.56it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.34it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 31/1000 [00:04<03:00,  5.37it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  6.01it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.59it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.76it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.38it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.76it/s]
+
+  5%|▌         | 51/1000 [00:07<01:39,  9.56it/s]
+
+  5%|▌         | 53/1000 [00:07<01:28, 10.64it/s]
+
+  6%|▌         | 55/1000 [00:07<01:29, 10.56it/s]
+
+  6%|▌         | 57/1000 [00:07<01:42,  9.21it/s]
+
+  6%|▌         | 58/1000 [00:07<01:43,  9.12it/s]
+
+  6%|▌         | 59/1000 [00:08<01:59,  7.86it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:35,  9.83it/s]
+
+  6%|▋         | 65/1000 [00:08<01:52,  8.30it/s]
+
+  7%|▋         | 67/1000 [00:09<02:04,  7.49it/s]
+
+  7%|▋         | 69/1000 [00:09<01:53,  8.23it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.95it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.69it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.21it/s]
+
+  8%|▊         | 84/1000 [00:11<01:53,  8.05it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.96it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.26it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:26,  6.18it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.53it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:12,  6.80it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.38it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.64it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.76it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.83it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.34it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.47it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.23it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.05it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.55it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.83it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.25it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.11it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.72it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.16it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.65it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.90it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.05it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.91it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.09it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:40,  8.45it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.51it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.94it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.07it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.43it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.67it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.99it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.63it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.83it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.42it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:26,  5.69it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.19it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:23,  5.79it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.86it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.16it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.23it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.75it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.46it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.82it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.59it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.52it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:22,  5.73it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:44,  7.76it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:42,  7.92it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.40it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.13it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:01,  6.65it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.51it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:05,  6.40it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.77it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.22it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.71it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.80it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.65it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.94it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 210/1000 [00:27<01:32,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.69it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.75it/s]
+
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.16it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.79it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.32it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.56it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.06it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:06,  6.11it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.43it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.92it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.39it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.05it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.24it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16, 10.00it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.49it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.99it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.38it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.03it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.27it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.87it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:05,  5.97it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.04it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.76it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:40,  7.38it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.70it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.70it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.54it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.45it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.61it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:45,  6.93it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:36,  7.56it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.71it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.07it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.76it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.80it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.01it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.20it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:41,  6.90it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.20it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.48it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.13it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.78it/s]
+
+ 30%|███       | 304/1000 [00:40<01:25,  8.10it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.17it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.97it/s]
+
+ 31%|███       | 307/1000 [00:41<01:41,  6.81it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.82it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.22it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.28it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.57it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.76it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.71it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.59it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.97it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.39it/s]
+
+ 34%|███▎      | 336/1000 [00:44<01:22,  8.06it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.84it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.62it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.74it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.23it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.37it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.56it/s]
+
+ 36%|███▌      | 361/1000 [00:47<01:07,  9.41it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:48<02:08,  4.93it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.38it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.04it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.57it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.74it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.23it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.93it/s]
+
+ 38%|███▊      | 380/1000 [00:50<01:10,  8.76it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.21it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.20it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.15it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.66it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.38it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:08,  8.80it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.74it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.50it/s]
+
+ 41%|████      | 408/1000 [00:54<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.46it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.85it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.37it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.07it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.26it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.51it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.70it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.94it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.38it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.97it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.55it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.85it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.56it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.03it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:48, 11.74it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.48it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.31it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.26it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.74it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.20it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:37,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.20it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.52it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.82it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.28it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.84it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.64it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.76it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.02it/s]
+
+ 47%|████▋     | 467/1000 [01:02<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:23,  6.32it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.87it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.22it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.37it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:25,  6.19it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.19it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.30it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.09it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.52it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.31it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.01it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.23it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.29it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.42it/s]
+
+ 50%|████▉     | 498/1000 [01:06<00:59,  8.45it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.94it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.56it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:12,  6.82it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.58it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.82it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.45it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.98it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:07,  7.12it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.09it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.05it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.69it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:21,  5.82it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.57it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.55it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.42it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.14it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.71it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.77it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.95it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.51it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.71it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.93it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.52it/s]
+
+ 55%|█████▌    | 552/1000 [01:14<01:01,  7.28it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.20it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.97it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.04it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.86it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.07it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.40it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.99it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.76it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.57it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.46it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.32it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.36it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.85it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.26it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.99it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.57it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.61it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.93it/s]
+
+ 59%|█████▊    | 587/1000 [01:19<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.76it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:50,  8.16it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.40it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.49it/s]
+
+ 60%|██████    | 601/1000 [01:21<01:02,  6.41it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.72it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.70it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.12it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.86it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:45,  8.49it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.74it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.82it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.64it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.11it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.05it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.22it/s]
+
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.47it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.20it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.23it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.87it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:39,  9.41it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.18it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.60it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:27<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.58it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.34it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.72it/s]
+
+ 66%|██████▌   | 656/1000 [01:28<00:38,  8.93it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.14it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.58it/s]
+
+ 66%|██████▌   | 662/1000 [01:29<00:52,  6.50it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.79it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.34it/s]
+
+ 67%|██████▋   | 667/1000 [01:30<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.21it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.65it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.70it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.96it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:32<00:31,  9.90it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:33<00:45,  6.83it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.86it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.28it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.44it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.35it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.96it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.06it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.62it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:35<00:40,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.51it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.70it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.37it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.95it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.49it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:33,  8.18it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.27it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:32,  8.34it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.12it/s]
+
+ 73%|███████▎  | 729/1000 [01:38<00:24, 11.10it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.18it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.40it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:27,  9.81it/s]
+
+ 74%|███████▎  | 737/1000 [01:39<00:27,  9.44it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.74it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.59it/s]
+
+ 75%|███████▍  | 747/1000 [01:40<00:24, 10.42it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.70it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:41<00:34,  7.24it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:42<00:27,  8.71it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.67it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.62it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.57it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.14it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.40it/s]
+
+ 77%|███████▋  | 774/1000 [01:44<00:33,  6.79it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.91it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.64it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.96it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:26,  8.38it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.00it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.98it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.20it/s]
+
+ 80%|███████▉  | 797/1000 [01:47<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.03it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.73it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.66it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:24,  7.86it/s]
+
+ 81%|████████  | 806/1000 [01:48<00:21,  9.19it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.99it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.23it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.25it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.07it/s]
+
+ 82%|████████▏ | 815/1000 [01:49<00:19,  9.33it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.15it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.22it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.06it/s]
+
+ 82%|████████▏ | 823/1000 [01:50<00:22,  7.83it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:52<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.13it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.47it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:53<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.43it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.78it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.27it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.57it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.51it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.89it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.49it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:13, 10.22it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.53it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.78it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.65it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.72it/s]
+
+ 87%|████████▋ | 867/1000 [01:56<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.17it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.21it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:57<00:16,  7.62it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.88it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.62it/s]
+
+ 88%|████████▊ | 882/1000 [01:58<00:17,  6.56it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.97it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.19it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.37it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.78it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.78it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.62it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.86it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.55it/s]
+
+ 90%|█████████ | 900/1000 [02:00<00:12,  7.72it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.87it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.08it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.06it/s]
+
+ 91%|█████████ | 908/1000 [02:01<00:11,  8.34it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.59it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.40it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.10it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.73it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.60it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.95it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.11it/s]
+
+ 93%|█████████▎| 928/1000 [02:04<00:10,  6.67it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.50it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.84it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.63it/s]
+
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.19it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.33it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:06,  8.86it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.15it/s]
+
+ 94%|█████████▍| 942/1000 [02:06<00:09,  6.35it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.10it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.70it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.42it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.41it/s]
+
+ 96%|█████████▌| 957/1000 [02:08<00:06,  6.42it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.79it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.77it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.02it/s]
+
+ 96%|█████████▋| 964/1000 [02:09<00:04,  7.48it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.33it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:04,  7.99it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.41it/s]
+
+ 97%|█████████▋| 972/1000 [02:10<00:03,  8.03it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.21it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.81it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.80it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.45it/s]
+
+ 98%|█████████▊| 981/1000 [02:11<00:02,  7.08it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.37it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.75it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:13<00:01,  4.99it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.54it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.78it/s]
+
+100%|██████████| 1000/1000 [02:14<00:00,  6.69it/s]
+                                                    
+
+                                                   
+{'eval_loss': '0.8577', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.379', 'eval_steps_per_second': '7.379', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312}
+
+ 17%|█▋        | 300/1813 [33:24<2:29:20,  5.92s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.69it/s]
+
+                                                   
+ 17%|█▋        | 301/1813 [33:29<19:29:52, 46.42s/it]
+ 17%|█▋        | 302/1813 [33:34<14:15:58, 33.99s/it]
+ 17%|█▋        | 303/1813 [33:41<10:51:44, 25.90s/it]
+ 17%|█▋        | 304/1813 [33:49<8:33:38, 20.42s/it] 
+ 17%|█▋        | 305/1813 [33:55<6:45:59, 16.15s/it]
+ 17%|█▋        | 306/1813 [34:01<5:27:46, 13.05s/it]
+ 17%|█▋        | 307/1813 [34:06<4:30:47, 10.79s/it]
+ 17%|█▋        | 308/1813 [34:12<3:52:25,  9.27s/it]
+ 17%|█▋        | 309/1813 [34:18<3:30:55,  8.41s/it]
+ 17%|█▋        | 310/1813 [34:23<3:04:14,  7.36s/it]
+                                                    
+{'loss': '0.9699', 'grad_norm': '1.305', 'learning_rate': '6.117e-05', 'epoch': '0.171', 'num_input_tokens_seen': 2050672, 'train_runtime': '2064', 'train_tokens_per_second': '993.7'}
+
+ 17%|█▋        | 310/1813 [34:23<3:04:14,  7.36s/it]
+ 17%|█▋        | 311/1813 [34:30<3:02:29,  7.29s/it]
+ 17%|█▋        | 312/1813 [34:38<3:03:57,  7.35s/it]
+ 17%|█▋        | 313/1813 [34:45<2:59:53,  7.20s/it]
+ 17%|█▋        | 314/1813 [34:50<2:48:37,  6.75s/it]
+ 17%|█▋        | 315/1813 [34:57<2:49:10,  6.78s/it]
+ 17%|█▋        | 316/1813 [35:04<2:46:34,  6.68s/it]
+ 17%|█▋        | 317/1813 [35:10<2:40:34,  6.44s/it]
+ 18%|█▊        | 318/1813 [35:15<2:33:37,  6.17s/it]
+ 18%|█▊        | 319/1813 [35:20<2:25:15,  5.83s/it]
+ 18%|█▊        | 320/1813 [35:26<2:24:41,  5.81s/it]
+                                                    
+{'loss': '1.064', 'grad_norm': '0.6859', 'learning_rate': '6.099e-05', 'epoch': '0.1766', 'num_input_tokens_seen': 2118000, 'train_runtime': '2126', 'train_tokens_per_second': '996.1'}
+
+ 18%|█▊        | 320/1813 [35:26<2:24:41,  5.81s/it]
+ 18%|█▊        | 321/1813 [35:32<2:24:28,  5.81s/it]
+ 18%|█▊        | 322/1813 [35:37<2:23:57,  5.79s/it]
+ 18%|█▊        | 323/1813 [35:44<2:26:57,  5.92s/it]
+ 18%|█▊        | 324/1813 [35:51<2:34:27,  6.22s/it]
+ 18%|█▊        | 325/1813 [35:58<2:40:25,  6.47s/it]
+ 18%|█▊        | 326/1813 [36:04<2:38:14,  6.38s/it]
+ 18%|█▊        | 327/1813 [36:10<2:34:09,  6.22s/it]
+ 18%|█▊        | 328/1813 [36:16<2:37:07,  6.35s/it]
+ 18%|█▊        | 329/1813 [36:22<2:33:20,  6.20s/it]
+ 18%|█▊        | 330/1813 [36:28<2:30:25,  6.09s/it]
+                                                    
+{'loss': '0.9703', 'grad_norm': '1.079', 'learning_rate': '6.08e-05', 'epoch': '0.1821', 'num_input_tokens_seen': 2184640, 'train_runtime': '2188', 'train_tokens_per_second': '998.3'}
+
+ 18%|█▊        | 330/1813 [36:28<2:30:25,  6.09s/it]
+ 18%|█▊        | 331/1813 [36:34<2:33:13,  6.20s/it]
+ 18%|█▊        | 332/1813 [36:40<2:25:18,  5.89s/it]
+ 18%|█▊        | 333/1813 [36:45<2:25:19,  5.89s/it]
+ 18%|█▊        | 334/1813 [36:51<2:22:50,  5.79s/it]
+ 18%|█▊        | 335/1813 [36:59<2:36:11,  6.34s/it]
+ 19%|█▊        | 336/1813 [37:06<2:40:09,  6.51s/it]
+ 19%|█▊        | 337/1813 [37:11<2:32:45,  6.21s/it]
+ 19%|█▊        | 338/1813 [37:17<2:31:50,  6.18s/it]
+ 19%|█▊        | 339/1813 [37:23<2:26:27,  5.96s/it]
+ 19%|█▉        | 340/1813 [37:30<2:33:47,  6.26s/it]
+                                                    
+{'loss': '0.9498', 'grad_norm': '1.224', 'learning_rate': '6.06e-05', 'epoch': '0.1876', 'num_input_tokens_seen': 2251392, 'train_runtime': '2250', 'train_tokens_per_second': '1001'}
+
+ 19%|█▉        | 340/1813 [37:30<2:33:47,  6.26s/it]
+ 19%|█▉        | 341/1813 [37:36<2:33:26,  6.25s/it]
+ 19%|█▉        | 342/1813 [37:43<2:41:32,  6.59s/it]
+ 19%|█▉        | 343/1813 [37:49<2:34:34,  6.31s/it]
+ 19%|█▉        | 344/1813 [37:55<2:29:42,  6.11s/it]
+ 19%|█▉        | 345/1813 [38:02<2:38:26,  6.48s/it]
+ 19%|█▉        | 346/1813 [38:08<2:38:12,  6.47s/it]
+ 19%|█▉        | 347/1813 [38:13<2:26:49,  6.01s/it]
+ 19%|█▉        | 348/1813 [38:20<2:30:05,  6.15s/it]
+ 19%|█▉        | 349/1813 [38:28<2:46:32,  6.83s/it]
+ 19%|█▉        | 350/1813 [38:36<2:53:29,  7.11s/it]
+                                                    
+{'loss': '1.058', 'grad_norm': '0.837', 'learning_rate': '6.039e-05', 'epoch': '0.1931', 'num_input_tokens_seen': 2324712, 'train_runtime': '2316', 'train_tokens_per_second': '1004'}
+
+ 19%|█▉        | 350/1813 [38:36<2:53:29,  7.11s/it]
+ 19%|█▉        | 351/1813 [38:43<2:53:10,  7.11s/it]
+ 19%|█▉        | 352/1813 [38:49<2:44:14,  6.75s/it]
+ 19%|█▉        | 353/1813 [38:54<2:34:43,  6.36s/it]
+ 20%|█▉        | 354/1813 [39:00<2:28:41,  6.11s/it]
+ 20%|█▉        | 355/1813 [39:06<2:26:55,  6.05s/it]
+ 20%|█▉        | 356/1813 [39:13<2:32:36,  6.28s/it]
+ 20%|█▉        | 357/1813 [39:18<2:27:44,  6.09s/it]
+ 20%|█▉        | 358/1813 [39:24<2:28:24,  6.12s/it]
+ 20%|█▉        | 359/1813 [39:30<2:24:41,  5.97s/it]
+ 20%|█▉        | 360/1813 [39:37<2:30:58,  6.23s/it]
+                                                    
+{'loss': '1.005', 'grad_norm': '0.9685', 'learning_rate': '6.017e-05', 'epoch': '0.1986', 'num_input_tokens_seen': 2390776, 'train_runtime': '2377', 'train_tokens_per_second': '1006'}
+
+ 20%|█▉        | 360/1813 [39:37<2:30:58,  6.23s/it]
+ 20%|█▉        | 361/1813 [39:42<2:24:27,  5.97s/it]
+ 20%|█▉        | 362/1813 [39:48<2:22:43,  5.90s/it]
+ 20%|██        | 363/1813 [39:55<2:29:40,  6.19s/it]
+ 20%|██        | 364/1813 [40:01<2:32:20,  6.31s/it]
+ 20%|██        | 365/1813 [40:07<2:29:15,  6.18s/it]
+ 20%|██        | 366/1813 [40:14<2:34:50,  6.42s/it]
+ 20%|██        | 367/1813 [40:20<2:31:14,  6.28s/it]
+ 20%|██        | 368/1813 [40:26<2:24:37,  6.00s/it]
+ 20%|██        | 369/1813 [40:30<2:16:03,  5.65s/it]
+ 20%|██        | 370/1813 [40:37<2:19:14,  5.79s/it]
+                                                    
+{'loss': '0.9962', 'grad_norm': '1.293', 'learning_rate': '5.994e-05', 'epoch': '0.2041', 'num_input_tokens_seen': 2454808, 'train_runtime': '2437', 'train_tokens_per_second': '1007'}
+
+ 20%|██        | 370/1813 [40:37<2:19:14,  5.79s/it]
+ 20%|██        | 371/1813 [40:42<2:15:17,  5.63s/it]
+ 21%|██        | 372/1813 [40:50<2:34:25,  6.43s/it]
+ 21%|██        | 373/1813 [40:57<2:34:12,  6.43s/it]
+ 21%|██        | 374/1813 [41:04<2:42:29,  6.78s/it]
+ 21%|██        | 375/1813 [41:11<2:44:37,  6.87s/it]
+ 21%|██        | 376/1813 [41:17<2:35:03,  6.47s/it]
+ 21%|██        | 377/1813 [41:23<2:33:53,  6.43s/it]
+ 21%|██        | 378/1813 [41:29<2:31:31,  6.34s/it]
+ 21%|██        | 379/1813 [41:34<2:22:18,  5.95s/it]
+ 21%|██        | 380/1813 [41:40<2:17:44,  5.77s/it]
+                                                    
+{'loss': '0.9715', 'grad_norm': '1.365', 'learning_rate': '5.969e-05', 'epoch': '0.2097', 'num_input_tokens_seen': 2520472, 'train_runtime': '2500', 'train_tokens_per_second': '1008'}
+
+ 21%|██        | 380/1813 [41:40<2:17:44,  5.77s/it]
+ 21%|██        | 381/1813 [41:47<2:27:01,  6.16s/it]
+ 21%|██        | 382/1813 [41:54<2:32:33,  6.40s/it]
+ 21%|██        | 383/1813 [42:00<2:31:17,  6.35s/it]
+ 21%|██        | 384/1813 [42:06<2:29:47,  6.29s/it]
+ 21%|██        | 385/1813 [42:12<2:30:27,  6.32s/it]
+ 21%|██▏       | 386/1813 [42:17<2:19:59,  5.89s/it]
+ 21%|██▏       | 387/1813 [42:24<2:22:17,  5.99s/it]
+ 21%|██▏       | 388/1813 [42:31<2:30:18,  6.33s/it]
+ 21%|██▏       | 389/1813 [42:38<2:40:59,  6.78s/it]
+ 22%|██▏       | 390/1813 [42:45<2:40:39,  6.77s/it]
+                                                    
+{'loss': '1.008', 'grad_norm': '0.8606', 'learning_rate': '5.944e-05', 'epoch': '0.2152', 'num_input_tokens_seen': 2591512, 'train_runtime': '2566', 'train_tokens_per_second': '1010'}
+
+ 22%|██▏       | 390/1813 [42:45<2:40:39,  6.77s/it]
+ 22%|██▏       | 391/1813 [42:51<2:35:43,  6.57s/it]
+ 22%|██▏       | 392/1813 [42:58<2:33:42,  6.49s/it]
+ 22%|██▏       | 393/1813 [43:04<2:34:24,  6.52s/it]
+ 22%|██▏       | 394/1813 [43:09<2:23:25,  6.06s/it]
+ 22%|██▏       | 395/1813 [43:15<2:19:28,  5.90s/it]
+ 22%|██▏       | 396/1813 [43:21<2:22:03,  6.02s/it]
+ 22%|██▏       | 397/1813 [43:26<2:14:25,  5.70s/it]
+ 22%|██▏       | 398/1813 [43:32<2:18:49,  5.89s/it]
+ 22%|██▏       | 399/1813 [43:40<2:31:10,  6.41s/it]
+ 22%|██▏       | 400/1813 [43:48<2:43:56,  6.96s/it]
+                                                    
+{'loss': '0.9869', 'grad_norm': '0.9102', 'learning_rate': '5.918e-05', 'epoch': '0.2207', 'num_input_tokens_seen': 2660336, 'train_runtime': '2629', 'train_tokens_per_second': '1012'}
+
+ 22%|██▏       | 400/1813 [43:48<2:43:56,  6.96s/it]
+ 22%|██▏       | 401/1813 [43:56<2:46:30,  7.08s/it]
+ 22%|██▏       | 402/1813 [44:01<2:32:15,  6.47s/it]
+ 22%|██▏       | 403/1813 [44:07<2:31:01,  6.43s/it]
+ 22%|██▏       | 404/1813 [44:14<2:34:22,  6.57s/it]
+ 22%|██▏       | 405/1813 [44:21<2:37:34,  6.71s/it]
+ 22%|██▏       | 406/1813 [44:26<2:23:57,  6.14s/it]
+ 22%|██▏       | 407/1813 [44:30<2:11:10,  5.60s/it]
+ 23%|██▎       | 408/1813 [44:36<2:12:02,  5.64s/it]
+ 23%|██▎       | 409/1813 [44:42<2:13:47,  5.72s/it]
+ 23%|██▎       | 410/1813 [44:47<2:10:04,  5.56s/it]
+                                                    
+{'loss': '1.01', 'grad_norm': '1.29', 'learning_rate': '5.89e-05', 'epoch': '0.2262', 'num_input_tokens_seen': 2722056, 'train_runtime': '2687', 'train_tokens_per_second': '1013'}
+
+ 23%|██▎       | 410/1813 [44:47<2:10:04,  5.56s/it]
+ 23%|██▎       | 411/1813 [44:53<2:14:35,  5.76s/it]
+ 23%|██▎       | 412/1813 [45:00<2:20:31,  6.02s/it]
+ 23%|██▎       | 413/1813 [45:07<2:29:09,  6.39s/it]
+ 23%|██▎       | 414/1813 [45:13<2:23:26,  6.15s/it]
+ 23%|██▎       | 415/1813 [45:19<2:23:33,  6.16s/it]
+ 23%|██▎       | 416/1813 [45:25<2:24:57,  6.23s/it]
+ 23%|██▎       | 417/1813 [45:33<2:33:59,  6.62s/it]
+ 23%|██▎       | 418/1813 [45:38<2:27:05,  6.33s/it]
+ 23%|██▎       | 419/1813 [45:45<2:28:27,  6.39s/it]
+ 23%|██▎       | 420/1813 [45:50<2:22:46,  6.15s/it]
+                                                    
+{'loss': '1.004', 'grad_norm': '0.7712', 'learning_rate': '5.862e-05', 'epoch': '0.2317', 'num_input_tokens_seen': 2790960, 'train_runtime': '2751', 'train_tokens_per_second': '1015'}
+
+ 23%|██▎       | 420/1813 [45:50<2:22:46,  6.15s/it]
+ 23%|██▎       | 421/1813 [45:57<2:23:02,  6.17s/it]
+ 23%|██▎       | 422/1813 [46:04<2:30:24,  6.49s/it]
+ 23%|██▎       | 423/1813 [46:10<2:27:26,  6.36s/it]
+ 23%|██▎       | 424/1813 [46:16<2:25:44,  6.30s/it]
+ 23%|██▎       | 425/1813 [46:22<2:21:06,  6.10s/it]
+ 23%|██▎       | 426/1813 [46:29<2:28:46,  6.44s/it]
+ 24%|██▎       | 427/1813 [46:35<2:24:48,  6.27s/it]
+ 24%|██▎       | 428/1813 [46:41<2:23:34,  6.22s/it]
+ 24%|██▎       | 429/1813 [46:46<2:16:40,  5.93s/it]
+ 24%|██▎       | 430/1813 [46:54<2:27:19,  6.39s/it]
+                                                    
+{'loss': '0.9908', 'grad_norm': '0.9487', 'learning_rate': '5.833e-05', 'epoch': '0.2372', 'num_input_tokens_seen': 2860096, 'train_runtime': '2814', 'train_tokens_per_second': '1016'}
+
+ 24%|██▎       | 430/1813 [46:54<2:27:19,  6.39s/it]
+ 24%|██▍       | 431/1813 [46:59<2:23:22,  6.22s/it]
+ 24%|██▍       | 432/1813 [47:05<2:19:51,  6.08s/it]
+ 24%|██▍       | 433/1813 [47:11<2:15:17,  5.88s/it]
+ 24%|██▍       | 434/1813 [47:17<2:19:40,  6.08s/it]
+ 24%|██▍       | 435/1813 [47:23<2:18:14,  6.02s/it]
+ 24%|██▍       | 436/1813 [47:29<2:20:35,  6.13s/it]
+ 24%|██▍       | 437/1813 [47:35<2:17:37,  6.00s/it]
+ 24%|██▍       | 438/1813 [47:42<2:20:58,  6.15s/it]
+ 24%|██▍       | 439/1813 [47:48<2:20:01,  6.11s/it]
+ 24%|██▍       | 440/1813 [47:54<2:22:19,  6.22s/it]
+                                                    
+{'loss': '0.9067', 'grad_norm': '1.323', 'learning_rate': '5.803e-05', 'epoch': '0.2428', 'num_input_tokens_seen': 2924856, 'train_runtime': '2875', 'train_tokens_per_second': '1017'}
+
+ 24%|██▍       | 440/1813 [47:54<2:22:19,  6.22s/it]
+ 24%|██▍       | 441/1813 [48:00<2:20:05,  6.13s/it]
+ 24%|██▍       | 442/1813 [48:06<2:19:00,  6.08s/it]
+ 24%|██▍       | 443/1813 [48:13<2:23:16,  6.28s/it]
+ 24%|██▍       | 444/1813 [48:18<2:17:47,  6.04s/it]
+ 25%|██▍       | 445/1813 [48:25<2:25:06,  6.36s/it]
+ 25%|██▍       | 446/1813 [48:32<2:25:51,  6.40s/it]
+ 25%|██▍       | 447/1813 [48:38<2:24:46,  6.36s/it]
+ 25%|██▍       | 448/1813 [48:45<2:25:13,  6.38s/it]
+ 25%|██▍       | 449/1813 [48:50<2:19:10,  6.12s/it]
+ 25%|██▍       | 450/1813 [48:56<2:16:15,  6.00s/it]
+                                                    
+{'loss': '1.037', 'grad_norm': '1.016', 'learning_rate': '5.772e-05', 'epoch': '0.2483', 'num_input_tokens_seen': 2990040, 'train_runtime': '2936', 'train_tokens_per_second': '1018'}
+
+ 25%|██▍       | 450/1813 [48:56<2:16:15,  6.00s/it]
+ 25%|██▍       | 451/1813 [49:03<2:26:36,  6.46s/it]
+ 25%|██▍       | 452/1813 [49:09<2:22:00,  6.26s/it]
+ 25%|██▍       | 453/1813 [49:14<2:14:32,  5.94s/it]
+ 25%|██▌       | 454/1813 [49:21<2:20:50,  6.22s/it]
+ 25%|██▌       | 455/1813 [49:28<2:21:39,  6.26s/it]
+ 25%|██▌       | 456/1813 [49:32<2:09:23,  5.72s/it]
+ 25%|██▌       | 457/1813 [49:38<2:13:21,  5.90s/it]
+ 25%|██▌       | 458/1813 [49:46<2:28:49,  6.59s/it]
+ 25%|██▌       | 459/1813 [49:53<2:31:18,  6.70s/it]
+ 25%|██▌       | 460/1813 [50:00<2:30:08,  6.66s/it]
+                                                    
+{'loss': '1.098', 'grad_norm': '0.7865', 'learning_rate': '5.739e-05', 'epoch': '0.2538', 'num_input_tokens_seen': 3058848, 'train_runtime': '3001', 'train_tokens_per_second': '1019'}
+
+ 25%|██▌       | 460/1813 [50:00<2:30:08,  6.66s/it]
+ 25%|██▌       | 461/1813 [50:06<2:25:48,  6.47s/it]
+ 25%|██▌       | 462/1813 [50:13<2:31:52,  6.75s/it]
+ 26%|██▌       | 463/1813 [50:19<2:25:54,  6.48s/it]
+ 26%|██▌       | 464/1813 [50:24<2:15:57,  6.05s/it]
+ 26%|██▌       | 465/1813 [50:30<2:12:39,  5.90s/it]
+ 26%|██▌       | 466/1813 [50:36<2:17:05,  6.11s/it]
+ 26%|██▌       | 467/1813 [50:42<2:14:25,  5.99s/it]
+ 26%|██▌       | 468/1813 [50:49<2:17:38,  6.14s/it]
+ 26%|██▌       | 469/1813 [50:55<2:20:37,  6.28s/it]
+ 26%|██▌       | 470/1813 [51:02<2:22:26,  6.36s/it]
+                                                    
+{'loss': '0.9614', 'grad_norm': '0.738', 'learning_rate': '5.706e-05', 'epoch': '0.2593', 'num_input_tokens_seen': 3126680, 'train_runtime': '3062', 'train_tokens_per_second': '1021'}
+
+ 26%|██▌       | 470/1813 [51:02<2:22:26,  6.36s/it]
+ 26%|██▌       | 471/1813 [51:07<2:16:56,  6.12s/it]
+ 26%|██▌       | 472/1813 [51:14<2:19:40,  6.25s/it]
+ 26%|██▌       | 473/1813 [51:21<2:22:13,  6.37s/it]
+ 26%|██▌       | 474/1813 [51:27<2:19:24,  6.25s/it]
+ 26%|██▌       | 475/1813 [51:32<2:11:58,  5.92s/it]
+ 26%|██▋       | 476/1813 [51:38<2:11:54,  5.92s/it]
+ 26%|██▋       | 477/1813 [51:44<2:15:51,  6.10s/it]
+ 26%|██▋       | 478/1813 [51:51<2:21:20,  6.35s/it]
+ 26%|██▋       | 479/1813 [51:57<2:16:07,  6.12s/it]
+ 26%|██▋       | 480/1813 [52:03<2:18:02,  6.21s/it]
+                                                    
+{'loss': '1.068', 'grad_norm': '0.9879', 'learning_rate': '5.672e-05', 'epoch': '0.2648', 'num_input_tokens_seen': 3193704, 'train_runtime': '3124', 'train_tokens_per_second': '1022'}
+
+ 26%|██▋       | 480/1813 [52:03<2:18:02,  6.21s/it]
+ 27%|██▋       | 481/1813 [52:09<2:18:24,  6.23s/it]
+ 27%|██▋       | 482/1813 [52:16<2:21:36,  6.38s/it]
+ 27%|██▋       | 483/1813 [52:24<2:29:49,  6.76s/it]
+ 27%|██▋       | 484/1813 [52:29<2:22:44,  6.44s/it]
+ 27%|██▋       | 485/1813 [52:37<2:31:02,  6.82s/it]
+ 27%|██▋       | 486/1813 [52:43<2:27:28,  6.67s/it]
+ 27%|██▋       | 487/1813 [52:50<2:28:48,  6.73s/it]
+ 27%|██▋       | 488/1813 [52:56<2:22:33,  6.46s/it]
+ 27%|██▋       | 489/1813 [53:03<2:26:42,  6.65s/it]
+ 27%|██▋       | 490/1813 [53:09<2:23:22,  6.50s/it]
+                                                    
+{'loss': '1.021', 'grad_norm': '0.9932', 'learning_rate': '5.638e-05', 'epoch': '0.2703', 'num_input_tokens_seen': 3268064, 'train_runtime': '3190', 'train_tokens_per_second': '1024'}
+
+ 27%|██▋       | 490/1813 [53:09<2:23:22,  6.50s/it]
+ 27%|██▋       | 491/1813 [53:16<2:25:50,  6.62s/it]
+ 27%|██▋       | 492/1813 [53:24<2:32:22,  6.92s/it]
+ 27%|██▋       | 493/1813 [53:30<2:23:26,  6.52s/it]
+ 27%|██▋       | 494/1813 [53:35<2:17:04,  6.24s/it]
+ 27%|██▋       | 495/1813 [53:42<2:22:15,  6.48s/it]
+ 27%|██▋       | 496/1813 [53:49<2:22:12,  6.48s/it]
+ 27%|██▋       | 497/1813 [53:54<2:15:59,  6.20s/it]
+ 27%|██▋       | 498/1813 [54:01<2:22:10,  6.49s/it]
+ 28%|██▊       | 499/1813 [54:07<2:19:45,  6.38s/it]
+ 28%|██▊       | 500/1813 [54:17<2:37:42,  7.21s/it]
+                                                    
+{'loss': '1.108', 'grad_norm': '0.7344', 'learning_rate': '5.602e-05', 'epoch': '0.2759', 'num_input_tokens_seen': 3341536, 'train_runtime': '3257', 'train_tokens_per_second': '1026'}
+
+ 28%|██▊       | 500/1813 [54:17<2:37:42,  7.21s/it]
+ 28%|██▊       | 501/1813 [54:23<2:31:30,  6.93s/it]
+ 28%|██▊       | 502/1813 [54:29<2:23:25,  6.56s/it]
+ 28%|██▊       | 503/1813 [54:34<2:13:48,  6.13s/it]
+ 28%|██▊       | 504/1813 [54:40<2:14:25,  6.16s/it]
+ 28%|██▊       | 505/1813 [54:47<2:21:51,  6.51s/it]
+ 28%|██▊       | 506/1813 [54:53<2:18:07,  6.34s/it]
+ 28%|██▊       | 507/1813 [55:01<2:24:32,  6.64s/it]
+ 28%|██▊       | 508/1813 [55:06<2:14:23,  6.18s/it]
+ 28%|██▊       | 509/1813 [55:11<2:07:52,  5.88s/it]
+ 28%|██▊       | 510/1813 [55:16<2:03:55,  5.71s/it]
+                                                    
+{'loss': '0.9385', 'grad_norm': '1.364', 'learning_rate': '5.565e-05', 'epoch': '0.2814', 'num_input_tokens_seen': 3402712, 'train_runtime': '3317', 'train_tokens_per_second': '1026'}
+
+ 28%|██▊       | 510/1813 [55:16<2:03:55,  5.71s/it]
+ 28%|██▊       | 511/1813 [55:22<2:01:37,  5.60s/it]
+ 28%|██▊       | 512/1813 [55:28<2:09:25,  5.97s/it]
+ 28%|██▊       | 513/1813 [55:35<2:17:00,  6.32s/it]
+ 28%|██▊       | 514/1813 [55:42<2:19:30,  6.44s/it]
+ 28%|██▊       | 515/1813 [55:47<2:10:12,  6.02s/it]
+ 28%|██▊       | 516/1813 [55:53<2:08:43,  5.96s/it]
+ 29%|██▊       | 517/1813 [56:00<2:13:25,  6.18s/it]
+ 29%|██▊       | 518/1813 [56:06<2:13:38,  6.19s/it]
+ 29%|██▊       | 519/1813 [56:12<2:14:41,  6.25s/it]
+ 29%|██▊       | 520/1813 [56:19<2:17:40,  6.39s/it]
+                                                    
+{'loss': '0.9798', 'grad_norm': '0.9405', 'learning_rate': '5.528e-05', 'epoch': '0.2869', 'num_input_tokens_seen': 3469624, 'train_runtime': '3380', 'train_tokens_per_second': '1027'}
+
+ 29%|██▊       | 520/1813 [56:19<2:17:40,  6.39s/it]
+ 29%|██▊       | 521/1813 [56:23<2:01:51,  5.66s/it]
+ 29%|██▉       | 522/1813 [56:29<2:03:16,  5.73s/it]
+ 29%|██▉       | 523/1813 [56:35<2:06:09,  5.87s/it]
+ 29%|██▉       | 524/1813 [56:41<2:08:39,  5.99s/it]
+ 29%|██▉       | 525/1813 [56:46<2:02:09,  5.69s/it]
+ 29%|██▉       | 526/1813 [56:53<2:08:23,  5.99s/it]
+ 29%|██▉       | 527/1813 [57:00<2:12:07,  6.16s/it]
+ 29%|██▉       | 528/1813 [57:07<2:16:45,  6.39s/it]
+ 29%|██▉       | 529/1813 [57:14<2:20:27,  6.56s/it]
+ 29%|██▉       | 530/1813 [57:19<2:15:21,  6.33s/it]
+                                                    
+{'loss': '0.9581', 'grad_norm': '1.294', 'learning_rate': '5.489e-05', 'epoch': '0.2924', 'num_input_tokens_seen': 3534904, 'train_runtime': '3440', 'train_tokens_per_second': '1028'}
+
+ 29%|██▉       | 530/1813 [57:19<2:15:21,  6.33s/it]
+ 29%|██▉       | 531/1813 [57:27<2:27:11,  6.89s/it]
+ 29%|██▉       | 532/1813 [57:34<2:22:56,  6.70s/it]
+ 29%|██▉       | 533/1813 [57:41<2:25:19,  6.81s/it]
+ 29%|██▉       | 534/1813 [57:48<2:27:43,  6.93s/it]
+ 30%|██▉       | 535/1813 [57:53<2:12:53,  6.24s/it]
+ 30%|██▉       | 536/1813 [57:58<2:05:31,  5.90s/it]
+ 30%|██▉       | 537/1813 [58:04<2:10:48,  6.15s/it]
+ 30%|██▉       | 538/1813 [58:10<2:05:50,  5.92s/it]
+ 30%|██▉       | 539/1813 [58:16<2:07:06,  5.99s/it]
+ 30%|██▉       | 540/1813 [58:22<2:07:07,  5.99s/it]
+                                                    
+{'loss': '0.9747', 'grad_norm': '0.8806', 'learning_rate': '5.45e-05', 'epoch': '0.2979', 'num_input_tokens_seen': 3603488, 'train_runtime': '3503', 'train_tokens_per_second': '1029'}
+
+ 30%|██▉       | 540/1813 [58:22<2:07:07,  5.99s/it]
+ 30%|██▉       | 541/1813 [58:27<2:01:16,  5.72s/it]
+ 30%|██▉       | 542/1813 [58:33<1:59:30,  5.64s/it]
+ 30%|██▉       | 543/1813 [58:40<2:08:52,  6.09s/it]
+ 30%|███       | 544/1813 [58:48<2:22:20,  6.73s/it]
+ 30%|███       | 545/1813 [58:54<2:20:24,  6.64s/it]
+ 30%|███       | 546/1813 [59:01<2:18:40,  6.57s/it]
+ 30%|███       | 547/1813 [59:08<2:25:06,  6.88s/it]
+ 30%|███       | 548/1813 [59:16<2:28:30,  7.04s/it]
+ 30%|███       | 549/1813 [59:21<2:18:44,  6.59s/it]
+ 30%|███       | 550/1813 [59:27<2:13:01,  6.32s/it]
+                                                    
+{'loss': '0.9173', 'grad_norm': '1.029', 'learning_rate': '5.41e-05', 'epoch': '0.3034', 'num_input_tokens_seen': 3676576, 'train_runtime': '3568', 'train_tokens_per_second': '1031'}
+
+ 30%|███       | 550/1813 [59:27<2:13:01,  6.32s/it]
+ 30%|███       | 551/1813 [59:33<2:10:14,  6.19s/it]
+ 30%|███       | 552/1813 [59:38<2:04:54,  5.94s/it]
+ 31%|███       | 553/1813 [59:43<1:57:51,  5.61s/it]
+ 31%|███       | 554/1813 [59:50<2:04:38,  5.94s/it]
+ 31%|███       | 555/1813 [59:55<1:58:07,  5.63s/it]
+ 31%|███       | 556/1813 [1:00:01<1:59:58,  5.73s/it]
+ 31%|███       | 557/1813 [1:00:07<2:06:25,  6.04s/it]
+ 31%|███       | 558/1813 [1:00:13<2:03:07,  5.89s/it]
+ 31%|███       | 559/1813 [1:00:18<1:57:11,  5.61s/it]
+ 31%|███       | 560/1813 [1:00:24<2:02:53,  5.88s/it]
+                                                      
+{'loss': '0.9869', 'grad_norm': '0.6366', 'learning_rate': '5.369e-05', 'epoch': '0.309', 'num_input_tokens_seen': 3738056, 'train_runtime': '3625', 'train_tokens_per_second': '1031'}
+
+ 31%|███       | 560/1813 [1:00:24<2:02:53,  5.88s/it]
+ 31%|███       | 561/1813 [1:00:30<2:02:22,  5.86s/it]
+ 31%|███       | 562/1813 [1:00:37<2:06:57,  6.09s/it]
+ 31%|███       | 563/1813 [1:00:44<2:11:40,  6.32s/it]
+ 31%|███       | 564/1813 [1:00:50<2:08:56,  6.19s/it]
+ 31%|███       | 565/1813 [1:00:55<2:04:00,  5.96s/it]
+ 31%|███       | 566/1813 [1:01:02<2:08:15,  6.17s/it]
+ 31%|███▏      | 567/1813 [1:01:07<2:03:37,  5.95s/it]
+ 31%|███▏      | 568/1813 [1:01:14<2:09:47,  6.26s/it]
+ 31%|███▏      | 569/1813 [1:01:22<2:17:57,  6.65s/it]
+ 31%|███▏      | 570/1813 [1:01:29<2:23:13,  6.91s/it]
+                                                      
+{'loss': '0.985', 'grad_norm': '0.7328', 'learning_rate': '5.327e-05', 'epoch': '0.3145', 'num_input_tokens_seen': 3808208, 'train_runtime': '3690', 'train_tokens_per_second': '1032'}
+
+ 31%|███▏      | 570/1813 [1:01:29<2:23:13,  6.91s/it]
+ 31%|███▏      | 571/1813 [1:01:36<2:22:51,  6.90s/it]
+ 32%|███▏      | 572/1813 [1:01:42<2:18:23,  6.69s/it]
+ 32%|███▏      | 573/1813 [1:01:48<2:09:30,  6.27s/it]
+ 32%|███▏      | 574/1813 [1:01:53<2:05:30,  6.08s/it]
+ 32%|███▏      | 575/1813 [1:02:00<2:08:17,  6.22s/it]
+ 32%|███▏      | 576/1813 [1:02:04<1:57:26,  5.70s/it]
+ 32%|███▏      | 577/1813 [1:02:11<2:03:11,  5.98s/it]
+ 32%|███▏      | 578/1813 [1:02:17<2:03:53,  6.02s/it]
+ 32%|███▏      | 579/1813 [1:02:23<2:05:10,  6.09s/it]
+ 32%|███▏      | 580/1813 [1:02:30<2:06:35,  6.16s/it]
+                                                      
+{'loss': '0.9868', 'grad_norm': '0.8879', 'learning_rate': '5.285e-05', 'epoch': '0.32', 'num_input_tokens_seen': 3872328, 'train_runtime': '3750', 'train_tokens_per_second': '1033'}
+
+ 32%|███▏      | 580/1813 [1:02:30<2:06:35,  6.16s/it]
+ 32%|███▏      | 581/1813 [1:02:35<2:04:38,  6.07s/it]
+ 32%|███▏      | 582/1813 [1:02:41<2:01:10,  5.91s/it]
+ 32%|███▏      | 583/1813 [1:02:47<2:01:09,  5.91s/it]
+ 32%|███▏      | 584/1813 [1:02:53<2:04:20,  6.07s/it]
+ 32%|███▏      | 585/1813 [1:02:59<2:03:52,  6.05s/it]
+ 32%|███▏      | 586/1813 [1:03:06<2:06:09,  6.17s/it]
+ 32%|███▏      | 587/1813 [1:03:12<2:04:38,  6.10s/it]
+ 32%|███▏      | 588/1813 [1:03:18<2:05:12,  6.13s/it]
+ 32%|███▏      | 589/1813 [1:03:24<2:06:57,  6.22s/it]
+ 33%|███▎      | 590/1813 [1:03:31<2:07:15,  6.24s/it]
+                                                      
+{'loss': '0.8934', 'grad_norm': '1.101', 'learning_rate': '5.242e-05', 'epoch': '0.3255', 'num_input_tokens_seen': 3936024, 'train_runtime': '3811', 'train_tokens_per_second': '1033'}
+
+ 33%|███▎      | 590/1813 [1:03:31<2:07:15,  6.24s/it]
+ 33%|███▎      | 591/1813 [1:03:36<2:03:10,  6.05s/it]
+ 33%|███▎      | 592/1813 [1:03:42<2:02:51,  6.04s/it]
+ 33%|███▎      | 593/1813 [1:03:49<2:05:39,  6.18s/it]
+ 33%|███▎      | 594/1813 [1:03:55<2:05:27,  6.18s/it]
+ 33%|███▎      | 595/1813 [1:04:01<2:03:01,  6.06s/it]
+ 33%|███▎      | 596/1813 [1:04:07<2:01:52,  6.01s/it]
+ 33%|███▎      | 597/1813 [1:04:12<1:58:10,  5.83s/it]
+ 33%|███▎      | 598/1813 [1:04:18<1:56:15,  5.74s/it]
+ 33%|███▎      | 599/1813 [1:04:24<1:57:46,  5.82s/it]
+ 33%|███▎      | 600/1813 [1:04:31<2:05:13,  6.19s/it]
+                                                      
+{'loss': '0.9521', 'grad_norm': '0.7894', 'learning_rate': '5.198e-05', 'epoch': '0.331', 'num_input_tokens_seen': 3999176, 'train_runtime': '3871', 'train_tokens_per_second': '1033'}
+
+ 33%|███▎      | 600/1813 [1:04:31<2:05:13,  6.19s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.80it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.25it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.34it/s]
+
+  1%|          | 9/1000 [00:01<02:32,  6.48it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.08it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.00it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.98it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.74it/s]
+
+  2%|▏         | 18/1000 [00:02<02:41,  6.07it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.16it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.61it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.26it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.37it/s]
+
+  3%|▎         | 26/1000 [00:03<02:35,  6.27it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  6.00it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 31/1000 [00:04<03:00,  5.36it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  6.00it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.68it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.13it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.59it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.77it/s]
+
+  4%|▍         | 42/1000 [00:06<01:44,  9.13it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.35it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.07it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.38it/s]
+
+  5%|▍         | 49/1000 [00:06<01:50,  8.64it/s]
+
+  5%|▌         | 51/1000 [00:07<01:36,  9.83it/s]
+
+  5%|▌         | 53/1000 [00:07<01:27, 10.88it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.72it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.28it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.28it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.02it/s]
+
+  7%|▋         | 72/1000 [00:09<02:17,  6.73it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.13it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.88it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.69it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.34it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.88it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.22it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.03it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.97it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.40it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.02it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.27it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.10it/s]
+
+  9%|▉         | 92/1000 [00:12<02:26,  6.19it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.54it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:12,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.83it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.35it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.48it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.23it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.04it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.54it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.83it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.23it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.10it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.45it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:21,  6.16it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.82it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.30it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.11it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.72it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.66it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:26,  9.93it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.98it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.05it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.90it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.08it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:40,  8.45it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.92it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.04it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.43it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.66it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.99it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.63it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.83it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.52it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.42it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:26,  5.69it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.19it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:23,  5.80it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.86it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.17it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.17it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.24it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.76it/s]
+
+ 18%|█▊        | 176/1000 [00:22<01:37,  8.48it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.82it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.59it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.52it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:00,  6.77it/s]
+
+ 18%|█▊        | 182/1000 [00:23<02:22,  5.73it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.76it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:42,  7.91it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.12it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.64it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.39it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.76it/s]
+
+ 20%|█▉        | 198/1000 [00:25<01:27,  9.21it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.69it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.80it/s]
+
+ 20%|██        | 204/1000 [00:26<01:31,  8.67it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.98it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.54it/s]
+
+ 21%|██        | 210/1000 [00:27<01:32,  8.51it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.71it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.76it/s]
+
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.17it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.52it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.81it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.32it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.56it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:06,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.43it/s]
+
+ 23%|██▎       | 228/1000 [00:29<01:53,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.23it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.99it/s]
+
+ 24%|██▎       | 237/1000 [00:30<01:20,  9.48it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.98it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.37it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.03it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.48it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.68it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:30,  8.28it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:05,  5.97it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.88it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.04it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.70it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.76it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.70it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.53it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.44it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.04it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:13,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.71it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:47,  6.71it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.08it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.83it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.64it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.04it/s]
+
+ 28%|██▊       | 283/1000 [00:37<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.72it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.46it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.77it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:38<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 293/1000 [00:39<01:46,  6.64it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:40,  7.01it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:44,  6.76it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.24it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:50,  6.36it/s]
+
+ 30%|███       | 300/1000 [00:40<01:43,  6.79it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.10it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.80it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.09it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.17it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.78it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.80it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.21it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.28it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.27it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.61it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:33,  7.25it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.91it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:44,  6.46it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:26,  7.78it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.71it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.42it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.18it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.80it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.60it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.97it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.39it/s]
+
+ 34%|███▎      | 336/1000 [00:44<01:22,  8.06it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.83it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.73it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.23it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.42it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:33,  6.93it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:48,  5.97it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.65it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.56it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.37it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.97it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.61it/s]
+
+ 36%|███▌      | 361/1000 [00:47<01:07,  9.44it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:29,  7.15it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.26it/s]
+
+ 36%|███▋      | 365/1000 [00:48<02:09,  4.91it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.54it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:50<01:10,  8.77it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.20it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.19it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.15it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.67it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.39it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:08,  8.80it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.49it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.77it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.25it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.70it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.94it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.27it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.51it/s]
+
+ 41%|████      | 408/1000 [00:54<01:32,  6.41it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.46it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.37it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.83it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.27it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.46it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.52it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.70it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:36,  5.95it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.39it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.97it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.56it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.85it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.57it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.05it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.83it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.53it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.10it/s]
+
+ 44%|████▍     | 441/1000 [00:59<00:59,  9.33it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.13it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.27it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.74it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.50it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.03it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.20it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:37,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:00<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.51it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.83it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.76it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.03it/s]
+
+ 47%|████▋     | 467/1000 [01:02<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:23,  6.32it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.58it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.52it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.41it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:34,  5.51it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.33it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.33it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.01it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.29it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+
+ 50%|████▉     | 498/1000 [01:06<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.56it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:12,  6.81it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.58it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.82it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.44it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.98it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48, 10.00it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.30it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.55it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:07,  7.13it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.10it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.40it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.04it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.69it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:21,  5.83it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.57it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.60it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.13it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.68it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.67it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.95it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.77it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.95it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.08it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.51it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.71it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.93it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:14<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.20it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.96it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:56,  7.79it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.85it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.05it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.39it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  8.00it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.76it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.46it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.46it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.32it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:06,  6.36it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.85it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.26it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.99it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.58it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.62it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.94it/s]
+
+ 59%|█████▊    | 587/1000 [01:19<00:57,  7.17it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.55it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.77it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.20it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.82it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.40it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.89it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.49it/s]
+
+ 60%|██████    | 601/1000 [01:21<01:02,  6.41it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.72it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.70it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.74it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.82it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.77it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.11it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.04it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.23it/s]
+
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.47it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.65it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.21it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.24it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.89it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.46it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.73it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.08it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.97it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.20it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.98it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.62it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.13it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.60it/s]
+
+ 65%|██████▍   | 648/1000 [01:27<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.33it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.72it/s]
+
+ 66%|██████▌   | 656/1000 [01:28<00:38,  8.93it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.02it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.58it/s]
+
+ 66%|██████▌   | 662/1000 [01:29<00:52,  6.49it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.34it/s]
+
+ 67%|██████▋   | 667/1000 [01:30<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.21it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.90it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.65it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.67it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.71it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:28, 11.00it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.94it/s]
+
+ 68%|██████▊   | 685/1000 [01:32<00:31,  9.90it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.88it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:33<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.85it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.28it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:40,  7.44it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.35it/s]
+
+ 70%|██████▉   | 698/1000 [01:34<00:43,  6.96it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:52,  5.63it/s]
+
+ 70%|███████   | 704/1000 [01:35<00:40,  7.23it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.04it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.51it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.73it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.88it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.40it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.08it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.82it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.95it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.48it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.17it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:38<00:23, 11.63it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.35it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:27,  9.55it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.91it/s]
+
+ 74%|███████▎  | 737/1000 [01:39<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.78it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.32it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.38it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.63it/s]
+
+ 75%|███████▍  | 747/1000 [01:40<00:24, 10.47it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.70it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.18it/s]
+
+ 75%|███████▌  | 753/1000 [01:41<00:34,  7.26it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.24it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.62it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.33it/s]
+
+ 76%|███████▌  | 761/1000 [01:42<00:27,  8.73it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.67it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.62it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:43<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.57it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.14it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.40it/s]
+
+ 77%|███████▋  | 774/1000 [01:44<00:33,  6.80it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.91it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.63it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.96it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:25,  8.39it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.17it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.56it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.39it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.01it/s]
+
+ 79%|███████▉  | 791/1000 [01:46<00:25,  8.24it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.75it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  8.00it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.22it/s]
+
+ 80%|███████▉  | 797/1000 [01:47<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.04it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.74it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.66it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:25,  7.66it/s]
+
+ 81%|████████  | 806/1000 [01:48<00:21,  9.00it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.84it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.12it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.16it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.01it/s]
+
+ 82%|████████▏ | 815/1000 [01:49<00:19,  9.29it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:50<00:22,  7.83it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.81it/s]
+
+ 83%|████████▎ | 828/1000 [01:51<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.76it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:52<00:27,  6.03it/s]
+
+ 84%|████████▎ | 835/1000 [01:52<00:25,  6.55it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.12it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.47it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:53<00:19,  8.17it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.78it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:54<00:19,  7.56it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.50it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.88it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.49it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.08it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:13, 10.23it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.54it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.80it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.66it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.74it/s]
+
+ 87%|████████▋ | 867/1000 [01:56<00:14,  9.34it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.28it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.30it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.81it/s]
+
+ 88%|████████▊ | 875/1000 [01:57<00:16,  7.66it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.90it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.89it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.62it/s]
+
+ 88%|████████▊ | 882/1000 [01:58<00:17,  6.56it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.98it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.19it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.37it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.77it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.84it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.55it/s]
+
+ 90%|█████████ | 900/1000 [02:00<00:13,  7.63it/s]
+
+ 90%|█████████ | 901/1000 [02:01<00:12,  7.64it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.87it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.38it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  6.96it/s]
+
+ 91%|█████████ | 908/1000 [02:01<00:10,  8.37it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:09,  8.84it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.40it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.38it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.59it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.95it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.18it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.11it/s]
+
+ 93%|█████████▎| 928/1000 [02:04<00:10,  6.66it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.63it/s]
+
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.19it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.33it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.85it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.06it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.16it/s]
+
+ 94%|█████████▍| 942/1000 [02:06<00:09,  6.35it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.10it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.41it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.75it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.85it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.43it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.18it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.41it/s]
+
+ 96%|█████████▌| 957/1000 [02:08<00:06,  6.42it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.79it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.83it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.06it/s]
+
+ 96%|█████████▋| 964/1000 [02:09<00:04,  7.51it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.46it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:10<00:03,  8.03it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.08it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.71it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.75it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.42it/s]
+
+ 98%|█████████▊| 981/1000 [02:11<00:02,  7.05it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.75it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.01it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:13<00:01,  4.99it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.54it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.07it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.78it/s]
+
+100%|██████████| 1000/1000 [02:14<00:00,  6.68it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.8176', 'eval_runtime': '135.1', 'eval_samples_per_second': '7.4', 'eval_steps_per_second': '7.4', 'epoch': '0.331', 'num_input_tokens_seen': 3999176}
+
+ 33%|███▎      | 600/1813 [1:06:46<2:05:13,  6.19s/it]
+
+100%|██████████| 1000/1000 [02:14<00:00,  6.68it/s]
+
+                                                   
+ 33%|███▎      | 601/1813 [1:06:50<15:33:27, 46.21s/it]
+ 33%|███▎      | 602/1813 [1:06:55<11:22:34, 33.82s/it]
+ 33%|███▎      | 603/1813 [1:07:01<8:30:21, 25.31s/it] 
+ 33%|███▎      | 604/1813 [1:07:07<6:33:40, 19.54s/it]
+ 33%|███▎      | 605/1813 [1:07:15<5:23:21, 16.06s/it]
+ 33%|███▎      | 606/1813 [1:07:22<4:30:30, 13.45s/it]
+ 33%|███▎      | 607/1813 [1:07:28<3:46:00, 11.24s/it]
+ 34%|███▎      | 608/1813 [1:07:34<3:15:59,  9.76s/it]
+ 34%|███▎      | 609/1813 [1:07:41<2:54:26,  8.69s/it]
+ 34%|███▎      | 610/1813 [1:07:47<2:39:49,  7.97s/it]
+                                                      
+{'loss': '0.987', 'grad_norm': '0.6577', 'learning_rate': '5.153e-05', 'epoch': '0.3366', 'num_input_tokens_seen': 4066376, 'train_runtime': '4067', 'train_tokens_per_second': '999.8'}
+
+ 34%|███▎      | 610/1813 [1:07:47<2:39:49,  7.97s/it]
+ 34%|███▎      | 611/1813 [1:07:52<2:23:22,  7.16s/it]
+ 34%|███▍      | 612/1813 [1:07:56<2:05:13,  6.26s/it]
+ 34%|███▍      | 613/1813 [1:08:02<2:02:47,  6.14s/it]
+ 34%|███▍      | 614/1813 [1:08:08<2:01:33,  6.08s/it]
+ 34%|███▍      | 615/1813 [1:08:13<1:56:32,  5.84s/it]
+ 34%|███▍      | 616/1813 [1:08:18<1:51:09,  5.57s/it]
+ 34%|███▍      | 617/1813 [1:08:24<1:49:19,  5.48s/it]
+ 34%|███▍      | 618/1813 [1:08:31<1:58:33,  5.95s/it]
+ 34%|███▍      | 619/1813 [1:08:35<1:51:56,  5.63s/it]
+ 34%|███▍      | 620/1813 [1:08:42<1:57:06,  5.89s/it]
+                                                      
+{'loss': '0.8772', 'grad_norm': '0.9207', 'learning_rate': '5.108e-05', 'epoch': '0.3421', 'num_input_tokens_seen': 4123976, 'train_runtime': '4122', 'train_tokens_per_second': '1000'}
+
+ 34%|███▍      | 620/1813 [1:08:42<1:57:06,  5.89s/it]
+ 34%|███▍      | 621/1813 [1:08:49<2:02:37,  6.17s/it]
+ 34%|███▍      | 622/1813 [1:08:56<2:11:21,  6.62s/it]
+ 34%|███▍      | 623/1813 [1:09:01<2:01:03,  6.10s/it]
+ 34%|███▍      | 624/1813 [1:09:08<2:02:02,  6.16s/it]
+ 34%|███▍      | 625/1813 [1:09:13<1:54:27,  5.78s/it]
+ 35%|███▍      | 626/1813 [1:09:17<1:45:40,  5.34s/it]
+ 35%|███▍      | 627/1813 [1:09:23<1:51:05,  5.62s/it]
+ 35%|███▍      | 628/1813 [1:09:29<1:54:45,  5.81s/it]
+ 35%|███▍      | 629/1813 [1:09:38<2:09:50,  6.58s/it]
+ 35%|███▍      | 630/1813 [1:09:45<2:11:46,  6.68s/it]
+                                                      
+{'loss': '0.9564', 'grad_norm': '0.7174', 'learning_rate': '5.062e-05', 'epoch': '0.3476', 'num_input_tokens_seen': 4191992, 'train_runtime': '4185', 'train_tokens_per_second': '1002'}
+
+ 35%|███▍      | 630/1813 [1:09:45<2:11:46,  6.68s/it]
+ 35%|███▍      | 631/1813 [1:09:51<2:11:10,  6.66s/it]
+ 35%|███▍      | 632/1813 [1:09:57<2:07:32,  6.48s/it]
+ 35%|███▍      | 633/1813 [1:10:03<2:04:01,  6.31s/it]
+ 35%|███▍      | 634/1813 [1:10:11<2:10:21,  6.63s/it]
+ 35%|███▌      | 635/1813 [1:10:18<2:11:44,  6.71s/it]
+ 35%|███▌      | 636/1813 [1:10:23<2:03:27,  6.29s/it]
+ 35%|███▌      | 637/1813 [1:10:30<2:07:59,  6.53s/it]
+ 35%|███▌      | 638/1813 [1:10:36<2:03:32,  6.31s/it]
+ 35%|███▌      | 639/1813 [1:10:41<1:58:32,  6.06s/it]
+ 35%|███▌      | 640/1813 [1:10:47<1:59:15,  6.10s/it]
+                                                      
+{'loss': '0.9738', 'grad_norm': '0.9641', 'learning_rate': '5.015e-05', 'epoch': '0.3531', 'num_input_tokens_seen': 4257760, 'train_runtime': '4248', 'train_tokens_per_second': '1002'}
+
+ 35%|███▌      | 640/1813 [1:10:47<1:59:15,  6.10s/it]
+ 35%|███▌      | 641/1813 [1:10:53<1:55:21,  5.91s/it]
+ 35%|███▌      | 642/1813 [1:11:00<2:03:05,  6.31s/it]
+ 35%|███▌      | 643/1813 [1:11:07<2:05:41,  6.45s/it]
+ 36%|███▌      | 644/1813 [1:11:13<2:01:11,  6.22s/it]
+ 36%|███▌      | 645/1813 [1:11:19<2:01:07,  6.22s/it]
+ 36%|███▌      | 646/1813 [1:11:25<2:00:26,  6.19s/it]
+ 36%|███▌      | 647/1813 [1:11:31<2:02:15,  6.29s/it]
+ 36%|███▌      | 648/1813 [1:11:39<2:07:28,  6.57s/it]
+ 36%|███▌      | 649/1813 [1:11:44<2:02:28,  6.31s/it]
+ 36%|███▌      | 650/1813 [1:11:50<1:56:41,  6.02s/it]
+                                                      
+{'loss': '0.9725', 'grad_norm': '1.052', 'learning_rate': '4.967e-05', 'epoch': '0.3586', 'num_input_tokens_seen': 4322136, 'train_runtime': '4310', 'train_tokens_per_second': '1003'}
+
+ 36%|███▌      | 650/1813 [1:11:50<1:56:41,  6.02s/it]
+ 36%|███▌      | 651/1813 [1:11:56<2:00:19,  6.21s/it]
+ 36%|███▌      | 652/1813 [1:12:03<2:04:02,  6.41s/it]
+ 36%|███▌      | 653/1813 [1:12:09<1:59:34,  6.19s/it]
+ 36%|███▌      | 654/1813 [1:12:14<1:54:27,  5.93s/it]
+ 36%|███▌      | 655/1813 [1:12:20<1:56:17,  6.03s/it]
+ 36%|███▌      | 656/1813 [1:12:26<1:52:06,  5.81s/it]
+ 36%|███▌      | 657/1813 [1:12:31<1:47:44,  5.59s/it]
+ 36%|███▋      | 658/1813 [1:12:38<1:53:56,  5.92s/it]
+ 36%|███▋      | 659/1813 [1:12:44<1:57:19,  6.10s/it]
+ 36%|███▋      | 660/1813 [1:12:50<1:54:02,  5.93s/it]
+                                                      
+{'loss': '0.8971', 'grad_norm': '1.024', 'learning_rate': '4.919e-05', 'epoch': '0.3641', 'num_input_tokens_seen': 4385688, 'train_runtime': '4370', 'train_tokens_per_second': '1004'}
+
+ 36%|███▋      | 660/1813 [1:12:50<1:54:02,  5.93s/it]
+ 36%|███▋      | 661/1813 [1:12:56<1:53:54,  5.93s/it]
+ 37%|███▋      | 662/1813 [1:13:02<1:54:33,  5.97s/it]
+ 37%|███▋      | 663/1813 [1:13:08<1:59:08,  6.22s/it]
+ 37%|███▋      | 664/1813 [1:13:15<2:00:19,  6.28s/it]
+ 37%|███▋      | 665/1813 [1:13:20<1:52:59,  5.91s/it]
+ 37%|███▋      | 666/1813 [1:13:25<1:50:05,  5.76s/it]
+ 37%|███▋      | 667/1813 [1:13:32<1:55:56,  6.07s/it]
+ 37%|███▋      | 668/1813 [1:13:39<1:57:55,  6.18s/it]
+ 37%|███▋      | 669/1813 [1:13:44<1:56:43,  6.12s/it]
+ 37%|███▋      | 670/1813 [1:13:50<1:54:11,  5.99s/it]
+                                                      
+{'loss': '0.9935', 'grad_norm': '1.02', 'learning_rate': '4.87e-05', 'epoch': '0.3697', 'num_input_tokens_seen': 4449456, 'train_runtime': '4431', 'train_tokens_per_second': '1004'}
+
+ 37%|███▋      | 670/1813 [1:13:50<1:54:11,  5.99s/it]
+ 37%|███▋      | 671/1813 [1:13:57<1:56:31,  6.12s/it]
+ 37%|███▋      | 672/1813 [1:14:03<1:59:45,  6.30s/it]
+ 37%|███▋      | 673/1813 [1:14:10<1:59:05,  6.27s/it]
+ 37%|███▋      | 674/1813 [1:14:16<2:01:57,  6.42s/it]
+ 37%|███▋      | 675/1813 [1:14:22<1:59:17,  6.29s/it]
+ 37%|███▋      | 676/1813 [1:14:28<1:54:47,  6.06s/it]
+ 37%|███▋      | 677/1813 [1:14:36<2:06:10,  6.66s/it]
+ 37%|███▋      | 678/1813 [1:14:41<1:57:12,  6.20s/it]
+ 37%|███▋      | 679/1813 [1:14:46<1:51:51,  5.92s/it]
+ 38%|███▊      | 680/1813 [1:14:52<1:50:47,  5.87s/it]
+                                                      
+{'loss': '0.9527', 'grad_norm': '0.9265', 'learning_rate': '4.821e-05', 'epoch': '0.3752', 'num_input_tokens_seen': 4514984, 'train_runtime': '4493', 'train_tokens_per_second': '1005'}
+
+ 38%|███▊      | 680/1813 [1:14:52<1:50:47,  5.87s/it]
+ 38%|███▊      | 681/1813 [1:14:59<1:55:01,  6.10s/it]
+ 38%|███▊      | 682/1813 [1:15:04<1:50:01,  5.84s/it]
+ 38%|███▊      | 683/1813 [1:15:10<1:50:56,  5.89s/it]
+ 38%|███▊      | 684/1813 [1:15:17<1:55:22,  6.13s/it]
+ 38%|███▊      | 685/1813 [1:15:24<2:04:27,  6.62s/it]
+ 38%|███▊      | 686/1813 [1:15:31<2:03:15,  6.56s/it]
+ 38%|███▊      | 687/1813 [1:15:39<2:10:02,  6.93s/it]
+ 38%|███▊      | 688/1813 [1:15:43<1:58:44,  6.33s/it]
+ 38%|███▊      | 689/1813 [1:15:52<2:12:19,  7.06s/it]
+ 38%|███▊      | 690/1813 [1:15:59<2:12:36,  7.09s/it]
+                                                      
+{'loss': '0.987', 'grad_norm': '1.002', 'learning_rate': '4.771e-05', 'epoch': '0.3807', 'num_input_tokens_seen': 4587840, 'train_runtime': '4560', 'train_tokens_per_second': '1006'}
+
+ 38%|███▊      | 690/1813 [1:15:59<2:12:36,  7.09s/it]
+ 38%|███▊      | 691/1813 [1:16:05<2:03:21,  6.60s/it]
+ 38%|███▊      | 692/1813 [1:16:11<1:59:20,  6.39s/it]
+ 38%|███▊      | 693/1813 [1:16:16<1:53:30,  6.08s/it]
+ 38%|███▊      | 694/1813 [1:16:22<1:53:25,  6.08s/it]
+ 38%|███▊      | 695/1813 [1:16:28<1:52:31,  6.04s/it]
+ 38%|███▊      | 696/1813 [1:16:36<2:02:27,  6.58s/it]
+ 38%|███▊      | 697/1813 [1:16:41<1:52:00,  6.02s/it]
+ 38%|███▊      | 698/1813 [1:16:46<1:47:41,  5.80s/it]
+ 39%|███▊      | 699/1813 [1:16:53<1:53:09,  6.09s/it]
+ 39%|███▊      | 700/1813 [1:16:59<1:55:01,  6.20s/it]
+                                                      
+{'loss': '1.004', 'grad_norm': '0.9849', 'learning_rate': '4.72e-05', 'epoch': '0.3862', 'num_input_tokens_seen': 4653312, 'train_runtime': '4620', 'train_tokens_per_second': '1007'}
+
+ 39%|███▊      | 700/1813 [1:16:59<1:55:01,  6.20s/it]
+ 39%|███▊      | 701/1813 [1:17:06<2:00:53,  6.52s/it]
+ 39%|███▊      | 702/1813 [1:17:13<1:59:05,  6.43s/it]
+ 39%|███▉      | 703/1813 [1:17:21<2:06:59,  6.86s/it]
+ 39%|███▉      | 704/1813 [1:17:26<2:00:09,  6.50s/it]
+ 39%|███▉      | 705/1813 [1:17:33<2:02:13,  6.62s/it]
+ 39%|███▉      | 706/1813 [1:17:39<2:00:08,  6.51s/it]
+ 39%|███▉      | 707/1813 [1:17:49<2:17:03,  7.44s/it]
+ 39%|███▉      | 708/1813 [1:17:56<2:16:23,  7.41s/it]
+ 39%|███▉      | 709/1813 [1:18:03<2:11:24,  7.14s/it]
+ 39%|███▉      | 710/1813 [1:18:08<2:02:59,  6.69s/it]
+                                                      
+{'loss': '0.9624', 'grad_norm': '0.7239', 'learning_rate': '4.669e-05', 'epoch': '0.3917', 'num_input_tokens_seen': 4732080, 'train_runtime': '4689', 'train_tokens_per_second': '1009'}
+
+ 39%|███▉      | 710/1813 [1:18:08<2:02:59,  6.69s/it]
+ 39%|███▉      | 711/1813 [1:18:15<2:03:00,  6.70s/it]
+ 39%|███▉      | 712/1813 [1:18:22<2:02:05,  6.65s/it]
+ 39%|███▉      | 713/1813 [1:18:27<1:55:54,  6.32s/it]
+ 39%|███▉      | 714/1813 [1:18:33<1:54:10,  6.23s/it]
+ 39%|███▉      | 715/1813 [1:18:40<1:55:45,  6.33s/it]
+ 39%|███▉      | 716/1813 [1:18:47<1:58:20,  6.47s/it]
+ 40%|███▉      | 717/1813 [1:18:53<1:55:49,  6.34s/it]
+ 40%|███▉      | 718/1813 [1:18:59<1:55:13,  6.31s/it]
+ 40%|███▉      | 719/1813 [1:19:04<1:47:48,  5.91s/it]
+ 40%|███▉      | 720/1813 [1:19:10<1:48:46,  5.97s/it]
+                                                      
+{'loss': '1.008', 'grad_norm': '1.08', 'learning_rate': '4.617e-05', 'epoch': '0.3972', 'num_input_tokens_seen': 4799552, 'train_runtime': '4751', 'train_tokens_per_second': '1010'}
+
+ 40%|███▉      | 720/1813 [1:19:10<1:48:46,  5.97s/it]
+ 40%|███▉      | 721/1813 [1:19:17<1:51:35,  6.13s/it]
+ 40%|███▉      | 722/1813 [1:19:22<1:45:36,  5.81s/it]
+ 40%|███▉      | 723/1813 [1:19:27<1:41:59,  5.61s/it]
+ 40%|███▉      | 724/1813 [1:19:35<1:54:05,  6.29s/it]
+ 40%|███▉      | 725/1813 [1:19:43<2:05:23,  6.92s/it]
+ 40%|████      | 726/1813 [1:19:49<2:02:46,  6.78s/it]
+ 40%|████      | 727/1813 [1:19:55<1:58:35,  6.55s/it]
+ 40%|████      | 728/1813 [1:20:01<1:51:45,  6.18s/it]
+ 40%|████      | 729/1813 [1:20:07<1:49:54,  6.08s/it]
+ 40%|████      | 730/1813 [1:20:12<1:46:23,  5.89s/it]
+                                                      
+{'loss': '0.9773', 'grad_norm': '1.841', 'learning_rate': '4.565e-05', 'epoch': '0.4028', 'num_input_tokens_seen': 4866864, 'train_runtime': '4813', 'train_tokens_per_second': '1011'}
+
+ 40%|████      | 730/1813 [1:20:12<1:46:23,  5.89s/it]
+ 40%|████      | 731/1813 [1:20:19<1:51:55,  6.21s/it]
+ 40%|████      | 732/1813 [1:20:25<1:51:41,  6.20s/it]
+ 40%|████      | 733/1813 [1:20:33<1:57:43,  6.54s/it]
+ 40%|████      | 734/1813 [1:20:39<1:59:37,  6.65s/it]
+ 41%|████      | 735/1813 [1:20:46<1:59:26,  6.65s/it]
+ 41%|████      | 736/1813 [1:20:52<1:54:27,  6.38s/it]
+ 41%|████      | 737/1813 [1:20:58<1:50:52,  6.18s/it]
+ 41%|████      | 738/1813 [1:21:03<1:44:46,  5.85s/it]
+ 41%|████      | 739/1813 [1:21:09<1:45:20,  5.89s/it]
+ 41%|████      | 740/1813 [1:21:15<1:49:10,  6.10s/it]
+                                                      
+{'loss': '0.9082', 'grad_norm': '0.7211', 'learning_rate': '4.512e-05', 'epoch': '0.4083', 'num_input_tokens_seen': 4936352, 'train_runtime': '4876', 'train_tokens_per_second': '1012'}
+
+ 41%|████      | 740/1813 [1:21:15<1:49:10,  6.10s/it]
+ 41%|████      | 741/1813 [1:21:22<1:50:36,  6.19s/it]
+ 41%|████      | 742/1813 [1:21:29<1:57:04,  6.56s/it]
+ 41%|████      | 743/1813 [1:21:36<2:01:28,  6.81s/it]
+ 41%|████      | 744/1813 [1:21:43<2:00:06,  6.74s/it]
+ 41%|████      | 745/1813 [1:21:50<2:01:15,  6.81s/it]
+ 41%|████      | 746/1813 [1:21:56<1:58:09,  6.64s/it]
+ 41%|████      | 747/1813 [1:22:04<2:04:41,  7.02s/it]
+ 41%|████▏     | 748/1813 [1:22:10<1:56:16,  6.55s/it]
+ 41%|████▏     | 749/1813 [1:22:16<1:55:26,  6.51s/it]
+ 41%|████▏     | 750/1813 [1:22:22<1:51:19,  6.28s/it]
+                                                      
+{'loss': '0.9874', 'grad_norm': '0.8073', 'learning_rate': '4.459e-05', 'epoch': '0.4138', 'num_input_tokens_seen': 5011064, 'train_runtime': '4942', 'train_tokens_per_second': '1014'}
+
+ 41%|████▏     | 750/1813 [1:22:22<1:51:19,  6.28s/it]
+ 41%|████▏     | 751/1813 [1:22:28<1:49:11,  6.17s/it]
+ 41%|████▏     | 752/1813 [1:22:33<1:47:11,  6.06s/it]
+ 42%|████▏     | 753/1813 [1:22:40<1:49:28,  6.20s/it]
+ 42%|████▏     | 754/1813 [1:22:49<2:01:46,  6.90s/it]
+ 42%|████▏     | 755/1813 [1:22:53<1:51:10,  6.30s/it]
+ 42%|████▏     | 756/1813 [1:22:59<1:47:00,  6.07s/it]
+ 42%|████▏     | 757/1813 [1:23:05<1:46:37,  6.06s/it]
+ 42%|████▏     | 758/1813 [1:23:12<1:50:32,  6.29s/it]
+ 42%|████▏     | 759/1813 [1:23:19<1:54:12,  6.50s/it]
+ 42%|████▏     | 760/1813 [1:23:25<1:52:25,  6.41s/it]
+                                                      
+{'loss': '1.001', 'grad_norm': '1.073', 'learning_rate': '4.405e-05', 'epoch': '0.4193', 'num_input_tokens_seen': 5078960, 'train_runtime': '5006', 'train_tokens_per_second': '1015'}
+
+ 42%|████▏     | 760/1813 [1:23:25<1:52:25,  6.41s/it]
+ 42%|████▏     | 761/1813 [1:23:32<1:53:45,  6.49s/it]
+ 42%|████▏     | 762/1813 [1:23:38<1:54:43,  6.55s/it]
+ 42%|████▏     | 763/1813 [1:23:46<1:58:16,  6.76s/it]
+ 42%|████▏     | 764/1813 [1:23:52<1:58:42,  6.79s/it]
+ 42%|████▏     | 765/1813 [1:24:00<2:02:08,  6.99s/it]
+ 42%|████▏     | 766/1813 [1:24:06<1:58:46,  6.81s/it]
+ 42%|████▏     | 767/1813 [1:24:12<1:51:11,  6.38s/it]
+ 42%|████▏     | 768/1813 [1:24:16<1:39:18,  5.70s/it]
+ 42%|████▏     | 769/1813 [1:24:22<1:44:12,  5.99s/it]
+ 42%|████▏     | 770/1813 [1:24:29<1:45:54,  6.09s/it]
+                                                      
+{'loss': '0.9338', 'grad_norm': '0.9095', 'learning_rate': '4.351e-05', 'epoch': '0.4248', 'num_input_tokens_seen': 5148504, 'train_runtime': '5069', 'train_tokens_per_second': '1016'}
+
+ 42%|████▏     | 770/1813 [1:24:29<1:45:54,  6.09s/it]
+ 43%|████▎     | 771/1813 [1:24:35<1:48:12,  6.23s/it]
+ 43%|████▎     | 772/1813 [1:24:43<1:53:26,  6.54s/it]
+ 43%|████▎     | 773/1813 [1:24:49<1:50:54,  6.40s/it]
+ 43%|████▎     | 774/1813 [1:24:54<1:43:28,  5.98s/it]
+ 43%|████▎     | 775/1813 [1:25:00<1:44:30,  6.04s/it]
+ 43%|████▎     | 776/1813 [1:25:07<1:49:07,  6.31s/it]
+ 43%|████▎     | 777/1813 [1:25:15<1:56:29,  6.75s/it]
+ 43%|████▎     | 778/1813 [1:25:20<1:47:21,  6.22s/it]
+ 43%|████▎     | 779/1813 [1:25:25<1:41:14,  5.87s/it]
+ 43%|████▎     | 780/1813 [1:25:31<1:44:55,  6.09s/it]
+                                                      
+{'loss': '0.8896', 'grad_norm': '0.9677', 'learning_rate': '4.297e-05', 'epoch': '0.4303', 'num_input_tokens_seen': 5215784, 'train_runtime': '5132', 'train_tokens_per_second': '1016'}
+
+ 43%|████▎     | 780/1813 [1:25:31<1:44:55,  6.09s/it]
+ 43%|████▎     | 781/1813 [1:25:38<1:47:50,  6.27s/it]
+ 43%|████▎     | 782/1813 [1:25:46<1:57:39,  6.85s/it]
+ 43%|████▎     | 783/1813 [1:25:53<1:59:06,  6.94s/it]
+ 43%|████▎     | 784/1813 [1:25:59<1:54:26,  6.67s/it]
+ 43%|████▎     | 785/1813 [1:26:06<1:56:37,  6.81s/it]
+ 43%|████▎     | 786/1813 [1:26:14<1:58:36,  6.93s/it]
+ 43%|████▎     | 787/1813 [1:26:19<1:50:10,  6.44s/it]
+ 43%|████▎     | 788/1813 [1:26:25<1:45:49,  6.19s/it]
+ 44%|████▎     | 789/1813 [1:26:30<1:41:27,  5.94s/it]
+ 44%|████▎     | 790/1813 [1:26:36<1:40:36,  5.90s/it]
+                                                      
+{'loss': '1.006', 'grad_norm': '0.9625', 'learning_rate': '4.242e-05', 'epoch': '0.4359', 'num_input_tokens_seen': 5285160, 'train_runtime': '5196', 'train_tokens_per_second': '1017'}
+
+ 44%|████▎     | 790/1813 [1:26:36<1:40:36,  5.90s/it]
+ 44%|████▎     | 791/1813 [1:26:42<1:41:28,  5.96s/it]
+ 44%|████▎     | 792/1813 [1:26:49<1:47:20,  6.31s/it]
+ 44%|████▎     | 793/1813 [1:26:56<1:52:59,  6.65s/it]
+ 44%|████▍     | 794/1813 [1:27:03<1:51:50,  6.59s/it]
+ 44%|████▍     | 795/1813 [1:27:09<1:47:52,  6.36s/it]
+ 44%|████▍     | 796/1813 [1:27:15<1:49:54,  6.48s/it]
+ 44%|████▍     | 797/1813 [1:27:23<1:55:18,  6.81s/it]
+ 44%|████▍     | 798/1813 [1:27:29<1:51:29,  6.59s/it]
+ 44%|████▍     | 799/1813 [1:27:35<1:49:28,  6.48s/it]
+ 44%|████▍     | 800/1813 [1:27:41<1:47:14,  6.35s/it]
+                                                      
+{'loss': '0.9486', 'grad_norm': '0.8092', 'learning_rate': '4.186e-05', 'epoch': '0.4414', 'num_input_tokens_seen': 5357720, 'train_runtime': '5262', 'train_tokens_per_second': '1018'}
+
+ 44%|████▍     | 800/1813 [1:27:41<1:47:14,  6.35s/it]
+ 44%|████▍     | 801/1813 [1:27:47<1:43:27,  6.13s/it]
+ 44%|████▍     | 802/1813 [1:27:53<1:43:28,  6.14s/it]
+ 44%|████▍     | 803/1813 [1:28:01<1:52:12,  6.67s/it]
+ 44%|████▍     | 804/1813 [1:28:07<1:49:15,  6.50s/it]
+ 44%|████▍     | 805/1813 [1:28:13<1:45:19,  6.27s/it]
+ 44%|████▍     | 806/1813 [1:28:19<1:43:27,  6.16s/it]
+ 45%|████▍     | 807/1813 [1:28:24<1:40:43,  6.01s/it]
+ 45%|████▍     | 808/1813 [1:28:32<1:49:39,  6.55s/it]
+ 45%|████▍     | 809/1813 [1:28:39<1:51:04,  6.64s/it]
+ 45%|████▍     | 810/1813 [1:28:47<1:56:53,  6.99s/it]
+                                                      
+{'loss': '0.9632', 'grad_norm': '4.587', 'learning_rate': '4.13e-05', 'epoch': '0.4469', 'num_input_tokens_seen': 5430288, 'train_runtime': '5327', 'train_tokens_per_second': '1019'}
+
+ 45%|████▍     | 810/1813 [1:28:47<1:56:53,  6.99s/it]
+ 45%|████▍     | 811/1813 [1:28:54<1:55:41,  6.93s/it]
+ 45%|████▍     | 812/1813 [1:29:00<1:50:02,  6.60s/it]
+ 45%|████▍     | 813/1813 [1:29:05<1:45:39,  6.34s/it]
+ 45%|████▍     | 814/1813 [1:29:11<1:44:35,  6.28s/it]
+ 45%|████▍     | 815/1813 [1:29:16<1:37:56,  5.89s/it]
+ 45%|████▌     | 816/1813 [1:29:23<1:42:41,  6.18s/it]
+ 45%|████▌     | 817/1813 [1:29:29<1:39:49,  6.01s/it]
+ 45%|████▌     | 818/1813 [1:29:36<1:44:56,  6.33s/it]
+ 45%|████▌     | 819/1813 [1:29:42<1:45:09,  6.35s/it]
+ 45%|████▌     | 820/1813 [1:29:50<1:49:47,  6.63s/it]
+                                                      
+{'loss': '0.9111', 'grad_norm': '0.9569', 'learning_rate': '4.074e-05', 'epoch': '0.4524', 'num_input_tokens_seen': 5497552, 'train_runtime': '5390', 'train_tokens_per_second': '1020'}
+
+ 45%|████▌     | 820/1813 [1:29:50<1:49:47,  6.63s/it]
+ 45%|████▌     | 821/1813 [1:29:56<1:48:56,  6.59s/it]
+ 45%|████▌     | 822/1813 [1:30:01<1:41:16,  6.13s/it]
+ 45%|████▌     | 823/1813 [1:30:06<1:35:03,  5.76s/it]
+ 45%|████▌     | 824/1813 [1:30:13<1:38:14,  5.96s/it]
+ 46%|████▌     | 825/1813 [1:30:19<1:40:12,  6.09s/it]
+ 46%|████▌     | 826/1813 [1:30:26<1:44:08,  6.33s/it]
+ 46%|████▌     | 827/1813 [1:30:32<1:42:14,  6.22s/it]
+ 46%|████▌     | 828/1813 [1:30:38<1:42:05,  6.22s/it]
+ 46%|████▌     | 829/1813 [1:30:44<1:39:32,  6.07s/it]
+ 46%|████▌     | 830/1813 [1:30:51<1:45:50,  6.46s/it]
+                                                      
+{'loss': '0.8041', 'grad_norm': '0.8592', 'learning_rate': '4.018e-05', 'epoch': '0.4579', 'num_input_tokens_seen': 5563144, 'train_runtime': '5452', 'train_tokens_per_second': '1020'}
+
+ 46%|████▌     | 830/1813 [1:30:51<1:45:50,  6.46s/it]
+ 46%|████▌     | 831/1813 [1:30:57<1:44:19,  6.37s/it]
+ 46%|████▌     | 832/1813 [1:31:05<1:50:01,  6.73s/it]
+ 46%|████▌     | 833/1813 [1:31:11<1:48:32,  6.64s/it]
+ 46%|████▌     | 834/1813 [1:31:18<1:48:46,  6.67s/it]
+ 46%|████▌     | 835/1813 [1:31:24<1:46:06,  6.51s/it]
+ 46%|████▌     | 836/1813 [1:31:31<1:47:13,  6.59s/it]
+ 46%|████▌     | 837/1813 [1:31:39<1:53:10,  6.96s/it]
+ 46%|████▌     | 838/1813 [1:31:45<1:48:06,  6.65s/it]
+ 46%|████▋     | 839/1813 [1:31:51<1:45:55,  6.52s/it]
+ 46%|████▋     | 840/1813 [1:31:57<1:42:12,  6.30s/it]
+                                                      
+{'loss': '0.9765', 'grad_norm': '1.009', 'learning_rate': '3.961e-05', 'epoch': '0.4634', 'num_input_tokens_seen': 5634104, 'train_runtime': '5517', 'train_tokens_per_second': '1021'}
+
+ 46%|████▋     | 840/1813 [1:31:57<1:42:12,  6.30s/it]
+ 46%|████▋     | 841/1813 [1:32:03<1:40:38,  6.21s/it]
+ 46%|████▋     | 842/1813 [1:32:09<1:42:36,  6.34s/it]
+ 46%|████▋     | 843/1813 [1:32:15<1:40:34,  6.22s/it]
+ 47%|████▋     | 844/1813 [1:32:21<1:36:22,  5.97s/it]
+ 47%|████▋     | 845/1813 [1:32:27<1:38:20,  6.10s/it]
+ 47%|████▋     | 846/1813 [1:32:34<1:41:26,  6.29s/it]
+ 47%|████▋     | 847/1813 [1:32:41<1:45:31,  6.55s/it]
+ 47%|████▋     | 848/1813 [1:32:48<1:49:10,  6.79s/it]
+ 47%|████▋     | 849/1813 [1:32:54<1:42:16,  6.37s/it]
+ 47%|████▋     | 850/1813 [1:33:02<1:51:49,  6.97s/it]
+                                                      
+{'loss': '0.9322', 'grad_norm': '0.7625', 'learning_rate': '3.904e-05', 'epoch': '0.469', 'num_input_tokens_seen': 5704744, 'train_runtime': '5583', 'train_tokens_per_second': '1022'}
+
+ 47%|████▋     | 850/1813 [1:33:02<1:51:49,  6.97s/it]
+ 47%|████▋     | 851/1813 [1:33:09<1:50:30,  6.89s/it]
+ 47%|████▋     | 852/1813 [1:33:14<1:41:04,  6.31s/it]
+ 47%|████▋     | 853/1813 [1:33:20<1:40:37,  6.29s/it]
+ 47%|████▋     | 854/1813 [1:33:27<1:42:39,  6.42s/it]
+ 47%|████▋     | 855/1813 [1:33:33<1:40:56,  6.32s/it]
+ 47%|████▋     | 856/1813 [1:33:39<1:40:30,  6.30s/it]
+ 47%|████▋     | 857/1813 [1:33:44<1:35:36,  6.00s/it]
+ 47%|████▋     | 858/1813 [1:33:48<1:25:10,  5.35s/it]
+ 47%|████▋     | 859/1813 [1:33:54<1:28:45,  5.58s/it]
+ 47%|████▋     | 860/1813 [1:34:01<1:33:02,  5.86s/it]
+                                                      
+{'loss': '0.9544', 'grad_norm': '0.955', 'learning_rate': '3.846e-05', 'epoch': '0.4745', 'num_input_tokens_seen': 5766424, 'train_runtime': '5641', 'train_tokens_per_second': '1022'}
+
+ 47%|████▋     | 860/1813 [1:34:01<1:33:02,  5.86s/it]
+ 47%|████▋     | 861/1813 [1:34:06<1:29:41,  5.65s/it]
+ 48%|████▊     | 862/1813 [1:34:13<1:35:10,  6.01s/it]
+ 48%|████▊     | 863/1813 [1:34:19<1:35:32,  6.03s/it]
+ 48%|████▊     | 864/1813 [1:34:23<1:27:43,  5.55s/it]
+ 48%|████▊     | 865/1813 [1:34:30<1:33:35,  5.92s/it]
+ 48%|████▊     | 866/1813 [1:34:36<1:34:07,  5.96s/it]
+ 48%|████▊     | 867/1813 [1:34:42<1:32:00,  5.84s/it]
+ 48%|████▊     | 868/1813 [1:34:47<1:29:33,  5.69s/it]
+ 48%|████▊     | 869/1813 [1:34:54<1:36:33,  6.14s/it]
+ 48%|████▊     | 870/1813 [1:35:02<1:43:58,  6.62s/it]
+                                                      
+{'loss': '0.9383', 'grad_norm': '0.7322', 'learning_rate': '3.789e-05', 'epoch': '0.48', 'num_input_tokens_seen': 5831352, 'train_runtime': '5702', 'train_tokens_per_second': '1023'}
+
+ 48%|████▊     | 870/1813 [1:35:02<1:43:58,  6.62s/it]
+ 48%|████▊     | 871/1813 [1:35:07<1:38:50,  6.30s/it]
+ 48%|████▊     | 872/1813 [1:35:15<1:43:54,  6.63s/it]
+ 48%|████▊     | 873/1813 [1:35:22<1:44:58,  6.70s/it]
+ 48%|████▊     | 874/1813 [1:35:29<1:48:51,  6.96s/it]
+ 48%|████▊     | 875/1813 [1:35:37<1:52:29,  7.20s/it]
+ 48%|████▊     | 876/1813 [1:35:44<1:51:22,  7.13s/it]
+ 48%|████▊     | 877/1813 [1:35:52<1:55:30,  7.40s/it]
+ 48%|████▊     | 878/1813 [1:35:59<1:51:14,  7.14s/it]
+ 48%|████▊     | 879/1813 [1:36:06<1:53:41,  7.30s/it]
+ 49%|████▊     | 880/1813 [1:36:11<1:40:41,  6.48s/it]
+                                                      
+{'loss': '0.9612', 'grad_norm': '1.316', 'learning_rate': '3.731e-05', 'epoch': '0.4855', 'num_input_tokens_seen': 5907208, 'train_runtime': '5771', 'train_tokens_per_second': '1024'}
+
+ 49%|████▊     | 880/1813 [1:36:11<1:40:41,  6.48s/it]
+ 49%|████▊     | 881/1813 [1:36:16<1:34:56,  6.11s/it]
+ 49%|████▊     | 882/1813 [1:36:22<1:34:53,  6.12s/it]
+ 49%|████▊     | 883/1813 [1:36:29<1:37:38,  6.30s/it]
+ 49%|████▉     | 884/1813 [1:36:35<1:36:08,  6.21s/it]
+ 49%|████▉     | 885/1813 [1:36:41<1:34:42,  6.12s/it]
+ 49%|████▉     | 886/1813 [1:36:47<1:35:33,  6.18s/it]
+ 49%|████▉     | 887/1813 [1:36:55<1:44:08,  6.75s/it]
+ 49%|████▉     | 888/1813 [1:37:02<1:44:23,  6.77s/it]
+ 49%|████▉     | 889/1813 [1:37:09<1:46:41,  6.93s/it]
+ 49%|████▉     | 890/1813 [1:37:15<1:42:44,  6.68s/it]
+                                                      
+{'loss': '0.9312', 'grad_norm': '0.7889', 'learning_rate': '3.673e-05', 'epoch': '0.491', 'num_input_tokens_seen': 5978048, 'train_runtime': '5836', 'train_tokens_per_second': '1024'}
+
+ 49%|████▉     | 890/1813 [1:37:15<1:42:44,  6.68s/it]
+ 49%|████▉     | 891/1813 [1:37:21<1:39:25,  6.47s/it]
+ 49%|████▉     | 892/1813 [1:37:29<1:43:11,  6.72s/it]
+ 49%|████▉     | 893/1813 [1:37:35<1:39:39,  6.50s/it]
+ 49%|████▉     | 894/1813 [1:37:40<1:35:25,  6.23s/it]
+ 49%|████▉     | 895/1813 [1:37:48<1:40:09,  6.55s/it]
+ 49%|████▉     | 896/1813 [1:37:53<1:34:47,  6.20s/it]
+ 49%|████▉     | 897/1813 [1:37:59<1:33:49,  6.15s/it]
+ 50%|████▉     | 898/1813 [1:38:06<1:38:33,  6.46s/it]
+ 50%|████▉     | 899/1813 [1:38:14<1:42:22,  6.72s/it]
+ 50%|████▉     | 900/1813 [1:38:21<1:46:46,  7.02s/it]
+                                                      
+{'loss': '0.9929', 'grad_norm': '0.8777', 'learning_rate': '3.614e-05', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856, 'train_runtime': '5902', 'train_tokens_per_second': '1025'}
+
+ 50%|████▉     | 900/1813 [1:38:21<1:46:46,  7.02s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.73it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.21it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.08it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  7.00it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.83it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.06it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:40,  6.09it/s]
+
+  2%|▏         | 23/1000 [00:03<02:57,  5.50it/s]
+
+  2%|▎         | 25/1000 [00:03<02:14,  7.27it/s]
+
+  3%|▎         | 26/1000 [00:04<02:36,  6.22it/s]
+
+  3%|▎         | 28/1000 [00:04<02:46,  5.82it/s]
+
+  3%|▎         | 29/1000 [00:04<02:36,  6.22it/s]
+
+  3%|▎         | 30/1000 [00:04<02:45,  5.88it/s]
+
+  3%|▎         | 31/1000 [00:04<03:02,  5.30it/s]
+
+  3%|▎         | 32/1000 [00:05<02:43,  5.93it/s]
+
+  3%|▎         | 33/1000 [00:05<02:49,  5.71it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.63it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.09it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:06<02:04,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.09it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.32it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.38it/s]
+
+  5%|▍         | 49/1000 [00:07<01:48,  8.75it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.89it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.75it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.31it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.21it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.25it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:17,  6.73it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.13it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.20it/s]
+
+  8%|▊         | 84/1000 [00:11<01:53,  8.04it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.96it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.02it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.26it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.09it/s]
+
+  9%|▉         | 92/1000 [00:12<02:26,  6.19it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.53it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:12,  6.80it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.62it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.30it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.43it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.20it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.03it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.51it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.80it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.22it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.44it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.65it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.91it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.05it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.90it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.08it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.92it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.04it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.41it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.66it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.99it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.62it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.42it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:26,  5.68it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.17it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.17it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.23it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.44it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.59it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.52it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.10it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.64it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.20it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.71it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.79it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.64it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.94it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.53it/s]
+
+ 21%|██        | 210/1000 [00:27<01:32,  8.50it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.70it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.73it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.51it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.80it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:23,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.80it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.02it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.20it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.94it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.45it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.96it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.36it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.27it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:25,  8.71it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:55,  6.41it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.32it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.58it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.63it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.05it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.77it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.79it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.02it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.20it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.90it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.20it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.15it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:41,  6.80it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.81it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.27it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.59it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.83it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.04it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.86it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.97it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.60it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.43it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.53it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.18it/s]
+
+ 38%|███▊      | 379/1000 [00:50<01:16,  8.15it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.18it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.54it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.07it/s]
+
+ 39%|███▊      | 387/1000 [00:51<01:21,  7.56it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.30it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.83it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.48it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  9.00it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.53it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.84it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.51it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.26it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.49it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.68it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.94it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.55it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.02it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:48, 11.53it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:54, 10.40it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.04it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.27it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.08it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.23it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.70it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.47it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.18it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:50, 10.75it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.60it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.50it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.40it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.33it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.23it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.29it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.94it/s]
+
+ 50%|████▉     | 496/1000 [01:06<01:00,  8.32it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.38it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.63it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.91it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.53it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.79it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.97it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.04it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.55it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.68it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.68it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.35it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.49it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.96it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.86it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.99it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.84it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.59it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.81it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.40it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.41it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.81it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.62it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.10it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.22it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.47it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.65it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.20it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.23it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.88it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.44it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.08it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.96it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.19it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.97it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.57it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.33it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.71it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.57it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.67it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.71it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:28, 11.01it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.95it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.90it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.82it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.62it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.23it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.51it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.72it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.87it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.39it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.08it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.48it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.12it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.60it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.51it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.62it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.46it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:28,  8.66it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.70it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.18it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.25it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.42it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.63it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.72it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.66it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.78it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.79it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.64it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.96it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:25,  8.39it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.18it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.55it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.38it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.01it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.24it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.75it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.99it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.21it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.03it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.73it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:25,  7.84it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.16it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.20it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.23it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.31it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.12it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.46it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.56it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.50it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:13, 10.22it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.53it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.78it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.64it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.68it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.31it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.27it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.29it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.61it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.97it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.76it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.75it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.84it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.06it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.33it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.60it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.78it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.40it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.18it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.09it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.62it/s]
+
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.32it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.85it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.06it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.41it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.73it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.42it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.83it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.19it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.81it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.83it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.44it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.07it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.45it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.95it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.7902', 'eval_runtime': '135.3', 'eval_samples_per_second': '7.392', 'eval_steps_per_second': '7.392', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856}
+
+ 50%|████▉     | 900/1813 [1:40:37<1:46:46,  7.02s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 50%|████▉     | 901/1813 [1:40:43<11:59:53, 47.36s/it]
+ 50%|████▉     | 902/1813 [1:40:49<8:50:00, 34.91s/it] 
+ 50%|████▉     | 903/1813 [1:40:55<6:37:47, 26.23s/it]
+ 50%|████▉     | 904/1813 [1:41:01<5:06:27, 20.23s/it]
+ 50%|████▉     | 905/1813 [1:41:09<4:10:24, 16.55s/it]
+ 50%|████▉     | 906/1813 [1:41:14<3:19:29, 13.20s/it]
+ 50%|█████     | 907/1813 [1:41:19<2:40:09, 10.61s/it]
+ 50%|█████     | 908/1813 [1:41:24<2:17:48,  9.14s/it]
+ 50%|█████     | 909/1813 [1:41:31<2:06:24,  8.39s/it]
+ 50%|█████     | 910/1813 [1:41:37<1:55:50,  7.70s/it]
+                                                      
+{'loss': '0.9268', 'grad_norm': '1.065', 'learning_rate': '3.556e-05', 'epoch': '0.5021', 'num_input_tokens_seen': 6115160, 'train_runtime': '6098', 'train_tokens_per_second': '1003'}
+
+ 50%|█████     | 910/1813 [1:41:37<1:55:50,  7.70s/it]
+ 50%|█████     | 911/1813 [1:41:43<1:46:19,  7.07s/it]
+ 50%|█████     | 912/1813 [1:41:48<1:39:52,  6.65s/it]
+ 50%|█████     | 913/1813 [1:41:54<1:34:30,  6.30s/it]
+ 50%|█████     | 914/1813 [1:41:59<1:28:03,  5.88s/it]
+ 50%|█████     | 915/1813 [1:42:07<1:37:04,  6.49s/it]
+ 51%|█████     | 916/1813 [1:42:14<1:40:15,  6.71s/it]
+ 51%|█████     | 917/1813 [1:42:21<1:40:18,  6.72s/it]
+ 51%|█████     | 918/1813 [1:42:26<1:35:37,  6.41s/it]
+ 51%|█████     | 919/1813 [1:42:33<1:35:05,  6.38s/it]
+ 51%|█████     | 920/1813 [1:42:38<1:31:08,  6.12s/it]
+                                                      
+{'loss': '0.9623', 'grad_norm': '0.7494', 'learning_rate': '3.497e-05', 'epoch': '0.5076', 'num_input_tokens_seen': 6178904, 'train_runtime': '6159', 'train_tokens_per_second': '1003'}
+
+ 51%|█████     | 920/1813 [1:42:38<1:31:08,  6.12s/it]
+ 51%|█████     | 921/1813 [1:42:44<1:29:42,  6.03s/it]
+ 51%|█████     | 922/1813 [1:42:50<1:27:33,  5.90s/it]
+ 51%|█████     | 923/1813 [1:42:55<1:26:16,  5.82s/it]
+ 51%|█████     | 924/1813 [1:43:02<1:30:45,  6.13s/it]
+ 51%|█████     | 925/1813 [1:43:08<1:29:03,  6.02s/it]
+ 51%|█████     | 926/1813 [1:43:14<1:31:29,  6.19s/it]
+ 51%|█████     | 927/1813 [1:43:20<1:30:22,  6.12s/it]
+ 51%|█████     | 928/1813 [1:43:26<1:27:43,  5.95s/it]
+ 51%|█████     | 929/1813 [1:43:32<1:27:38,  5.95s/it]
+ 51%|█████▏    | 930/1813 [1:43:39<1:31:33,  6.22s/it]
+                                                      
+{'loss': '0.9239', 'grad_norm': '1.004', 'learning_rate': '3.439e-05', 'epoch': '0.5131', 'num_input_tokens_seen': 6244360, 'train_runtime': '6219', 'train_tokens_per_second': '1004'}
+
+ 51%|█████▏    | 930/1813 [1:43:39<1:31:33,  6.22s/it]
+ 51%|█████▏    | 931/1813 [1:43:46<1:34:11,  6.41s/it]
+ 51%|█████▏    | 932/1813 [1:43:51<1:29:46,  6.11s/it]
+ 51%|█████▏    | 933/1813 [1:43:57<1:28:48,  6.05s/it]
+ 52%|█████▏    | 934/1813 [1:44:04<1:32:20,  6.30s/it]
+ 52%|█████▏    | 935/1813 [1:44:13<1:43:32,  7.08s/it]
+ 52%|█████▏    | 936/1813 [1:44:19<1:41:08,  6.92s/it]
+ 52%|█████▏    | 937/1813 [1:44:26<1:38:16,  6.73s/it]
+ 52%|█████▏    | 938/1813 [1:44:33<1:40:15,  6.87s/it]
+ 52%|█████▏    | 939/1813 [1:44:39<1:35:33,  6.56s/it]
+ 52%|█████▏    | 940/1813 [1:44:46<1:37:29,  6.70s/it]
+                                                      
+{'loss': '0.9064', 'grad_norm': '1.104', 'learning_rate': '3.38e-05', 'epoch': '0.5186', 'num_input_tokens_seen': 6318968, 'train_runtime': '6286', 'train_tokens_per_second': '1005'}
+
+ 52%|█████▏    | 940/1813 [1:44:46<1:37:29,  6.70s/it]
+ 52%|█████▏    | 941/1813 [1:44:51<1:30:17,  6.21s/it]
+ 52%|█████▏    | 942/1813 [1:44:56<1:27:06,  6.00s/it]
+ 52%|█████▏    | 943/1813 [1:45:02<1:24:56,  5.86s/it]
+ 52%|█████▏    | 944/1813 [1:45:08<1:25:01,  5.87s/it]
+ 52%|█████▏    | 945/1813 [1:45:13<1:23:05,  5.74s/it]
+ 52%|█████▏    | 946/1813 [1:45:19<1:25:26,  5.91s/it]
+ 52%|█████▏    | 947/1813 [1:45:26<1:26:58,  6.03s/it]
+ 52%|█████▏    | 948/1813 [1:45:32<1:28:19,  6.13s/it]
+ 52%|█████▏    | 949/1813 [1:45:37<1:23:37,  5.81s/it]
+ 52%|█████▏    | 950/1813 [1:45:43<1:25:26,  5.94s/it]
+                                                      
+{'loss': '0.9587', 'grad_norm': '1.037', 'learning_rate': '3.321e-05', 'epoch': '0.5241', 'num_input_tokens_seen': 6379704, 'train_runtime': '6344', 'train_tokens_per_second': '1006'}
+
+ 52%|█████▏    | 950/1813 [1:45:43<1:25:26,  5.94s/it]
+ 52%|█████▏    | 951/1813 [1:45:49<1:23:15,  5.80s/it]
+ 53%|█████▎    | 952/1813 [1:45:54<1:21:57,  5.71s/it]
+ 53%|█████▎    | 953/1813 [1:46:00<1:22:46,  5.77s/it]
+ 53%|█████▎    | 954/1813 [1:46:06<1:22:47,  5.78s/it]
+ 53%|█████▎    | 955/1813 [1:46:12<1:22:22,  5.76s/it]
+ 53%|█████▎    | 956/1813 [1:46:17<1:21:25,  5.70s/it]
+ 53%|█████▎    | 957/1813 [1:46:23<1:20:37,  5.65s/it]
+ 53%|█████▎    | 958/1813 [1:46:27<1:15:46,  5.32s/it]
+ 53%|█████▎    | 959/1813 [1:46:33<1:17:51,  5.47s/it]
+ 53%|█████▎    | 960/1813 [1:46:41<1:29:16,  6.28s/it]
+                                                      
+{'loss': '0.9669', 'grad_norm': '0.7124', 'learning_rate': '3.262e-05', 'epoch': '0.5297', 'num_input_tokens_seen': 6441656, 'train_runtime': '6402', 'train_tokens_per_second': '1006'}
+
+ 53%|█████▎    | 960/1813 [1:46:41<1:29:16,  6.28s/it]
+ 53%|█████▎    | 961/1813 [1:46:48<1:30:31,  6.38s/it]
+ 53%|█████▎    | 962/1813 [1:46:56<1:36:18,  6.79s/it]
+ 53%|█████▎    | 963/1813 [1:47:03<1:36:15,  6.80s/it]
+ 53%|█████▎    | 964/1813 [1:47:08<1:31:45,  6.48s/it]
+ 53%|█████▎    | 965/1813 [1:47:14<1:28:59,  6.30s/it]
+ 53%|█████▎    | 966/1813 [1:47:22<1:34:03,  6.66s/it]
+ 53%|█████▎    | 967/1813 [1:47:29<1:37:01,  6.88s/it]
+ 53%|█████▎    | 968/1813 [1:47:35<1:31:33,  6.50s/it]
+ 53%|█████▎    | 969/1813 [1:47:41<1:30:14,  6.42s/it]
+ 54%|█████▎    | 970/1813 [1:47:46<1:23:49,  5.97s/it]
+                                                      
+{'loss': '1.012', 'grad_norm': '1.124', 'learning_rate': '3.203e-05', 'epoch': '0.5352', 'num_input_tokens_seen': 6512320, 'train_runtime': '6466', 'train_tokens_per_second': '1007'}
+
+ 54%|█████▎    | 970/1813 [1:47:46<1:23:49,  5.97s/it]
+ 54%|█████▎    | 971/1813 [1:47:53<1:27:29,  6.23s/it]
+ 54%|█████▎    | 972/1813 [1:47:59<1:28:38,  6.32s/it]
+ 54%|█████▎    | 973/1813 [1:48:05<1:28:02,  6.29s/it]
+ 54%|█████▎    | 974/1813 [1:48:14<1:36:47,  6.92s/it]
+ 54%|█████▍    | 975/1813 [1:48:20<1:33:51,  6.72s/it]
+ 54%|█████▍    | 976/1813 [1:48:26<1:28:27,  6.34s/it]
+ 54%|█████▍    | 977/1813 [1:48:33<1:34:05,  6.75s/it]
+ 54%|█████▍    | 978/1813 [1:48:40<1:32:10,  6.62s/it]
+ 54%|█████▍    | 979/1813 [1:48:47<1:34:32,  6.80s/it]
+ 54%|█████▍    | 980/1813 [1:48:53<1:33:53,  6.76s/it]
+                                                      
+{'loss': '0.9394', 'grad_norm': '1.628', 'learning_rate': '3.144e-05', 'epoch': '0.5407', 'num_input_tokens_seen': 6586672, 'train_runtime': '6534', 'train_tokens_per_second': '1008'}
+
+ 54%|█████▍    | 980/1813 [1:48:53<1:33:53,  6.76s/it]
+ 54%|█████▍    | 981/1813 [1:49:00<1:32:58,  6.70s/it]
+ 54%|█████▍    | 982/1813 [1:49:05<1:26:11,  6.22s/it]
+ 54%|█████▍    | 983/1813 [1:49:10<1:21:58,  5.93s/it]
+ 54%|█████▍    | 984/1813 [1:49:16<1:19:31,  5.76s/it]
+ 54%|█████▍    | 985/1813 [1:49:22<1:22:54,  6.01s/it]
+ 54%|█████▍    | 986/1813 [1:49:30<1:28:42,  6.44s/it]
+ 54%|█████▍    | 987/1813 [1:49:36<1:26:46,  6.30s/it]
+ 54%|█████▍    | 988/1813 [1:49:41<1:23:09,  6.05s/it]
+ 55%|█████▍    | 989/1813 [1:49:46<1:18:38,  5.73s/it]
+ 55%|█████▍    | 990/1813 [1:49:54<1:27:34,  6.38s/it]
+                                                      
+{'loss': '0.9136', 'grad_norm': '0.9393', 'learning_rate': '3.084e-05', 'epoch': '0.5462', 'num_input_tokens_seen': 6653096, 'train_runtime': '6595', 'train_tokens_per_second': '1009'}
+
+ 55%|█████▍    | 990/1813 [1:49:54<1:27:34,  6.38s/it]
+ 55%|█████▍    | 991/1813 [1:49:59<1:22:16,  6.01s/it]
+ 55%|█████▍    | 992/1813 [1:50:06<1:27:30,  6.40s/it]
+ 55%|█████▍    | 993/1813 [1:50:12<1:25:39,  6.27s/it]
+ 55%|█████▍    | 994/1813 [1:50:21<1:34:23,  6.91s/it]
+ 55%|█████▍    | 995/1813 [1:50:27<1:29:26,  6.56s/it]
+ 55%|█████▍    | 996/1813 [1:50:33<1:28:15,  6.48s/it]
+ 55%|█████▍    | 997/1813 [1:50:39<1:25:45,  6.31s/it]
+ 55%|█████▌    | 998/1813 [1:50:44<1:21:48,  6.02s/it]
+ 55%|█████▌    | 999/1813 [1:50:50<1:20:42,  5.95s/it]
+ 55%|█████▌    | 1000/1813 [1:50:55<1:17:48,  5.74s/it]
+                                                       
+{'loss': '1.01', 'grad_norm': '0.8547', 'learning_rate': '3.025e-05', 'epoch': '0.5517', 'num_input_tokens_seen': 6716784, 'train_runtime': '6656', 'train_tokens_per_second': '1009'}
+
+ 55%|█████▌    | 1000/1813 [1:50:55<1:17:48,  5.74s/it]
+ 55%|█████▌    | 1001/1813 [1:51:02<1:21:29,  6.02s/it]
+ 55%|█████▌    | 1002/1813 [1:51:08<1:22:46,  6.12s/it]
+ 55%|█████▌    | 1003/1813 [1:51:15<1:26:23,  6.40s/it]
+ 55%|█████▌    | 1004/1813 [1:51:21<1:21:41,  6.06s/it]
+ 55%|█████▌    | 1005/1813 [1:51:28<1:25:30,  6.35s/it]
+ 55%|█████▌    | 1006/1813 [1:51:34<1:25:46,  6.38s/it]
+ 56%|█████▌    | 1007/1813 [1:51:40<1:22:27,  6.14s/it]
+ 56%|█████▌    | 1008/1813 [1:51:45<1:20:44,  6.02s/it]
+ 56%|█████▌    | 1009/1813 [1:51:51<1:20:38,  6.02s/it]
+ 56%|█████▌    | 1010/1813 [1:51:57<1:19:39,  5.95s/it]
+                                                       
+{'loss': '0.9565', 'grad_norm': '1.152', 'learning_rate': '2.966e-05', 'epoch': '0.5572', 'num_input_tokens_seen': 6783768, 'train_runtime': '6718', 'train_tokens_per_second': '1010'}
+
+ 56%|█████▌    | 1010/1813 [1:51:57<1:19:39,  5.95s/it]
+ 56%|█████▌    | 1011/1813 [1:52:04<1:23:07,  6.22s/it]
+ 56%|█████▌    | 1012/1813 [1:52:09<1:17:09,  5.78s/it]
+ 56%|█████▌    | 1013/1813 [1:52:13<1:11:14,  5.34s/it]
+ 56%|█████▌    | 1014/1813 [1:52:21<1:22:59,  6.23s/it]
+ 56%|█████▌    | 1015/1813 [1:52:27<1:18:39,  5.91s/it]
+ 56%|█████▌    | 1016/1813 [1:52:32<1:15:45,  5.70s/it]
+ 56%|█████▌    | 1017/1813 [1:52:39<1:20:05,  6.04s/it]
+ 56%|█████▌    | 1018/1813 [1:52:45<1:20:50,  6.10s/it]
+ 56%|█████▌    | 1019/1813 [1:52:50<1:15:55,  5.74s/it]
+ 56%|█████▋    | 1020/1813 [1:52:55<1:14:11,  5.61s/it]
+                                                       
+{'loss': '0.8749', 'grad_norm': '1.051', 'learning_rate': '2.907e-05', 'epoch': '0.5628', 'num_input_tokens_seen': 6844992, 'train_runtime': '6776', 'train_tokens_per_second': '1010'}
+
+ 56%|█████▋    | 1020/1813 [1:52:55<1:14:11,  5.61s/it]
+ 56%|█████▋    | 1021/1813 [1:53:02<1:19:53,  6.05s/it]
+ 56%|█████▋    | 1022/1813 [1:53:08<1:20:44,  6.12s/it]
+ 56%|█████▋    | 1023/1813 [1:53:16<1:26:33,  6.57s/it]
+ 56%|█████▋    | 1024/1813 [1:53:22<1:25:15,  6.48s/it]
+ 57%|█████▋    | 1025/1813 [1:53:31<1:35:29,  7.27s/it]
+ 57%|█████▋    | 1026/1813 [1:53:37<1:28:40,  6.76s/it]
+ 57%|█████▋    | 1027/1813 [1:53:43<1:26:57,  6.64s/it]
+ 57%|█████▋    | 1028/1813 [1:53:51<1:30:21,  6.91s/it]
+ 57%|█████▋    | 1029/1813 [1:53:57<1:28:59,  6.81s/it]
+ 57%|█████▋    | 1030/1813 [1:54:04<1:29:09,  6.83s/it]
+                                                       
+{'loss': '1.009', 'grad_norm': '0.7467', 'learning_rate': '2.848e-05', 'epoch': '0.5683', 'num_input_tokens_seen': 6920856, 'train_runtime': '6845', 'train_tokens_per_second': '1011'}
+
+ 57%|█████▋    | 1030/1813 [1:54:04<1:29:09,  6.83s/it]
+ 57%|█████▋    | 1031/1813 [1:54:10<1:24:48,  6.51s/it]
+ 57%|█████▋    | 1032/1813 [1:54:17<1:25:46,  6.59s/it]
+ 57%|█████▋    | 1033/1813 [1:54:23<1:24:27,  6.50s/it]
+ 57%|█████▋    | 1034/1813 [1:54:29<1:22:05,  6.32s/it]
+ 57%|█████▋    | 1035/1813 [1:54:35<1:21:56,  6.32s/it]
+ 57%|█████▋    | 1036/1813 [1:54:42<1:23:16,  6.43s/it]
+ 57%|█████▋    | 1037/1813 [1:54:48<1:20:46,  6.25s/it]
+ 57%|█████▋    | 1038/1813 [1:54:54<1:21:03,  6.28s/it]
+ 57%|█████▋    | 1039/1813 [1:55:01<1:23:18,  6.46s/it]
+ 57%|█████▋    | 1040/1813 [1:55:08<1:24:44,  6.58s/it]
+                                                       
+{'loss': '0.9405', 'grad_norm': '0.7767', 'learning_rate': '2.79e-05', 'epoch': '0.5738', 'num_input_tokens_seen': 6988568, 'train_runtime': '6908', 'train_tokens_per_second': '1012'}
+
+ 57%|█████▋    | 1040/1813 [1:55:08<1:24:44,  6.58s/it]
+ 57%|█████▋    | 1041/1813 [1:55:14<1:22:27,  6.41s/it]
+ 57%|█████▋    | 1042/1813 [1:55:20<1:22:41,  6.44s/it]
+ 58%|█████▊    | 1043/1813 [1:55:26<1:19:48,  6.22s/it]
+ 58%|█████▊    | 1044/1813 [1:55:32<1:16:29,  5.97s/it]
+ 58%|█████▊    | 1045/1813 [1:55:37<1:15:47,  5.92s/it]
+ 58%|█████▊    | 1046/1813 [1:55:43<1:14:46,  5.85s/it]
+ 58%|█████▊    | 1047/1813 [1:55:49<1:16:25,  5.99s/it]
+ 58%|█████▊    | 1048/1813 [1:55:56<1:19:28,  6.23s/it]
+ 58%|█████▊    | 1049/1813 [1:56:02<1:16:10,  5.98s/it]
+ 58%|█████▊    | 1050/1813 [1:56:07<1:15:26,  5.93s/it]
+                                                       
+{'loss': '0.9069', 'grad_norm': '1.095', 'learning_rate': '2.731e-05', 'epoch': '0.5793', 'num_input_tokens_seen': 7051808, 'train_runtime': '6968', 'train_tokens_per_second': '1012'}
+
+ 58%|█████▊    | 1050/1813 [1:56:07<1:15:26,  5.93s/it]
+ 58%|█████▊    | 1051/1813 [1:56:12<1:11:56,  5.66s/it]
+ 58%|█████▊    | 1052/1813 [1:56:18<1:12:25,  5.71s/it]
+ 58%|█████▊    | 1053/1813 [1:56:24<1:12:26,  5.72s/it]
+ 58%|█████▊    | 1054/1813 [1:56:30<1:12:53,  5.76s/it]
+ 58%|█████▊    | 1055/1813 [1:56:35<1:10:53,  5.61s/it]
+ 58%|█████▊    | 1056/1813 [1:56:41<1:12:06,  5.72s/it]
+ 58%|█████▊    | 1057/1813 [1:56:48<1:15:21,  5.98s/it]
+ 58%|█████▊    | 1058/1813 [1:56:53<1:11:15,  5.66s/it]
+ 58%|█████▊    | 1059/1813 [1:56:57<1:07:11,  5.35s/it]
+ 58%|█████▊    | 1060/1813 [1:57:04<1:13:02,  5.82s/it]
+                                                       
+{'loss': '0.9281', 'grad_norm': '0.7002', 'learning_rate': '2.672e-05', 'epoch': '0.5848', 'num_input_tokens_seen': 7110624, 'train_runtime': '7025', 'train_tokens_per_second': '1012'}
+
+ 58%|█████▊    | 1060/1813 [1:57:04<1:13:02,  5.82s/it]
+ 59%|█████▊    | 1061/1813 [1:57:09<1:09:13,  5.52s/it]
+ 59%|█████▊    | 1062/1813 [1:57:14<1:07:03,  5.36s/it]
+ 59%|█████▊    | 1063/1813 [1:57:21<1:11:47,  5.74s/it]
+ 59%|█████▊    | 1064/1813 [1:57:28<1:18:10,  6.26s/it]
+ 59%|█████▊    | 1065/1813 [1:57:35<1:19:01,  6.34s/it]
+ 59%|█████▉    | 1066/1813 [1:57:42<1:23:01,  6.67s/it]
+ 59%|█████▉    | 1067/1813 [1:57:49<1:22:30,  6.64s/it]
+ 59%|█████▉    | 1068/1813 [1:57:54<1:16:53,  6.19s/it]
+ 59%|█████▉    | 1069/1813 [1:58:02<1:23:33,  6.74s/it]
+ 59%|█████▉    | 1070/1813 [1:58:07<1:19:42,  6.44s/it]
+                                                       
+{'loss': '0.9254', 'grad_norm': '1.007', 'learning_rate': '2.614e-05', 'epoch': '0.5903', 'num_input_tokens_seen': 7179728, 'train_runtime': '7088', 'train_tokens_per_second': '1013'}
+
+ 59%|█████▉    | 1070/1813 [1:58:07<1:19:42,  6.44s/it]
+ 59%|█████▉    | 1071/1813 [1:58:13<1:15:31,  6.11s/it]
+ 59%|█████▉    | 1072/1813 [1:58:19<1:16:24,  6.19s/it]
+ 59%|█████▉    | 1073/1813 [1:58:25<1:15:04,  6.09s/it]
+ 59%|█████▉    | 1074/1813 [1:58:31<1:13:43,  5.99s/it]
+ 59%|█████▉    | 1075/1813 [1:58:37<1:15:56,  6.17s/it]
+ 59%|█████▉    | 1076/1813 [1:58:44<1:17:50,  6.34s/it]
+ 59%|█████▉    | 1077/1813 [1:58:49<1:12:07,  5.88s/it]
+ 59%|█████▉    | 1078/1813 [1:58:54<1:09:11,  5.65s/it]
+ 60%|█████▉    | 1079/1813 [1:59:01<1:12:28,  5.92s/it]
+ 60%|█████▉    | 1080/1813 [1:59:07<1:14:56,  6.13s/it]
+                                                       
+{'loss': '0.9224', 'grad_norm': '0.8788', 'learning_rate': '2.556e-05', 'epoch': '0.5959', 'num_input_tokens_seen': 7240984, 'train_runtime': '7148', 'train_tokens_per_second': '1013'}
+
+ 60%|█████▉    | 1080/1813 [1:59:07<1:14:56,  6.13s/it]
+ 60%|█████▉    | 1081/1813 [1:59:14<1:15:31,  6.19s/it]
+ 60%|█████▉    | 1082/1813 [1:59:19<1:13:38,  6.04s/it]
+ 60%|█████▉    | 1083/1813 [1:59:25<1:13:00,  6.00s/it]
+ 60%|█████▉    | 1084/1813 [1:59:31<1:12:14,  5.95s/it]
+ 60%|█████▉    | 1085/1813 [1:59:37<1:11:55,  5.93s/it]
+ 60%|█████▉    | 1086/1813 [1:59:41<1:04:30,  5.32s/it]
+ 60%|█████▉    | 1087/1813 [1:59:46<1:05:36,  5.42s/it]
+ 60%|██████    | 1088/1813 [1:59:52<1:06:20,  5.49s/it]
+ 60%|██████    | 1089/1813 [1:59:59<1:10:20,  5.83s/it]
+ 60%|██████    | 1090/1813 [2:00:05<1:10:58,  5.89s/it]
+                                                       
+{'loss': '0.855', 'grad_norm': '1.041', 'learning_rate': '2.498e-05', 'epoch': '0.6014', 'num_input_tokens_seen': 7301824, 'train_runtime': '7205', 'train_tokens_per_second': '1013'}
+
+ 60%|██████    | 1090/1813 [2:00:05<1:10:58,  5.89s/it]
+ 60%|██████    | 1091/1813 [2:00:11<1:11:59,  5.98s/it]
+ 60%|██████    | 1092/1813 [2:00:17<1:11:59,  5.99s/it]
+ 60%|██████    | 1093/1813 [2:00:22<1:10:02,  5.84s/it]
+ 60%|██████    | 1094/1813 [2:00:29<1:11:57,  6.01s/it]
+ 60%|██████    | 1095/1813 [2:00:35<1:12:11,  6.03s/it]
+ 60%|██████    | 1096/1813 [2:00:40<1:09:36,  5.83s/it]
+ 61%|██████    | 1097/1813 [2:00:46<1:09:34,  5.83s/it]
+ 61%|██████    | 1098/1813 [2:00:53<1:13:42,  6.19s/it]
+ 61%|██████    | 1099/1813 [2:00:59<1:13:12,  6.15s/it]
+ 61%|██████    | 1100/1813 [2:01:07<1:18:59,  6.65s/it]
+                                                       
+{'loss': '0.9036', 'grad_norm': '0.9833', 'learning_rate': '2.44e-05', 'epoch': '0.6069', 'num_input_tokens_seen': 7369528, 'train_runtime': '7267', 'train_tokens_per_second': '1014'}
+
+ 61%|██████    | 1100/1813 [2:01:07<1:18:59,  6.65s/it]
+ 61%|██████    | 1101/1813 [2:01:13<1:17:50,  6.56s/it]
+ 61%|██████    | 1102/1813 [2:01:20<1:19:36,  6.72s/it]
+ 61%|██████    | 1103/1813 [2:01:26<1:14:34,  6.30s/it]
+ 61%|██████    | 1104/1813 [2:01:33<1:18:32,  6.65s/it]
+ 61%|██████    | 1105/1813 [2:01:38<1:11:20,  6.05s/it]
+ 61%|██████    | 1106/1813 [2:01:43<1:09:33,  5.90s/it]
+ 61%|██████    | 1107/1813 [2:01:52<1:18:06,  6.64s/it]
+ 61%|██████    | 1108/1813 [2:01:58<1:14:56,  6.38s/it]
+ 61%|██████    | 1109/1813 [2:02:04<1:15:04,  6.40s/it]
+ 61%|██████    | 1110/1813 [2:02:12<1:19:16,  6.77s/it]
+                                                       
+{'loss': '0.945', 'grad_norm': '0.9524', 'learning_rate': '2.382e-05', 'epoch': '0.6124', 'num_input_tokens_seen': 7441040, 'train_runtime': '7332', 'train_tokens_per_second': '1015'}
+
+ 61%|██████    | 1110/1813 [2:02:12<1:19:16,  6.77s/it]
+ 61%|██████▏   | 1111/1813 [2:02:16<1:11:22,  6.10s/it]
+ 61%|██████▏   | 1112/1813 [2:02:22<1:09:39,  5.96s/it]
+ 61%|██████▏   | 1113/1813 [2:02:28<1:10:29,  6.04s/it]
+ 61%|██████▏   | 1114/1813 [2:02:35<1:14:00,  6.35s/it]
+ 62%|██████▏   | 1115/1813 [2:02:42<1:14:59,  6.45s/it]
+ 62%|██████▏   | 1116/1813 [2:02:48<1:14:02,  6.37s/it]
+ 62%|██████▏   | 1117/1813 [2:02:53<1:10:29,  6.08s/it]
+ 62%|██████▏   | 1118/1813 [2:02:58<1:05:53,  5.69s/it]
+ 62%|██████▏   | 1119/1813 [2:03:04<1:07:40,  5.85s/it]
+ 62%|██████▏   | 1120/1813 [2:03:11<1:09:54,  6.05s/it]
+                                                       
+{'loss': '0.9021', 'grad_norm': '1.44', 'learning_rate': '2.325e-05', 'epoch': '0.6179', 'num_input_tokens_seen': 7504976, 'train_runtime': '7391', 'train_tokens_per_second': '1015'}
+
+ 62%|██████▏   | 1120/1813 [2:03:11<1:09:54,  6.05s/it]
+ 62%|██████▏   | 1121/1813 [2:03:19<1:15:52,  6.58s/it]
+ 62%|██████▏   | 1122/1813 [2:03:23<1:09:05,  6.00s/it]
+ 62%|██████▏   | 1123/1813 [2:03:31<1:14:11,  6.45s/it]
+ 62%|██████▏   | 1124/1813 [2:03:36<1:10:27,  6.14s/it]
+ 62%|██████▏   | 1125/1813 [2:03:43<1:12:06,  6.29s/it]
+ 62%|██████▏   | 1126/1813 [2:03:49<1:10:46,  6.18s/it]
+ 62%|██████▏   | 1127/1813 [2:03:54<1:07:32,  5.91s/it]
+ 62%|██████▏   | 1128/1813 [2:04:00<1:05:58,  5.78s/it]
+ 62%|██████▏   | 1129/1813 [2:04:05<1:04:50,  5.69s/it]
+ 62%|██████▏   | 1130/1813 [2:04:10<1:02:53,  5.52s/it]
+                                                       
+{'loss': '0.8986', 'grad_norm': '0.8304', 'learning_rate': '2.268e-05', 'epoch': '0.6234', 'num_input_tokens_seen': 7568576, 'train_runtime': '7451', 'train_tokens_per_second': '1016'}
+
+ 62%|██████▏   | 1130/1813 [2:04:10<1:02:53,  5.52s/it]
+ 62%|██████▏   | 1131/1813 [2:04:16<1:03:39,  5.60s/it]
+ 62%|██████▏   | 1132/1813 [2:04:22<1:05:28,  5.77s/it]
+ 62%|██████▏   | 1133/1813 [2:04:29<1:07:50,  5.99s/it]
+ 63%|██████▎   | 1134/1813 [2:04:37<1:14:10,  6.56s/it]
+ 63%|██████▎   | 1135/1813 [2:04:43<1:14:45,  6.62s/it]
+ 63%|██████▎   | 1136/1813 [2:04:50<1:15:28,  6.69s/it]
+ 63%|██████▎   | 1137/1813 [2:04:57<1:14:47,  6.64s/it]
+ 63%|██████▎   | 1138/1813 [2:05:04<1:15:59,  6.75s/it]
+ 63%|██████▎   | 1139/1813 [2:05:09<1:12:07,  6.42s/it]
+ 63%|██████▎   | 1140/1813 [2:05:15<1:09:26,  6.19s/it]
+                                                       
+{'loss': '0.9676', 'grad_norm': '0.9219', 'learning_rate': '2.212e-05', 'epoch': '0.629', 'num_input_tokens_seen': 7639168, 'train_runtime': '7515', 'train_tokens_per_second': '1016'}
+
+ 63%|██████▎   | 1140/1813 [2:05:15<1:09:26,  6.19s/it]
+ 63%|██████▎   | 1141/1813 [2:05:21<1:09:20,  6.19s/it]
+ 63%|██████▎   | 1142/1813 [2:05:27<1:08:32,  6.13s/it]
+ 63%|██████▎   | 1143/1813 [2:05:33<1:07:47,  6.07s/it]
+ 63%|██████▎   | 1144/1813 [2:05:40<1:09:54,  6.27s/it]
+ 63%|██████▎   | 1145/1813 [2:05:46<1:10:01,  6.29s/it]
+ 63%|██████▎   | 1146/1813 [2:05:51<1:06:09,  5.95s/it]
+ 63%|██████▎   | 1147/1813 [2:05:59<1:11:14,  6.42s/it]
+ 63%|██████▎   | 1148/1813 [2:06:06<1:13:02,  6.59s/it]
+ 63%|██████▎   | 1149/1813 [2:06:12<1:09:58,  6.32s/it]
+ 63%|██████▎   | 1150/1813 [2:06:19<1:13:50,  6.68s/it]
+                                                       
+{'loss': '0.9091', 'grad_norm': '0.857', 'learning_rate': '2.155e-05', 'epoch': '0.6345', 'num_input_tokens_seen': 7707832, 'train_runtime': '7580', 'train_tokens_per_second': '1017'}
+
+ 63%|██████▎   | 1150/1813 [2:06:19<1:13:50,  6.68s/it]
+ 63%|██████▎   | 1151/1813 [2:06:23<1:04:48,  5.87s/it]
+ 64%|██████▎   | 1152/1813 [2:06:29<1:03:51,  5.80s/it]
+ 64%|██████▎   | 1153/1813 [2:06:36<1:07:56,  6.18s/it]
+ 64%|██████▎   | 1154/1813 [2:06:42<1:07:29,  6.15s/it]
+ 64%|██████▎   | 1155/1813 [2:06:48<1:09:13,  6.31s/it]
+ 64%|██████▍   | 1156/1813 [2:06:55<1:09:27,  6.34s/it]
+ 64%|██████▍   | 1157/1813 [2:07:02<1:13:27,  6.72s/it]
+ 64%|██████▍   | 1158/1813 [2:07:09<1:13:10,  6.70s/it]
+ 64%|██████▍   | 1159/1813 [2:07:16<1:14:30,  6.84s/it]
+ 64%|██████▍   | 1160/1813 [2:07:22<1:12:01,  6.62s/it]
+                                                       
+{'loss': '0.9366', 'grad_norm': '0.673', 'learning_rate': '2.099e-05', 'epoch': '0.64', 'num_input_tokens_seen': 7775872, 'train_runtime': '7643', 'train_tokens_per_second': '1017'}
+
+ 64%|██████▍   | 1160/1813 [2:07:22<1:12:01,  6.62s/it]
+ 64%|██████▍   | 1161/1813 [2:07:30<1:14:00,  6.81s/it]
+ 64%|██████▍   | 1162/1813 [2:07:36<1:13:31,  6.78s/it]
+ 64%|██████▍   | 1163/1813 [2:07:43<1:11:51,  6.63s/it]
+ 64%|██████▍   | 1164/1813 [2:07:49<1:10:23,  6.51s/it]
+ 64%|██████▍   | 1165/1813 [2:07:55<1:10:03,  6.49s/it]
+ 64%|██████▍   | 1166/1813 [2:08:01<1:08:25,  6.35s/it]
+ 64%|██████▍   | 1167/1813 [2:08:08<1:10:53,  6.58s/it]
+ 64%|██████▍   | 1168/1813 [2:08:15<1:10:55,  6.60s/it]
+ 64%|██████▍   | 1169/1813 [2:08:21<1:09:39,  6.49s/it]
+ 65%|██████▍   | 1170/1813 [2:08:27<1:07:35,  6.31s/it]
+                                                       
+{'loss': '0.9243', 'grad_norm': '0.9508', 'learning_rate': '2.044e-05', 'epoch': '0.6455', 'num_input_tokens_seen': 7846688, 'train_runtime': '7708', 'train_tokens_per_second': '1018'}
+
+ 65%|██████▍   | 1170/1813 [2:08:27<1:07:35,  6.31s/it]
+ 65%|██████▍   | 1171/1813 [2:08:35<1:11:23,  6.67s/it]
+ 65%|██████▍   | 1172/1813 [2:08:42<1:12:33,  6.79s/it]
+ 65%|██████▍   | 1173/1813 [2:08:47<1:08:13,  6.40s/it]
+ 65%|██████▍   | 1174/1813 [2:08:52<1:04:16,  6.04s/it]
+ 65%|██████▍   | 1175/1813 [2:08:58<1:03:26,  5.97s/it]
+ 65%|██████▍   | 1176/1813 [2:09:05<1:04:13,  6.05s/it]
+ 65%|██████▍   | 1177/1813 [2:09:10<1:02:49,  5.93s/it]
+ 65%|██████▍   | 1178/1813 [2:09:16<1:02:02,  5.86s/it]
+ 65%|██████▌   | 1179/1813 [2:09:22<1:01:58,  5.87s/it]
+ 65%|██████▌   | 1180/1813 [2:09:28<1:04:14,  6.09s/it]
+                                                       
+{'loss': '0.9063', 'grad_norm': '0.9371', 'learning_rate': '1.988e-05', 'epoch': '0.651', 'num_input_tokens_seen': 7914072, 'train_runtime': '7769', 'train_tokens_per_second': '1019'}
+
+ 65%|██████▌   | 1180/1813 [2:09:28<1:04:14,  6.09s/it]
+ 65%|██████▌   | 1181/1813 [2:09:36<1:09:19,  6.58s/it]
+ 65%|██████▌   | 1182/1813 [2:09:44<1:12:45,  6.92s/it]
+ 65%|██████▌   | 1183/1813 [2:09:51<1:12:46,  6.93s/it]
+ 65%|██████▌   | 1184/1813 [2:09:57<1:09:19,  6.61s/it]
+ 65%|██████▌   | 1185/1813 [2:10:04<1:12:43,  6.95s/it]
+ 65%|██████▌   | 1186/1813 [2:10:12<1:13:42,  7.05s/it]
+ 65%|██████▌   | 1187/1813 [2:10:19<1:13:14,  7.02s/it]
+ 66%|██████▌   | 1188/1813 [2:10:24<1:07:51,  6.51s/it]
+ 66%|██████▌   | 1189/1813 [2:10:29<1:04:09,  6.17s/it]
+ 66%|██████▌   | 1190/1813 [2:10:36<1:05:05,  6.27s/it]
+                                                       
+{'loss': '0.9584', 'grad_norm': '1.052', 'learning_rate': '1.933e-05', 'epoch': '0.6566', 'num_input_tokens_seen': 7989064, 'train_runtime': '7836', 'train_tokens_per_second': '1019'}
+
+ 66%|██████▌   | 1190/1813 [2:10:36<1:05:05,  6.27s/it]
+ 66%|██████▌   | 1191/1813 [2:10:42<1:04:07,  6.19s/it]
+ 66%|██████▌   | 1192/1813 [2:10:49<1:05:45,  6.35s/it]
+ 66%|██████▌   | 1193/1813 [2:10:55<1:04:26,  6.24s/it]
+ 66%|██████▌   | 1194/1813 [2:11:01<1:05:08,  6.31s/it]
+ 66%|██████▌   | 1195/1813 [2:11:06<1:00:04,  5.83s/it]
+ 66%|██████▌   | 1196/1813 [2:11:12<1:00:58,  5.93s/it]
+ 66%|██████▌   | 1197/1813 [2:11:18<1:01:36,  6.00s/it]
+ 66%|██████▌   | 1198/1813 [2:11:24<1:01:25,  5.99s/it]
+ 66%|██████▌   | 1199/1813 [2:11:30<1:01:17,  5.99s/it]
+ 66%|██████▌   | 1200/1813 [2:11:37<1:05:20,  6.39s/it]
+                                                       
+{'loss': '0.8693', 'grad_norm': '0.8853', 'learning_rate': '1.879e-05', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352, 'train_runtime': '7898', 'train_tokens_per_second': '1020'}
+
+ 66%|██████▌   | 1200/1813 [2:11:37<1:05:20,  6.39s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.82it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.22it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.31it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.08it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.01it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.83it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.74it/s]
+
+  2%|▏         | 18/1000 [00:02<02:41,  6.06it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.38it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  6.00it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  6.00it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.68it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.11it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.10it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.32it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.76it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.31it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.76it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.01it/s]
+
+  7%|▋         | 72/1000 [00:09<02:17,  6.73it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.13it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.34it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.88it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.21it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.09it/s]
+
+  9%|▉         | 92/1000 [00:12<02:26,  6.18it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.64it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.30it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.43it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.20it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.02it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.52it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.82it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.22it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.09it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.44it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.14it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.87it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.95it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.06it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.51it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.94it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.06it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.42it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.66it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.98it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.63it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.82it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:26,  5.68it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.19it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.23it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.46it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.51it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:22,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.81it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:42,  7.91it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.12it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:01,  6.64it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.39it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.76it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.21it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.69it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.79it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.65it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.92it/s]
+
+ 21%|██        | 208/1000 [00:27<01:36,  8.25it/s]
+
+ 21%|██        | 210/1000 [00:27<01:36,  8.16it/s]
+
+ 21%|██        | 212/1000 [00:27<01:23,  9.39it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:31,  8.59it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:26,  9.01it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:45,  7.39it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:29,  8.69it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:04,  6.29it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.29it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.53it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.03it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.43it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.23it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.99it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.48it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.99it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.37it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.03it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.48it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.67it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.87it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:05,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:40,  7.38it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.70it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.70it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.53it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.44it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.92it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:36,  7.55it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.04it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:13,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.71it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.71it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.07it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.82it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.63it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.04it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.75it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.78it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.01it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.20it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:41,  6.90it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.20it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.48it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:37,  7.14it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.77it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.09it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.16it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.96it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.81it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.22it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.27it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.26it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.58it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.73it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.97it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.39it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.06it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.62it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.73it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.23it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.42it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.65it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.58it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.41it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.23it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.93it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.78it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.21it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.20it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.16it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.66it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.38it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.79it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.48it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.76it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:10,  8.48it/s]
+
+ 41%|████      | 408/1000 [00:54<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.07it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.27it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.47it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.52it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.96it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.55it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.53it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.00it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.77it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.51it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.09it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.20it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:37,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.20it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.51it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.83it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.28it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.85it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.02it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.32it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.32it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.33it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.29it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50, 10.01it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.42it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.44it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.81it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.97it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.07it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.13it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:47,  9.74it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.67it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:58,  7.65it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  6.96it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:50,  8.78it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43,  9.99it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.35it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.96it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.33it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.82it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.72it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.24it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.79it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.40it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.72it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.86it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:45,  8.48it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.80it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.75it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.10it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.23it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.41it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.60it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.55it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.31it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.91it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.57it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.34it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.67it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.70it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.98it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.94it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.90it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:33<00:45,  6.82it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.27it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:35<00:40,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.49it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.69it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.36it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.07it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.48it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.26it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.14it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.62it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.34it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:27,  9.54it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.90it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.78it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.32it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.63it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:28,  8.66it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.70it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.18it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.24it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.39it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.71it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.78it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.14it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.79it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.96it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:25,  8.39it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.17it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.55it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  8.99it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.74it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.20it/s]
+
+ 80%|███████▉  | 797/1000 [01:47<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.03it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.73it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.22it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.24it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.06it/s]
+
+ 82%|████████▏ | 815/1000 [01:49<00:19,  9.32it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.15it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.21it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.06it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.83it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.02it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.81it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.76it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:52<00:27,  6.03it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.47it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.64it/s]
+
+ 84%|████████▍ | 842/1000 [01:53<00:19,  8.17it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.78it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.56it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.51it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.88it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.49it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.07it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:13, 10.22it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.54it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.79it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.65it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.72it/s]
+
+ 87%|████████▋ | 867/1000 [01:56<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.28it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.30it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+
+ 88%|████████▊ | 875/1000 [01:57<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.97it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.19it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.37it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.78it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.78it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.63it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.86it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.55it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.72it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.88it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.09it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.07it/s]
+
+ 91%|█████████ | 908/1000 [02:01<00:11,  8.34it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.61it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.79it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.40it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.38it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.61it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.95it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.18it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:04<00:10,  6.67it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.49it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.84it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.62it/s]
+
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.19it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.33it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:06,  8.86it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.06it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.15it/s]
+
+ 94%|█████████▍| 942/1000 [02:06<00:09,  6.35it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.10it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.84it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.43it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:08<00:06,  6.41it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.79it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.83it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:09<00:04,  7.51it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.46it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.36it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.03it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.43it/s]
+
+ 97%|█████████▋| 972/1000 [02:10<00:03,  8.04it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.22it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.80it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.84it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.45it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.08it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.45it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.01it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.54it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                       
+
+                                                   
+{'eval_loss': '0.7751', 'eval_runtime': '135.2', 'eval_samples_per_second': '7.394', 'eval_steps_per_second': '7.394', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352}
+
+ 66%|██████▌   | 1200/1813 [2:13:53<1:05:20,  6.39s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 66%|██████▌   | 1201/1813 [2:13:58<7:55:46, 46.65s/it]
+ 66%|██████▋   | 1202/1813 [2:14:04<5:52:32, 34.62s/it]
+ 66%|██████▋   | 1203/1813 [2:14:11<4:25:57, 26.16s/it]
+ 66%|██████▋   | 1204/1813 [2:14:18<3:26:14, 20.32s/it]
+ 66%|██████▋   | 1205/1813 [2:14:24<2:44:09, 16.20s/it]
+ 67%|██████▋   | 1206/1813 [2:14:30<2:13:42, 13.22s/it]
+ 67%|██████▋   | 1207/1813 [2:14:37<1:52:44, 11.16s/it]
+ 67%|██████▋   | 1208/1813 [2:14:44<1:42:01, 10.12s/it]
+ 67%|██████▋   | 1209/1813 [2:14:51<1:32:14,  9.16s/it]
+ 67%|██████▋   | 1210/1813 [2:14:58<1:22:54,  8.25s/it]
+                                                       
+{'loss': '0.9386', 'grad_norm': '1.044', 'learning_rate': '1.825e-05', 'epoch': '0.6676', 'num_input_tokens_seen': 8125224, 'train_runtime': '8098', 'train_tokens_per_second': '1003'}
+
+ 67%|██████▋   | 1210/1813 [2:14:58<1:22:54,  8.25s/it]
+ 67%|██████▋   | 1211/1813 [2:15:04<1:18:46,  7.85s/it]
+ 67%|██████▋   | 1212/1813 [2:15:09<1:08:34,  6.85s/it]
+ 67%|██████▋   | 1213/1813 [2:15:16<1:08:08,  6.81s/it]
+ 67%|██████▋   | 1214/1813 [2:15:24<1:13:15,  7.34s/it]
+ 67%|██████▋   | 1215/1813 [2:15:31<1:10:04,  7.03s/it]
+ 67%|██████▋   | 1216/1813 [2:15:38<1:10:19,  7.07s/it]
+ 67%|██████▋   | 1217/1813 [2:15:44<1:07:34,  6.80s/it]
+ 67%|██████▋   | 1218/1813 [2:15:50<1:05:32,  6.61s/it]
+ 67%|██████▋   | 1219/1813 [2:15:55<1:01:47,  6.24s/it]
+ 67%|██████▋   | 1220/1813 [2:16:04<1:07:06,  6.79s/it]
+                                                       
+{'loss': '0.9729', 'grad_norm': '8.26', 'learning_rate': '1.772e-05', 'epoch': '0.6731', 'num_input_tokens_seen': 8197488, 'train_runtime': '8164', 'train_tokens_per_second': '1004'}
+
+ 67%|██████▋   | 1220/1813 [2:16:04<1:07:06,  6.79s/it]
+ 67%|██████▋   | 1221/1813 [2:16:09<1:02:58,  6.38s/it]
+ 67%|██████▋   | 1222/1813 [2:16:14<59:19,  6.02s/it]  
+ 67%|██████▋   | 1223/1813 [2:16:20<57:36,  5.86s/it]
+ 68%|██████▊   | 1224/1813 [2:16:25<55:11,  5.62s/it]
+ 68%|██████▊   | 1225/1813 [2:16:31<56:07,  5.73s/it]
+ 68%|██████▊   | 1226/1813 [2:16:37<56:54,  5.82s/it]
+ 68%|██████▊   | 1227/1813 [2:16:43<58:57,  6.04s/it]
+ 68%|██████▊   | 1228/1813 [2:16:48<55:59,  5.74s/it]
+ 68%|██████▊   | 1229/1813 [2:16:55<57:46,  5.94s/it]
+ 68%|██████▊   | 1230/1813 [2:17:01<57:25,  5.91s/it]
+                                                     
+{'loss': '0.8312', 'grad_norm': '0.8146', 'learning_rate': '1.719e-05', 'epoch': '0.6786', 'num_input_tokens_seen': 8257288, 'train_runtime': '8221', 'train_tokens_per_second': '1004'}
+
+ 68%|██████▊   | 1230/1813 [2:17:01<57:25,  5.91s/it]
+ 68%|██████▊   | 1231/1813 [2:17:08<1:02:18,  6.42s/it]
+ 68%|██████▊   | 1232/1813 [2:17:16<1:05:05,  6.72s/it]
+ 68%|██████▊   | 1233/1813 [2:17:22<1:03:55,  6.61s/it]
+ 68%|██████▊   | 1234/1813 [2:17:28<1:02:32,  6.48s/it]
+ 68%|██████▊   | 1235/1813 [2:17:34<1:01:41,  6.40s/it]
+ 68%|██████▊   | 1236/1813 [2:17:43<1:08:01,  7.07s/it]
+ 68%|██████▊   | 1237/1813 [2:17:49<1:06:11,  6.90s/it]
+ 68%|██████▊   | 1238/1813 [2:17:55<1:02:01,  6.47s/it]
+ 68%|██████▊   | 1239/1813 [2:18:00<57:59,  6.06s/it]  
+ 68%|██████▊   | 1240/1813 [2:18:05<55:07,  5.77s/it]
+                                                     
+{'loss': '0.8382', 'grad_norm': '0.8944', 'learning_rate': '1.666e-05', 'epoch': '0.6841', 'num_input_tokens_seen': 8327600, 'train_runtime': '8286', 'train_tokens_per_second': '1005'}
+
+ 68%|██████▊   | 1240/1813 [2:18:05<55:07,  5.77s/it]
+ 68%|██████▊   | 1241/1813 [2:18:11<54:11,  5.69s/it]
+ 69%|██████▊   | 1242/1813 [2:18:17<55:20,  5.82s/it]
+ 69%|██████▊   | 1243/1813 [2:18:23<55:48,  5.87s/it]
+ 69%|██████▊   | 1244/1813 [2:18:29<58:17,  6.15s/it]
+ 69%|██████▊   | 1245/1813 [2:18:37<1:01:23,  6.49s/it]
+ 69%|██████▊   | 1246/1813 [2:18:43<59:17,  6.27s/it]  
+ 69%|██████▉   | 1247/1813 [2:18:49<1:01:00,  6.47s/it]
+ 69%|██████▉   | 1248/1813 [2:18:56<1:01:59,  6.58s/it]
+ 69%|██████▉   | 1249/1813 [2:19:03<1:01:37,  6.56s/it]
+ 69%|██████▉   | 1250/1813 [2:19:08<56:45,  6.05s/it]  
+                                                     
+{'loss': '0.9732', 'grad_norm': '1.19', 'learning_rate': '1.614e-05', 'epoch': '0.6897', 'num_input_tokens_seen': 8394824, 'train_runtime': '8348', 'train_tokens_per_second': '1006'}
+
+ 69%|██████▉   | 1250/1813 [2:19:08<56:45,  6.05s/it]
+ 69%|██████▉   | 1251/1813 [2:19:14<57:27,  6.13s/it]
+ 69%|██████▉   | 1252/1813 [2:19:20<56:29,  6.04s/it]
+ 69%|██████▉   | 1253/1813 [2:19:26<56:43,  6.08s/it]
+ 69%|██████▉   | 1254/1813 [2:19:33<57:49,  6.21s/it]
+ 69%|██████▉   | 1255/1813 [2:19:39<57:19,  6.16s/it]
+ 69%|██████▉   | 1256/1813 [2:19:45<57:09,  6.16s/it]
+ 69%|██████▉   | 1257/1813 [2:19:51<56:08,  6.06s/it]
+ 69%|██████▉   | 1258/1813 [2:19:57<58:09,  6.29s/it]
+ 69%|██████▉   | 1259/1813 [2:20:04<59:17,  6.42s/it]
+ 69%|██████▉   | 1260/1813 [2:20:11<1:01:27,  6.67s/it]
+                                                       
+{'loss': '0.9263', 'grad_norm': '1.174', 'learning_rate': '1.563e-05', 'epoch': '0.6952', 'num_input_tokens_seen': 8461096, 'train_runtime': '8412', 'train_tokens_per_second': '1006'}
+
+ 69%|██████▉   | 1260/1813 [2:20:11<1:01:27,  6.67s/it]
+ 70%|██████▉   | 1261/1813 [2:20:20<1:05:36,  7.13s/it]
+ 70%|██████▉   | 1262/1813 [2:20:27<1:06:53,  7.28s/it]
+ 70%|██████▉   | 1263/1813 [2:20:35<1:06:59,  7.31s/it]
+ 70%|██████▉   | 1264/1813 [2:20:40<1:02:23,  6.82s/it]
+ 70%|██████▉   | 1265/1813 [2:20:45<57:32,  6.30s/it]  
+ 70%|██████▉   | 1266/1813 [2:20:53<1:01:33,  6.75s/it]
+ 70%|██████▉   | 1267/1813 [2:20:58<56:43,  6.23s/it]  
+ 70%|██████▉   | 1268/1813 [2:21:03<52:39,  5.80s/it]
+ 70%|██████▉   | 1269/1813 [2:21:08<51:14,  5.65s/it]
+ 70%|███████   | 1270/1813 [2:21:14<51:51,  5.73s/it]
+                                                     
+{'loss': '0.925', 'grad_norm': '1.069', 'learning_rate': '1.512e-05', 'epoch': '0.7007', 'num_input_tokens_seen': 8529224, 'train_runtime': '8475', 'train_tokens_per_second': '1006'}
+
+ 70%|███████   | 1270/1813 [2:21:14<51:51,  5.73s/it]
+ 70%|███████   | 1271/1813 [2:21:19<48:47,  5.40s/it]
+ 70%|███████   | 1272/1813 [2:21:26<52:57,  5.87s/it]
+ 70%|███████   | 1273/1813 [2:21:33<55:19,  6.15s/it]
+ 70%|███████   | 1274/1813 [2:21:40<58:32,  6.52s/it]
+ 70%|███████   | 1275/1813 [2:21:47<59:15,  6.61s/it]
+ 70%|███████   | 1276/1813 [2:21:53<57:01,  6.37s/it]
+ 70%|███████   | 1277/1813 [2:22:00<58:57,  6.60s/it]
+ 70%|███████   | 1278/1813 [2:22:05<56:01,  6.28s/it]
+ 71%|███████   | 1279/1813 [2:22:10<52:29,  5.90s/it]
+ 71%|███████   | 1280/1813 [2:22:17<55:43,  6.27s/it]
+                                                     
+{'loss': '0.9456', 'grad_norm': '1.008', 'learning_rate': '1.461e-05', 'epoch': '0.7062', 'num_input_tokens_seen': 8596456, 'train_runtime': '8538', 'train_tokens_per_second': '1007'}
+
+ 71%|███████   | 1280/1813 [2:22:17<55:43,  6.27s/it]
+ 71%|███████   | 1281/1813 [2:22:23<53:30,  6.03s/it]
+ 71%|███████   | 1282/1813 [2:22:29<54:25,  6.15s/it]
+ 71%|███████   | 1283/1813 [2:22:36<55:57,  6.34s/it]
+ 71%|███████   | 1284/1813 [2:22:42<54:19,  6.16s/it]
+ 71%|███████   | 1285/1813 [2:22:47<51:26,  5.85s/it]
+ 71%|███████   | 1286/1813 [2:22:54<54:39,  6.22s/it]
+ 71%|███████   | 1287/1813 [2:23:00<53:45,  6.13s/it]
+ 71%|███████   | 1288/1813 [2:23:05<51:10,  5.85s/it]
+ 71%|███████   | 1289/1813 [2:23:12<53:13,  6.10s/it]
+ 71%|███████   | 1290/1813 [2:23:19<55:39,  6.39s/it]
+                                                     
+{'loss': '0.9972', 'grad_norm': '0.7022', 'learning_rate': '1.412e-05', 'epoch': '0.7117', 'num_input_tokens_seen': 8662472, 'train_runtime': '8599', 'train_tokens_per_second': '1007'}
+
+ 71%|███████   | 1290/1813 [2:23:19<55:39,  6.39s/it]
+ 71%|███████   | 1291/1813 [2:23:25<54:46,  6.30s/it]
+ 71%|███████▏  | 1292/1813 [2:23:32<56:00,  6.45s/it]
+ 71%|███████▏  | 1293/1813 [2:23:39<57:13,  6.60s/it]
+ 71%|███████▏  | 1294/1813 [2:23:45<56:09,  6.49s/it]
+ 71%|███████▏  | 1295/1813 [2:23:51<54:10,  6.27s/it]
+ 71%|███████▏  | 1296/1813 [2:23:56<51:17,  5.95s/it]
+ 72%|███████▏  | 1297/1813 [2:24:01<49:17,  5.73s/it]
+ 72%|███████▏  | 1298/1813 [2:24:08<51:17,  5.98s/it]
+ 72%|███████▏  | 1299/1813 [2:24:15<53:45,  6.28s/it]
+ 72%|███████▏  | 1300/1813 [2:24:21<53:42,  6.28s/it]
+                                                     
+{'loss': '0.8905', 'grad_norm': '1.207', 'learning_rate': '1.363e-05', 'epoch': '0.7172', 'num_input_tokens_seen': 8730600, 'train_runtime': '8661', 'train_tokens_per_second': '1008'}
+
+ 72%|███████▏  | 1300/1813 [2:24:21<53:42,  6.28s/it]
+ 72%|███████▏  | 1301/1813 [2:24:27<52:07,  6.11s/it]
+ 72%|███████▏  | 1302/1813 [2:24:33<53:10,  6.24s/it]
+ 72%|███████▏  | 1303/1813 [2:24:41<56:36,  6.66s/it]
+ 72%|███████▏  | 1304/1813 [2:24:46<53:13,  6.27s/it]
+ 72%|███████▏  | 1305/1813 [2:24:54<55:43,  6.58s/it]
+ 72%|███████▏  | 1306/1813 [2:24:59<53:18,  6.31s/it]
+ 72%|███████▏  | 1307/1813 [2:25:05<52:53,  6.27s/it]
+ 72%|███████▏  | 1308/1813 [2:25:12<53:09,  6.32s/it]
+ 72%|███████▏  | 1309/1813 [2:25:18<52:49,  6.29s/it]
+ 72%|███████▏  | 1310/1813 [2:25:25<54:25,  6.49s/it]
+                                                     
+{'loss': '0.9771', 'grad_norm': '0.8627', 'learning_rate': '1.314e-05', 'epoch': '0.7228', 'num_input_tokens_seen': 8802000, 'train_runtime': '8726', 'train_tokens_per_second': '1009'}
+
+ 72%|███████▏  | 1310/1813 [2:25:25<54:25,  6.49s/it]
+ 72%|███████▏  | 1311/1813 [2:25:31<53:55,  6.45s/it]
+ 72%|███████▏  | 1312/1813 [2:25:37<51:31,  6.17s/it]
+ 72%|███████▏  | 1313/1813 [2:25:43<50:51,  6.10s/it]
+ 72%|███████▏  | 1314/1813 [2:25:49<51:05,  6.14s/it]
+ 73%|███████▎  | 1315/1813 [2:25:55<49:39,  5.98s/it]
+ 73%|███████▎  | 1316/1813 [2:26:00<48:58,  5.91s/it]
+ 73%|███████▎  | 1317/1813 [2:26:06<49:02,  5.93s/it]
+ 73%|███████▎  | 1318/1813 [2:26:12<48:08,  5.84s/it]
+ 73%|███████▎  | 1319/1813 [2:26:18<49:36,  6.03s/it]
+ 73%|███████▎  | 1320/1813 [2:26:25<51:01,  6.21s/it]
+                                                     
+{'loss': '0.9675', 'grad_norm': '0.8346', 'learning_rate': '1.266e-05', 'epoch': '0.7283', 'num_input_tokens_seen': 8866824, 'train_runtime': '8786', 'train_tokens_per_second': '1009'}
+
+ 73%|███████▎  | 1320/1813 [2:26:25<51:01,  6.21s/it]
+ 73%|███████▎  | 1321/1813 [2:26:32<51:48,  6.32s/it]
+ 73%|███████▎  | 1322/1813 [2:26:38<50:55,  6.22s/it]
+ 73%|███████▎  | 1323/1813 [2:26:44<52:16,  6.40s/it]
+ 73%|███████▎  | 1324/1813 [2:26:51<51:31,  6.32s/it]
+ 73%|███████▎  | 1325/1813 [2:26:57<50:33,  6.22s/it]
+ 73%|███████▎  | 1326/1813 [2:27:03<50:53,  6.27s/it]
+ 73%|███████▎  | 1327/1813 [2:27:11<54:27,  6.72s/it]
+ 73%|███████▎  | 1328/1813 [2:27:18<54:54,  6.79s/it]
+ 73%|███████▎  | 1329/1813 [2:27:24<53:18,  6.61s/it]
+ 73%|███████▎  | 1330/1813 [2:27:30<51:06,  6.35s/it]
+                                                     
+{'loss': '0.9085', 'grad_norm': '1.044', 'learning_rate': '1.219e-05', 'epoch': '0.7338', 'num_input_tokens_seen': 8935584, 'train_runtime': '8850', 'train_tokens_per_second': '1010'}
+
+ 73%|███████▎  | 1330/1813 [2:27:30<51:06,  6.35s/it]
+ 73%|███████▎  | 1331/1813 [2:27:36<49:50,  6.20s/it]
+ 73%|███████▎  | 1332/1813 [2:27:41<48:35,  6.06s/it]
+ 74%|███████▎  | 1333/1813 [2:27:47<47:57,  6.00s/it]
+ 74%|███████▎  | 1334/1813 [2:27:52<46:23,  5.81s/it]
+ 74%|███████▎  | 1335/1813 [2:27:58<45:14,  5.68s/it]
+ 74%|███████▎  | 1336/1813 [2:28:04<46:27,  5.84s/it]
+ 74%|███████▎  | 1337/1813 [2:28:12<50:19,  6.34s/it]
+ 74%|███████▍  | 1338/1813 [2:28:19<52:42,  6.66s/it]
+ 74%|███████▍  | 1339/1813 [2:28:24<49:37,  6.28s/it]
+ 74%|███████▍  | 1340/1813 [2:28:31<51:00,  6.47s/it]
+                                                     
+{'loss': '0.9939', 'grad_norm': '0.8717', 'learning_rate': '1.173e-05', 'epoch': '0.7393', 'num_input_tokens_seen': 9000416, 'train_runtime': '8912', 'train_tokens_per_second': '1010'}
+
+ 74%|███████▍  | 1340/1813 [2:28:31<51:00,  6.47s/it]
+ 74%|███████▍  | 1341/1813 [2:28:38<51:13,  6.51s/it]
+ 74%|███████▍  | 1342/1813 [2:28:43<48:33,  6.19s/it]
+ 74%|███████▍  | 1343/1813 [2:28:49<46:44,  5.97s/it]
+ 74%|███████▍  | 1344/1813 [2:28:55<46:20,  5.93s/it]
+ 74%|███████▍  | 1345/1813 [2:29:02<50:27,  6.47s/it]
+ 74%|███████▍  | 1346/1813 [2:29:09<50:49,  6.53s/it]
+ 74%|███████▍  | 1347/1813 [2:29:15<49:43,  6.40s/it]
+ 74%|███████▍  | 1348/1813 [2:29:22<49:46,  6.42s/it]
+ 74%|███████▍  | 1349/1813 [2:29:28<50:31,  6.53s/it]
+ 74%|███████▍  | 1350/1813 [2:29:34<48:50,  6.33s/it]
+                                                     
+{'loss': '0.9475', 'grad_norm': '1.095', 'learning_rate': '1.127e-05', 'epoch': '0.7448', 'num_input_tokens_seen': 9066864, 'train_runtime': '8975', 'train_tokens_per_second': '1010'}
+
+ 74%|███████▍  | 1350/1813 [2:29:34<48:50,  6.33s/it]
+ 75%|███████▍  | 1351/1813 [2:29:42<52:24,  6.81s/it]
+ 75%|███████▍  | 1352/1813 [2:29:48<51:11,  6.66s/it]
+ 75%|███████▍  | 1353/1813 [2:29:55<51:23,  6.70s/it]
+ 75%|███████▍  | 1354/1813 [2:30:02<51:27,  6.73s/it]
+ 75%|███████▍  | 1355/1813 [2:30:11<55:38,  7.29s/it]
+ 75%|███████▍  | 1356/1813 [2:30:17<52:16,  6.86s/it]
+ 75%|███████▍  | 1357/1813 [2:30:23<50:42,  6.67s/it]
+ 75%|███████▍  | 1358/1813 [2:30:29<50:33,  6.67s/it]
+ 75%|███████▍  | 1359/1813 [2:30:35<48:12,  6.37s/it]
+ 75%|███████▌  | 1360/1813 [2:30:41<47:03,  6.23s/it]
+                                                     
+{'loss': '0.9194', 'grad_norm': '1.214', 'learning_rate': '1.082e-05', 'epoch': '0.7503', 'num_input_tokens_seen': 9140520, 'train_runtime': '9042', 'train_tokens_per_second': '1011'}
+
+ 75%|███████▌  | 1360/1813 [2:30:41<47:03,  6.23s/it]
+ 75%|███████▌  | 1361/1813 [2:30:48<48:22,  6.42s/it]
+ 75%|███████▌  | 1362/1813 [2:30:54<48:39,  6.47s/it]
+ 75%|███████▌  | 1363/1813 [2:31:02<50:50,  6.78s/it]
+ 75%|███████▌  | 1364/1813 [2:31:08<49:33,  6.62s/it]
+ 75%|███████▌  | 1365/1813 [2:31:16<51:19,  6.87s/it]
+ 75%|███████▌  | 1366/1813 [2:31:22<49:04,  6.59s/it]
+ 75%|███████▌  | 1367/1813 [2:31:28<48:56,  6.59s/it]
+ 75%|███████▌  | 1368/1813 [2:31:33<45:22,  6.12s/it]
+ 76%|███████▌  | 1369/1813 [2:31:41<49:28,  6.69s/it]
+ 76%|███████▌  | 1370/1813 [2:31:47<47:45,  6.47s/it]
+                                                     
+{'loss': '0.9058', 'grad_norm': '0.9825', 'learning_rate': '1.038e-05', 'epoch': '0.7559', 'num_input_tokens_seen': 9213048, 'train_runtime': '9108', 'train_tokens_per_second': '1012'}
+
+ 76%|███████▌  | 1370/1813 [2:31:47<47:45,  6.47s/it]
+ 76%|███████▌  | 1371/1813 [2:31:55<51:01,  6.93s/it]
+ 76%|███████▌  | 1372/1813 [2:32:01<47:24,  6.45s/it]
+ 76%|███████▌  | 1373/1813 [2:32:07<46:50,  6.39s/it]
+ 76%|███████▌  | 1374/1813 [2:32:13<45:50,  6.27s/it]
+ 76%|███████▌  | 1375/1813 [2:32:18<44:23,  6.08s/it]
+ 76%|███████▌  | 1376/1813 [2:32:24<43:03,  5.91s/it]
+ 76%|███████▌  | 1377/1813 [2:32:30<43:39,  6.01s/it]
+ 76%|███████▌  | 1378/1813 [2:32:36<43:08,  5.95s/it]
+ 76%|███████▌  | 1379/1813 [2:32:42<42:51,  5.92s/it]
+ 76%|███████▌  | 1380/1813 [2:32:49<45:19,  6.28s/it]
+                                                     
+{'loss': '0.8941', 'grad_norm': '1.059', 'learning_rate': '9.94e-06', 'epoch': '0.7614', 'num_input_tokens_seen': 9279088, 'train_runtime': '9169', 'train_tokens_per_second': '1012'}
+
+ 76%|███████▌  | 1380/1813 [2:32:49<45:19,  6.28s/it]
+ 76%|███████▌  | 1381/1813 [2:32:56<47:01,  6.53s/it]
+ 76%|███████▌  | 1382/1813 [2:33:02<44:48,  6.24s/it]
+ 76%|███████▋  | 1383/1813 [2:33:08<44:26,  6.20s/it]
+ 76%|███████▋  | 1384/1813 [2:33:13<43:23,  6.07s/it]
+ 76%|███████▋  | 1385/1813 [2:33:20<44:34,  6.25s/it]
+ 76%|███████▋  | 1386/1813 [2:33:27<44:48,  6.30s/it]
+ 77%|███████▋  | 1387/1813 [2:33:34<46:14,  6.51s/it]
+ 77%|███████▋  | 1388/1813 [2:33:39<44:05,  6.23s/it]
+ 77%|███████▋  | 1389/1813 [2:33:44<40:35,  5.74s/it]
+ 77%|███████▋  | 1390/1813 [2:33:51<43:00,  6.10s/it]
+                                                     
+{'loss': '0.9589', 'grad_norm': '1.013', 'learning_rate': '9.511e-06', 'epoch': '0.7669', 'num_input_tokens_seen': 9344880, 'train_runtime': '9231', 'train_tokens_per_second': '1012'}
+
+ 77%|███████▋  | 1390/1813 [2:33:51<43:00,  6.10s/it]
+ 77%|███████▋  | 1391/1813 [2:33:56<41:47,  5.94s/it]
+ 77%|███████▋  | 1392/1813 [2:34:03<43:33,  6.21s/it]
+ 77%|███████▋  | 1393/1813 [2:34:09<42:02,  6.01s/it]
+ 77%|███████▋  | 1394/1813 [2:34:15<43:35,  6.24s/it]
+ 77%|███████▋  | 1395/1813 [2:34:22<43:59,  6.31s/it]
+ 77%|███████▋  | 1396/1813 [2:34:28<43:21,  6.24s/it]
+ 77%|███████▋  | 1397/1813 [2:34:35<44:40,  6.44s/it]
+ 77%|███████▋  | 1398/1813 [2:34:40<41:13,  5.96s/it]
+ 77%|███████▋  | 1399/1813 [2:34:45<39:12,  5.68s/it]
+ 77%|███████▋  | 1400/1813 [2:34:52<41:32,  6.04s/it]
+                                                     
+{'loss': '0.8902', 'grad_norm': '0.795', 'learning_rate': '9.091e-06', 'epoch': '0.7724', 'num_input_tokens_seen': 9408280, 'train_runtime': '9292', 'train_tokens_per_second': '1013'}
+
+ 77%|███████▋  | 1400/1813 [2:34:52<41:32,  6.04s/it]
+ 77%|███████▋  | 1401/1813 [2:34:57<40:31,  5.90s/it]
+ 77%|███████▋  | 1402/1813 [2:35:03<40:56,  5.98s/it]
+ 77%|███████▋  | 1403/1813 [2:35:10<42:05,  6.16s/it]
+ 77%|███████▋  | 1404/1813 [2:35:16<42:19,  6.21s/it]
+ 77%|███████▋  | 1405/1813 [2:35:21<39:40,  5.83s/it]
+ 78%|███████▊  | 1406/1813 [2:35:27<39:28,  5.82s/it]
+ 78%|███████▊  | 1407/1813 [2:35:34<41:17,  6.10s/it]
+ 78%|███████▊  | 1408/1813 [2:35:39<39:38,  5.87s/it]
+ 78%|███████▊  | 1409/1813 [2:35:44<37:34,  5.58s/it]
+ 78%|███████▊  | 1410/1813 [2:35:50<37:51,  5.64s/it]
+                                                     
+{'loss': '0.952', 'grad_norm': '1.049', 'learning_rate': '8.678e-06', 'epoch': '0.7779', 'num_input_tokens_seen': 9467408, 'train_runtime': '9350', 'train_tokens_per_second': '1013'}
+
+ 78%|███████▊  | 1410/1813 [2:35:50<37:51,  5.64s/it]
+ 78%|███████▊  | 1411/1813 [2:35:55<37:28,  5.59s/it]
+ 78%|███████▊  | 1412/1813 [2:36:02<39:13,  5.87s/it]
+ 78%|███████▊  | 1413/1813 [2:36:08<39:10,  5.88s/it]
+ 78%|███████▊  | 1414/1813 [2:36:14<39:45,  5.98s/it]
+ 78%|███████▊  | 1415/1813 [2:36:21<41:52,  6.31s/it]
+ 78%|███████▊  | 1416/1813 [2:36:27<41:39,  6.30s/it]
+ 78%|███████▊  | 1417/1813 [2:36:33<40:04,  6.07s/it]
+ 78%|███████▊  | 1418/1813 [2:36:39<39:33,  6.01s/it]
+ 78%|███████▊  | 1419/1813 [2:36:47<44:47,  6.82s/it]
+ 78%|███████▊  | 1420/1813 [2:36:52<40:40,  6.21s/it]
+                                                     
+{'loss': '0.9012', 'grad_norm': '1.04', 'learning_rate': '8.274e-06', 'epoch': '0.7834', 'num_input_tokens_seen': 9532720, 'train_runtime': '9413', 'train_tokens_per_second': '1013'}
+
+ 78%|███████▊  | 1420/1813 [2:36:52<40:40,  6.21s/it]
+ 78%|███████▊  | 1421/1813 [2:36:59<41:48,  6.40s/it]
+ 78%|███████▊  | 1422/1813 [2:37:04<39:48,  6.11s/it]
+ 78%|███████▊  | 1423/1813 [2:37:10<38:31,  5.93s/it]
+ 79%|███████▊  | 1424/1813 [2:37:17<41:24,  6.39s/it]
+ 79%|███████▊  | 1425/1813 [2:37:22<38:33,  5.96s/it]
+ 79%|███████▊  | 1426/1813 [2:37:31<43:35,  6.76s/it]
+ 79%|███████▊  | 1427/1813 [2:37:37<41:18,  6.42s/it]
+ 79%|███████▉  | 1428/1813 [2:37:43<41:54,  6.53s/it]
+ 79%|███████▉  | 1429/1813 [2:37:50<41:22,  6.46s/it]
+ 79%|███████▉  | 1430/1813 [2:37:57<42:04,  6.59s/it]
+                                                     
+{'loss': '0.8954', 'grad_norm': '0.9881', 'learning_rate': '7.878e-06', 'epoch': '0.789', 'num_input_tokens_seen': 9602112, 'train_runtime': '9477', 'train_tokens_per_second': '1013'}
+
+ 79%|███████▉  | 1430/1813 [2:37:57<42:04,  6.59s/it]
+ 79%|███████▉  | 1431/1813 [2:38:03<41:23,  6.50s/it]
+ 79%|███████▉  | 1432/1813 [2:38:09<40:34,  6.39s/it]
+ 79%|███████▉  | 1433/1813 [2:38:15<40:31,  6.40s/it]
+ 79%|███████▉  | 1434/1813 [2:38:22<40:33,  6.42s/it]
+ 79%|███████▉  | 1435/1813 [2:38:27<38:41,  6.14s/it]
+ 79%|███████▉  | 1436/1813 [2:38:34<39:45,  6.33s/it]
+ 79%|███████▉  | 1437/1813 [2:38:40<38:50,  6.20s/it]
+ 79%|███████▉  | 1438/1813 [2:38:47<40:42,  6.51s/it]
+ 79%|███████▉  | 1439/1813 [2:38:53<38:55,  6.25s/it]
+ 79%|███████▉  | 1440/1813 [2:38:58<36:40,  5.90s/it]
+                                                     
+{'loss': '0.9015', 'grad_norm': '1.035', 'learning_rate': '7.49e-06', 'epoch': '0.7945', 'num_input_tokens_seen': 9667112, 'train_runtime': '9538', 'train_tokens_per_second': '1013'}
+
+ 79%|███████▉  | 1440/1813 [2:38:58<36:40,  5.90s/it]
+ 79%|███████▉  | 1441/1813 [2:39:04<37:26,  6.04s/it]
+ 80%|███████▉  | 1442/1813 [2:39:10<35:47,  5.79s/it]
+ 80%|███████▉  | 1443/1813 [2:39:15<34:28,  5.59s/it]
+ 80%|███████▉  | 1444/1813 [2:39:22<36:55,  6.00s/it]
+ 80%|███████▉  | 1445/1813 [2:39:27<36:05,  5.88s/it]
+ 80%|███████▉  | 1446/1813 [2:39:33<35:38,  5.83s/it]
+ 80%|███████▉  | 1447/1813 [2:39:41<40:06,  6.58s/it]
+ 80%|███████▉  | 1448/1813 [2:39:49<41:20,  6.80s/it]
+ 80%|███████▉  | 1449/1813 [2:39:55<40:58,  6.75s/it]
+ 80%|███████▉  | 1450/1813 [2:40:02<40:30,  6.69s/it]
+                                                     
+{'loss': '0.9865', 'grad_norm': '1.008', 'learning_rate': '7.11e-06', 'epoch': '0.8', 'num_input_tokens_seen': 9735792, 'train_runtime': '9602', 'train_tokens_per_second': '1014'}
+
+ 80%|███████▉  | 1450/1813 [2:40:02<40:30,  6.69s/it]
+ 80%|████████  | 1451/1813 [2:40:09<41:07,  6.82s/it]
+ 80%|████████  | 1452/1813 [2:40:14<37:10,  6.18s/it]
+ 80%|████████  | 1453/1813 [2:40:22<40:52,  6.81s/it]
+ 80%|████████  | 1454/1813 [2:40:27<38:29,  6.43s/it]
+ 80%|████████  | 1455/1813 [2:40:34<38:14,  6.41s/it]
+ 80%|████████  | 1456/1813 [2:40:41<39:27,  6.63s/it]
+ 80%|████████  | 1457/1813 [2:40:45<33:57,  5.72s/it]
+ 80%|████████  | 1458/1813 [2:40:50<33:28,  5.66s/it]
+ 80%|████████  | 1459/1813 [2:40:57<35:45,  6.06s/it]
+ 81%|████████  | 1460/1813 [2:41:04<37:45,  6.42s/it]
+                                                     
+{'loss': '0.9308', 'grad_norm': '0.9971', 'learning_rate': '6.739e-06', 'epoch': '0.8055', 'num_input_tokens_seen': 9802968, 'train_runtime': '9665', 'train_tokens_per_second': '1014'}
+
+ 81%|████████  | 1460/1813 [2:41:04<37:45,  6.42s/it]
+ 81%|████████  | 1461/1813 [2:41:10<36:58,  6.30s/it]
+ 81%|████████  | 1462/1813 [2:41:16<35:14,  6.03s/it]
+ 81%|████████  | 1463/1813 [2:41:23<37:38,  6.45s/it]
+ 81%|████████  | 1464/1813 [2:41:30<37:55,  6.52s/it]
+ 81%|████████  | 1465/1813 [2:41:38<40:52,  7.05s/it]
+ 81%|████████  | 1466/1813 [2:41:44<38:28,  6.65s/it]
+ 81%|████████  | 1467/1813 [2:41:51<39:46,  6.90s/it]
+ 81%|████████  | 1468/1813 [2:41:58<39:09,  6.81s/it]
+ 81%|████████  | 1469/1813 [2:42:07<42:21,  7.39s/it]
+ 81%|████████  | 1470/1813 [2:42:13<41:08,  7.20s/it]
+                                                     
+{'loss': '0.926', 'grad_norm': '0.9611', 'learning_rate': '6.377e-06', 'epoch': '0.811', 'num_input_tokens_seen': 9879232, 'train_runtime': '9734', 'train_tokens_per_second': '1015'}
+
+ 81%|████████  | 1470/1813 [2:42:13<41:08,  7.20s/it]
+ 81%|████████  | 1471/1813 [2:42:20<39:31,  6.93s/it]
+ 81%|████████  | 1472/1813 [2:42:27<39:17,  6.91s/it]
+ 81%|████████  | 1473/1813 [2:42:33<37:40,  6.65s/it]
+ 81%|████████▏ | 1474/1813 [2:42:39<37:45,  6.68s/it]
+ 81%|████████▏ | 1475/1813 [2:42:46<36:55,  6.55s/it]
+ 81%|████████▏ | 1476/1813 [2:42:52<36:40,  6.53s/it]
+ 81%|████████▏ | 1477/1813 [2:42:59<36:30,  6.52s/it]
+ 82%|████████▏ | 1478/1813 [2:43:04<35:11,  6.30s/it]
+ 82%|████████▏ | 1479/1813 [2:43:11<35:00,  6.29s/it]
+ 82%|████████▏ | 1480/1813 [2:43:18<36:09,  6.52s/it]
+                                                     
+{'loss': '0.9053', 'grad_norm': '0.7409', 'learning_rate': '6.024e-06', 'epoch': '0.8166', 'num_input_tokens_seen': 9948032, 'train_runtime': '9798', 'train_tokens_per_second': '1015'}
+
+ 82%|████████▏ | 1480/1813 [2:43:18<36:09,  6.52s/it]
+ 82%|████████▏ | 1481/1813 [2:43:24<36:13,  6.55s/it]
+ 82%|████████▏ | 1482/1813 [2:43:30<34:40,  6.29s/it]
+ 82%|████████▏ | 1483/1813 [2:43:37<35:22,  6.43s/it]
+ 82%|████████▏ | 1484/1813 [2:43:43<35:12,  6.42s/it]
+ 82%|████████▏ | 1485/1813 [2:43:50<35:49,  6.55s/it]
+ 82%|████████▏ | 1486/1813 [2:43:55<33:54,  6.22s/it]
+ 82%|████████▏ | 1487/1813 [2:44:03<36:40,  6.75s/it]
+ 82%|████████▏ | 1488/1813 [2:44:10<36:30,  6.74s/it]
+ 82%|████████▏ | 1489/1813 [2:44:18<37:33,  6.96s/it]
+ 82%|████████▏ | 1490/1813 [2:44:25<37:28,  6.96s/it]
+                                                     
+{'loss': '0.9602', 'grad_norm': '1.078', 'learning_rate': '5.68e-06', 'epoch': '0.8221', 'num_input_tokens_seen': 10021400, 'train_runtime': '9865', 'train_tokens_per_second': '1016'}
+
+ 82%|████████▏ | 1490/1813 [2:44:25<37:28,  6.96s/it]
+ 82%|████████▏ | 1491/1813 [2:44:31<35:49,  6.67s/it]
+ 82%|████████▏ | 1492/1813 [2:44:39<38:26,  7.18s/it]
+ 82%|████████▏ | 1493/1813 [2:44:45<36:39,  6.87s/it]
+ 82%|████████▏ | 1494/1813 [2:44:51<35:27,  6.67s/it]
+ 82%|████████▏ | 1495/1813 [2:44:59<37:31,  7.08s/it]
+ 83%|████████▎ | 1496/1813 [2:45:05<34:34,  6.54s/it]
+ 83%|████████▎ | 1497/1813 [2:45:10<32:10,  6.11s/it]
+ 83%|████████▎ | 1498/1813 [2:45:16<32:29,  6.19s/it]
+ 83%|████████▎ | 1499/1813 [2:45:21<29:45,  5.69s/it]
+ 83%|████████▎ | 1500/1813 [2:45:28<32:27,  6.22s/it]
+                                                     
+{'loss': '0.874', 'grad_norm': '0.9083', 'learning_rate': '5.345e-06', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904, 'train_runtime': '9929', 'train_tokens_per_second': '1016'}
+
+ 83%|████████▎ | 1500/1813 [2:45:28<32:27,  6.22s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.78it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.22it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.31it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.09it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.00it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.98it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.34it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.68it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.12it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.10it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.32it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.04it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.90it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.91it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.29it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.76it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:02,  7.59it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.13it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.22it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.78it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.62it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.34it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.46it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.22it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.06it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.53it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.82it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.23it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.69it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.13it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.89it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.90it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.08it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.05it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.43it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.66it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:46,  7.92it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.58it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.79it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.68it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.83it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:56,  7.14it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.21it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.44it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.51it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.64it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.76it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.18it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.65it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.77it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.63it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.93it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.51it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.47it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.67it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.72it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.13it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.49it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.38it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.22it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.96it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.96it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.37it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.03it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.04it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.70it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.53it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.61it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.92it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.75it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.01it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.20it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.90it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.20it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.48it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.13it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.78it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.09it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.94it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.81it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.22it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.28it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.23it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.57it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.16it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.78it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.59it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.97it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.05it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.87it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.62it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.73it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.56it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.41it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.23it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.78it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.92it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.22it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.19it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.15it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.66it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.38it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.79it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.48it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+
+ 41%|████      | 408/1000 [00:54<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.37it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.07it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.26it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.51it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.55it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.01it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.78it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.50it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.09it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.31it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.19it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.20it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.52it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.28it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.84it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.64it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.50it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.32it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.01it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:16,  6.64it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:51,  9.92it/s]
+
+ 50%|████▉     | 496/1000 [01:06<01:00,  8.39it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.97it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48, 10.00it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.07it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.57it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.50it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.49it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.37it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.09it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.70it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.35it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:18,  5.85it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:20,  5.70it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.23it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:06,  6.90it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.03it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:10,  6.48it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.91it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.49it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.25it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.85it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.03it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.99it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.93it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.52it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.46it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.81it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.62it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.74it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.08it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.45it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.62it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.23it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.43it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.97it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.19it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.97it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.60it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.57it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.33it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.71it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.93it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.58it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.49it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.34it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.65it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.97it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.94it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.91it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.82it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.85it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.27it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.35it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.96it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.51it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.72it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.87it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.37it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.07it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.48it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.17it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.26it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.14it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.63it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.54it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.79it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.63it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.47it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.69it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.25it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.39it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.59it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:33,  7.30it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.70it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.02it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.34it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.76it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.32it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:32,  6.85it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:26,  8.29it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.12it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.50it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.33it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  8.96it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.21it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:26,  7.95it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.73it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.66it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:24,  7.85it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.18it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.98it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.09it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.14it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.03it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.30it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.15it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.21it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.06it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.83it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.81it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.22it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.72it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.00it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.12it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.46it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.17it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.78it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.57it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.50it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.88it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.35it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14,  9.96it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:14, 10.08it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.47it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.73it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.58it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.53it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.25it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.19it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.21it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.75it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.62it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.86it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.79it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.77it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.62it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.85it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.55it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.72it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.88it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.08it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.06it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.33it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.60it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.61it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.18it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.11it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.63it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.84it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.06it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.16it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.35it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.73it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.84it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.41it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.83it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.06it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.51it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.46it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.03it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.43it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.04it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.21it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.81it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.83it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.45it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.07it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.45it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.37it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7682', 'eval_runtime': '135.3', 'eval_samples_per_second': '7.39', 'eval_steps_per_second': '7.39', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904}
+
+ 83%|████████▎ | 1500/1813 [2:47:43<32:27,  6.22s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 83%|████████▎ | 1501/1813 [2:47:50<4:03:56, 46.91s/it]
+ 83%|████████▎ | 1502/1813 [2:47:54<2:57:05, 34.17s/it]
+ 83%|████████▎ | 1503/1813 [2:48:01<2:13:33, 25.85s/it]
+ 83%|████████▎ | 1504/1813 [2:48:06<1:41:27, 19.70s/it]
+ 83%|████████▎ | 1505/1813 [2:48:12<1:19:20, 15.46s/it]
+ 83%|████████▎ | 1506/1813 [2:48:19<1:06:04, 12.91s/it]
+ 83%|████████▎ | 1507/1813 [2:48:24<53:45, 10.54s/it]  
+ 83%|████████▎ | 1508/1813 [2:48:31<48:03,  9.45s/it]
+ 83%|████████▎ | 1509/1813 [2:48:36<42:11,  8.33s/it]
+ 83%|████████▎ | 1510/1813 [2:48:42<38:12,  7.57s/it]
+                                                     
+{'loss': '0.88', 'grad_norm': '1.116', 'learning_rate': '5.019e-06', 'epoch': '0.8331', 'num_input_tokens_seen': 10151928, 'train_runtime': '1.012e+04', 'train_tokens_per_second': '1003'}
+
+ 83%|████████▎ | 1510/1813 [2:48:42<38:12,  7.57s/it]
+ 83%|████████▎ | 1511/1813 [2:48:48<36:05,  7.17s/it]
+ 83%|████████▎ | 1512/1813 [2:48:54<33:36,  6.70s/it]
+ 83%|████████▎ | 1513/1813 [2:48:59<31:34,  6.31s/it]
+ 84%|████████▎ | 1514/1813 [2:49:06<31:48,  6.38s/it]
+ 84%|████████▎ | 1515/1813 [2:49:13<33:12,  6.69s/it]
+ 84%|████████▎ | 1516/1813 [2:49:20<33:22,  6.74s/it]
+ 84%|████████▎ | 1517/1813 [2:49:27<33:09,  6.72s/it]
+ 84%|████████▎ | 1518/1813 [2:49:35<34:33,  7.03s/it]
+ 84%|████████▍ | 1519/1813 [2:49:42<34:35,  7.06s/it]
+ 84%|████████▍ | 1520/1813 [2:49:48<33:11,  6.80s/it]
+                                                     
+{'loss': '0.9027', 'grad_norm': '1.196', 'learning_rate': '4.703e-06', 'epoch': '0.8386', 'num_input_tokens_seen': 10225024, 'train_runtime': '1.019e+04', 'train_tokens_per_second': '1004'}
+
+ 84%|████████▍ | 1520/1813 [2:49:48<33:11,  6.80s/it]
+ 84%|████████▍ | 1521/1813 [2:49:55<33:08,  6.81s/it]
+ 84%|████████▍ | 1522/1813 [2:50:01<32:21,  6.67s/it]
+ 84%|████████▍ | 1523/1813 [2:50:09<33:26,  6.92s/it]
+ 84%|████████▍ | 1524/1813 [2:50:15<32:27,  6.74s/it]
+ 84%|████████▍ | 1525/1813 [2:50:20<29:58,  6.25s/it]
+ 84%|████████▍ | 1526/1813 [2:50:26<30:03,  6.29s/it]
+ 84%|████████▍ | 1527/1813 [2:50:33<30:22,  6.37s/it]
+ 84%|████████▍ | 1528/1813 [2:50:39<30:14,  6.37s/it]
+ 84%|████████▍ | 1529/1813 [2:50:46<30:57,  6.54s/it]
+ 84%|████████▍ | 1530/1813 [2:50:52<29:28,  6.25s/it]
+                                                     
+{'loss': '0.9452', 'grad_norm': '0.8013', 'learning_rate': '4.396e-06', 'epoch': '0.8441', 'num_input_tokens_seen': 10294368, 'train_runtime': '1.025e+04', 'train_tokens_per_second': '1004'}
+
+ 84%|████████▍ | 1530/1813 [2:50:52<29:28,  6.25s/it]
+ 84%|████████▍ | 1531/1813 [2:50:57<27:47,  5.91s/it]
+ 85%|████████▍ | 1532/1813 [2:51:04<29:06,  6.22s/it]
+ 85%|████████▍ | 1533/1813 [2:51:09<27:18,  5.85s/it]
+ 85%|████████▍ | 1534/1813 [2:51:17<30:19,  6.52s/it]
+ 85%|████████▍ | 1535/1813 [2:51:22<28:38,  6.18s/it]
+ 85%|████████▍ | 1536/1813 [2:51:30<30:20,  6.57s/it]
+ 85%|████████▍ | 1537/1813 [2:51:36<30:12,  6.57s/it]
+ 85%|████████▍ | 1538/1813 [2:51:41<27:18,  5.96s/it]
+ 85%|████████▍ | 1539/1813 [2:51:47<26:39,  5.84s/it]
+ 85%|████████▍ | 1540/1813 [2:51:52<26:03,  5.73s/it]
+                                                     
+{'loss': '0.9252', 'grad_norm': '1.211', 'learning_rate': '4.099e-06', 'epoch': '0.8497', 'num_input_tokens_seen': 10359416, 'train_runtime': '1.031e+04', 'train_tokens_per_second': '1005'}
+
+ 85%|████████▍ | 1540/1813 [2:51:52<26:03,  5.73s/it]
+ 85%|████████▍ | 1541/1813 [2:51:59<27:05,  5.98s/it]
+ 85%|████████▌ | 1542/1813 [2:52:05<27:17,  6.04s/it]
+ 85%|████████▌ | 1543/1813 [2:52:10<25:36,  5.69s/it]
+ 85%|████████▌ | 1544/1813 [2:52:15<24:25,  5.45s/it]
+ 85%|████████▌ | 1545/1813 [2:52:22<27:27,  6.15s/it]
+ 85%|████████▌ | 1546/1813 [2:52:28<27:04,  6.09s/it]
+ 85%|████████▌ | 1547/1813 [2:52:33<25:45,  5.81s/it]
+ 85%|████████▌ | 1548/1813 [2:52:42<28:44,  6.51s/it]
+ 85%|████████▌ | 1549/1813 [2:52:47<26:59,  6.13s/it]
+ 85%|████████▌ | 1550/1813 [2:52:54<28:22,  6.47s/it]
+                                                     
+{'loss': '1.021', 'grad_norm': '0.9534', 'learning_rate': '3.811e-06', 'epoch': '0.8552', 'num_input_tokens_seen': 10427368, 'train_runtime': '1.037e+04', 'train_tokens_per_second': '1005'}
+
+ 85%|████████▌ | 1550/1813 [2:52:54<28:22,  6.47s/it]
+ 86%|████████▌ | 1551/1813 [2:53:02<29:46,  6.82s/it]
+ 86%|████████▌ | 1552/1813 [2:53:08<28:34,  6.57s/it]
+ 86%|████████▌ | 1553/1813 [2:53:14<27:36,  6.37s/it]
+ 86%|████████▌ | 1554/1813 [2:53:20<27:50,  6.45s/it]
+ 86%|████████▌ | 1555/1813 [2:53:27<28:01,  6.52s/it]
+ 86%|████████▌ | 1556/1813 [2:53:33<27:39,  6.46s/it]
+ 86%|████████▌ | 1557/1813 [2:53:41<29:22,  6.88s/it]
+ 86%|████████▌ | 1558/1813 [2:53:47<28:21,  6.67s/it]
+ 86%|████████▌ | 1559/1813 [2:53:54<27:59,  6.61s/it]
+ 86%|████████▌ | 1560/1813 [2:53:59<26:18,  6.24s/it]
+                                                     
+{'loss': '0.9989', 'grad_norm': '0.8428', 'learning_rate': '3.533e-06', 'epoch': '0.8607', 'num_input_tokens_seen': 10497472, 'train_runtime': '1.044e+04', 'train_tokens_per_second': '1006'}
+
+ 86%|████████▌ | 1560/1813 [2:53:59<26:18,  6.24s/it]
+ 86%|████████▌ | 1561/1813 [2:54:06<27:03,  6.44s/it]
+ 86%|████████▌ | 1562/1813 [2:54:12<26:59,  6.45s/it]
+ 86%|████████▌ | 1563/1813 [2:54:19<26:40,  6.40s/it]
+ 86%|████████▋ | 1564/1813 [2:54:25<26:15,  6.33s/it]
+ 86%|████████▋ | 1565/1813 [2:54:30<24:00,  5.81s/it]
+ 86%|████████▋ | 1566/1813 [2:54:36<24:20,  5.91s/it]
+ 86%|████████▋ | 1567/1813 [2:54:42<24:40,  6.02s/it]
+ 86%|████████▋ | 1568/1813 [2:54:47<23:19,  5.71s/it]
+ 87%|████████▋ | 1569/1813 [2:54:55<25:37,  6.30s/it]
+ 87%|████████▋ | 1570/1813 [2:55:02<26:50,  6.63s/it]
+                                                     
+{'loss': '0.8522', 'grad_norm': '1.024', 'learning_rate': '3.265e-06', 'epoch': '0.8662', 'num_input_tokens_seen': 10564336, 'train_runtime': '1.05e+04', 'train_tokens_per_second': '1006'}
+
+ 87%|████████▋ | 1570/1813 [2:55:02<26:50,  6.63s/it]
+ 87%|████████▋ | 1571/1813 [2:55:07<25:15,  6.26s/it]
+ 87%|████████▋ | 1572/1813 [2:55:13<24:29,  6.10s/it]
+ 87%|████████▋ | 1573/1813 [2:55:19<24:06,  6.03s/it]
+ 87%|████████▋ | 1574/1813 [2:55:26<24:43,  6.21s/it]
+ 87%|████████▋ | 1575/1813 [2:55:31<24:04,  6.07s/it]
+ 87%|████████▋ | 1576/1813 [2:55:39<25:25,  6.44s/it]
+ 87%|████████▋ | 1577/1813 [2:55:45<25:43,  6.54s/it]
+ 87%|████████▋ | 1578/1813 [2:55:52<25:26,  6.50s/it]
+ 87%|████████▋ | 1579/1813 [2:55:58<24:53,  6.38s/it]
+ 87%|████████▋ | 1580/1813 [2:56:03<23:11,  5.97s/it]
+                                                     
+{'loss': '0.9463', 'grad_norm': '1.367', 'learning_rate': '3.007e-06', 'epoch': '0.8717', 'num_input_tokens_seen': 10628400, 'train_runtime': '1.056e+04', 'train_tokens_per_second': '1006'}
+
+ 87%|████████▋ | 1580/1813 [2:56:03<23:11,  5.97s/it]
+ 87%|████████▋ | 1581/1813 [2:56:09<23:07,  5.98s/it]
+ 87%|████████▋ | 1582/1813 [2:56:13<21:21,  5.55s/it]
+ 87%|████████▋ | 1583/1813 [2:56:20<22:19,  5.82s/it]
+ 87%|████████▋ | 1584/1813 [2:56:26<22:57,  6.01s/it]
+ 87%|████████▋ | 1585/1813 [2:56:34<24:22,  6.42s/it]
+ 87%|████████▋ | 1586/1813 [2:56:41<25:26,  6.72s/it]
+ 88%|████████▊ | 1587/1813 [2:56:49<26:13,  6.96s/it]
+ 88%|████████▊ | 1588/1813 [2:56:54<24:44,  6.60s/it]
+ 88%|████████▊ | 1589/1813 [2:57:00<23:04,  6.18s/it]
+ 88%|████████▊ | 1590/1813 [2:57:06<23:33,  6.34s/it]
+                                                     
+{'loss': '0.9044', 'grad_norm': '0.9844', 'learning_rate': '2.76e-06', 'epoch': '0.8772', 'num_input_tokens_seen': 10695288, 'train_runtime': '1.063e+04', 'train_tokens_per_second': '1006'}
+
+ 88%|████████▊ | 1590/1813 [2:57:06<23:33,  6.34s/it]
+ 88%|████████▊ | 1591/1813 [2:57:12<23:06,  6.24s/it]
+ 88%|████████▊ | 1592/1813 [2:57:19<23:05,  6.27s/it]
+ 88%|████████▊ | 1593/1813 [2:57:25<23:20,  6.37s/it]
+ 88%|████████▊ | 1594/1813 [2:57:32<23:32,  6.45s/it]
+ 88%|████████▊ | 1595/1813 [2:57:39<24:24,  6.72s/it]
+ 88%|████████▊ | 1596/1813 [2:57:48<26:33,  7.34s/it]
+ 88%|████████▊ | 1597/1813 [2:57:55<25:49,  7.18s/it]
+ 88%|████████▊ | 1598/1813 [2:58:02<25:05,  7.00s/it]
+ 88%|████████▊ | 1599/1813 [2:58:07<23:44,  6.65s/it]
+ 88%|████████▊ | 1600/1813 [2:58:12<21:46,  6.13s/it]
+                                                     
+{'loss': '0.9374', 'grad_norm': '0.9498', 'learning_rate': '2.522e-06', 'epoch': '0.8828', 'num_input_tokens_seen': 10767512, 'train_runtime': '1.069e+04', 'train_tokens_per_second': '1007'}
+
+ 88%|████████▊ | 1600/1813 [2:58:12<21:46,  6.13s/it]
+ 88%|████████▊ | 1601/1813 [2:58:20<23:45,  6.72s/it]
+ 88%|████████▊ | 1602/1813 [2:58:27<23:03,  6.55s/it]
+ 88%|████████▊ | 1603/1813 [2:58:33<23:15,  6.64s/it]
+ 88%|████████▊ | 1604/1813 [2:58:40<23:23,  6.72s/it]
+ 89%|████████▊ | 1605/1813 [2:58:49<24:59,  7.21s/it]
+ 89%|████████▊ | 1606/1813 [2:58:54<22:39,  6.57s/it]
+ 89%|████████▊ | 1607/1813 [2:59:00<22:38,  6.60s/it]
+ 89%|████████▊ | 1608/1813 [2:59:08<23:22,  6.84s/it]
+ 89%|████████▊ | 1609/1813 [2:59:13<21:46,  6.40s/it]
+ 89%|████████▉ | 1610/1813 [2:59:19<21:11,  6.27s/it]
+                                                     
+{'loss': '0.928', 'grad_norm': '1.293', 'learning_rate': '2.295e-06', 'epoch': '0.8883', 'num_input_tokens_seen': 10840800, 'train_runtime': '1.076e+04', 'train_tokens_per_second': '1008'}
+
+ 89%|████████▉ | 1610/1813 [2:59:19<21:11,  6.27s/it]
+ 89%|████████▉ | 1611/1813 [2:59:24<19:54,  5.91s/it]
+ 89%|████████▉ | 1612/1813 [2:59:30<19:53,  5.94s/it]
+ 89%|████████▉ | 1613/1813 [2:59:35<18:23,  5.52s/it]
+ 89%|████████▉ | 1614/1813 [2:59:41<18:34,  5.60s/it]
+ 89%|████████▉ | 1615/1813 [2:59:47<19:19,  5.86s/it]
+ 89%|████████▉ | 1616/1813 [2:59:54<19:57,  6.08s/it]
+ 89%|████████▉ | 1617/1813 [3:00:00<20:21,  6.23s/it]
+ 89%|████████▉ | 1618/1813 [3:00:05<19:13,  5.92s/it]
+ 89%|████████▉ | 1619/1813 [3:00:11<19:00,  5.88s/it]
+ 89%|████████▉ | 1620/1813 [3:00:16<17:53,  5.56s/it]
+                                                     
+{'loss': '0.9323', 'grad_norm': '1.038', 'learning_rate': '2.078e-06', 'epoch': '0.8938', 'num_input_tokens_seen': 10899640, 'train_runtime': '1.082e+04', 'train_tokens_per_second': '1008'}
+
+ 89%|████████▉ | 1620/1813 [3:00:16<17:53,  5.56s/it]
+ 89%|████████▉ | 1621/1813 [3:00:21<17:32,  5.48s/it]
+ 89%|████████▉ | 1622/1813 [3:00:29<19:25,  6.10s/it]
+ 90%|████████▉ | 1623/1813 [3:00:36<19:58,  6.31s/it]
+ 90%|████████▉ | 1624/1813 [3:00:41<19:25,  6.17s/it]
+ 90%|████████▉ | 1625/1813 [3:00:49<20:22,  6.50s/it]
+ 90%|████████▉ | 1626/1813 [3:00:55<19:46,  6.35s/it]
+ 90%|████████▉ | 1627/1813 [3:01:01<19:10,  6.18s/it]
+ 90%|████████▉ | 1628/1813 [3:01:06<18:31,  6.01s/it]
+ 90%|████████▉ | 1629/1813 [3:01:13<19:27,  6.35s/it]
+ 90%|████████▉ | 1630/1813 [3:01:18<18:20,  6.01s/it]
+                                                     
+{'loss': '0.9393', 'grad_norm': '1.522', 'learning_rate': '1.871e-06', 'epoch': '0.8993', 'num_input_tokens_seen': 10968528, 'train_runtime': '1.088e+04', 'train_tokens_per_second': '1008'}
+
+ 90%|████████▉ | 1630/1813 [3:01:18<18:20,  6.01s/it]
+ 90%|████████▉ | 1631/1813 [3:01:24<17:39,  5.82s/it]
+ 90%|█████████ | 1632/1813 [3:01:29<16:36,  5.51s/it]
+ 90%|█████████ | 1633/1813 [3:01:36<18:11,  6.06s/it]
+ 90%|█████████ | 1634/1813 [3:01:41<17:00,  5.70s/it]
+ 90%|█████████ | 1635/1813 [3:01:48<17:48,  6.00s/it]
+ 90%|█████████ | 1636/1813 [3:01:53<17:12,  5.83s/it]
+ 90%|█████████ | 1637/1813 [3:01:59<17:30,  5.97s/it]
+ 90%|█████████ | 1638/1813 [3:02:04<16:41,  5.72s/it]
+ 90%|█████████ | 1639/1813 [3:02:11<17:45,  6.12s/it]
+ 90%|█████████ | 1640/1813 [3:02:17<16:55,  5.87s/it]
+                                                     
+{'loss': '0.9436', 'grad_norm': '0.8202', 'learning_rate': '1.675e-06', 'epoch': '0.9048', 'num_input_tokens_seen': 11030048, 'train_runtime': '1.094e+04', 'train_tokens_per_second': '1008'}
+
+ 90%|█████████ | 1640/1813 [3:02:17<16:55,  5.87s/it]
+ 91%|█████████ | 1641/1813 [3:02:23<16:54,  5.90s/it]
+ 91%|█████████ | 1642/1813 [3:02:29<17:12,  6.04s/it]
+ 91%|█████████ | 1643/1813 [3:02:35<17:05,  6.03s/it]
+ 91%|█████████ | 1644/1813 [3:02:42<17:33,  6.23s/it]
+ 91%|█████████ | 1645/1813 [3:02:51<20:09,  7.20s/it]
+ 91%|█████████ | 1646/1813 [3:02:58<19:38,  7.06s/it]
+ 91%|█████████ | 1647/1813 [3:03:04<18:44,  6.77s/it]
+ 91%|█████████ | 1648/1813 [3:03:11<18:37,  6.77s/it]
+ 91%|█████████ | 1649/1813 [3:03:16<17:13,  6.30s/it]
+ 91%|█████████ | 1650/1813 [3:03:21<16:01,  5.90s/it]
+                                                     
+{'loss': '0.9065', 'grad_norm': '1.027', 'learning_rate': '1.49e-06', 'epoch': '0.9103', 'num_input_tokens_seen': 11098768, 'train_runtime': '1.1e+04', 'train_tokens_per_second': '1009'}
+
+ 91%|█████████ | 1650/1813 [3:03:21<16:01,  5.90s/it]
+ 91%|█████████ | 1651/1813 [3:03:27<16:15,  6.02s/it]
+ 91%|█████████ | 1652/1813 [3:03:34<16:29,  6.14s/it]
+ 91%|█████████ | 1653/1813 [3:03:40<16:46,  6.29s/it]
+ 91%|█████████ | 1654/1813 [3:03:46<16:24,  6.19s/it]
+ 91%|█████████▏| 1655/1813 [3:03:53<16:34,  6.30s/it]
+ 91%|█████████▏| 1656/1813 [3:03:59<16:09,  6.18s/it]
+ 91%|█████████▏| 1657/1813 [3:04:07<17:15,  6.64s/it]
+ 91%|█████████▏| 1658/1813 [3:04:13<16:55,  6.55s/it]
+ 92%|█████████▏| 1659/1813 [3:04:20<17:09,  6.68s/it]
+ 92%|█████████▏| 1660/1813 [3:04:26<16:45,  6.57s/it]
+                                                     
+{'loss': '0.9588', 'grad_norm': '1.17', 'learning_rate': '1.315e-06', 'epoch': '0.9159', 'num_input_tokens_seen': 11168848, 'train_runtime': '1.107e+04', 'train_tokens_per_second': '1009'}
+
+ 92%|█████████▏| 1660/1813 [3:04:26<16:45,  6.57s/it]
+ 92%|█████████▏| 1661/1813 [3:04:33<17:02,  6.73s/it]
+ 92%|█████████▏| 1662/1813 [3:04:40<16:53,  6.71s/it]
+ 92%|█████████▏| 1663/1813 [3:04:47<17:02,  6.82s/it]
+ 92%|█████████▏| 1664/1813 [3:04:54<17:07,  6.90s/it]
+ 92%|█████████▏| 1665/1813 [3:05:02<17:32,  7.11s/it]
+ 92%|█████████▏| 1666/1813 [3:05:06<15:26,  6.31s/it]
+ 92%|█████████▏| 1667/1813 [3:05:13<15:41,  6.45s/it]
+ 92%|█████████▏| 1668/1813 [3:05:19<15:18,  6.33s/it]
+ 92%|█████████▏| 1669/1813 [3:05:29<17:48,  7.42s/it]
+ 92%|█████████▏| 1670/1813 [3:05:34<16:08,  6.77s/it]
+                                                     
+{'loss': '0.9207', 'grad_norm': '1.138', 'learning_rate': '1.151e-06', 'epoch': '0.9214', 'num_input_tokens_seen': 11244288, 'train_runtime': '1.113e+04', 'train_tokens_per_second': '1010'}
+
+ 92%|█████████▏| 1670/1813 [3:05:34<16:08,  6.77s/it]
+ 92%|█████████▏| 1671/1813 [3:05:39<14:48,  6.26s/it]
+ 92%|█████████▏| 1672/1813 [3:05:45<14:18,  6.09s/it]
+ 92%|█████████▏| 1673/1813 [3:05:51<13:58,  5.99s/it]
+ 92%|█████████▏| 1674/1813 [3:05:58<14:29,  6.26s/it]
+ 92%|█████████▏| 1675/1813 [3:06:05<15:08,  6.59s/it]
+ 92%|█████████▏| 1676/1813 [3:06:12<15:41,  6.87s/it]
+ 92%|█████████▏| 1677/1813 [3:06:19<15:12,  6.71s/it]
+ 93%|█████████▎| 1678/1813 [3:06:25<14:54,  6.63s/it]
+ 93%|█████████▎| 1679/1813 [3:06:31<14:26,  6.46s/it]
+ 93%|█████████▎| 1680/1813 [3:06:40<15:52,  7.16s/it]
+                                                     
+{'loss': '0.9799', 'grad_norm': '0.9285', 'learning_rate': '9.971e-07', 'epoch': '0.9269', 'num_input_tokens_seen': 11317056, 'train_runtime': '1.12e+04', 'train_tokens_per_second': '1010'}
+
+ 93%|█████████▎| 1680/1813 [3:06:40<15:52,  7.16s/it]
+ 93%|█████████▎| 1681/1813 [3:06:45<14:10,  6.44s/it]
+ 93%|█████████▎| 1682/1813 [3:06:50<13:11,  6.04s/it]
+ 93%|█████████▎| 1683/1813 [3:06:55<12:08,  5.60s/it]
+ 93%|█████████▎| 1684/1813 [3:07:01<12:48,  5.95s/it]
+ 93%|█████████▎| 1685/1813 [3:07:08<13:08,  6.16s/it]
+ 93%|█████████▎| 1686/1813 [3:07:15<13:27,  6.36s/it]
+ 93%|█████████▎| 1687/1813 [3:07:21<13:20,  6.36s/it]
+ 93%|█████████▎| 1688/1813 [3:07:28<13:30,  6.49s/it]
+ 93%|█████████▎| 1689/1813 [3:07:33<12:22,  5.99s/it]
+ 93%|█████████▎| 1690/1813 [3:07:39<12:41,  6.19s/it]
+                                                     
+{'loss': '0.9026', 'grad_norm': '0.8212', 'learning_rate': '8.545e-07', 'epoch': '0.9324', 'num_input_tokens_seen': 11380560, 'train_runtime': '1.126e+04', 'train_tokens_per_second': '1011'}
+
+ 93%|█████████▎| 1690/1813 [3:07:39<12:41,  6.19s/it]
+ 93%|█████████▎| 1691/1813 [3:07:46<12:37,  6.21s/it]
+ 93%|█████████▎| 1692/1813 [3:07:53<13:25,  6.66s/it]
+ 93%|█████████▎| 1693/1813 [3:08:00<13:04,  6.54s/it]
+ 93%|█████████▎| 1694/1813 [3:08:08<14:03,  7.09s/it]
+ 93%|█████████▎| 1695/1813 [3:08:14<13:23,  6.81s/it]
+ 94%|█████████▎| 1696/1813 [3:08:20<12:45,  6.54s/it]
+ 94%|█████████▎| 1697/1813 [3:08:27<12:37,  6.53s/it]
+ 94%|█████████▎| 1698/1813 [3:08:32<11:53,  6.21s/it]
+ 94%|█████████▎| 1699/1813 [3:08:38<11:51,  6.24s/it]
+ 94%|█████████▍| 1700/1813 [3:08:45<11:45,  6.24s/it]
+                                                     
+{'loss': '0.9169', 'grad_norm': '1.118', 'learning_rate': '7.227e-07', 'epoch': '0.9379', 'num_input_tokens_seen': 11450712, 'train_runtime': '1.133e+04', 'train_tokens_per_second': '1011'}
+
+ 94%|█████████▍| 1700/1813 [3:08:45<11:45,  6.24s/it]
+ 94%|█████████▍| 1701/1813 [3:08:49<10:39,  5.71s/it]
+ 94%|█████████▍| 1702/1813 [3:08:54<10:20,  5.59s/it]
+ 94%|█████████▍| 1703/1813 [3:09:00<10:19,  5.63s/it]
+ 94%|█████████▍| 1704/1813 [3:09:08<11:17,  6.21s/it]
+ 94%|█████████▍| 1705/1813 [3:09:14<11:24,  6.34s/it]
+ 94%|█████████▍| 1706/1813 [3:09:21<11:21,  6.37s/it]
+ 94%|█████████▍| 1707/1813 [3:09:26<10:52,  6.16s/it]
+ 94%|█████████▍| 1708/1813 [3:09:31<10:10,  5.82s/it]
+ 94%|█████████▍| 1709/1813 [3:09:38<10:19,  5.96s/it]
+ 94%|█████████▍| 1710/1813 [3:09:44<10:10,  5.93s/it]
+                                                     
+{'loss': '0.9487', 'grad_norm': '1.224', 'learning_rate': '6.019e-07', 'epoch': '0.9434', 'num_input_tokens_seen': 11511984, 'train_runtime': '1.138e+04', 'train_tokens_per_second': '1011'}
+
+ 94%|█████████▍| 1710/1813 [3:09:44<10:10,  5.93s/it]
+ 94%|█████████▍| 1711/1813 [3:09:49<09:50,  5.79s/it]
+ 94%|█████████▍| 1712/1813 [3:09:55<09:39,  5.74s/it]
+ 94%|█████████▍| 1713/1813 [3:10:00<09:34,  5.75s/it]
+ 95%|█████████▍| 1714/1813 [3:10:06<09:25,  5.72s/it]
+ 95%|█████████▍| 1715/1813 [3:10:11<09:10,  5.62s/it]
+ 95%|█████████▍| 1716/1813 [3:10:16<08:44,  5.41s/it]
+ 95%|█████████▍| 1717/1813 [3:10:22<08:33,  5.34s/it]
+ 95%|█████████▍| 1718/1813 [3:10:28<08:50,  5.58s/it]
+ 95%|█████████▍| 1719/1813 [3:10:34<09:03,  5.78s/it]
+ 95%|█████████▍| 1720/1813 [3:10:40<09:02,  5.83s/it]
+                                                     
+{'loss': '0.9179', 'grad_norm': '0.8832', 'learning_rate': '4.92e-07', 'epoch': '0.949', 'num_input_tokens_seen': 11571368, 'train_runtime': '1.144e+04', 'train_tokens_per_second': '1011'}
+
+ 95%|█████████▍| 1720/1813 [3:10:40<09:02,  5.83s/it]
+ 95%|█████████▍| 1721/1813 [3:10:47<09:36,  6.27s/it]
+ 95%|█████████▍| 1722/1813 [3:10:53<09:23,  6.19s/it]
+ 95%|█████████▌| 1723/1813 [3:10:59<09:12,  6.14s/it]
+ 95%|█████████▌| 1724/1813 [3:11:05<08:59,  6.06s/it]
+ 95%|█████████▌| 1725/1813 [3:11:12<09:23,  6.40s/it]
+ 95%|█████████▌| 1726/1813 [3:11:20<09:41,  6.69s/it]
+ 95%|█████████▌| 1727/1813 [3:11:26<09:16,  6.47s/it]
+ 95%|█████████▌| 1728/1813 [3:11:31<08:30,  6.00s/it]
+ 95%|█████████▌| 1729/1813 [3:11:36<08:03,  5.76s/it]
+ 95%|█████████▌| 1730/1813 [3:11:42<08:17,  5.99s/it]
+                                                     
+{'loss': '0.9475', 'grad_norm': '0.7417', 'learning_rate': '3.931e-07', 'epoch': '0.9545', 'num_input_tokens_seen': 11640392, 'train_runtime': '1.15e+04', 'train_tokens_per_second': '1012'}
+
+ 95%|█████████▌| 1730/1813 [3:11:42<08:17,  5.99s/it]
+ 95%|█████████▌| 1731/1813 [3:11:48<08:13,  6.02s/it]
+ 96%|█████████▌| 1732/1813 [3:11:55<08:28,  6.27s/it]
+ 96%|█████████▌| 1733/1813 [3:12:00<07:52,  5.91s/it]
+ 96%|█████████▌| 1734/1813 [3:12:06<07:45,  5.89s/it]
+ 96%|█████████▌| 1735/1813 [3:12:13<07:51,  6.04s/it]
+ 96%|█████████▌| 1736/1813 [3:12:19<08:03,  6.27s/it]
+ 96%|█████████▌| 1737/1813 [3:12:26<07:56,  6.26s/it]
+ 96%|█████████▌| 1738/1813 [3:12:31<07:26,  5.95s/it]
+ 96%|█████████▌| 1739/1813 [3:12:39<08:06,  6.58s/it]
+ 96%|█████████▌| 1740/1813 [3:12:45<07:59,  6.56s/it]
+                                                     
+{'loss': '0.9345', 'grad_norm': '1.107', 'learning_rate': '3.052e-07', 'epoch': '0.96', 'num_input_tokens_seen': 11709120, 'train_runtime': '1.157e+04', 'train_tokens_per_second': '1012'}
+
+ 96%|█████████▌| 1740/1813 [3:12:45<07:59,  6.56s/it]
+ 96%|█████████▌| 1741/1813 [3:12:51<07:34,  6.31s/it]
+ 96%|█████████▌| 1742/1813 [3:12:58<07:36,  6.43s/it]
+ 96%|█████████▌| 1743/1813 [3:13:05<07:42,  6.61s/it]
+ 96%|█████████▌| 1744/1813 [3:13:12<07:45,  6.75s/it]
+ 96%|█████████▌| 1745/1813 [3:13:18<07:21,  6.50s/it]
+ 96%|█████████▋| 1746/1813 [3:13:25<07:31,  6.75s/it]
+ 96%|█████████▋| 1747/1813 [3:13:30<06:45,  6.15s/it]
+ 96%|█████████▋| 1748/1813 [3:13:37<07:02,  6.51s/it]
+ 96%|█████████▋| 1749/1813 [3:13:43<06:35,  6.17s/it]
+ 97%|█████████▋| 1750/1813 [3:13:48<06:16,  5.98s/it]
+                                                     
+{'loss': '0.91', 'grad_norm': '0.8465', 'learning_rate': '2.284e-07', 'epoch': '0.9655', 'num_input_tokens_seen': 11777768, 'train_runtime': '1.163e+04', 'train_tokens_per_second': '1013'}
+
+ 97%|█████████▋| 1750/1813 [3:13:48<06:16,  5.98s/it]
+ 97%|█████████▋| 1751/1813 [3:13:54<06:14,  6.03s/it]
+ 97%|█████████▋| 1752/1813 [3:14:00<06:04,  5.98s/it]
+ 97%|█████████▋| 1753/1813 [3:14:08<06:27,  6.45s/it]
+ 97%|█████████▋| 1754/1813 [3:14:15<06:35,  6.71s/it]
+ 97%|█████████▋| 1755/1813 [3:14:21<06:23,  6.61s/it]
+ 97%|█████████▋| 1756/1813 [3:14:27<05:59,  6.30s/it]
+ 97%|█████████▋| 1757/1813 [3:14:33<05:43,  6.13s/it]
+ 97%|█████████▋| 1758/1813 [3:14:39<05:34,  6.08s/it]
+ 97%|█████████▋| 1759/1813 [3:14:45<05:29,  6.10s/it]
+ 97%|█████████▋| 1760/1813 [3:14:50<05:09,  5.84s/it]
+                                                     
+{'loss': '0.9204', 'grad_norm': '1.112', 'learning_rate': '1.627e-07', 'epoch': '0.971', 'num_input_tokens_seen': 11843192, 'train_runtime': '1.169e+04', 'train_tokens_per_second': '1013'}
+
+ 97%|█████████▋| 1760/1813 [3:14:50<05:09,  5.84s/it]
+ 97%|█████████▋| 1761/1813 [3:14:55<04:55,  5.68s/it]
+ 97%|█████████▋| 1762/1813 [3:15:00<04:33,  5.36s/it]
+ 97%|█████████▋| 1763/1813 [3:15:08<05:04,  6.08s/it]
+ 97%|█████████▋| 1764/1813 [3:15:13<04:46,  5.85s/it]
+ 97%|█████████▋| 1765/1813 [3:15:18<04:34,  5.71s/it]
+ 97%|█████████▋| 1766/1813 [3:15:24<04:27,  5.68s/it]
+ 97%|█████████▋| 1767/1813 [3:15:30<04:21,  5.69s/it]
+ 98%|█████████▊| 1768/1813 [3:15:36<04:20,  5.79s/it]
+ 98%|█████████▊| 1769/1813 [3:15:42<04:16,  5.83s/it]
+ 98%|█████████▊| 1770/1813 [3:15:48<04:22,  6.10s/it]
+                                                     
+{'loss': '0.9343', 'grad_norm': '0.9116', 'learning_rate': '1.08e-07', 'epoch': '0.9766', 'num_input_tokens_seen': 11903728, 'train_runtime': '1.175e+04', 'train_tokens_per_second': '1013'}
+
+ 98%|█████████▊| 1770/1813 [3:15:48<04:22,  6.10s/it]
+ 98%|█████████▊| 1771/1813 [3:15:55<04:20,  6.21s/it]
+ 98%|█████████▊| 1772/1813 [3:16:01<04:18,  6.30s/it]
+ 98%|█████████▊| 1773/1813 [3:16:09<04:21,  6.54s/it]
+ 98%|█████████▊| 1774/1813 [3:16:13<03:55,  6.04s/it]
+ 98%|█████████▊| 1775/1813 [3:16:21<04:06,  6.49s/it]
+ 98%|█████████▊| 1776/1813 [3:16:28<04:09,  6.76s/it]
+ 98%|█████████▊| 1777/1813 [3:16:36<04:10,  6.96s/it]
+ 98%|█████████▊| 1778/1813 [3:16:41<03:48,  6.54s/it]
+ 98%|█████████▊| 1779/1813 [3:16:47<03:32,  6.26s/it]
+ 98%|█████████▊| 1780/1813 [3:16:53<03:24,  6.20s/it]
+                                                     
+{'loss': '0.9347', 'grad_norm': '0.7866', 'learning_rate': '6.452e-08', 'epoch': '0.9821', 'num_input_tokens_seen': 11972864, 'train_runtime': '1.181e+04', 'train_tokens_per_second': '1013'}
+
+ 98%|█████████▊| 1780/1813 [3:16:53<03:24,  6.20s/it]
+ 98%|█████████▊| 1781/1813 [3:16:59<03:20,  6.26s/it]
+ 98%|█████████▊| 1782/1813 [3:17:05<03:12,  6.20s/it]
+ 98%|█████████▊| 1783/1813 [3:17:12<03:05,  6.18s/it]
+ 98%|█████████▊| 1784/1813 [3:17:18<03:03,  6.33s/it]
+ 98%|█████████▊| 1785/1813 [3:17:24<02:53,  6.20s/it]
+ 99%|█████████▊| 1786/1813 [3:17:30<02:46,  6.17s/it]
+ 99%|█████████▊| 1787/1813 [3:17:36<02:33,  5.90s/it]
+ 99%|█████████▊| 1788/1813 [3:17:42<02:32,  6.11s/it]
+ 99%|█████████▊| 1789/1813 [3:17:51<02:44,  6.84s/it]
+ 99%|█████████▊| 1790/1813 [3:17:58<02:39,  6.95s/it]
+                                                     
+{'loss': '0.9259', 'grad_norm': '0.9169', 'learning_rate': '3.215e-08', 'epoch': '0.9876', 'num_input_tokens_seen': 12043496, 'train_runtime': '1.188e+04', 'train_tokens_per_second': '1014'}
+
+ 99%|█████████▊| 1790/1813 [3:17:58<02:39,  6.95s/it]
+ 99%|█████████▉| 1791/1813 [3:18:03<02:23,  6.54s/it]
+ 99%|█████████▉| 1792/1813 [3:18:10<02:15,  6.44s/it]
+ 99%|█████████▉| 1793/1813 [3:18:16<02:10,  6.53s/it]
+ 99%|█████████▉| 1794/1813 [3:18:23<02:07,  6.70s/it]
+ 99%|█████████▉| 1795/1813 [3:18:30<02:00,  6.72s/it]
+ 99%|█████████▉| 1796/1813 [3:18:37<01:54,  6.76s/it]
+ 99%|█████████▉| 1797/1813 [3:18:43<01:45,  6.60s/it]
+ 99%|█████████▉| 1798/1813 [3:18:53<01:51,  7.42s/it]
+ 99%|█████████▉| 1799/1813 [3:18:59<01:40,  7.17s/it]
+ 99%|█████████▉| 1800/1813 [3:19:04<01:22,  6.37s/it]
+                                                     
+{'loss': '0.9041', 'grad_norm': '0.9477', 'learning_rate': '1.094e-08', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448, 'train_runtime': '1.194e+04', 'train_tokens_per_second': '1014'}
+
+ 99%|█████████▉| 1800/1813 [3:19:04<01:22,  6.37s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.80it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.21it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.31it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:08,  7.70it/s]
+
+  1%|          | 12/1000 [00:01<02:05,  7.84it/s]
+
+  1%|▏         | 13/1000 [00:01<02:31,  6.52it/s]
+
+  1%|▏         | 14/1000 [00:01<02:32,  6.46it/s]
+
+  2%|▏         | 15/1000 [00:02<02:55,  5.62it/s]
+
+  2%|▏         | 16/1000 [00:02<03:00,  5.45it/s]
+
+  2%|▏         | 18/1000 [00:02<02:46,  5.90it/s]
+
+  2%|▏         | 19/1000 [00:02<03:15,  5.02it/s]
+
+  2%|▏         | 20/1000 [00:03<03:37,  4.51it/s]
+
+  2%|▏         | 22/1000 [00:03<02:37,  6.20it/s]
+
+  2%|▏         | 23/1000 [00:03<02:56,  5.55it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.35it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.35it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  6.00it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.68it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.09it/s]
+
+  4%|▍         | 39/1000 [00:05<02:07,  7.56it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.34it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:07<01:48,  8.75it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.90it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.94it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.76it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.30it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:35,  9.76it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+
+  7%|▋         | 72/1000 [00:09<02:17,  6.73it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.13it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.86it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.86it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.19it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.01it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:12<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.09it/s]
+
+  9%|▉         | 92/1000 [00:12<02:26,  6.18it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:12,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.33it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.43it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.19it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.03it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.52it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.80it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.20it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.14it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.65it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.89it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.07it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.51it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.06it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.41it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.66it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.82it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.85it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.15it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.75it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.46it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.82it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.59it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.76it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.50it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.39it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.76it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.21it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.69it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.79it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.65it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.89it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.50it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.67it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.73it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.14it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.22it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.94it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.96it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.37it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.46it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.04it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:40,  7.38it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.70it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.70it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.53it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.04it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:13,  5.45it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.71it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.71it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.07it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.73it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.90it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.20it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.48it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.15it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.81it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.24it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.24it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.58it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.59it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.97it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.39it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.02it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.87it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.73it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.23it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.42it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.97it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.61it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.42it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.57it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.74it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.10it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.24it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.76it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.90it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.20it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.19it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.66it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.39it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:08,  8.80it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.49it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.76it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.24it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.50it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.83it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.27it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.46it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.51it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.96it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.53it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.00it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.79it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.50it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.07it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.29it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.26it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.03it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.19it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:17,  7.01it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:05,  8.34it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:10,  7.73it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.21it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:50, 10.76it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.59it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.02it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.58it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.40it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.33it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.33it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:16,  6.64it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.42it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.44it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.94it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.56it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.57it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.82it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.44it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.98it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.99it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.04it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.55it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.42it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.14it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.72it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.67it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.77it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.95it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.51it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.71it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<01:01,  7.35it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:02,  7.14it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.13it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.91it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.74it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  6.99it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.82it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.02it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.46it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.32it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.84it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.26it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.57it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.61it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.93it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.76it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.81it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.40it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.89it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.40it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.72it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.70it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.12it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:45,  8.48it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.74it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.82it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.64it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.11it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.04it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.22it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.47it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.20it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.44it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.73it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.08it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.96it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.18it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.97it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.61it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:45,  7.61it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.83it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.08it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  5.99it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.55it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.67it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.70it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.99it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.94it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.91it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.88it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.18it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.83it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.86it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.28it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.44it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.35it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.97it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.07it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.63it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.23it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.43it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.04it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.70it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.38it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.07it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.48it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.17it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.26it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.26it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.12it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.60it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.34it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:27,  9.55it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.91it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.52it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.78it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.32it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.60it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.43it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.69it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.24it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.42it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.62it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.72it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.67it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.62it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.48it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.80it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.65it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:34,  6.57it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.14it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.41it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.81it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.91it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.64it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.96it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:26,  8.38it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.17it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  8.99it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.98it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.20it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.73it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.66it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:24,  7.85it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.18it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.98it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.22it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.24it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.08it/s]
+
+ 82%|████████▏ | 815/1000 [01:49<00:19,  9.33it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.83it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.03it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.55it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.13it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.17it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.56it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.50it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:14, 10.21it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.53it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.64it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.71it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.31it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.25it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.27it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.61it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.97it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.19it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.37it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.78it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.77it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.83it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.32it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.61it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.18it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.67it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:08,  8.11it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.66it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.84it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.63it/s]
+
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.19it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.15it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.71it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.41it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.41it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.83it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.33it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.41it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.80it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.44it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.07it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.00it/s]
+
+ 99%|█████████▊| 987/1000 [02:12<00:01,  7.56it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.33it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  5.60it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  5.90it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.76it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.45it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.43it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.01it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.74it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7667', 'eval_runtime': '135.3', 'eval_samples_per_second': '7.391', 'eval_steps_per_second': '7.391', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448}
+
+ 99%|█████████▉| 1800/1813 [3:21:19<01:22,  6.37s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_006_lr_6.257e-05/checkpoint-1800
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+ 99%|█████████▉| 1801/1813 [3:21:28<09:32, 47.71s/it]
+ 99%|█████████▉| 1802/1813 [3:21:33<06:25, 35.02s/it]
+ 99%|█████████▉| 1803/1813 [3:21:39<04:22, 26.24s/it]
+100%|█████████▉| 1804/1813 [3:21:47<03:06, 20.68s/it]
+100%|█████████▉| 1805/1813 [3:21:53<02:10, 16.34s/it]
+100%|█████████▉| 1806/1813 [3:22:00<01:33, 13.41s/it]
+100%|█████████▉| 1807/1813 [3:22:07<01:09, 11.57s/it]
+100%|█████████▉| 1808/1813 [3:22:14<00:51, 10.34s/it]
+100%|█████████▉| 1809/1813 [3:22:21<00:36,  9.19s/it]
+100%|█████████▉| 1810/1813 [3:22:26<00:24,  8.06s/it]
+                                                     
+{'loss': '0.8792', 'grad_norm': '0.9834', 'learning_rate': '8.933e-10', 'epoch': '0.9986', 'num_input_tokens_seen': 12187432, 'train_runtime': '1.215e+04', 'train_tokens_per_second': '1003'}
+
+100%|█████████▉| 1810/1813 [3:22:26<00:24,  8.06s/it]
+100%|█████████▉| 1811/1813 [3:22:32<00:14,  7.26s/it]
+100%|█████████▉| 1812/1813 [3:22:37<00:06,  6.71s/it]
+100%|██████████| 1813/1813 [3:22:40<00:00,  5.58s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.77it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.34it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.06it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.99it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.82it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.27it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.32it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.04it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.35it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.73it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.90it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.92it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.74it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.30it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:35,  9.76it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.38it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.27it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.01it/s]
+
+  7%|▋         | 72/1000 [00:09<02:17,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.14it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.86it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.20it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.03it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.53it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:12,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.38it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.83it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.33it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.46it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.22it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.05it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.54it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.81it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.22it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.09it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.81it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.69it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.13it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.65it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:26,  9.91it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.90it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.08it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:40,  8.45it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.51it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.92it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.04it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.44it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.67it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.99it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.82it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.52it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.42it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:23,  5.79it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.17it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.23it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.75it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.82it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.51it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:22,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:01,  6.64it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.76it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.21it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.71it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.80it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.65it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.96it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.54it/s]
+
+ 21%|██        | 210/1000 [00:27<01:32,  8.50it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.70it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.79it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:06,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.43it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:52,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.92it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.39it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.22it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.97it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.47it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.36it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.87it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:05,  5.97it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.70it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.74it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.44it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.90it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.63it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.04it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.72it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.99it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.75it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.07it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.81it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.25it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.24it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.56it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.76it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.97it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:44<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.83it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.86it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.65it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.56it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.37it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.60it/s]
+
+ 36%|███▌      | 361/1000 [00:47<01:07,  9.43it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.29it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.93it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.23it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.91it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.77it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.21it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.19it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.65it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.37it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.79it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.48it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.76it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.23it/s]
+
+ 41%|████      | 406/1000 [00:54<01:10,  8.47it/s]
+
+ 41%|████      | 408/1000 [00:54<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.26it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.51it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.70it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.94it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.55it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.85it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.56it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.04it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:48, 11.72it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.47it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<00:59,  9.32it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:05,  8.45it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:15,  7.36it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:19,  6.96it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:29,  6.13it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.54it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.16it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.48it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.82it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.79it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.61it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.32it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.87it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.31it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:05,  7.86it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.58it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.17it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.23it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.94it/s]
+
+ 50%|████▉     | 496/1000 [01:06<01:00,  8.39it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.94it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.81it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.82it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.44it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.97it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.99it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.55it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.69it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.51it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.76it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.01it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:50,  8.81it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.00it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.36it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.46it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.84it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.57it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.61it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.76it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.79it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.37it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.86it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.68it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.80it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.62it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:42,  8.93it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:48,  7.81it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.06it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.37it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.54it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:52,  7.15it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.20it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.84it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.42it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.33it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.71it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.57it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.49it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.65it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.97it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.90it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.82it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:48,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.49it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.68it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.36it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.07it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.30it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.61it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.90it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.54it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:28,  8.66it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.69it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.24it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:39,  6.25it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:32,  7.47it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:33,  7.22it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.62it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.61it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.58it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.46it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.77it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.54it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.03it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.35it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.77it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.88it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.54it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.92it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.35it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.00it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.64it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.23it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.24it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.31it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.15it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.21it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.81it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.76it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.46it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.78it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.56it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.50it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.88it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.48it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.21it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.53it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.78it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.65it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.68it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.20it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.23it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.62it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.85it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.54it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.95it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.78it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.78it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.62it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:12,  8.63it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.47it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.64it/s]
+
+ 90%|█████████ | 901/1000 [02:01<00:12,  7.64it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.37it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  6.96it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.28it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:12,  7.49it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.75it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.35it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.05it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.35it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.58it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.69it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.57it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.93it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.18it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.66it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.49it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.32it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.85it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.06it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.84it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.42it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.43it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.03it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.80it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.45it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.07it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.45it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.37it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.04it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.93it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.48it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.48it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.01it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.73it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.64it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7667', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.387', 'eval_steps_per_second': '7.387', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:24:55<00:00,  5.58s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.64it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_006_lr_6.257e-05/checkpoint-1813
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+
+Training completed. Do not forget to share your model on huggingface.co/models =)
+
+
+
+                                                     
+{'train_runtime': '1.23e+04', 'train_samples_per_second': '2.358', 'train_steps_per_second': '0.147', 'train_loss': '0.9795', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:24:56<00:00,  5.58s/it]
+100%|██████████| 1813/1813 [3:24:56<00:00,  6.78s/it]
+Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_006_lr_6.257e-05
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+***** train metrics *****
+  epoch                    =         1.0
+  num_input_tokens_seen    =    12201816
+  total_flos               = 487685654GF
+  train_loss               =      0.9795
+  train_runtime            =  3:24:56.88
+  train_samples_per_second =       2.358
+  train_steps_per_second   =       0.147
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_006_lr_6.257e-05/training_loss.png
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_006_lr_6.257e-05/training_eval_loss.png
+[WARNING|2026-10-15 22:05:20] llamafactory.extras.ploting:149 >> No metric eval_accuracy to plot.
+
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+  0%|          | 3/1000 [00:00<01:02, 15.95it/s]
+  0%|          | 5/1000 [00:00<02:16,  7.27it/s]
+  1%|          | 7/1000 [00:00<01:58,  8.36it/s]
+  1%|          | 9/1000 [00:01<02:32,  6.48it/s]
+  1%|          | 11/1000 [00:01<02:01,  8.11it/s]
+  1%|▏         | 13/1000 [00:01<02:20,  7.03it/s]
+  1%|▏         | 14/1000 [00:01<02:23,  6.85it/s]
+  2%|▏         | 15/1000 [00:02<02:44,  5.98it/s]
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+  2%|▏         | 18/1000 [00:02<02:42,  6.06it/s]
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+  2%|▏         | 20/1000 [00:03<03:32,  4.61it/s]
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+  2%|▎         | 25/1000 [00:03<02:12,  7.37it/s]
+  3%|▎         | 26/1000 [00:03<02:35,  6.27it/s]
+  3%|▎         | 28/1000 [00:04<02:41,  6.00it/s]
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+  3%|▎         | 32/1000 [00:05<02:41,  6.00it/s]
+  3%|▎         | 33/1000 [00:05<02:47,  5.77it/s]
+  4%|▎         | 35/1000 [00:05<02:05,  7.69it/s]
+  4%|▎         | 37/1000 [00:05<01:45,  9.11it/s]
+  4%|▍         | 39/1000 [00:05<02:06,  7.59it/s]
+  4%|▍         | 40/1000 [00:05<02:03,  7.76it/s]
+  4%|▍         | 42/1000 [00:06<01:45,  9.12it/s]
+  4%|▍         | 44/1000 [00:06<01:32, 10.34it/s]
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+  5%|▍         | 47/1000 [00:06<02:09,  7.38it/s]
+  5%|▍         | 49/1000 [00:06<01:48,  8.76it/s]
+  5%|▌         | 51/1000 [00:07<01:35,  9.92it/s]
+  5%|▌         | 53/1000 [00:07<01:26, 10.94it/s]
+  6%|▌         | 55/1000 [00:07<01:27, 10.75it/s]
+  6%|▌         | 57/1000 [00:07<01:41,  9.28it/s]
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+  6%|▋         | 63/1000 [00:08<01:36,  9.76it/s]
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+  7%|▋         | 67/1000 [00:08<02:02,  7.59it/s]
+  7%|▋         | 69/1000 [00:09<01:52,  8.28it/s]
+  7%|▋         | 70/1000 [00:09<02:12,  7.01it/s]
+  7%|▋         | 72/1000 [00:09<02:17,  6.73it/s]
+  7%|▋         | 74/1000 [00:09<01:53,  8.14it/s]
+  8%|▊         | 75/1000 [00:10<02:14,  6.88it/s]
+  8%|▊         | 76/1000 [00:10<02:18,  6.69it/s]
+  8%|▊         | 78/1000 [00:10<02:25,  6.34it/s]
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+  8%|▊         | 82/1000 [00:10<01:39,  9.22it/s]
+  8%|▊         | 84/1000 [00:11<01:53,  8.04it/s]
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+  9%|▊         | 86/1000 [00:11<02:33,  5.96it/s]
+  9%|▊         | 87/1000 [00:11<02:49,  5.40it/s]
+  9%|▉         | 89/1000 [00:12<02:09,  7.02it/s]
+  9%|▉         | 90/1000 [00:12<02:05,  7.26it/s]
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+  9%|▉         | 92/1000 [00:12<02:26,  6.18it/s]
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+ 10%|▉         | 95/1000 [00:13<02:43,  5.53it/s]
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+ 10%|█         | 103/1000 [00:14<01:55,  7.76it/s]
+ 10%|█         | 104/1000 [00:14<01:54,  7.83it/s]
+ 11%|█         | 106/1000 [00:14<01:35,  9.33it/s]
+ 11%|█         | 108/1000 [00:14<01:25, 10.47it/s]
+ 11%|█         | 110/1000 [00:14<01:19, 11.22it/s]
+ 11%|█         | 112/1000 [00:14<01:13, 12.05it/s]
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.55it/s]
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.83it/s]
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.22it/s]
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.09it/s]
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.44it/s]
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.16it/s]
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.94it/s]
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.03it/s]
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.90it/s]
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.07it/s]
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.92it/s]
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.04it/s]
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.41it/s]
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.98it/s]
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.62it/s]
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.82it/s]
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.42it/s]
+ 16%|█▋        | 165/1000 [00:21<02:26,  5.69it/s]
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.19it/s]
+ 17%|█▋        | 167/1000 [00:21<02:23,  5.79it/s]
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.85it/s]
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.16it/s]
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.24it/s]
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.75it/s]
+ 18%|█▊        | 176/1000 [00:22<01:37,  8.46it/s]
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.51it/s]
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+ 18%|█▊        | 182/1000 [00:23<02:22,  5.72it/s]
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.40it/s]
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.13it/s]
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.39it/s]
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.77it/s]
+ 20%|█▉        | 198/1000 [00:25<01:27,  9.20it/s]
+ 20%|██        | 200/1000 [00:26<01:14, 10.68it/s]
+ 20%|██        | 202/1000 [00:26<01:30,  8.79it/s]
+ 20%|██        | 204/1000 [00:26<01:31,  8.66it/s]
+ 21%|██        | 206/1000 [00:26<01:19,  9.96it/s]
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+ 21%|██        | 210/1000 [00:27<01:32,  8.50it/s]
+ 21%|██        | 212/1000 [00:27<01:21,  9.70it/s]
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.75it/s]
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.16it/s]
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+ 23%|██▎       | 228/1000 [00:29<01:53,  6.83it/s]
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.03it/s]
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.22it/s]
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.97it/s]
+ 24%|██▎       | 237/1000 [00:30<01:20,  9.48it/s]
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.98it/s]
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.36it/s]
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.03it/s]
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.48it/s]
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.67it/s]
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.27it/s]
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.87it/s]
+ 25%|██▍       | 249/1000 [00:32<02:05,  5.97it/s]
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.88it/s]
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.04it/s]
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.70it/s]
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.92it/s]
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.04it/s]
+ 27%|██▋       | 271/1000 [00:36<02:13,  5.45it/s]
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.71it/s]
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.71it/s]
+ 28%|██▊       | 277/1000 [00:36<01:30,  7.96it/s]
+ 28%|██▊       | 278/1000 [00:37<01:47,  6.74it/s]
+ 28%|██▊       | 280/1000 [00:37<01:49,  6.59it/s]
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.00it/s]
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.84it/s]
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.42it/s]
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.72it/s]
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+ 30%|███       | 301/1000 [00:40<01:37,  7.14it/s]
+ 30%|███       | 303/1000 [00:40<01:19,  8.77it/s]
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+ 30%|███       | 305/1000 [00:40<01:25,  8.16it/s]
+ 31%|███       | 306/1000 [00:41<01:39,  6.96it/s]
+ 31%|███       | 307/1000 [00:41<01:41,  6.79it/s]
+ 31%|███       | 309/1000 [00:41<01:18,  8.82it/s]
+ 31%|███       | 311/1000 [00:41<01:07, 10.23it/s]
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.29it/s]
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.27it/s]
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.60it/s]
+ 32%|███▏      | 319/1000 [00:42<01:33,  7.25it/s]
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.45it/s]
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.76it/s]
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.71it/s]
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.42it/s]
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.18it/s]
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.59it/s]
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+ 34%|███▎      | 336/1000 [00:44<01:24,  7.89it/s]
+ 34%|███▍      | 338/1000 [00:45<01:16,  8.64it/s]
+ 34%|███▍      | 340/1000 [00:45<01:06,  9.89it/s]
+ 34%|███▍      | 342/1000 [00:45<01:01, 10.77it/s]
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.56it/s]
+ 34%|███▍      | 345/1000 [00:46<01:25,  7.68it/s]
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.21it/s]
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+ 35%|███▌      | 352/1000 [00:47<01:25,  7.62it/s]
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.54it/s]
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.60it/s]
+ 36%|███▌      | 361/1000 [00:47<01:07,  9.44it/s]
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.20it/s]
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.93it/s]
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.38it/s]
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.74it/s]
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.10it/s]
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.23it/s]
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.93it/s]
+ 38%|███▊      | 380/1000 [00:51<01:12,  8.53it/s]
+ 38%|███▊      | 381/1000 [00:51<01:11,  8.66it/s]
+ 38%|███▊      | 382/1000 [00:51<01:22,  7.53it/s]
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.18it/s]
+ 38%|███▊      | 385/1000 [00:51<01:08,  8.98it/s]
+ 39%|███▊      | 387/1000 [00:51<01:24,  7.30it/s]
+ 39%|███▉      | 388/1000 [00:52<01:26,  7.06it/s]
+ 39%|███▉      | 390/1000 [00:52<01:18,  7.73it/s]
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.39it/s]
+ 39%|███▉      | 393/1000 [00:52<01:07,  9.00it/s]
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.50it/s]
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.85it/s]
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.21it/s]
+ 40%|████      | 400/1000 [00:53<01:46,  5.66it/s]
+ 40%|████      | 402/1000 [00:54<01:41,  5.91it/s]
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+ 41%|████      | 406/1000 [00:54<01:09,  8.51it/s]
+ 41%|████      | 408/1000 [00:54<01:32,  6.40it/s]
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+ 41%|████      | 411/1000 [00:55<01:41,  5.83it/s]
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.07it/s]
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+ 42%|████▏     | 418/1000 [00:56<01:12,  8.05it/s]
+ 42%|████▏     | 420/1000 [00:56<01:09,  8.31it/s]
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.43it/s]
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.64it/s]
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.90it/s]
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.34it/s]
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.93it/s]
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.53it/s]
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.85it/s]
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.56it/s]
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.00it/s]
+ 44%|████▎     | 435/1000 [00:58<00:48, 11.61it/s]
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.44it/s]
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.07it/s]
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.19it/s]
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.50it/s]
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.82it/s]
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.28it/s]
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.85it/s]
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.64it/s]
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.76it/s]
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.02it/s]
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.83it/s]
+ 47%|████▋     | 469/1000 [01:03<01:23,  6.32it/s]
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.58it/s]
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.52it/s]
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.40it/s]
+ 48%|████▊     | 477/1000 [01:04<01:34,  5.51it/s]
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.07it/s]
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.33it/s]
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.33it/s]
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+ 49%|████▉     | 488/1000 [01:05<01:04,  8.00it/s]
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.64it/s]
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.21it/s]
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.98it/s]
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.45it/s]
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.68it/s]
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+ 50%|█████     | 502/1000 [01:07<01:05,  7.56it/s]
+ 50%|█████     | 503/1000 [01:07<01:12,  6.81it/s]
+ 50%|█████     | 505/1000 [01:08<01:15,  6.58it/s]
+ 51%|█████     | 506/1000 [01:08<01:24,  5.82it/s]
+ 51%|█████     | 508/1000 [01:08<01:06,  7.44it/s]
+ 51%|█████     | 510/1000 [01:08<00:54,  8.97it/s]
+ 51%|█████     | 512/1000 [01:08<00:48,  9.97it/s]
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.05it/s]
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.69it/s]
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.57it/s]
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.60it/s]
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.55it/s]
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.72it/s]
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.71it/s]
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.38it/s]
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.77it/s]
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.07it/s]
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.51it/s]
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.71it/s]
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.93it/s]
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.96it/s]
+ 56%|█████▌    | 556/1000 [01:15<00:56,  7.79it/s]
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.04it/s]
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.86it/s]
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.99it/s]
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.25it/s]
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.13it/s]
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.52it/s]
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.40it/s]
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+ 60%|██████    | 601/1000 [01:22<01:02,  6.40it/s]
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+ 61%|██████    | 608/1000 [01:22<00:49,  7.86it/s]
+ 61%|██████    | 610/1000 [01:23<00:45,  8.48it/s]
+ 61%|██████    | 612/1000 [01:23<00:39,  9.75it/s]
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.82it/s]
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.62it/s]
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.75it/s]
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.10it/s]
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.05it/s]
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.22it/s]
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.46it/s]
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.20it/s]
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.23it/s]
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.87it/s]
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.45it/s]
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.73it/s]
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.08it/s]
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.97it/s]
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.20it/s]
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.10it/s]
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.60it/s]
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.32it/s]
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.71it/s]
+ 66%|██████▌   | 656/1000 [01:28<00:38,  8.92it/s]
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.58it/s]
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.49it/s]
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.69it/s]
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.64it/s]
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.67it/s]
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.91it/s]
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.90it/s]
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.87it/s]
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+ 69%|██████▉   | 690/1000 [01:33<00:45,  6.81it/s]
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.85it/s]
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.27it/s]
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+ 70%|███████   | 704/1000 [01:35<00:40,  7.23it/s]
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+ 71%|███████   | 709/1000 [01:36<00:39,  7.42it/s]
+ 71%|███████   | 710/1000 [01:36<00:45,  6.39it/s]
+ 71%|███████   | 712/1000 [01:37<00:37,  7.59it/s]
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.76it/s]
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.28it/s]
+ 72%|███████▏  | 717/1000 [01:37<00:35,  7.99it/s]
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.77it/s]
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.92it/s]
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.13it/s]
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.20it/s]
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.22it/s]
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.30it/s]
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.13it/s]
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.61it/s]
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.54it/s]
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.78it/s]
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.62it/s]
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.46it/s]
+ 75%|███████▍  | 749/1000 [01:41<00:28,  8.66it/s]
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.70it/s]
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.18it/s]
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.25it/s]
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.24it/s]
+ 76%|███████▌  | 756/1000 [01:42<00:37,  6.43it/s]
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.63it/s]
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.33it/s]
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.71it/s]
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.66it/s]
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.47it/s]
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.39it/s]
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.79it/s]
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.96it/s]
+ 78%|███████▊  | 782/1000 [01:45<00:26,  8.38it/s]
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.17it/s]
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.55it/s]
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.38it/s]
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.01it/s]
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.98it/s]
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.20it/s]
+ 80%|███████▉  | 797/1000 [01:47<00:29,  6.87it/s]
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+ 80%|████████  | 804/1000 [01:48<00:24,  7.84it/s]
+ 81%|████████  | 806/1000 [01:49<00:21,  9.18it/s]
+ 81%|████████  | 807/1000 [01:49<00:21,  8.98it/s]
+ 81%|████████  | 809/1000 [01:49<00:18, 10.21it/s]
+ 81%|████████  | 811/1000 [01:49<00:16, 11.23it/s]
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.06it/s]
+ 82%|████████▏ | 815/1000 [01:49<00:19,  9.31it/s]
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.21it/s]
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.06it/s]
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.83it/s]
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.81it/s]
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.25it/s]
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.76it/s]
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.25it/s]
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.12it/s]
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.44it/s]
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.62it/s]
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.15it/s]
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.26it/s]
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.55it/s]
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.50it/s]
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.07it/s]
+ 86%|████████▌ | 857/1000 [01:55<00:13, 10.22it/s]
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.53it/s]
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.78it/s]
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.64it/s]
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.72it/s]
+ 87%|████████▋ | 867/1000 [01:56<00:14,  9.33it/s]
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.23it/s]
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.24it/s]
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+ 88%|████████▊ | 875/1000 [01:57<00:16,  7.63it/s]
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.61it/s]
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.95it/s]
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.28it/s]
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.73it/s]
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.73it/s]
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.83it/s]
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.54it/s]
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.71it/s]
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.87it/s]
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.08it/s]
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.06it/s]
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.33it/s]
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.59it/s]
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.79it/s]
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.40it/s]
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.10it/s]
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.38it/s]
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.61it/s]
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.93it/s]
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.64it/s]
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.09it/s]
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.66it/s]
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.84it/s]
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.63it/s]
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.19it/s]
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.33it/s]
+ 94%|█████████▍| 938/1000 [02:06<00:06,  8.86it/s]
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.06it/s]
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.16it/s]
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.71it/s]
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.82it/s]
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.39it/s]
+ 96%|█████████▌| 957/1000 [02:08<00:06,  6.40it/s]
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.04it/s]
+ 96%|█████████▋| 964/1000 [02:09<00:04,  7.48it/s]
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.44it/s]
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.03it/s]
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.43it/s]
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.81it/s]
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.86it/s]
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.44it/s]
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.07it/s]
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.45it/s]
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.95it/s]
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+100%|█████████▉| 995/1000 [02:14<00:00,  5.54it/s]
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+100%|█████████▉| 999/1000 [02:14<00:00,  6.77it/s]
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+100%|██████████| 1000/1000 [02:15<00:00,  7.40it/s]
+***** eval metrics *****
+  epoch                   =        1.0
+  eval_loss               =     0.7667
+  eval_runtime            = 0:02:15.26
+  eval_samples_per_second =      7.393
+  eval_steps_per_second   =      7.393
+  num_input_tokens_seen   =   12201816
+Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
diff --git a/optuna_30k_experiments/logs/trial_007_lr_2.028e-05.log b/optuna_30k_experiments/logs/trial_007_lr_2.028e-05.log
new file mode 100644
index 0000000..40c9fe2
--- /dev/null
+++ b/optuna_30k_experiments/logs/trial_007_lr_2.028e-05.log
@@ -0,0 +1,12411 @@
+[INFO|2026-10-15 22:07:42] llamafactory.hparams.parser:651 >> Process rank: 0, world size: 1, device: cuda:0, distributed training: False, compute dtype: torch.bfloat16
+Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
+[INFO|configuration_utils.py:765] 2026-10-15 22:07:42,727 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-15 22:07:42,736 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|configuration_utils.py:765] 2026-10-15 22:07:44,335 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-15 22:07:44,339 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|configuration_utils.py:765] 2026-10-15 22:07:44,482 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-15 22:07:44,485 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-15 22:07:45] llamafactory.data.loader:144 >> Loading dataset aya_slovak_hpo_30k.jsonl...
+training example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28828, 2176, 28711, 28863, 29114, 10268, 28709, 6612, 28714, 2892, 493, 363, 6294, 10719, 363, 3811, 28723, 7379, 626, 28889, 307, 14637, 311, 28768, 2218, 22885, 3748, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 284, 8341, 6729, 345, 28738, 28739, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28723, 524, 4634, 28711, 570, 1879, 261, 28863, 532, 613, 14483, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 345, 15962, 28739, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 1879, 15173, 537, 284, 8341, 705, 28723, 367, 280, 28879, 570, 4415, 28716, 28797, 9335, 3175, 29826, 626, 323, 412, 8514, 16986, 29826, 28708, 345, 15962, 28739, 1879, 15173, 537, 1221, 4566, 28718, 325, 28707, 28723, 461, 28723, 724, 494, 8891, 24160, 28725, 8582, 28862, 24160, 28725, 20532, 28764, 271, 28725, 438, 29365, 15745, 1319, 12035, 28862, 8269, 284, 28827, 2762, 28709, 264, 4798, 8359, 1450, 8325, 12121, 1798, 420, 762, 1089, 290, 28906, 18765, 724, 2331, 29114, 1221, 4304, 1780, 363, 13881, 29826, 316, 2085, 28708, 28723, 22627, 28873, 263, 335, 20208, 284, 28827, 2762, 28709, 264, 4798, 5424, 10056, 1286, 290, 28906, 18765, 724, 2331, 29114, 11323, 10719, 12490, 363, 13881, 29826, 316, 284, 8341, 2390, 28723, 22127, 280, 28879, 21022, 1150, 424, 724, 28827, 1144, 28873, 340, 17066, 270, 21022, 485, 284, 8341, 705, 28725, 446, 6909, 28797, 637, 28705, 29114, 28708, 28879, 4798, 6612, 28827, 1100, 28768, 28863, 28723, 320, 412, 28879, 637, 15612, 28716, 28889, 28726, 1150, 424, 19523, 412, 28873, 276, 28797, 2390, 284, 8341, 2390, 363, 23407, 11277, 9538, 2085, 28708, 28723, 9532, 2144, 686, 7865, 28718, 341, 14475, 1687, 28706, 18037, 452, 270, 8269, 284, 8341, 705, 28725, 290, 4827, 486, 2710, 2144, 332, 1800, 16344, 29114, 6588, 269, 28862, 28711, 28863, 686, 1653, 5388, 28718, 710, 261, 1296, 28709, 284, 8341, 705, 28723, 351, 28711, 1371, 28797, 1776, 29826, 8269, 284, 8341, 705, 268, 28863, 1776, 29826, 8269, 1879, 3497, 598, 8269, 8631, 28879, 279, 412, 28725, 12121, 7988, 268, 28863, 24270, 12115, 28797, 1879, 5626, 263, 28862, 8269, 8631, 28879, 279, 412, 28723, 318, 15612, 1473, 276, 15690, 2245, 626, 15690, 307, 14637, 311, 28768, 300, 15469, 28711, 570, 446, 450, 4798, 29826, 333, 28729, 1879, 319, 3976, 294, 405, 28768, 4882, 405, 28725, 534, 28724, 2710, 724, 313, 4827, 2245, 626, 28889, 20532, 28764, 271, 28723, 10650, 300, 12235, 28827, 28873, 424, 268, 27172, 2245, 28723, 1054, 15504, 28879, 424, 724, 313, 8768, 4545, 504, 270, 8269, 2124, 2245, 626, 12689, 4584, 10906, 28718, 710, 18782, 28879, 28715, 28797, 1135, 10356, 28723, 2645, 28764, 12275, 28820, 28873, 29114, 13067, 549, 2892, 5872, 3811, 8631, 25991, 3026, 29114, 408, 28906, 28764, 485, 2082, 1403, 12121, 1798, 1773, 29826, 28729, 22160, 710, 408, 28906, 28764, 485, 1135, 10874, 1776, 4536, 28873, 300, 23407, 11277, 9538, 2085, 28708, 28723, 22127, 3361, 13470, 1150, 317, 28873, 424, 2082, 7330, 511, 2085, 28708, 28723, 401, 492, 13591, 28820, 261, 2220, 28862, 10766, 381, 28827, 486, 29114, 363, 28879, 4689, 3418, 28879, 21775, 29114, 280, 28723, 418, 28820, 11277, 28716, 6294, 26177, 903, 12689, 6612, 20122, 28733, 12868, 291, 2124, 2085, 28708, 1879, 6670, 28862, 9348, 493, 686, 375, 5620, 386, 28862, 28827, 28725, 23374, 4536, 25306, 16388, 957, 450, 15612, 28764, 2126, 29114, 511, 2152, 363, 25769, 14110, 17413, 26177, 903, 28873, 15808, 19396, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 7379, 1879, 277, 7939, 1879, 724, 313, 8768, 2245, 28718, 363, 446, 551, 9538, 12121, 1798, 24473, 21694, 300, 261, 5869, 28723, 384, 28728, 491, 28712, 4566, 15469, 28711, 570, 1879, 2245, 28725, 534, 28724, 2710, 7463, 15612, 13335, 28728, 4371, 1144, 28723, 10650, 300, 15469, 28711, 570, 1879, 1160, 28709, 28879, 3907, 13288, 491, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 28725, 534, 28724, 637, 686, 598, 5237, 5945, 17984, 28718, 2117, 18904, 3146, 515, 13288, 25208, 28723, 367, 280, 28879, 570, 12472, 1006, 5872, 28863, 358, 446, 311, 4845, 1879, 6149, 783, 494, 28718, 2245, 28718, 363, 17984, 28718, 405, 13288, 491, 28723, 13343, 29826, 28729, 385, 29114, 284, 8341, 705, 28747, 399, 3711, 3175, 29826, 626, 12990, 17984, 28718, 2117, 268, 6125, 15690, 345, 28738, 28739, 16986, 29826, 28708, 1773, 29826, 10641, 2124, 345, 28738, 28739, 582, 20524, 12214, 1773, 29826, 28729, 385, 29114, 284, 8341, 705, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 15173, 537, 28718, 325, 849, 3748, 28725, 446, 6909, 28820, 3061, 28720, 300, 28827, 1520, 6612, 749, 1780, 20532, 28764, 271, 586, 28873, 28710, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28731, 1879, 686, 6729, 1773, 29826, 28729, 22160, 284, 8341, 705, 28723, 550, 11245, 412, 28747, 550, 11245, 412, 1683, 28827, 16032, 271, 1342, 4295, 3191, 316, 28729, 6449, 2245, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 345, 28741, 28739, 1879, 22954, 338, 28718, 297, 12689, 345, 28741, 28739, 686, 2129, 28710, 29114, 16986, 28863, 358, 28723, 17563, 971, 28747, 17563, 971, 582, 20524, 12214, 1342, 28764, 1193, 1342, 4295, 5765, 6175, 2959, 15416, 28710, 13288, 491, 3589, 28723, 9532, 484, 28717, 3811, 18904, 17101, 29114, 22399, 971, 28725, 15612, 13335, 28728, 4371, 28711, 570, 13288, 20996, 28725, 446, 6909, 28797, 484, 28717, 3811, 18904, 17101, 29114, 28725, 264, 2513, 300, 8631, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 268, 345, 28741, 28739, 264, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Zhrnúť tento článok v jednej vete. Textový nástroj je ikona, ktorá sa podobá písmenu "T" v paneli nástrojov vľavo. Kliknite na túto ikonu. Použite rozbaľovaciu ponuku vedľa "Character" v hornej časti obrazovky na výber písma. Použite druhé rozbaľovacie menu vedľa "Character" na výber formátu (t. j. pravidelný, tučný, kurzor, atď.). Tradičné písmo ako Times New Roman alebo Garamond môže pridať formálny vzhľad loga. Bezšerifové písmo ako Helvetica môže pridať modernejší vzhľad písmu. Nepoužívajte príliš dekoratívne písma, ktoré sa ťažko čítajú. Tiež sa vyhýbajte zmiešanému písmu v návrhu loga. Ak si z internetu stiahnete bezplatné písma, mali by ste si uvedomiť licenčnú zmluvu pre tieto písma. Mnohé voľné písma sú voľné na osobné použitie, ale nie sú povolené na komerčné použitie. S vybraným textovým nástrojom kliknite kdekoľvek na grafickej doske, aby ste pridali textový kurzor. Potom napíšte svoj text. Zvážte pridanie samostatného textového objektu pre každé slovo. Obzvlášť ak plánujete používať rôzne farby alebo veľkosti pre rôzne slová vo vašom návrhu loga. Nepridávaj ešte farbu do loga. Farebná tlač nemusí byť vždy možnosťou. Návrh jednoduchého čierno-bieleho loga na začiatok zabezpečí, že vaše logo bude vyzerať dobre v jeho najjednoduchšej forme. Môžete tiež použiť nástroj Text na ceste na pridanie textu v kruhu alebo zakrivenom tvare. Dvakrát kliknite na text, aby ste ho vyzdvihli. Potom kliknite na položku znak v hornej časti obrazovky, aby sa zobrazila ponuka nastavenia znakov. Použite nasledujúce kroky na úpravu textu v ponuke znak. Veľkosť písma: Rozbaľovacia ponuka s malým "T" vedľa veľkého "T" upravuje veľkosť písma. Môžete tiež použiť nástroj výberu (ikona, ktorá pripomína čierny kurzor myši v paneli nástrojov vľavo) na zmenu veľkosti písma. Vedenie: Vedenie mení priestor medzi riadkami textu. Použite rozbaľovaciu ponuku vedľa ikony, ktorá sa podobá "A" na vrchu iného "A" zmeniť vedúce. Kerning: Kerning upravuje medzery medzi konkrétnymi znakmi. Ak chcete nastaviť kerning, vyzdvihnite znaky, ktoré chcete nastaviť, a potom použite rozbaľovaciu ponuku vedľa ikony s "A" aVyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+labels:
+Vyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+eval example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28965, 28709, 637, 8631, 25991, 3026, 28725, 13067, 10531, 28729, 350, 2591, 686, 3023, 28862, 28827, 284, 28873, 269, 13096, 28711, 28863, 6149, 3399, 28804, 550, 13881, 29826, 316, 300, 1879, 7871, 338, 28820, 6936, 1150, 28863, 7340, 6957, 8905, 3907, 12235, 28827, 28873, 424, 5765, 772, 28725, 446, 6909, 28889, 14848, 912, 12214, 4321, 28720, 751, 29365, 28723, 351, 28906, 18765, 298, 486, 29114, 28705, 28740, 387, 28705, 28750, 28734, 22116, 28827, 28723, 8294, 772, 28747, 13, 13, 27332, 12107, 28747, 13, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Čo sa používa, ak škodca zničí pšeničnú úrodu? Vzhľadom na predchádzajúcu otázku napíšte kontext, ktorý obsahuje odpoveď. Môže to byť 1 - 20 vetí. Kontext:
+
+### Response:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+labels:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+[INFO|configuration_utils.py:765] 2026-10-15 22:07:45,941 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-15 22:07:45,944 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-15 22:07:45] llamafactory.model.model_utils.kv_cache:144 >> KV cache is disabled during training.
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/unsloth/__init__.py:165: UserWarning: WARNING: Unsloth should be imported before [trl, transformers, peft] to ensure all optimizations are applied. Your code may run slower or encounter memory issues without these optimizations.
+
+Please restructure your imports with 'import unsloth' at the top of your file.
+  from ._gpu_init import *
+🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
+🦥 Unsloth Zoo will now patch everything to make training faster!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: WARNING `trust_remote_code` is True.
+Are you certain you want to do remote code execution?
+==((====))==  Unsloth 2026.6.1: Fast Mistral patching. Transformers: 5.5.0.
+   \\   /|    NVIDIA RTX A4500. Num GPUs = 1. Max memory: 19.573 GB. Platform: Linux.
+O^O/ \_/ \    Torch: 2.6.0+cu124. CUDA: 8.6. CUDA Toolkit: 12.4. Triton: 3.2.0
+\        /    Bfloat16 = TRUE. FA [Xformers = 0.0.29.post3. FA2 = False]
+ "-____-"     Free license: http://github.com/unslothai/unsloth
+Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading weights file model.safetensors from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/model.safetensors.index.json
+Will use dtype=torch.bfloat16 as defined in model's config object
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 2,
+  "use_cache": true
+}
+
+
+Loading weights:   0%|          | 0/291 [00:00<?, ?it/s]
+Loading weights:   4%|▍         | 13/291 [00:00<00:02, 128.82it/s]
+Loading weights:   9%|▉         | 26/291 [00:00<00:02, 103.40it/s]
+Loading weights:  14%|█▎        | 40/291 [00:00<00:02, 108.86it/s]
+Loading weights:  18%|█▊        | 52/291 [00:00<00:02, 98.58it/s] 
+Loading weights:  22%|██▏       | 64/291 [00:00<00:02, 104.81it/s]
+Loading weights:  26%|██▌       | 76/291 [00:00<00:02, 104.42it/s]
+Loading weights:  30%|██▉       | 87/291 [00:00<00:02, 92.23it/s] 
+Loading weights:  34%|███▍      | 100/291 [00:00<00:01, 101.19it/s]
+Loading weights:  38%|███▊      | 111/291 [00:01<00:01, 103.40it/s]
+Loading weights:  42%|████▏     | 122/291 [00:01<00:01, 95.40it/s] 
+Loading weights:  48%|████▊     | 139/291 [00:01<00:01, 113.17it/s]
+Loading weights:  53%|█████▎    | 153/291 [00:01<00:01, 119.25it/s]
+Loading weights:  57%|█████▋    | 167/291 [00:01<00:00, 124.06it/s]
+Loading weights:  62%|██████▏   | 180/291 [00:01<00:00, 115.05it/s]
+Loading weights:  66%|██████▋   | 193/291 [00:01<00:00, 114.60it/s]
+Loading weights:  70%|███████   | 205/291 [00:01<00:00, 108.58it/s]
+Loading weights:  75%|███████▍  | 217/291 [00:02<00:00, 109.16it/s]
+Loading weights:  79%|███████▊  | 229/291 [00:02<00:00, 111.00it/s]
+Loading weights:  83%|████████▎ | 241/291 [00:02<00:00, 107.93it/s]
+Loading weights:  87%|████████▋ | 252/291 [00:02<00:00, 105.85it/s]
+Loading weights:  90%|█████████ | 263/291 [00:02<00:00, 104.56it/s]
+Loading weights:  95%|█████████▍| 275/291 [00:02<00:00, 102.32it/s]
+Loading weights:  98%|█████████▊| 286/291 [00:02<00:00, 97.12it/s] 
+Loading weights: 100%|██████████| 291/291 [00:02<00:00, 106.98it/s]
+loading configuration file generation_config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/generation_config.json
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2
+}
+
+Could not locate the custom_generate/generate.py inside slovak-nlp/mistral-sk-7b.
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: slovak-nlp/mistral-sk-7b had a bad pad_token (</s>). Using pad_token = <unk>.
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:68: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_fwd
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:85: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_bwd
+[INFO|2026-10-15 22:08:03] llamafactory.model.model_utils.checkpointing:144 >> Gradient checkpointing enabled.
+[INFO|2026-10-15 22:08:03] llamafactory.model.adapter:144 >> Upcasting trainable params to float32.
+[INFO|2026-10-15 22:08:03] llamafactory.model.adapter:144 >> Fine-tuning method: LoRA
+Unsloth 2026.6.1 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers.
+[INFO|2026-10-15 22:08:05] llamafactory.model.loader:144 >> trainable params: 41,943,040 || all params: 7,283,675,136 || trainable%: 0.5758
+The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'pad_token_id': 2}.
+==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 1
+   \\   /|    Num examples = 29,000 | Num Epochs = 1 | Total steps = 1,813
+O^O/ \_/ \    Batch size per device = 1 | Gradient accumulation steps = 16
+\        /    Data Parallel GPUs = 1 | Total batch size (1 x 16 x 1) = 16
+ "-____-"     Trainable parameters = 41,943,040 of 7,283,675,136 (0.58% trained)
+
+  0%|          | 0/1813 [00:00<?, ?it/s]`use_return_dict` is deprecated! Use `return_dict` instead!
+Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
+Unsloth: Will smartly offload gradients to save VRAM!
+
+  0%|          | 1/1813 [00:06<3:25:05,  6.79s/it]
+  0%|          | 2/1813 [00:13<3:22:38,  6.71s/it]
+  0%|          | 3/1813 [00:19<3:08:12,  6.24s/it]
+  0%|          | 4/1813 [00:23<2:47:04,  5.54s/it]
+  0%|          | 5/1813 [00:30<3:00:37,  5.99s/it]
+  0%|          | 6/1813 [00:38<3:17:32,  6.56s/it]
+  0%|          | 7/1813 [00:44<3:17:53,  6.57s/it]
+  0%|          | 8/1813 [00:49<3:03:57,  6.12s/it]
+  0%|          | 9/1813 [00:57<3:17:37,  6.57s/it]
+  1%|          | 10/1813 [01:03<3:10:40,  6.35s/it]
+                                                   
+{'loss': '1.804', 'grad_norm': '1.934', 'learning_rate': '1.217e-06', 'epoch': '0.005517', 'num_input_tokens_seen': 65928, 'train_runtime': '63.21', 'train_tokens_per_second': '1043'}
+
+  1%|          | 10/1813 [01:03<3:10:40,  6.35s/it]
+  1%|          | 11/1813 [01:09<3:12:23,  6.41s/it]
+  1%|          | 12/1813 [01:17<3:26:20,  6.87s/it]
+  1%|          | 13/1813 [01:22<3:05:27,  6.18s/it]
+  1%|          | 14/1813 [01:28<3:06:41,  6.23s/it]
+  1%|          | 15/1813 [01:33<2:58:52,  5.97s/it]
+  1%|          | 16/1813 [01:40<3:00:53,  6.04s/it]
+  1%|          | 17/1813 [01:46<3:05:47,  6.21s/it]
+  1%|          | 18/1813 [01:52<3:04:39,  6.17s/it]
+  1%|          | 19/1813 [01:59<3:09:10,  6.33s/it]
+  1%|          | 20/1813 [02:05<3:06:10,  6.23s/it]
+                                                   
+{'loss': '1.683', 'grad_norm': '2.435', 'learning_rate': '2.569e-06', 'epoch': '0.01103', 'num_input_tokens_seen': 131352, 'train_runtime': '125.6', 'train_tokens_per_second': '1046'}
+
+  1%|          | 20/1813 [02:05<3:06:10,  6.23s/it]
+  1%|          | 21/1813 [02:11<3:02:50,  6.12s/it]
+  1%|          | 22/1813 [02:16<2:51:59,  5.76s/it]
+  1%|▏         | 23/1813 [02:21<2:46:32,  5.58s/it]
+  1%|▏         | 24/1813 [02:27<2:47:20,  5.61s/it]
+  1%|▏         | 25/1813 [02:34<3:00:05,  6.04s/it]
+  1%|▏         | 26/1813 [02:41<3:10:18,  6.39s/it]
+  1%|▏         | 27/1813 [02:47<3:06:44,  6.27s/it]
+  2%|▏         | 28/1813 [02:54<3:11:59,  6.45s/it]
+  2%|▏         | 29/1813 [03:02<3:25:52,  6.92s/it]
+  2%|▏         | 30/1813 [03:07<3:09:17,  6.37s/it]
+                                                   
+{'loss': '1.687', 'grad_norm': '3.301', 'learning_rate': '3.92e-06', 'epoch': '0.01655', 'num_input_tokens_seen': 197376, 'train_runtime': '187.4', 'train_tokens_per_second': '1053'}
+
+  2%|▏         | 30/1813 [03:07<3:09:17,  6.37s/it]
+  2%|▏         | 31/1813 [03:13<3:09:47,  6.39s/it]
+  2%|▏         | 32/1813 [03:19<3:01:03,  6.10s/it]
+  2%|▏         | 33/1813 [03:25<2:59:24,  6.05s/it]
+  2%|▏         | 34/1813 [03:32<3:12:22,  6.49s/it]
+  2%|▏         | 35/1813 [03:39<3:12:33,  6.50s/it]
+  2%|▏         | 36/1813 [03:45<3:11:32,  6.47s/it]
+  2%|▏         | 37/1813 [03:50<2:59:27,  6.06s/it]
+  2%|▏         | 38/1813 [03:55<2:50:14,  5.75s/it]
+  2%|▏         | 39/1813 [04:01<2:49:54,  5.75s/it]
+  2%|▏         | 40/1813 [04:07<2:49:54,  5.75s/it]
+                                                   
+{'loss': '1.515', 'grad_norm': '1.379', 'learning_rate': '5.272e-06', 'epoch': '0.02207', 'num_input_tokens_seen': 261168, 'train_runtime': '247.3', 'train_tokens_per_second': '1056'}
+
+  2%|▏         | 40/1813 [04:07<2:49:54,  5.75s/it]
+  2%|▏         | 41/1813 [04:12<2:47:25,  5.67s/it]
+  2%|▏         | 42/1813 [04:18<2:51:06,  5.80s/it]
+  2%|▏         | 43/1813 [04:26<3:07:51,  6.37s/it]
+  2%|▏         | 44/1813 [04:33<3:09:54,  6.44s/it]
+  2%|▏         | 45/1813 [04:40<3:19:45,  6.78s/it]
+  3%|▎         | 46/1813 [04:46<3:14:14,  6.60s/it]
+  3%|▎         | 47/1813 [04:52<3:07:00,  6.35s/it]
+  3%|▎         | 48/1813 [04:59<3:11:47,  6.52s/it]
+  3%|▎         | 49/1813 [05:06<3:10:36,  6.48s/it]
+  3%|▎         | 50/1813 [05:12<3:10:41,  6.49s/it]
+                                                   
+{'loss': '1.533', 'grad_norm': '1.467', 'learning_rate': '6.624e-06', 'epoch': '0.02759', 'num_input_tokens_seen': 330104, 'train_runtime': '312.5', 'train_tokens_per_second': '1056'}
+
+  3%|▎         | 50/1813 [05:12<3:10:41,  6.49s/it]
+  3%|▎         | 51/1813 [05:20<3:19:41,  6.80s/it]
+  3%|▎         | 52/1813 [05:29<3:41:31,  7.55s/it]
+  3%|▎         | 53/1813 [05:35<3:27:00,  7.06s/it]
+  3%|▎         | 54/1813 [05:42<3:30:00,  7.16s/it]
+  3%|▎         | 55/1813 [05:48<3:21:52,  6.89s/it]
+  3%|▎         | 56/1813 [05:55<3:19:35,  6.82s/it]
+  3%|▎         | 57/1813 [06:00<3:05:46,  6.35s/it]
+  3%|▎         | 58/1813 [06:09<3:23:40,  6.96s/it]
+  3%|▎         | 59/1813 [06:16<3:25:20,  7.02s/it]
+  3%|▎         | 60/1813 [06:21<3:10:11,  6.51s/it]
+                                                   
+{'loss': '1.41', 'grad_norm': '1.321', 'learning_rate': '7.976e-06', 'epoch': '0.0331', 'num_input_tokens_seen': 407792, 'train_runtime': '381.7', 'train_tokens_per_second': '1068'}
+
+  3%|▎         | 60/1813 [06:21<3:10:11,  6.51s/it]
+  3%|▎         | 61/1813 [06:26<2:57:34,  6.08s/it]
+  3%|▎         | 62/1813 [06:32<2:54:53,  5.99s/it]
+  3%|▎         | 63/1813 [06:38<2:52:19,  5.91s/it]
+  4%|▎         | 64/1813 [06:43<2:50:29,  5.85s/it]
+  4%|▎         | 65/1813 [06:51<3:01:25,  6.23s/it]
+  4%|▎         | 66/1813 [06:57<3:02:06,  6.25s/it]
+  4%|▎         | 67/1813 [07:02<2:55:13,  6.02s/it]
+  4%|▍         | 68/1813 [07:08<2:55:54,  6.05s/it]
+  4%|▍         | 69/1813 [07:16<3:06:39,  6.42s/it]
+  4%|▍         | 70/1813 [07:23<3:16:38,  6.77s/it]
+                                                   
+{'loss': '1.261', 'grad_norm': '1.103', 'learning_rate': '9.328e-06', 'epoch': '0.03862', 'num_input_tokens_seen': 474368, 'train_runtime': '443.9', 'train_tokens_per_second': '1069'}
+
+  4%|▍         | 70/1813 [07:23<3:16:38,  6.77s/it]
+  4%|▍         | 71/1813 [07:29<3:05:17,  6.38s/it]
+  4%|▍         | 72/1813 [07:37<3:24:33,  7.05s/it]
+  4%|▍         | 73/1813 [07:43<3:10:33,  6.57s/it]
+  4%|▍         | 74/1813 [07:49<3:07:00,  6.45s/it]
+  4%|▍         | 75/1813 [07:56<3:07:57,  6.49s/it]
+  4%|▍         | 76/1813 [08:02<3:05:23,  6.40s/it]
+  4%|▍         | 77/1813 [08:09<3:09:26,  6.55s/it]
+  4%|▍         | 78/1813 [08:14<3:00:43,  6.25s/it]
+  4%|▍         | 79/1813 [08:20<2:54:35,  6.04s/it]
+  4%|▍         | 80/1813 [08:26<2:54:50,  6.05s/it]
+                                                   
+{'loss': '1.275', 'grad_norm': '1.353', 'learning_rate': '1.068e-05', 'epoch': '0.04414', 'num_input_tokens_seen': 541616, 'train_runtime': '506.4', 'train_tokens_per_second': '1069'}
+
+  4%|▍         | 80/1813 [08:26<2:54:50,  6.05s/it]
+  4%|▍         | 81/1813 [08:32<2:54:42,  6.05s/it]
+  5%|▍         | 82/1813 [08:39<3:03:47,  6.37s/it]
+  5%|▍         | 83/1813 [08:46<3:09:58,  6.59s/it]
+  5%|▍         | 84/1813 [08:53<3:14:29,  6.75s/it]
+  5%|▍         | 85/1813 [09:00<3:09:46,  6.59s/it]
+  5%|▍         | 86/1813 [09:06<3:08:19,  6.54s/it]
+  5%|▍         | 87/1813 [09:11<2:56:43,  6.14s/it]
+  5%|▍         | 88/1813 [09:18<2:58:26,  6.21s/it]
+  5%|▍         | 89/1813 [09:24<2:57:39,  6.18s/it]
+  5%|▍         | 90/1813 [09:30<3:00:38,  6.29s/it]
+                                                   
+{'loss': '1.182', 'grad_norm': '1.011', 'learning_rate': '1.203e-05', 'epoch': '0.04966', 'num_input_tokens_seen': 608792, 'train_runtime': '570.7', 'train_tokens_per_second': '1067'}
+
+  5%|▍         | 90/1813 [09:30<3:00:38,  6.29s/it]
+  5%|▌         | 91/1813 [09:36<2:56:46,  6.16s/it]
+  5%|▌         | 92/1813 [09:41<2:48:43,  5.88s/it]
+  5%|▌         | 93/1813 [09:47<2:50:50,  5.96s/it]
+  5%|▌         | 94/1813 [09:54<2:54:11,  6.08s/it]
+  5%|▌         | 95/1813 [10:00<2:56:54,  6.18s/it]
+  5%|▌         | 96/1813 [10:06<2:56:17,  6.16s/it]
+  5%|▌         | 97/1813 [10:13<3:00:26,  6.31s/it]
+  5%|▌         | 98/1813 [10:18<2:49:49,  5.94s/it]
+  5%|▌         | 99/1813 [10:24<2:46:00,  5.81s/it]
+  6%|▌         | 100/1813 [10:30<2:50:34,  5.97s/it]
+                                                    
+{'loss': '1.245', 'grad_norm': '1.095', 'learning_rate': '1.338e-05', 'epoch': '0.05517', 'num_input_tokens_seen': 670936, 'train_runtime': '630.4', 'train_tokens_per_second': '1064'}
+
+  6%|▌         | 100/1813 [10:30<2:50:34,  5.97s/it]
+  6%|▌         | 101/1813 [10:37<2:57:57,  6.24s/it]
+  6%|▌         | 102/1813 [10:44<3:02:59,  6.42s/it]
+  6%|▌         | 103/1813 [10:50<3:00:53,  6.35s/it]
+  6%|▌         | 104/1813 [10:56<3:02:39,  6.41s/it]
+  6%|▌         | 105/1813 [11:02<2:55:50,  6.18s/it]
+  6%|▌         | 106/1813 [11:08<2:54:56,  6.15s/it]
+  6%|▌         | 107/1813 [11:14<2:56:51,  6.22s/it]
+  6%|▌         | 108/1813 [11:21<2:57:51,  6.26s/it]
+  6%|▌         | 109/1813 [11:27<2:58:22,  6.28s/it]
+  6%|▌         | 110/1813 [11:34<3:02:08,  6.42s/it]
+                                                    
+{'loss': '1.202', 'grad_norm': '1.143', 'learning_rate': '1.474e-05', 'epoch': '0.06069', 'num_input_tokens_seen': 738632, 'train_runtime': '694.4', 'train_tokens_per_second': '1064'}
+
+  6%|▌         | 110/1813 [11:34<3:02:08,  6.42s/it]
+  6%|▌         | 111/1813 [11:41<3:04:32,  6.51s/it]
+  6%|▌         | 112/1813 [11:47<3:06:03,  6.56s/it]
+  6%|▌         | 113/1813 [11:53<3:02:29,  6.44s/it]
+  6%|▋         | 114/1813 [12:01<3:10:51,  6.74s/it]
+  6%|▋         | 115/1813 [12:07<3:02:43,  6.46s/it]
+  6%|▋         | 116/1813 [12:14<3:05:53,  6.57s/it]
+  6%|▋         | 117/1813 [12:19<2:56:01,  6.23s/it]
+  7%|▋         | 118/1813 [12:25<2:52:23,  6.10s/it]
+  7%|▋         | 119/1813 [12:31<2:53:46,  6.15s/it]
+  7%|▋         | 120/1813 [12:37<2:54:26,  6.18s/it]
+                                                    
+{'loss': '1.216', 'grad_norm': '1.656', 'learning_rate': '1.609e-05', 'epoch': '0.06621', 'num_input_tokens_seen': 808824, 'train_runtime': '757.8', 'train_tokens_per_second': '1067'}
+
+  7%|▋         | 120/1813 [12:37<2:54:26,  6.18s/it]
+  7%|▋         | 121/1813 [12:41<2:36:53,  5.56s/it]
+  7%|▋         | 122/1813 [12:46<2:31:37,  5.38s/it]
+  7%|▋         | 123/1813 [12:53<2:40:07,  5.68s/it]
+  7%|▋         | 124/1813 [12:58<2:36:08,  5.55s/it]
+  7%|▋         | 125/1813 [13:05<2:45:13,  5.87s/it]
+  7%|▋         | 126/1813 [13:11<2:49:02,  6.01s/it]
+  7%|▋         | 127/1813 [13:16<2:39:03,  5.66s/it]
+  7%|▋         | 128/1813 [13:21<2:31:59,  5.41s/it]
+  7%|▋         | 129/1813 [13:29<2:53:53,  6.20s/it]
+  7%|▋         | 130/1813 [13:35<2:59:10,  6.39s/it]
+                                                    
+{'loss': '1.15', 'grad_norm': '0.9926', 'learning_rate': '1.744e-05', 'epoch': '0.07172', 'num_input_tokens_seen': 869720, 'train_runtime': '816', 'train_tokens_per_second': '1066'}
+
+  7%|▋         | 130/1813 [13:35<2:59:10,  6.39s/it]
+  7%|▋         | 131/1813 [13:42<2:56:13,  6.29s/it]
+  7%|▋         | 132/1813 [13:47<2:50:11,  6.07s/it]
+  7%|▋         | 133/1813 [13:53<2:46:56,  5.96s/it]
+  7%|▋         | 134/1813 [13:58<2:41:13,  5.76s/it]
+  7%|▋         | 135/1813 [14:03<2:34:49,  5.54s/it]
+  8%|▊         | 136/1813 [14:09<2:35:02,  5.55s/it]
+  8%|▊         | 137/1813 [14:14<2:37:09,  5.63s/it]
+  8%|▊         | 138/1813 [14:21<2:47:59,  6.02s/it]
+  8%|▊         | 139/1813 [14:28<2:51:30,  6.15s/it]
+  8%|▊         | 140/1813 [14:35<2:57:33,  6.37s/it]
+                                                    
+{'loss': '1.146', 'grad_norm': '1.734', 'learning_rate': '1.879e-05', 'epoch': '0.07724', 'num_input_tokens_seen': 932000, 'train_runtime': '875.2', 'train_tokens_per_second': '1065'}
+
+  8%|▊         | 140/1813 [14:35<2:57:33,  6.37s/it]
+  8%|▊         | 141/1813 [14:40<2:50:44,  6.13s/it]
+  8%|▊         | 142/1813 [14:48<3:06:16,  6.69s/it]
+  8%|▊         | 143/1813 [14:54<2:55:55,  6.32s/it]
+  8%|▊         | 144/1813 [14:59<2:46:34,  5.99s/it]
+  8%|▊         | 145/1813 [15:06<2:55:50,  6.33s/it]
+  8%|▊         | 146/1813 [15:12<2:54:04,  6.27s/it]
+  8%|▊         | 147/1813 [15:18<2:46:15,  5.99s/it]
+  8%|▊         | 148/1813 [15:23<2:45:51,  5.98s/it]
+  8%|▊         | 149/1813 [15:29<2:44:39,  5.94s/it]
+  8%|▊         | 150/1813 [15:36<2:49:01,  6.10s/it]
+                                                    
+{'loss': '1.178', 'grad_norm': '1.543', 'learning_rate': '2.014e-05', 'epoch': '0.08276', 'num_input_tokens_seen': 997296, 'train_runtime': '936.3', 'train_tokens_per_second': '1065'}
+
+  8%|▊         | 150/1813 [15:36<2:49:01,  6.10s/it]
+  8%|▊         | 151/1813 [15:41<2:43:40,  5.91s/it]
+  8%|▊         | 152/1813 [15:47<2:39:13,  5.75s/it]
+  8%|▊         | 153/1813 [15:52<2:38:57,  5.75s/it]
+  8%|▊         | 154/1813 [15:59<2:43:47,  5.92s/it]
+  9%|▊         | 155/1813 [16:06<2:52:57,  6.26s/it]
+  9%|▊         | 156/1813 [16:12<2:48:54,  6.12s/it]
+  9%|▊         | 157/1813 [16:18<2:47:28,  6.07s/it]
+  9%|▊         | 158/1813 [16:23<2:38:30,  5.75s/it]
+  9%|▉         | 159/1813 [16:28<2:38:07,  5.74s/it]
+  9%|▉         | 160/1813 [16:34<2:37:03,  5.70s/it]
+                                                    
+{'loss': '1.11', 'grad_norm': '1.746', 'learning_rate': '2.028e-05', 'epoch': '0.08828', 'num_input_tokens_seen': 1060096, 'train_runtime': '994.4', 'train_tokens_per_second': '1066'}
+
+  9%|▉         | 160/1813 [16:34<2:37:03,  5.70s/it]
+  9%|▉         | 161/1813 [16:40<2:37:19,  5.71s/it]
+  9%|▉         | 162/1813 [16:44<2:26:36,  5.33s/it]
+  9%|▉         | 163/1813 [16:51<2:37:24,  5.72s/it]
+  9%|▉         | 164/1813 [16:58<2:50:03,  6.19s/it]
+  9%|▉         | 165/1813 [17:04<2:49:16,  6.16s/it]
+  9%|▉         | 166/1813 [17:12<3:01:59,  6.63s/it]
+  9%|▉         | 167/1813 [17:18<3:02:23,  6.65s/it]
+  9%|▉         | 168/1813 [17:26<3:06:13,  6.79s/it]
+  9%|▉         | 169/1813 [17:31<2:57:28,  6.48s/it]
+  9%|▉         | 170/1813 [17:38<3:00:47,  6.60s/it]
+                                                    
+{'loss': '1.116', 'grad_norm': '0.8596', 'learning_rate': '2.027e-05', 'epoch': '0.09379', 'num_input_tokens_seen': 1130368, 'train_runtime': '1059', 'train_tokens_per_second': '1068'}
+
+  9%|▉         | 170/1813 [17:38<3:00:47,  6.60s/it]
+  9%|▉         | 171/1813 [17:44<2:52:19,  6.30s/it]
+  9%|▉         | 172/1813 [17:49<2:44:34,  6.02s/it]
+ 10%|▉         | 173/1813 [17:56<2:47:34,  6.13s/it]
+ 10%|▉         | 174/1813 [18:01<2:44:56,  6.04s/it]
+ 10%|▉         | 175/1813 [18:08<2:47:13,  6.13s/it]
+ 10%|▉         | 176/1813 [18:13<2:41:01,  5.90s/it]
+ 10%|▉         | 177/1813 [18:21<2:53:31,  6.36s/it]
+ 10%|▉         | 178/1813 [18:25<2:41:11,  5.92s/it]
+ 10%|▉         | 179/1813 [18:31<2:40:44,  5.90s/it]
+ 10%|▉         | 180/1813 [18:37<2:41:47,  5.94s/it]
+                                                    
+{'loss': '1.103', 'grad_norm': '1.558', 'learning_rate': '2.026e-05', 'epoch': '0.09931', 'num_input_tokens_seen': 1193192, 'train_runtime': '1118', 'train_tokens_per_second': '1067'}
+
+ 10%|▉         | 180/1813 [18:37<2:41:47,  5.94s/it]
+ 10%|▉         | 181/1813 [18:43<2:38:05,  5.81s/it]
+ 10%|█         | 182/1813 [18:48<2:34:27,  5.68s/it]
+ 10%|█         | 183/1813 [18:54<2:33:14,  5.64s/it]
+ 10%|█         | 184/1813 [18:59<2:31:24,  5.58s/it]
+ 10%|█         | 185/1813 [19:07<2:51:35,  6.32s/it]
+ 10%|█         | 186/1813 [19:14<2:51:19,  6.32s/it]
+ 10%|█         | 187/1813 [19:20<2:49:42,  6.26s/it]
+ 10%|█         | 188/1813 [19:25<2:40:59,  5.94s/it]
+ 10%|█         | 189/1813 [19:31<2:46:08,  6.14s/it]
+ 10%|█         | 190/1813 [19:38<2:50:16,  6.29s/it]
+                                                    
+{'loss': '1.054', 'grad_norm': '1.564', 'learning_rate': '2.025e-05', 'epoch': '0.1048', 'num_input_tokens_seen': 1259336, 'train_runtime': '1179', 'train_tokens_per_second': '1068'}
+
+ 10%|█         | 190/1813 [19:38<2:50:16,  6.29s/it]
+ 11%|█         | 191/1813 [19:43<2:42:35,  6.01s/it]
+ 11%|█         | 192/1813 [19:50<2:46:31,  6.16s/it]
+ 11%|█         | 193/1813 [19:57<2:50:03,  6.30s/it]
+ 11%|█         | 194/1813 [20:03<2:46:41,  6.18s/it]
+ 11%|█         | 195/1813 [20:10<2:57:05,  6.57s/it]
+ 11%|█         | 196/1813 [20:15<2:45:54,  6.16s/it]
+ 11%|█         | 197/1813 [20:22<2:52:34,  6.41s/it]
+ 11%|█         | 198/1813 [20:29<2:58:56,  6.65s/it]
+ 11%|█         | 199/1813 [20:36<2:58:03,  6.62s/it]
+ 11%|█         | 200/1813 [20:42<2:56:29,  6.57s/it]
+                                                    
+{'loss': '1.149', 'grad_norm': '1.701', 'learning_rate': '2.023e-05', 'epoch': '0.1103', 'num_input_tokens_seen': 1331224, 'train_runtime': '1243', 'train_tokens_per_second': '1071'}
+
+ 11%|█         | 200/1813 [20:42<2:56:29,  6.57s/it]
+ 11%|█         | 201/1813 [20:48<2:46:28,  6.20s/it]
+ 11%|█         | 202/1813 [20:53<2:36:47,  5.84s/it]
+ 11%|█         | 203/1813 [20:58<2:28:23,  5.53s/it]
+ 11%|█▏        | 204/1813 [21:04<2:38:12,  5.90s/it]
+ 11%|█▏        | 205/1813 [21:10<2:32:58,  5.71s/it]
+ 11%|█▏        | 206/1813 [21:15<2:33:45,  5.74s/it]
+ 11%|█▏        | 207/1813 [21:21<2:33:57,  5.75s/it]
+ 11%|█▏        | 208/1813 [21:29<2:49:38,  6.34s/it]
+ 12%|█▏        | 209/1813 [21:34<2:38:52,  5.94s/it]
+ 12%|█▏        | 210/1813 [21:40<2:39:12,  5.96s/it]
+                                                    
+{'loss': '1.026', 'grad_norm': '1.938', 'learning_rate': '2.022e-05', 'epoch': '0.1159', 'num_input_tokens_seen': 1392472, 'train_runtime': '1300', 'train_tokens_per_second': '1071'}
+
+ 12%|█▏        | 210/1813 [21:40<2:39:12,  5.96s/it]
+ 12%|█▏        | 211/1813 [21:48<3:00:22,  6.76s/it]
+ 12%|█▏        | 212/1813 [21:54<2:50:44,  6.40s/it]
+ 12%|█▏        | 213/1813 [21:59<2:42:02,  6.08s/it]
+ 12%|█▏        | 214/1813 [22:04<2:33:59,  5.78s/it]
+ 12%|█▏        | 215/1813 [22:10<2:31:12,  5.68s/it]
+ 12%|█▏        | 216/1813 [22:14<2:21:54,  5.33s/it]
+ 12%|█▏        | 217/1813 [22:20<2:20:23,  5.28s/it]
+ 12%|█▏        | 218/1813 [22:26<2:26:51,  5.52s/it]
+ 12%|█▏        | 219/1813 [22:32<2:30:16,  5.66s/it]
+ 12%|█▏        | 220/1813 [22:38<2:34:27,  5.82s/it]
+                                                    
+{'loss': '1.115', 'grad_norm': '1.318', 'learning_rate': '2.019e-05', 'epoch': '0.1214', 'num_input_tokens_seen': 1454040, 'train_runtime': '1358', 'train_tokens_per_second': '1070'}
+
+ 12%|█▏        | 220/1813 [22:38<2:34:27,  5.82s/it]
+ 12%|█▏        | 221/1813 [22:44<2:38:19,  5.97s/it]
+ 12%|█▏        | 222/1813 [22:49<2:32:44,  5.76s/it]
+ 12%|█▏        | 223/1813 [22:57<2:47:48,  6.33s/it]
+ 12%|█▏        | 224/1813 [23:02<2:39:24,  6.02s/it]
+ 12%|█▏        | 225/1813 [23:09<2:47:56,  6.35s/it]
+ 12%|█▏        | 226/1813 [23:17<2:53:38,  6.57s/it]
+ 13%|█▎        | 227/1813 [23:23<2:48:53,  6.39s/it]
+ 13%|█▎        | 228/1813 [23:29<2:50:54,  6.47s/it]
+ 13%|█▎        | 229/1813 [23:36<2:55:12,  6.64s/it]
+ 13%|█▎        | 230/1813 [23:41<2:42:50,  6.17s/it]
+                                                    
+{'loss': '1.068', 'grad_norm': '2.175', 'learning_rate': '2.017e-05', 'epoch': '0.1269', 'num_input_tokens_seen': 1522056, 'train_runtime': '1422', 'train_tokens_per_second': '1070'}
+
+ 13%|█▎        | 230/1813 [23:41<2:42:50,  6.17s/it]
+ 13%|█▎        | 231/1813 [23:46<2:33:40,  5.83s/it]
+ 13%|█▎        | 232/1813 [23:54<2:46:50,  6.33s/it]
+ 13%|█▎        | 233/1813 [23:59<2:37:51,  5.99s/it]
+ 13%|█▎        | 234/1813 [24:04<2:33:17,  5.83s/it]
+ 13%|█▎        | 235/1813 [24:11<2:41:53,  6.16s/it]
+ 13%|█▎        | 236/1813 [24:19<2:53:21,  6.60s/it]
+ 13%|█▎        | 237/1813 [24:26<2:56:59,  6.74s/it]
+ 13%|█▎        | 238/1813 [24:32<2:49:40,  6.46s/it]
+ 13%|█▎        | 239/1813 [24:39<2:58:11,  6.79s/it]
+ 13%|█▎        | 240/1813 [24:46<2:53:12,  6.61s/it]
+                                                    
+{'loss': '1.049', 'grad_norm': '1.276', 'learning_rate': '2.014e-05', 'epoch': '0.1324', 'num_input_tokens_seen': 1590200, 'train_runtime': '1486', 'train_tokens_per_second': '1070'}
+
+ 13%|█▎        | 240/1813 [24:46<2:53:12,  6.61s/it]
+ 13%|█▎        | 241/1813 [24:52<2:50:28,  6.51s/it]
+ 13%|█▎        | 242/1813 [24:58<2:47:53,  6.41s/it]
+ 13%|█▎        | 243/1813 [25:04<2:40:14,  6.12s/it]
+ 13%|█▎        | 244/1813 [25:11<2:46:48,  6.38s/it]
+ 14%|█▎        | 245/1813 [25:18<2:51:27,  6.56s/it]
+ 14%|█▎        | 246/1813 [25:23<2:41:57,  6.20s/it]
+ 14%|█▎        | 247/1813 [25:30<2:46:59,  6.40s/it]
+ 14%|█▎        | 248/1813 [25:38<2:57:52,  6.82s/it]
+ 14%|█▎        | 249/1813 [25:43<2:45:27,  6.35s/it]
+ 14%|█▍        | 250/1813 [25:48<2:39:02,  6.11s/it]
+                                                    
+{'loss': '1.15', 'grad_norm': '1.537', 'learning_rate': '2.01e-05', 'epoch': '0.1379', 'num_input_tokens_seen': 1658648, 'train_runtime': '1549', 'train_tokens_per_second': '1071'}
+
+ 14%|█▍        | 250/1813 [25:48<2:39:02,  6.11s/it]
+ 14%|█▍        | 251/1813 [25:53<2:26:48,  5.64s/it]
+ 14%|█▍        | 252/1813 [25:58<2:24:12,  5.54s/it]
+ 14%|█▍        | 253/1813 [26:04<2:29:16,  5.74s/it]
+ 14%|█▍        | 254/1813 [26:12<2:42:27,  6.25s/it]
+ 14%|█▍        | 255/1813 [26:19<2:45:21,  6.37s/it]
+ 14%|█▍        | 256/1813 [26:24<2:40:03,  6.17s/it]
+ 14%|█▍        | 257/1813 [26:31<2:45:22,  6.38s/it]
+ 14%|█▍        | 258/1813 [26:38<2:48:21,  6.50s/it]
+ 14%|█▍        | 259/1813 [26:42<2:32:27,  5.89s/it]
+ 14%|█▍        | 260/1813 [26:48<2:33:11,  5.92s/it]
+                                                    
+{'loss': '0.9606', 'grad_norm': '2.409', 'learning_rate': '2.006e-05', 'epoch': '0.1434', 'num_input_tokens_seen': 1722792, 'train_runtime': '1609', 'train_tokens_per_second': '1071'}
+
+ 14%|█▍        | 260/1813 [26:48<2:33:11,  5.92s/it]
+ 14%|█▍        | 261/1813 [26:56<2:43:46,  6.33s/it]
+ 14%|█▍        | 262/1813 [27:02<2:47:01,  6.46s/it]
+ 15%|█▍        | 263/1813 [27:09<2:49:03,  6.54s/it]
+ 15%|█▍        | 264/1813 [27:15<2:46:32,  6.45s/it]
+ 15%|█▍        | 265/1813 [27:21<2:43:53,  6.35s/it]
+ 15%|█▍        | 266/1813 [27:26<2:33:11,  5.94s/it]
+ 15%|█▍        | 267/1813 [27:32<2:28:47,  5.77s/it]
+ 15%|█▍        | 268/1813 [27:37<2:24:59,  5.63s/it]
+ 15%|█▍        | 269/1813 [27:44<2:30:40,  5.86s/it]
+ 15%|█▍        | 270/1813 [27:49<2:30:32,  5.85s/it]
+                                                    
+{'loss': '0.9785', 'grad_norm': '1.807', 'learning_rate': '2.002e-05', 'epoch': '0.149', 'num_input_tokens_seen': 1786808, 'train_runtime': '1670', 'train_tokens_per_second': '1070'}
+
+ 15%|█▍        | 270/1813 [27:49<2:30:32,  5.85s/it]
+ 15%|█▍        | 271/1813 [27:57<2:46:27,  6.48s/it]
+ 15%|█▌        | 272/1813 [28:03<2:36:49,  6.11s/it]
+ 15%|█▌        | 273/1813 [28:11<2:53:12,  6.75s/it]
+ 15%|█▌        | 274/1813 [28:17<2:52:06,  6.71s/it]
+ 15%|█▌        | 275/1813 [28:23<2:39:49,  6.23s/it]
+ 15%|█▌        | 276/1813 [28:28<2:30:24,  5.87s/it]
+ 15%|█▌        | 277/1813 [28:34<2:32:30,  5.96s/it]
+ 15%|█▌        | 278/1813 [28:40<2:33:01,  5.98s/it]
+ 15%|█▌        | 279/1813 [28:47<2:44:22,  6.43s/it]
+ 15%|█▌        | 280/1813 [28:54<2:44:12,  6.43s/it]
+                                                    
+{'loss': '1.057', 'grad_norm': '1.593', 'learning_rate': '1.998e-05', 'epoch': '0.1545', 'num_input_tokens_seen': 1858112, 'train_runtime': '1734', 'train_tokens_per_second': '1071'}
+
+ 15%|█▌        | 280/1813 [28:54<2:44:12,  6.43s/it]
+ 15%|█▌        | 281/1813 [28:59<2:38:24,  6.20s/it]
+ 16%|█▌        | 282/1813 [29:05<2:31:40,  5.94s/it]
+ 16%|█▌        | 283/1813 [29:10<2:25:59,  5.73s/it]
+ 16%|█▌        | 284/1813 [29:15<2:24:28,  5.67s/it]
+ 16%|█▌        | 285/1813 [29:22<2:31:44,  5.96s/it]
+ 16%|█▌        | 286/1813 [29:28<2:31:24,  5.95s/it]
+ 16%|█▌        | 287/1813 [29:34<2:32:41,  6.00s/it]
+ 16%|█▌        | 288/1813 [29:39<2:27:39,  5.81s/it]
+ 16%|█▌        | 289/1813 [29:46<2:31:50,  5.98s/it]
+ 16%|█▌        | 290/1813 [29:52<2:36:06,  6.15s/it]
+                                                    
+{'loss': '1.031', 'grad_norm': '1.419', 'learning_rate': '1.993e-05', 'epoch': '0.16', 'num_input_tokens_seen': 1920600, 'train_runtime': '1793', 'train_tokens_per_second': '1071'}
+
+ 16%|█▌        | 290/1813 [29:52<2:36:06,  6.15s/it]
+ 16%|█▌        | 291/1813 [30:00<2:43:27,  6.44s/it]
+ 16%|█▌        | 292/1813 [30:07<2:50:34,  6.73s/it]
+ 16%|█▌        | 293/1813 [30:13<2:42:55,  6.43s/it]
+ 16%|█▌        | 294/1813 [30:20<2:48:20,  6.65s/it]
+ 16%|█▋        | 295/1813 [30:27<2:54:52,  6.91s/it]
+ 16%|█▋        | 296/1813 [30:33<2:44:23,  6.50s/it]
+ 16%|█▋        | 297/1813 [30:39<2:40:42,  6.36s/it]
+ 16%|█▋        | 298/1813 [30:44<2:28:00,  5.86s/it]
+ 16%|█▋        | 299/1813 [30:49<2:27:36,  5.85s/it]
+ 17%|█▋        | 300/1813 [30:55<2:26:16,  5.80s/it]
+                                                    
+{'loss': '1.087', 'grad_norm': '1.414', 'learning_rate': '1.988e-05', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312, 'train_runtime': '1856', 'train_tokens_per_second': '1072'}
+
+ 17%|█▋        | 300/1813 [30:55<2:26:16,  5.80s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.75it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.23it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.30it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.45it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.05it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.99it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.82it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:35,  6.24it/s]
+
+  3%|▎         | 30/1000 [00:04<02:44,  5.88it/s]
+
+  3%|▎         | 31/1000 [00:04<03:02,  5.30it/s]
+
+  3%|▎         | 32/1000 [00:05<02:42,  5.95it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.73it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.63it/s]
+
+  4%|▎         | 37/1000 [00:05<01:46,  9.07it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:06<02:04,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:47,  8.91it/s]
+
+  4%|▍         | 44/1000 [00:06<01:34, 10.17it/s]
+
+  5%|▍         | 46/1000 [00:06<01:46,  8.96it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.33it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.73it/s]
+
+  5%|▌         | 51/1000 [00:07<01:36,  9.87it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.90it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.72it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.28it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.18it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.27it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:20,  6.62it/s]
+
+  7%|▋         | 74/1000 [00:09<01:55,  8.03it/s]
+
+  8%|▊         | 75/1000 [00:10<02:15,  6.82it/s]
+
+  8%|▊         | 76/1000 [00:10<02:19,  6.65it/s]
+
+  8%|▊         | 77/1000 [00:10<02:08,  7.20it/s]
+
+  8%|▊         | 78/1000 [00:10<02:32,  6.04it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.82it/s]
+
+  8%|▊         | 82/1000 [00:10<01:37,  9.38it/s]
+
+  8%|▊         | 84/1000 [00:11<01:53,  8.04it/s]
+
+  8%|▊         | 85/1000 [00:11<02:15,  6.74it/s]
+
+  9%|▊         | 86/1000 [00:11<02:35,  5.87it/s]
+
+  9%|▊         | 87/1000 [00:12<02:51,  5.31it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  6.98it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.07it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.16it/s]
+
+  9%|▉         | 93/1000 [00:12<02:37,  5.76it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.51it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:42,  8.70it/s]
+
+ 11%|█         | 108/1000 [00:14<01:29,  9.94it/s]
+
+ 11%|█         | 110/1000 [00:14<01:22, 10.81it/s]
+
+ 11%|█         | 112/1000 [00:14<01:15, 11.72it/s]
+
+ 11%|█▏        | 114/1000 [00:15<01:12, 12.28it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.65it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:07, 13.11it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.88it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:49,  8.05it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.71it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.13it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:40,  8.61it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.07it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:30,  9.60it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.86it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.95it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.06it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.92it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.06it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.41it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.98it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.79it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.40it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.17it/s]
+
+ 17%|█▋        | 167/1000 [00:22<02:24,  5.77it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.82it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.82it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.59it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.52it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.76it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.62it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.74it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.69it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.76it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.64it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.95it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.51it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.67it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.14it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.49it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:29<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.04it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.90it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.03it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.93it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.45it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+
+ 24%|██▍       | 242/1000 [00:32<02:06,  6.01it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.84it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.70it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.42it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.92it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.71it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.72it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:21,  8.60it/s]
+
+ 30%|███       | 304/1000 [00:40<01:27,  7.92it/s]
+
+ 30%|███       | 305/1000 [00:41<01:26,  8.02it/s]
+
+ 31%|███       | 306/1000 [00:41<01:40,  6.88it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.74it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.76it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.15it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.22it/s]
+
+ 32%|███▏      | 315/1000 [00:42<01:07, 10.21it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.56it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:43<01:38,  6.88it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:44<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.37it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.83it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.05it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.90it/s]
+
+ 34%|███▍      | 344/1000 [00:46<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.42it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:33,  6.93it/s]
+
+ 35%|███▌      | 350/1000 [00:47<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.65it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.57it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.37it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.98it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.62it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.45it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.23it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.19it/s]
+
+ 38%|███▊      | 379/1000 [00:51<01:16,  8.16it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.18it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.55it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.08it/s]
+
+ 39%|███▊      | 387/1000 [00:52<01:21,  7.56it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.30it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.84it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.49it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  9.00it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.53it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.84it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:54<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.51it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:56<01:22,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.26it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.51it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.38it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.97it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.55it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:12,  7.85it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.54it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.01it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.78it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.53it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.19it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.49it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.80it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.82it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.61it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.05it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.31it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.30it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.20it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.26it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.96it/s]
+
+ 50%|████▉     | 496/1000 [01:06<01:00,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:08<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.93it/s]
+
+ 51%|█████     | 512/1000 [01:09<00:49,  9.85it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:07,  7.24it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.49it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.07it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<01:00,  7.90it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:16,  6.30it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<01:01,  7.80it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:12,  6.56it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.75it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:26,  5.51it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.51it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.39it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:12<00:48,  9.67it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.67it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.34it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.51it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:15<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.76it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.85it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.05it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.37it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.84it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.26it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.57it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.79it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.40it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.68it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.74it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.83it/s]
+
+ 62%|██████▏   | 616/1000 [01:24<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.11it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.04it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.42it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.96it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.18it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.97it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.12it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.60it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.57it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.33it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:44,  7.71it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.58it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.49it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.69it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.67it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.99it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.92it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.90it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.88it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.82it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.85it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.28it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:48,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+
+ 70%|███████   | 702/1000 [01:36<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.21it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.41it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.71it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.87it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.37it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.62it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.53it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.79it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.71it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.67it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.62it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.13it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.99it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.20it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.16it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.21it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.23it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.31it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.13it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.00it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.09it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.39it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.76it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.24it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.54it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.86it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.45it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.04it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.18it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.62it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.72it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.33it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.25it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.27it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.61it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.56it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.77it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.84it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.85it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.05it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.32it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.57it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.75it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.38it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.16it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.35it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  7.91it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.56it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.43it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.78it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.57it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.15it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.82it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.03it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.11it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.33it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.71it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.81it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.40it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.03it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.19it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.79it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.84it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.05it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                    
+
+                                                   
+{'eval_loss': '0.9061', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.378', 'eval_steps_per_second': '7.378', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312}
+
+ 17%|█▋        | 300/1813 [33:11<2:26:16,  5.80s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 17%|█▋        | 301/1813 [33:16<19:26:33, 46.29s/it]
+ 17%|█▋        | 302/1813 [33:20<14:10:34, 33.78s/it]
+ 17%|█▋        | 303/1813 [33:27<10:46:26, 25.69s/it]
+ 17%|█▋        | 304/1813 [33:35<8:28:39, 20.22s/it] 
+ 17%|█▋        | 305/1813 [33:41<6:42:40, 16.02s/it]
+ 17%|█▋        | 306/1813 [33:47<5:25:29, 12.96s/it]
+ 17%|█▋        | 307/1813 [33:52<4:28:34, 10.70s/it]
+ 17%|█▋        | 308/1813 [33:58<3:48:59,  9.13s/it]
+ 17%|█▋        | 309/1813 [34:04<3:27:57,  8.30s/it]
+ 17%|█▋        | 310/1813 [34:09<3:02:07,  7.27s/it]
+                                                    
+{'loss': '1.005', 'grad_norm': '2.01', 'learning_rate': '1.982e-05', 'epoch': '0.171', 'num_input_tokens_seen': 2050672, 'train_runtime': '2049', 'train_tokens_per_second': '1001'}
+
+ 17%|█▋        | 310/1813 [34:09<3:02:07,  7.27s/it]
+ 17%|█▋        | 311/1813 [34:16<2:59:50,  7.18s/it]
+ 17%|█▋        | 312/1813 [34:23<3:00:27,  7.21s/it]
+ 17%|█▋        | 313/1813 [34:30<2:56:29,  7.06s/it]
+ 17%|█▋        | 314/1813 [34:35<2:45:17,  6.62s/it]
+ 17%|█▋        | 315/1813 [34:42<2:47:41,  6.72s/it]
+ 17%|█▋        | 316/1813 [34:49<2:45:58,  6.65s/it]
+ 17%|█▋        | 317/1813 [34:55<2:39:48,  6.41s/it]
+ 18%|█▊        | 318/1813 [35:00<2:31:33,  6.08s/it]
+ 18%|█▊        | 319/1813 [35:05<2:21:18,  5.67s/it]
+ 18%|█▊        | 320/1813 [35:11<2:22:01,  5.71s/it]
+                                                    
+{'loss': '1.085', 'grad_norm': '1.099', 'learning_rate': '1.977e-05', 'epoch': '0.1766', 'num_input_tokens_seen': 2118000, 'train_runtime': '2111', 'train_tokens_per_second': '1003'}
+
+ 18%|█▊        | 320/1813 [35:11<2:22:01,  5.71s/it]
+ 18%|█▊        | 321/1813 [35:16<2:22:43,  5.74s/it]
+ 18%|█▊        | 322/1813 [35:22<2:24:08,  5.80s/it]
+ 18%|█▊        | 323/1813 [35:29<2:28:36,  5.98s/it]
+ 18%|█▊        | 324/1813 [35:36<2:37:24,  6.34s/it]
+ 18%|█▊        | 325/1813 [35:43<2:42:02,  6.53s/it]
+ 18%|█▊        | 326/1813 [35:49<2:39:21,  6.43s/it]
+ 18%|█▊        | 327/1813 [35:55<2:35:25,  6.28s/it]
+ 18%|█▊        | 328/1813 [36:01<2:36:37,  6.33s/it]
+ 18%|█▊        | 329/1813 [36:07<2:31:39,  6.13s/it]
+ 18%|█▊        | 330/1813 [36:13<2:27:43,  5.98s/it]
+                                                    
+{'loss': '1.002', 'grad_norm': '1.568', 'learning_rate': '1.97e-05', 'epoch': '0.1821', 'num_input_tokens_seen': 2184640, 'train_runtime': '2173', 'train_tokens_per_second': '1005'}
+
+ 18%|█▊        | 330/1813 [36:13<2:27:43,  5.98s/it]
+ 18%|█▊        | 331/1813 [36:19<2:29:13,  6.04s/it]
+ 18%|█▊        | 332/1813 [36:24<2:21:20,  5.73s/it]
+ 18%|█▊        | 333/1813 [36:30<2:22:37,  5.78s/it]
+ 18%|█▊        | 334/1813 [36:36<2:21:49,  5.75s/it]
+ 18%|█▊        | 335/1813 [36:43<2:37:11,  6.38s/it]
+ 19%|█▊        | 336/1813 [36:50<2:42:00,  6.58s/it]
+ 19%|█▊        | 337/1813 [36:56<2:35:12,  6.31s/it]
+ 19%|█▊        | 338/1813 [37:02<2:35:15,  6.32s/it]
+ 19%|█▊        | 339/1813 [37:08<2:31:13,  6.16s/it]
+ 19%|█▉        | 340/1813 [37:15<2:38:38,  6.46s/it]
+                                                    
+{'loss': '0.9893', 'grad_norm': '1.819', 'learning_rate': '1.964e-05', 'epoch': '0.1876', 'num_input_tokens_seen': 2251392, 'train_runtime': '2236', 'train_tokens_per_second': '1007'}
+
+ 19%|█▉        | 340/1813 [37:15<2:38:38,  6.46s/it]
+ 19%|█▉        | 341/1813 [37:22<2:38:32,  6.46s/it]
+ 19%|█▉        | 342/1813 [37:29<2:45:54,  6.77s/it]
+ 19%|█▉        | 343/1813 [37:35<2:37:10,  6.42s/it]
+ 19%|█▉        | 344/1813 [37:41<2:31:59,  6.21s/it]
+ 19%|█▉        | 345/1813 [37:48<2:40:22,  6.55s/it]
+ 19%|█▉        | 346/1813 [37:54<2:39:49,  6.54s/it]
+ 19%|█▉        | 347/1813 [38:00<2:29:40,  6.13s/it]
+ 19%|█▉        | 348/1813 [38:06<2:32:17,  6.24s/it]
+ 19%|█▉        | 349/1813 [38:15<2:47:51,  6.88s/it]
+ 19%|█▉        | 350/1813 [38:22<2:53:28,  7.11s/it]
+                                                    
+{'loss': '1.094', 'grad_norm': '1.303', 'learning_rate': '1.957e-05', 'epoch': '0.1931', 'num_input_tokens_seen': 2324712, 'train_runtime': '2303', 'train_tokens_per_second': '1010'}
+
+ 19%|█▉        | 350/1813 [38:22<2:53:28,  7.11s/it]
+ 19%|█▉        | 351/1813 [38:29<2:51:13,  7.03s/it]
+ 19%|█▉        | 352/1813 [38:35<2:42:03,  6.66s/it]
+ 19%|█▉        | 353/1813 [38:40<2:33:03,  6.29s/it]
+ 20%|█▉        | 354/1813 [38:46<2:27:31,  6.07s/it]
+ 20%|█▉        | 355/1813 [38:52<2:26:11,  6.02s/it]
+ 20%|█▉        | 356/1813 [38:59<2:32:10,  6.27s/it]
+ 20%|█▉        | 357/1813 [39:04<2:28:21,  6.11s/it]
+ 20%|█▉        | 358/1813 [39:11<2:29:02,  6.15s/it]
+ 20%|█▉        | 359/1813 [39:16<2:25:11,  5.99s/it]
+ 20%|█▉        | 360/1813 [39:23<2:31:21,  6.25s/it]
+                                                    
+{'loss': '1.043', 'grad_norm': '1.731', 'learning_rate': '1.95e-05', 'epoch': '0.1986', 'num_input_tokens_seen': 2390776, 'train_runtime': '2364', 'train_tokens_per_second': '1012'}
+
+ 20%|█▉        | 360/1813 [39:23<2:31:21,  6.25s/it]
+ 20%|█▉        | 361/1813 [39:28<2:22:42,  5.90s/it]
+ 20%|█▉        | 362/1813 [39:34<2:21:43,  5.86s/it]
+ 20%|██        | 363/1813 [39:41<2:29:25,  6.18s/it]
+ 20%|██        | 364/1813 [39:47<2:32:14,  6.30s/it]
+ 20%|██        | 365/1813 [39:53<2:29:12,  6.18s/it]
+ 20%|██        | 366/1813 [40:00<2:35:09,  6.43s/it]
+ 20%|██        | 367/1813 [40:06<2:33:04,  6.35s/it]
+ 20%|██        | 368/1813 [40:12<2:28:16,  6.16s/it]
+ 20%|██        | 369/1813 [40:17<2:18:58,  5.77s/it]
+ 20%|██        | 370/1813 [40:23<2:21:19,  5.88s/it]
+                                                    
+{'loss': '1.029', 'grad_norm': '2.252', 'learning_rate': '1.942e-05', 'epoch': '0.2041', 'num_input_tokens_seen': 2454808, 'train_runtime': '2424', 'train_tokens_per_second': '1013'}
+
+ 20%|██        | 370/1813 [40:23<2:21:19,  5.88s/it]
+ 20%|██        | 371/1813 [40:28<2:17:05,  5.70s/it]
+ 21%|██        | 372/1813 [40:37<2:35:51,  6.49s/it]
+ 21%|██        | 373/1813 [40:43<2:34:13,  6.43s/it]
+ 21%|██        | 374/1813 [40:51<2:41:58,  6.75s/it]
+ 21%|██        | 375/1813 [40:58<2:43:37,  6.83s/it]
+ 21%|██        | 376/1813 [41:03<2:34:38,  6.46s/it]
+ 21%|██        | 377/1813 [41:09<2:33:34,  6.42s/it]
+ 21%|██        | 378/1813 [41:16<2:31:10,  6.32s/it]
+ 21%|██        | 379/1813 [41:21<2:21:27,  5.92s/it]
+ 21%|██        | 380/1813 [41:26<2:17:02,  5.74s/it]
+                                                    
+{'loss': '1.004', 'grad_norm': '2.369', 'learning_rate': '1.934e-05', 'epoch': '0.2097', 'num_input_tokens_seen': 2520472, 'train_runtime': '2486', 'train_tokens_per_second': '1014'}
+
+ 21%|██        | 380/1813 [41:26<2:17:02,  5.74s/it]
+ 21%|██        | 381/1813 [41:33<2:25:37,  6.10s/it]
+ 21%|██        | 382/1813 [41:40<2:30:54,  6.33s/it]
+ 21%|██        | 383/1813 [41:46<2:30:24,  6.31s/it]
+ 21%|██        | 384/1813 [41:52<2:30:21,  6.31s/it]
+ 21%|██        | 385/1813 [41:59<2:30:59,  6.34s/it]
+ 21%|██▏       | 386/1813 [42:04<2:20:42,  5.92s/it]
+ 21%|██▏       | 387/1813 [42:10<2:22:52,  6.01s/it]
+ 21%|██▏       | 388/1813 [42:17<2:30:57,  6.36s/it]
+ 21%|██▏       | 389/1813 [42:25<2:41:24,  6.80s/it]
+ 22%|██▏       | 390/1813 [42:32<2:41:01,  6.79s/it]
+                                                    
+{'loss': '1.04', 'grad_norm': '1.434', 'learning_rate': '1.926e-05', 'epoch': '0.2152', 'num_input_tokens_seen': 2591512, 'train_runtime': '2552', 'train_tokens_per_second': '1015'}
+
+ 22%|██▏       | 390/1813 [42:32<2:41:01,  6.79s/it]
+ 22%|██▏       | 391/1813 [42:38<2:37:08,  6.63s/it]
+ 22%|██▏       | 392/1813 [42:44<2:36:07,  6.59s/it]
+ 22%|██▏       | 393/1813 [42:51<2:37:12,  6.64s/it]
+ 22%|██▏       | 394/1813 [42:56<2:26:30,  6.19s/it]
+ 22%|██▏       | 395/1813 [43:02<2:23:11,  6.06s/it]
+ 22%|██▏       | 396/1813 [43:09<2:26:42,  6.21s/it]
+ 22%|██▏       | 397/1813 [43:14<2:19:13,  5.90s/it]
+ 22%|██▏       | 398/1813 [43:20<2:22:43,  6.05s/it]
+ 22%|██▏       | 399/1813 [43:28<2:34:26,  6.55s/it]
+ 22%|██▏       | 400/1813 [43:36<2:47:10,  7.10s/it]
+                                                    
+{'loss': '1.023', 'grad_norm': '1.626', 'learning_rate': '1.918e-05', 'epoch': '0.2207', 'num_input_tokens_seen': 2660336, 'train_runtime': '2617', 'train_tokens_per_second': '1017'}
+
+ 22%|██▏       | 400/1813 [43:36<2:47:10,  7.10s/it]
+ 22%|██▏       | 401/1813 [43:44<2:48:42,  7.17s/it]
+ 22%|██▏       | 402/1813 [43:49<2:33:46,  6.54s/it]
+ 22%|██▏       | 403/1813 [43:55<2:30:57,  6.42s/it]
+ 22%|██▏       | 404/1813 [44:02<2:33:30,  6.54s/it]
+ 22%|██▏       | 405/1813 [44:09<2:38:05,  6.74s/it]
+ 22%|██▏       | 406/1813 [44:14<2:26:00,  6.23s/it]
+ 22%|██▏       | 407/1813 [44:19<2:15:58,  5.80s/it]
+ 23%|██▎       | 408/1813 [44:25<2:17:07,  5.86s/it]
+ 23%|██▎       | 409/1813 [44:31<2:19:33,  5.96s/it]
+ 23%|██▎       | 410/1813 [44:36<2:16:44,  5.85s/it]
+                                                    
+{'loss': '1.043', 'grad_norm': '2.34', 'learning_rate': '1.909e-05', 'epoch': '0.2262', 'num_input_tokens_seen': 2722056, 'train_runtime': '2677', 'train_tokens_per_second': '1017'}
+
+ 23%|██▎       | 410/1813 [44:36<2:16:44,  5.85s/it]
+ 23%|██▎       | 411/1813 [44:43<2:19:46,  5.98s/it]
+ 23%|██▎       | 412/1813 [44:49<2:23:27,  6.14s/it]
+ 23%|██▎       | 413/1813 [44:57<2:31:32,  6.49s/it]
+ 23%|██▎       | 414/1813 [45:02<2:22:32,  6.11s/it]
+ 23%|██▎       | 415/1813 [45:08<2:22:58,  6.14s/it]
+ 23%|██▎       | 416/1813 [45:14<2:24:36,  6.21s/it]
+ 23%|██▎       | 417/1813 [45:22<2:34:06,  6.62s/it]
+ 23%|██▎       | 418/1813 [45:28<2:28:06,  6.37s/it]
+ 23%|██▎       | 419/1813 [45:34<2:29:35,  6.44s/it]
+ 23%|██▎       | 420/1813 [45:40<2:25:38,  6.27s/it]
+                                                    
+{'loss': '1.031', 'grad_norm': '1.132', 'learning_rate': '1.9e-05', 'epoch': '0.2317', 'num_input_tokens_seen': 2790960, 'train_runtime': '2741', 'train_tokens_per_second': '1018'}
+
+ 23%|██▎       | 420/1813 [45:40<2:25:38,  6.27s/it]
+ 23%|██▎       | 421/1813 [45:47<2:25:40,  6.28s/it]
+ 23%|██▎       | 422/1813 [45:54<2:32:35,  6.58s/it]
+ 23%|██▎       | 423/1813 [46:00<2:27:22,  6.36s/it]
+ 23%|██▎       | 424/1813 [46:06<2:24:29,  6.24s/it]
+ 23%|██▎       | 425/1813 [46:11<2:19:00,  6.01s/it]
+ 23%|██▎       | 426/1813 [46:18<2:27:21,  6.37s/it]
+ 24%|██▎       | 427/1813 [46:24<2:23:53,  6.23s/it]
+ 24%|██▎       | 428/1813 [46:30<2:22:39,  6.18s/it]
+ 24%|██▎       | 429/1813 [46:35<2:15:55,  5.89s/it]
+ 24%|██▎       | 430/1813 [46:43<2:26:40,  6.36s/it]
+                                                    
+{'loss': '1.017', 'grad_norm': '1.387', 'learning_rate': '1.89e-05', 'epoch': '0.2372', 'num_input_tokens_seen': 2860096, 'train_runtime': '2803', 'train_tokens_per_second': '1020'}
+
+ 24%|██▎       | 430/1813 [46:43<2:26:40,  6.36s/it]
+ 24%|██▍       | 431/1813 [46:49<2:22:59,  6.21s/it]
+ 24%|██▍       | 432/1813 [46:54<2:19:17,  6.05s/it]
+ 24%|██▍       | 433/1813 [47:00<2:14:14,  5.84s/it]
+ 24%|██▍       | 434/1813 [47:06<2:18:38,  6.03s/it]
+ 24%|██▍       | 435/1813 [47:12<2:17:37,  5.99s/it]
+ 24%|██▍       | 436/1813 [47:19<2:20:11,  6.11s/it]
+ 24%|██▍       | 437/1813 [47:24<2:17:37,  6.00s/it]
+ 24%|██▍       | 438/1813 [47:31<2:21:37,  6.18s/it]
+ 24%|██▍       | 439/1813 [47:37<2:21:29,  6.18s/it]
+ 24%|██▍       | 440/1813 [47:44<2:23:00,  6.25s/it]
+                                                    
+{'loss': '0.9496', 'grad_norm': '2.501', 'learning_rate': '1.88e-05', 'epoch': '0.2428', 'num_input_tokens_seen': 2924856, 'train_runtime': '2864', 'train_tokens_per_second': '1021'}
+
+ 24%|██▍       | 440/1813 [47:44<2:23:00,  6.25s/it]
+ 24%|██▍       | 441/1813 [47:49<2:20:00,  6.12s/it]
+ 24%|██▍       | 442/1813 [47:55<2:18:56,  6.08s/it]
+ 24%|██▍       | 443/1813 [48:02<2:22:33,  6.24s/it]
+ 24%|██▍       | 444/1813 [48:07<2:17:13,  6.01s/it]
+ 25%|██▍       | 445/1813 [48:15<2:24:43,  6.35s/it]
+ 25%|██▍       | 446/1813 [48:21<2:24:31,  6.34s/it]
+ 25%|██▍       | 447/1813 [48:27<2:22:40,  6.27s/it]
+ 25%|██▍       | 448/1813 [48:33<2:23:41,  6.32s/it]
+ 25%|██▍       | 449/1813 [48:39<2:18:29,  6.09s/it]
+ 25%|██▍       | 450/1813 [48:45<2:15:20,  5.96s/it]
+                                                    
+{'loss': '1.071', 'grad_norm': '1.702', 'learning_rate': '1.87e-05', 'epoch': '0.2483', 'num_input_tokens_seen': 2990040, 'train_runtime': '2925', 'train_tokens_per_second': '1022'}
+
+ 25%|██▍       | 450/1813 [48:45<2:15:20,  5.96s/it]
+ 25%|██▍       | 451/1813 [48:52<2:26:08,  6.44s/it]
+ 25%|██▍       | 452/1813 [48:58<2:21:20,  6.23s/it]
+ 25%|██▍       | 453/1813 [49:03<2:14:52,  5.95s/it]
+ 25%|██▌       | 454/1813 [49:10<2:21:36,  6.25s/it]
+ 25%|██▌       | 455/1813 [49:17<2:23:45,  6.35s/it]
+ 25%|██▌       | 456/1813 [49:21<2:11:58,  5.84s/it]
+ 25%|██▌       | 457/1813 [49:28<2:14:42,  5.96s/it]
+ 25%|██▌       | 458/1813 [49:36<2:29:09,  6.60s/it]
+ 25%|██▌       | 459/1813 [49:43<2:31:27,  6.71s/it]
+ 25%|██▌       | 460/1813 [49:49<2:29:43,  6.64s/it]
+                                                    
+{'loss': '1.123', 'grad_norm': '1.272', 'learning_rate': '1.86e-05', 'epoch': '0.2538', 'num_input_tokens_seen': 3058848, 'train_runtime': '2990', 'train_tokens_per_second': '1023'}
+
+ 25%|██▌       | 460/1813 [49:49<2:29:43,  6.64s/it]
+ 25%|██▌       | 461/1813 [49:55<2:26:56,  6.52s/it]
+ 25%|██▌       | 462/1813 [50:03<2:33:06,  6.80s/it]
+ 26%|██▌       | 463/1813 [50:09<2:27:05,  6.54s/it]
+ 26%|██▌       | 464/1813 [50:14<2:17:47,  6.13s/it]
+ 26%|██▌       | 465/1813 [50:20<2:16:12,  6.06s/it]
+ 26%|██▌       | 466/1813 [50:27<2:21:21,  6.30s/it]
+ 26%|██▌       | 467/1813 [50:33<2:19:16,  6.21s/it]
+ 26%|██▌       | 468/1813 [50:39<2:22:20,  6.35s/it]
+ 26%|██▌       | 469/1813 [50:46<2:25:35,  6.50s/it]
+ 26%|██▌       | 470/1813 [50:53<2:27:25,  6.59s/it]
+                                                    
+{'loss': '1.004', 'grad_norm': '1.202', 'learning_rate': '1.849e-05', 'epoch': '0.2593', 'num_input_tokens_seen': 3126680, 'train_runtime': '3054', 'train_tokens_per_second': '1024'}
+
+ 26%|██▌       | 470/1813 [50:53<2:27:25,  6.59s/it]
+ 26%|██▌       | 471/1813 [50:59<2:21:44,  6.34s/it]
+ 26%|██▌       | 472/1813 [51:06<2:24:51,  6.48s/it]
+ 26%|██▌       | 473/1813 [51:13<2:27:41,  6.61s/it]
+ 26%|██▌       | 474/1813 [51:19<2:24:44,  6.49s/it]
+ 26%|██▌       | 475/1813 [51:24<2:17:36,  6.17s/it]
+ 26%|██▋       | 476/1813 [51:30<2:17:17,  6.16s/it]
+ 26%|██▋       | 477/1813 [51:37<2:20:25,  6.31s/it]
+ 26%|██▋       | 478/1813 [51:44<2:24:34,  6.50s/it]
+ 26%|██▋       | 479/1813 [51:50<2:19:28,  6.27s/it]
+ 26%|██▋       | 480/1813 [51:56<2:21:53,  6.39s/it]
+                                                    
+{'loss': '1.101', 'grad_norm': '1.7', 'learning_rate': '1.838e-05', 'epoch': '0.2648', 'num_input_tokens_seen': 3193704, 'train_runtime': '3117', 'train_tokens_per_second': '1025'}
+
+ 26%|██▋       | 480/1813 [51:56<2:21:53,  6.39s/it]
+ 27%|██▋       | 481/1813 [52:03<2:22:22,  6.41s/it]
+ 27%|██▋       | 482/1813 [52:10<2:25:33,  6.56s/it]
+ 27%|██▋       | 483/1813 [52:18<2:33:47,  6.94s/it]
+ 27%|██▋       | 484/1813 [52:23<2:25:23,  6.56s/it]
+ 27%|██▋       | 485/1813 [52:31<2:32:54,  6.91s/it]
+ 27%|██▋       | 486/1813 [52:37<2:28:40,  6.72s/it]
+ 27%|██▋       | 487/1813 [52:44<2:29:42,  6.77s/it]
+ 27%|██▋       | 488/1813 [52:50<2:23:13,  6.49s/it]
+ 27%|██▋       | 489/1813 [52:57<2:28:13,  6.72s/it]
+ 27%|██▋       | 490/1813 [53:04<2:26:02,  6.62s/it]
+                                                    
+{'loss': '1.045', 'grad_norm': '1.841', 'learning_rate': '1.827e-05', 'epoch': '0.2703', 'num_input_tokens_seen': 3268064, 'train_runtime': '3184', 'train_tokens_per_second': '1026'}
+
+ 27%|██▋       | 490/1813 [53:04<2:26:02,  6.62s/it]
+ 27%|██▋       | 491/1813 [53:10<2:27:26,  6.69s/it]
+ 27%|██▋       | 492/1813 [53:18<2:33:41,  6.98s/it]
+ 27%|██▋       | 493/1813 [53:24<2:24:10,  6.55s/it]
+ 27%|██▋       | 494/1813 [53:29<2:18:56,  6.32s/it]
+ 27%|██▋       | 495/1813 [53:36<2:23:40,  6.54s/it]
+ 27%|██▋       | 496/1813 [53:43<2:21:39,  6.45s/it]
+ 27%|██▋       | 497/1813 [53:48<2:14:13,  6.12s/it]
+ 27%|██▋       | 498/1813 [53:55<2:20:12,  6.40s/it]
+ 28%|██▊       | 499/1813 [54:01<2:18:06,  6.31s/it]
+ 28%|██▊       | 500/1813 [54:10<2:36:09,  7.14s/it]
+                                                    
+{'loss': '1.139', 'grad_norm': '1.2', 'learning_rate': '1.815e-05', 'epoch': '0.2759', 'num_input_tokens_seen': 3341536, 'train_runtime': '3251', 'train_tokens_per_second': '1028'}
+
+ 28%|██▊       | 500/1813 [54:10<2:36:09,  7.14s/it]
+ 28%|██▊       | 501/1813 [54:16<2:28:46,  6.80s/it]
+ 28%|██▊       | 502/1813 [54:22<2:20:03,  6.41s/it]
+ 28%|██▊       | 503/1813 [54:27<2:09:38,  5.94s/it]
+ 28%|██▊       | 504/1813 [54:33<2:10:05,  5.96s/it]
+ 28%|██▊       | 505/1813 [54:40<2:17:28,  6.31s/it]
+ 28%|██▊       | 506/1813 [54:46<2:15:21,  6.21s/it]
+ 28%|██▊       | 507/1813 [54:53<2:21:47,  6.51s/it]
+ 28%|██▊       | 508/1813 [54:58<2:11:40,  6.05s/it]
+ 28%|██▊       | 509/1813 [55:03<2:05:59,  5.80s/it]
+ 28%|██▊       | 510/1813 [55:09<2:04:17,  5.72s/it]
+                                                    
+{'loss': '0.9639', 'grad_norm': '2.259', 'learning_rate': '1.803e-05', 'epoch': '0.2814', 'num_input_tokens_seen': 3402712, 'train_runtime': '3309', 'train_tokens_per_second': '1028'}
+
+ 28%|██▊       | 510/1813 [55:09<2:04:17,  5.72s/it]
+ 28%|██▊       | 511/1813 [55:14<2:03:40,  5.70s/it]
+ 28%|██▊       | 512/1813 [55:21<2:11:56,  6.09s/it]
+ 28%|██▊       | 513/1813 [55:29<2:19:12,  6.42s/it]
+ 28%|██▊       | 514/1813 [55:35<2:21:17,  6.53s/it]
+ 28%|██▊       | 515/1813 [55:40<2:11:39,  6.09s/it]
+ 28%|██▊       | 516/1813 [55:46<2:08:16,  5.93s/it]
+ 29%|██▊       | 517/1813 [55:52<2:11:58,  6.11s/it]
+ 29%|██▊       | 518/1813 [55:59<2:11:57,  6.11s/it]
+ 29%|██▊       | 519/1813 [56:05<2:12:01,  6.12s/it]
+ 29%|██▊       | 520/1813 [56:12<2:16:34,  6.34s/it]
+                                                    
+{'loss': '1.006', 'grad_norm': '1.615', 'learning_rate': '1.791e-05', 'epoch': '0.2869', 'num_input_tokens_seen': 3469624, 'train_runtime': '3372', 'train_tokens_per_second': '1029'}
+
+ 29%|██▊       | 520/1813 [56:12<2:16:34,  6.34s/it]
+ 29%|██▊       | 521/1813 [56:16<2:01:13,  5.63s/it]
+ 29%|██▉       | 522/1813 [56:21<2:02:25,  5.69s/it]
+ 29%|██▉       | 523/1813 [56:27<2:05:02,  5.82s/it]
+ 29%|██▉       | 524/1813 [56:34<2:08:27,  5.98s/it]
+ 29%|██▉       | 525/1813 [56:39<2:02:07,  5.69s/it]
+ 29%|██▉       | 526/1813 [56:46<2:08:29,  5.99s/it]
+ 29%|██▉       | 527/1813 [56:52<2:12:18,  6.17s/it]
+ 29%|██▉       | 528/1813 [56:59<2:16:55,  6.39s/it]
+ 29%|██▉       | 529/1813 [57:06<2:20:39,  6.57s/it]
+ 29%|██▉       | 530/1813 [57:12<2:15:38,  6.34s/it]
+                                                    
+{'loss': '0.9794', 'grad_norm': '2.356', 'learning_rate': '1.779e-05', 'epoch': '0.2924', 'num_input_tokens_seen': 3534904, 'train_runtime': '3432', 'train_tokens_per_second': '1030'}
+
+ 29%|██▉       | 530/1813 [57:12<2:15:38,  6.34s/it]
+ 29%|██▉       | 531/1813 [57:20<2:27:10,  6.89s/it]
+ 29%|██▉       | 532/1813 [57:26<2:23:01,  6.70s/it]
+ 29%|██▉       | 533/1813 [57:34<2:26:31,  6.87s/it]
+ 29%|██▉       | 534/1813 [57:41<2:28:46,  6.98s/it]
+ 30%|██▉       | 535/1813 [57:45<2:12:17,  6.21s/it]
+ 30%|██▉       | 536/1813 [57:50<2:05:17,  5.89s/it]
+ 30%|██▉       | 537/1813 [57:57<2:09:34,  6.09s/it]
+ 30%|██▉       | 538/1813 [58:02<2:05:06,  5.89s/it]
+ 30%|██▉       | 539/1813 [58:08<2:06:39,  5.96s/it]
+ 30%|██▉       | 540/1813 [58:14<2:06:56,  5.98s/it]
+                                                    
+{'loss': '0.9965', 'grad_norm': '1.462', 'learning_rate': '1.766e-05', 'epoch': '0.2979', 'num_input_tokens_seen': 3603488, 'train_runtime': '3495', 'train_tokens_per_second': '1031'}
+
+ 30%|██▉       | 540/1813 [58:14<2:06:56,  5.98s/it]
+ 30%|██▉       | 541/1813 [58:20<2:01:12,  5.72s/it]
+ 30%|██▉       | 542/1813 [58:25<2:00:50,  5.70s/it]
+ 30%|██▉       | 543/1813 [58:32<2:09:52,  6.14s/it]
+ 30%|███       | 544/1813 [58:41<2:23:08,  6.77s/it]
+ 30%|███       | 545/1813 [58:47<2:22:16,  6.73s/it]
+ 30%|███       | 546/1813 [58:54<2:20:44,  6.66s/it]
+ 30%|███       | 547/1813 [59:01<2:26:38,  6.95s/it]
+ 30%|███       | 548/1813 [59:09<2:29:37,  7.10s/it]
+ 30%|███       | 549/1813 [59:14<2:19:38,  6.63s/it]
+ 30%|███       | 550/1813 [59:20<2:15:15,  6.43s/it]
+                                                    
+{'loss': '0.9516', 'grad_norm': '2.155', 'learning_rate': '1.753e-05', 'epoch': '0.3034', 'num_input_tokens_seen': 3676576, 'train_runtime': '3561', 'train_tokens_per_second': '1033'}
+
+ 30%|███       | 550/1813 [59:20<2:15:15,  6.43s/it]
+ 30%|███       | 551/1813 [59:26<2:12:57,  6.32s/it]
+ 30%|███       | 552/1813 [59:32<2:07:17,  6.06s/it]
+ 31%|███       | 553/1813 [59:37<1:59:33,  5.69s/it]
+ 31%|███       | 554/1813 [59:43<2:05:55,  6.00s/it]
+ 31%|███       | 555/1813 [59:48<1:59:02,  5.68s/it]
+ 31%|███       | 556/1813 [59:54<2:01:11,  5.78s/it]
+ 31%|███       | 557/1813 [1:00:01<2:07:25,  6.09s/it]
+ 31%|███       | 558/1813 [1:00:07<2:03:53,  5.92s/it]
+ 31%|███       | 559/1813 [1:00:12<1:57:47,  5.64s/it]
+ 31%|███       | 560/1813 [1:00:18<2:01:45,  5.83s/it]
+                                                      
+{'loss': '1.011', 'grad_norm': '1.079', 'learning_rate': '1.74e-05', 'epoch': '0.309', 'num_input_tokens_seen': 3738056, 'train_runtime': '3618', 'train_tokens_per_second': '1033'}
+
+ 31%|███       | 560/1813 [1:00:18<2:01:45,  5.83s/it]
+ 31%|███       | 561/1813 [1:00:24<2:00:32,  5.78s/it]
+ 31%|███       | 562/1813 [1:00:30<2:04:56,  5.99s/it]
+ 31%|███       | 563/1813 [1:00:37<2:11:17,  6.30s/it]
+ 31%|███       | 564/1813 [1:00:43<2:10:20,  6.26s/it]
+ 31%|███       | 565/1813 [1:00:49<2:06:49,  6.10s/it]
+ 31%|███       | 566/1813 [1:00:56<2:09:23,  6.23s/it]
+ 31%|███▏      | 567/1813 [1:01:01<2:05:20,  6.04s/it]
+ 31%|███▏      | 568/1813 [1:01:08<2:10:47,  6.30s/it]
+ 31%|███▏      | 569/1813 [1:01:16<2:18:29,  6.68s/it]
+ 31%|███▏      | 570/1813 [1:01:23<2:22:42,  6.89s/it]
+                                                      
+{'loss': '1.011', 'grad_norm': '1.204', 'learning_rate': '1.726e-05', 'epoch': '0.3145', 'num_input_tokens_seen': 3808208, 'train_runtime': '3683', 'train_tokens_per_second': '1034'}
+
+ 31%|███▏      | 570/1813 [1:01:23<2:22:42,  6.89s/it]
+ 31%|███▏      | 571/1813 [1:01:30<2:21:46,  6.85s/it]
+ 32%|███▏      | 572/1813 [1:01:36<2:16:10,  6.58s/it]
+ 32%|███▏      | 573/1813 [1:01:41<2:07:22,  6.16s/it]
+ 32%|███▏      | 574/1813 [1:01:47<2:04:14,  6.02s/it]
+ 32%|███▏      | 575/1813 [1:01:53<2:07:30,  6.18s/it]
+ 32%|███▏      | 576/1813 [1:01:58<1:56:56,  5.67s/it]
+ 32%|███▏      | 577/1813 [1:02:04<2:02:27,  5.94s/it]
+ 32%|███▏      | 578/1813 [1:02:10<2:02:49,  5.97s/it]
+ 32%|███▏      | 579/1813 [1:02:16<2:03:42,  6.01s/it]
+ 32%|███▏      | 580/1813 [1:02:23<2:05:29,  6.11s/it]
+                                                      
+{'loss': '1.012', 'grad_norm': '1.724', 'learning_rate': '1.713e-05', 'epoch': '0.32', 'num_input_tokens_seen': 3872328, 'train_runtime': '3743', 'train_tokens_per_second': '1035'}
+
+ 32%|███▏      | 580/1813 [1:02:23<2:05:29,  6.11s/it]
+ 32%|███▏      | 581/1813 [1:02:29<2:04:02,  6.04s/it]
+ 32%|███▏      | 582/1813 [1:02:34<2:00:57,  5.90s/it]
+ 32%|███▏      | 583/1813 [1:02:40<2:01:35,  5.93s/it]
+ 32%|███▏      | 584/1813 [1:02:47<2:04:41,  6.09s/it]
+ 32%|███▏      | 585/1813 [1:02:53<2:04:55,  6.10s/it]
+ 32%|███▏      | 586/1813 [1:02:59<2:06:42,  6.20s/it]
+ 32%|███▏      | 587/1813 [1:03:05<2:05:13,  6.13s/it]
+ 32%|███▏      | 588/1813 [1:03:11<2:04:31,  6.10s/it]
+ 32%|███▏      | 589/1813 [1:03:17<2:04:54,  6.12s/it]
+ 33%|███▎      | 590/1813 [1:03:23<2:04:51,  6.13s/it]
+                                                      
+{'loss': '0.9094', 'grad_norm': '1.827', 'learning_rate': '1.699e-05', 'epoch': '0.3255', 'num_input_tokens_seen': 3936024, 'train_runtime': '3804', 'train_tokens_per_second': '1035'}
+
+ 33%|███▎      | 590/1813 [1:03:23<2:04:51,  6.13s/it]
+ 33%|███▎      | 591/1813 [1:03:29<2:00:15,  5.90s/it]
+ 33%|███▎      | 592/1813 [1:03:35<1:59:59,  5.90s/it]
+ 33%|███▎      | 593/1813 [1:03:41<2:03:33,  6.08s/it]
+ 33%|███▎      | 594/1813 [1:03:47<2:03:47,  6.09s/it]
+ 33%|███▎      | 595/1813 [1:03:53<2:01:19,  5.98s/it]
+ 33%|███▎      | 596/1813 [1:03:59<2:00:35,  5.95s/it]
+ 33%|███▎      | 597/1813 [1:04:05<1:58:24,  5.84s/it]
+ 33%|███▎      | 598/1813 [1:04:10<1:58:43,  5.86s/it]
+ 33%|███▎      | 599/1813 [1:04:17<2:01:25,  6.00s/it]
+ 33%|███▎      | 600/1813 [1:04:24<2:07:26,  6.30s/it]
+                                                      
+{'loss': '0.9824', 'grad_norm': '1.284', 'learning_rate': '1.684e-05', 'epoch': '0.331', 'num_input_tokens_seen': 3999176, 'train_runtime': '3864', 'train_tokens_per_second': '1035'}
+
+ 33%|███▎      | 600/1813 [1:04:24<2:07:26,  6.30s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.79it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.25it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.34it/s]
+
+  1%|          | 9/1000 [00:01<02:32,  6.48it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.10it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.02it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 31/1000 [00:04<03:00,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.11it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.59it/s]
+
+  4%|▍         | 40/1000 [00:05<02:04,  7.72it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.08it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.30it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.04it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.37it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.73it/s]
+
+  5%|▌         | 51/1000 [00:07<01:36,  9.88it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.89it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.71it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.28it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.76it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.36it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+
+  8%|▊         | 82/1000 [00:10<01:40,  9.10it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  7.98it/s]
+
+  8%|▊         | 85/1000 [00:11<02:15,  6.77it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.94it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.37it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:37,  5.77it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.51it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.35it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.62it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.32it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.45it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.21it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.04it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.53it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.81it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.21it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.07it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:58,  7.42it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.13it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.62it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.88it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25, 10.05it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.91it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.04it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.39it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.98it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.85it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.43it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.50it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.71it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.13it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.64it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.39it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.66it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.77it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.64it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.94it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.68it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.16it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.51it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.38it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.03it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.96it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.48it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.99it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.37it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.73it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.44it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.32it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.71it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.70it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.43it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.74it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.78it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.06it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.11it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.15it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.96it/s]
+
+ 31%|███       | 307/1000 [00:41<01:41,  6.80it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.80it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.26it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.58it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.18it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.05it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.83it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.04it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.89it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.42it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:48,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.58it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.42it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.02it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:27,  7.16it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:24,  7.42it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:19,  7.87it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:42,  6.13it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:54,  5.43it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:41,  6.14it/s]
+
+ 38%|███▊      | 379/1000 [00:51<01:16,  8.12it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.16it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.54it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.06it/s]
+
+ 39%|███▊      | 387/1000 [00:51<01:20,  7.57it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.31it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.83it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.48it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  8.98it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.52it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.85it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.51it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  6.99it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:26,  6.78it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.23it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.43it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.48it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.68it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.83it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.53it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.01it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.81it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.52it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.09it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.19it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:09,  8.05it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.22it/s]
+
+ 44%|████▍     | 445/1000 [00:59<01:14,  7.48it/s]
+
+ 45%|████▍     | 447/1000 [00:59<01:00,  9.14it/s]
+
+ 45%|████▍     | 449/1000 [01:00<01:12,  7.55it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:24,  6.48it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:36,  5.70it/s]
+
+ 45%|████▌     | 452/1000 [01:01<01:45,  5.21it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:18,  6.98it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:04,  8.40it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:10,  7.74it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.25it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.81it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.60it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.73it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:29,  5.99it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.81it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.30it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.56it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.50it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.32it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.97it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.98it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.44it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.44it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.55it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:07,  7.12it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.04it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+
+ 52%|█████▏    | 524/1000 [01:11<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.72it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.52it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.96it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.84it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.03it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.36it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:55,  7.86it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.69it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:06,  6.52it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:07,  6.40it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.42it/s]
+
+ 57%|█████▋    | 572/1000 [01:18<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.81it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.72it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.59it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:02,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.68it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.48it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.74it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.80it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.62it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.73it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.08it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.12it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.41it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.60it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.17it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.84it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:39,  9.38it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.70it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.05it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.93it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.15it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.94it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:54,  6.38it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.22it/s]
+
+ 65%|██████▌   | 654/1000 [01:29<00:45,  7.60it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:39,  8.82it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.06it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  5.98it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.47it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.76it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.56it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.62it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.92it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.91it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.88it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.27it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+
+ 70%|██████▉   | 696/1000 [01:35<00:47,  6.35it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.06it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.62it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.68it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.83it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.34it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.05it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.12it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.62it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.73it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.29it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.69it/s]
+
+ 75%|███████▌  | 751/1000 [01:42<00:40,  6.18it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.25it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.39it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.31it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.71it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.80it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.13it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.79it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.96it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.38it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.52it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.99it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.85it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.63it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.83it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.16it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.22it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.24it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.32it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.83it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.03it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.10it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.44it/s]
+
+ 84%|████████▍ | 840/1000 [01:54<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.15it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.78it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.56it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.88it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.48it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.76it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.64it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.72it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.33it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.27it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.28it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.84it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.58it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.53it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.94it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.76it/s]
+
+ 89%|████████▉ | 892/1000 [02:00<00:10, 10.75it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.84it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.71it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.06it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.06it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.33it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.59it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:03<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.38it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.11it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.59it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.06it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.64it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.71it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.84it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.42it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.41it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.41it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.72it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.77it/s]
+
+ 96%|█████████▌| 962/1000 [02:10<00:06,  6.02it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.47it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.42it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.33it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:04,  7.68it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.14it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  7.92it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.10it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.71it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.71it/s]
+
+ 98%|█████████▊| 980/1000 [02:12<00:02,  8.41it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.43it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.36it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.75it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.14it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.99it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.52it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.04it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.8467', 'eval_runtime': '135.5', 'eval_samples_per_second': '7.382', 'eval_steps_per_second': '7.382', 'epoch': '0.331', 'num_input_tokens_seen': 3999176}
+
+ 33%|███▎      | 600/1813 [1:06:39<2:07:26,  6.30s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 33%|███▎      | 601/1813 [1:06:44<15:38:20, 46.45s/it]
+ 33%|███▎      | 602/1813 [1:06:49<11:27:16, 34.05s/it]
+ 33%|███▎      | 603/1813 [1:06:55<8:34:39, 25.52s/it] 
+ 33%|███▎      | 604/1813 [1:07:01<6:37:43, 19.74s/it]
+ 33%|███▎      | 605/1813 [1:07:09<5:26:17, 16.21s/it]
+ 33%|███▎      | 606/1813 [1:07:16<4:32:39, 13.55s/it]
+ 33%|███▎      | 607/1813 [1:07:22<3:48:11, 11.35s/it]
+ 34%|███▎      | 608/1813 [1:07:29<3:17:33,  9.84s/it]
+ 34%|███▎      | 609/1813 [1:07:35<2:55:39,  8.75s/it]
+ 34%|███▎      | 610/1813 [1:07:41<2:40:44,  8.02s/it]
+                                                      
+{'loss': '1.023', 'grad_norm': '1.087', 'learning_rate': '1.67e-05', 'epoch': '0.3366', 'num_input_tokens_seen': 4066376, 'train_runtime': '4062', 'train_tokens_per_second': '1001'}
+
+ 34%|███▎      | 610/1813 [1:07:41<2:40:44,  8.02s/it]
+ 34%|███▎      | 611/1813 [1:07:47<2:25:12,  7.25s/it]
+ 34%|███▍      | 612/1813 [1:07:51<2:06:55,  6.34s/it]
+ 34%|███▍      | 613/1813 [1:07:57<2:03:21,  6.17s/it]
+ 34%|███▍      | 614/1813 [1:08:03<2:02:40,  6.14s/it]
+ 34%|███▍      | 615/1813 [1:08:08<1:57:27,  5.88s/it]
+ 34%|███▍      | 616/1813 [1:08:13<1:53:05,  5.67s/it]
+ 34%|███▍      | 617/1813 [1:08:19<1:52:02,  5.62s/it]
+ 34%|███▍      | 618/1813 [1:08:26<2:02:01,  6.13s/it]
+ 34%|███▍      | 619/1813 [1:08:31<1:56:42,  5.86s/it]
+ 34%|███▍      | 620/1813 [1:08:38<2:01:51,  6.13s/it]
+                                                      
+{'loss': '0.9029', 'grad_norm': '1.72', 'learning_rate': '1.655e-05', 'epoch': '0.3421', 'num_input_tokens_seen': 4123976, 'train_runtime': '4119', 'train_tokens_per_second': '1001'}
+
+ 34%|███▍      | 620/1813 [1:08:38<2:01:51,  6.13s/it]
+ 34%|███▍      | 621/1813 [1:08:45<2:07:13,  6.40s/it]
+ 34%|███▍      | 622/1813 [1:08:53<2:14:27,  6.77s/it]
+ 34%|███▍      | 623/1813 [1:08:58<2:03:42,  6.24s/it]
+ 34%|███▍      | 624/1813 [1:09:04<2:05:32,  6.34s/it]
+ 34%|███▍      | 625/1813 [1:09:09<1:58:47,  6.00s/it]
+ 35%|███▍      | 626/1813 [1:09:14<1:50:38,  5.59s/it]
+ 35%|███▍      | 627/1813 [1:09:20<1:54:49,  5.81s/it]
+ 35%|███▍      | 628/1813 [1:09:27<1:57:56,  5.97s/it]
+ 35%|███▍      | 629/1813 [1:09:35<2:11:47,  6.68s/it]
+ 35%|███▍      | 630/1813 [1:09:42<2:12:53,  6.74s/it]
+                                                      
+{'loss': '0.9844', 'grad_norm': '1.11', 'learning_rate': '1.64e-05', 'epoch': '0.3476', 'num_input_tokens_seen': 4191992, 'train_runtime': '4182', 'train_tokens_per_second': '1002'}
+
+ 35%|███▍      | 630/1813 [1:09:42<2:12:53,  6.74s/it]
+ 35%|███▍      | 631/1813 [1:09:49<2:11:41,  6.68s/it]
+ 35%|███▍      | 632/1813 [1:09:55<2:07:36,  6.48s/it]
+ 35%|███▍      | 633/1813 [1:10:00<2:03:56,  6.30s/it]
+ 35%|███▍      | 634/1813 [1:10:08<2:10:30,  6.64s/it]
+ 35%|███▌      | 635/1813 [1:10:15<2:11:49,  6.71s/it]
+ 35%|███▌      | 636/1813 [1:10:20<2:03:41,  6.31s/it]
+ 35%|███▌      | 637/1813 [1:10:27<2:08:20,  6.55s/it]
+ 35%|███▌      | 638/1813 [1:10:33<2:02:18,  6.25s/it]
+ 35%|███▌      | 639/1813 [1:10:38<1:56:02,  5.93s/it]
+ 35%|███▌      | 640/1813 [1:10:44<1:57:26,  6.01s/it]
+                                                      
+{'loss': '1.003', 'grad_norm': '1.587', 'learning_rate': '1.625e-05', 'epoch': '0.3531', 'num_input_tokens_seen': 4257760, 'train_runtime': '4245', 'train_tokens_per_second': '1003'}
+
+ 35%|███▌      | 640/1813 [1:10:44<1:57:26,  6.01s/it]
+ 35%|███▌      | 641/1813 [1:10:50<1:54:30,  5.86s/it]
+ 35%|███▌      | 642/1813 [1:10:57<2:02:30,  6.28s/it]
+ 35%|███▌      | 643/1813 [1:11:03<2:04:03,  6.36s/it]
+ 36%|███▌      | 644/1813 [1:11:09<1:59:53,  6.15s/it]
+ 36%|███▌      | 645/1813 [1:11:15<1:58:56,  6.11s/it]
+ 36%|███▌      | 646/1813 [1:11:21<1:57:02,  6.02s/it]
+ 36%|███▌      | 647/1813 [1:11:27<1:58:13,  6.08s/it]
+ 36%|███▌      | 648/1813 [1:11:34<2:03:22,  6.35s/it]
+ 36%|███▌      | 649/1813 [1:11:40<1:59:07,  6.14s/it]
+ 36%|███▌      | 650/1813 [1:11:45<1:52:34,  5.81s/it]
+                                                      
+{'loss': '1.01', 'grad_norm': '1.915', 'learning_rate': '1.61e-05', 'epoch': '0.3586', 'num_input_tokens_seen': 4322136, 'train_runtime': '4305', 'train_tokens_per_second': '1004'}
+
+ 36%|███▌      | 650/1813 [1:11:45<1:52:34,  5.81s/it]
+ 36%|███▌      | 651/1813 [1:11:51<1:56:22,  6.01s/it]
+ 36%|███▌      | 652/1813 [1:11:58<2:01:00,  6.25s/it]
+ 36%|███▌      | 653/1813 [1:12:04<1:55:59,  6.00s/it]
+ 36%|███▌      | 654/1813 [1:12:09<1:51:49,  5.79s/it]
+ 36%|███▌      | 655/1813 [1:12:15<1:54:18,  5.92s/it]
+ 36%|███▌      | 656/1813 [1:12:20<1:50:46,  5.74s/it]
+ 36%|███▌      | 657/1813 [1:12:25<1:46:35,  5.53s/it]
+ 36%|███▋      | 658/1813 [1:12:32<1:51:27,  5.79s/it]
+ 36%|███▋      | 659/1813 [1:12:38<1:54:54,  5.97s/it]
+ 36%|███▋      | 660/1813 [1:12:44<1:52:28,  5.85s/it]
+                                                      
+{'loss': '0.917', 'grad_norm': '1.534', 'learning_rate': '1.594e-05', 'epoch': '0.3641', 'num_input_tokens_seen': 4385688, 'train_runtime': '4364', 'train_tokens_per_second': '1005'}
+
+ 36%|███▋      | 660/1813 [1:12:44<1:52:28,  5.85s/it]
+ 36%|███▋      | 661/1813 [1:12:50<1:52:56,  5.88s/it]
+ 37%|███▋      | 662/1813 [1:12:56<1:54:41,  5.98s/it]
+ 37%|███▋      | 663/1813 [1:13:03<1:59:30,  6.24s/it]
+ 37%|███▋      | 664/1813 [1:13:09<2:00:40,  6.30s/it]
+ 37%|███▋      | 665/1813 [1:13:14<1:54:13,  5.97s/it]
+ 37%|███▋      | 666/1813 [1:13:20<1:52:06,  5.86s/it]
+ 37%|███▋      | 667/1813 [1:13:27<1:57:56,  6.17s/it]
+ 37%|███▋      | 668/1813 [1:13:33<1:59:49,  6.28s/it]
+ 37%|███▋      | 669/1813 [1:13:40<1:58:41,  6.23s/it]
+ 37%|███▋      | 670/1813 [1:13:45<1:55:36,  6.07s/it]
+                                                      
+{'loss': '1.023', 'grad_norm': '1.738', 'learning_rate': '1.578e-05', 'epoch': '0.3697', 'num_input_tokens_seen': 4449456, 'train_runtime': '4426', 'train_tokens_per_second': '1005'}
+
+ 37%|███▋      | 670/1813 [1:13:45<1:55:36,  6.07s/it]
+ 37%|███▋      | 671/1813 [1:13:52<1:57:32,  6.18s/it]
+ 37%|███▋      | 672/1813 [1:13:58<2:00:40,  6.35s/it]
+ 37%|███▋      | 673/1813 [1:14:05<2:00:40,  6.35s/it]
+ 37%|███▋      | 674/1813 [1:14:12<2:02:32,  6.46s/it]
+ 37%|███▋      | 675/1813 [1:14:18<2:00:02,  6.33s/it]
+ 37%|███▋      | 676/1813 [1:14:23<1:56:40,  6.16s/it]
+ 37%|███▋      | 677/1813 [1:14:31<2:07:48,  6.75s/it]
+ 37%|███▋      | 678/1813 [1:14:37<1:59:30,  6.32s/it]
+ 37%|███▋      | 679/1813 [1:14:42<1:53:04,  5.98s/it]
+ 38%|███▊      | 680/1813 [1:14:48<1:52:18,  5.95s/it]
+                                                      
+{'loss': '0.988', 'grad_norm': '1.553', 'learning_rate': '1.562e-05', 'epoch': '0.3752', 'num_input_tokens_seen': 4514984, 'train_runtime': '4488', 'train_tokens_per_second': '1006'}
+
+ 38%|███▊      | 680/1813 [1:14:48<1:52:18,  5.95s/it]
+ 38%|███▊      | 681/1813 [1:14:54<1:55:53,  6.14s/it]
+ 38%|███▊      | 682/1813 [1:15:00<1:50:49,  5.88s/it]
+ 38%|███▊      | 683/1813 [1:15:06<1:51:49,  5.94s/it]
+ 38%|███▊      | 684/1813 [1:15:12<1:55:55,  6.16s/it]
+ 38%|███▊      | 685/1813 [1:15:20<2:04:48,  6.64s/it]
+ 38%|███▊      | 686/1813 [1:15:27<2:03:08,  6.56s/it]
+ 38%|███▊      | 687/1813 [1:15:34<2:09:09,  6.88s/it]
+ 38%|███▊      | 688/1813 [1:15:39<1:55:38,  6.17s/it]
+ 38%|███▊      | 689/1813 [1:15:47<2:09:25,  6.91s/it]
+ 38%|███▊      | 690/1813 [1:15:54<2:10:32,  6.97s/it]
+                                                      
+{'loss': '1.017', 'grad_norm': '1.636', 'learning_rate': '1.546e-05', 'epoch': '0.3807', 'num_input_tokens_seen': 4587840, 'train_runtime': '4555', 'train_tokens_per_second': '1007'}
+
+ 38%|███▊      | 690/1813 [1:15:54<2:10:32,  6.97s/it]
+ 38%|███▊      | 691/1813 [1:16:00<2:01:49,  6.51s/it]
+ 38%|███▊      | 692/1813 [1:16:06<1:58:16,  6.33s/it]
+ 38%|███▊      | 693/1813 [1:16:11<1:52:47,  6.04s/it]
+ 38%|███▊      | 694/1813 [1:16:17<1:53:48,  6.10s/it]
+ 38%|███▊      | 695/1813 [1:16:23<1:52:48,  6.05s/it]
+ 38%|███▊      | 696/1813 [1:16:31<2:02:35,  6.58s/it]
+ 38%|███▊      | 697/1813 [1:16:36<1:52:06,  6.03s/it]
+ 38%|███▊      | 698/1813 [1:16:41<1:47:52,  5.80s/it]
+ 39%|███▊      | 699/1813 [1:16:48<1:54:12,  6.15s/it]
+ 39%|███▊      | 700/1813 [1:16:55<1:55:47,  6.24s/it]
+                                                      
+{'loss': '1.028', 'grad_norm': '1.479', 'learning_rate': '1.53e-05', 'epoch': '0.3862', 'num_input_tokens_seen': 4653312, 'train_runtime': '4615', 'train_tokens_per_second': '1008'}
+
+ 39%|███▊      | 700/1813 [1:16:55<1:55:47,  6.24s/it]
+ 39%|███▊      | 701/1813 [1:17:02<2:01:28,  6.55s/it]
+ 39%|███▊      | 702/1813 [1:17:08<1:59:33,  6.46s/it]
+ 39%|███▉      | 703/1813 [1:17:16<2:07:20,  6.88s/it]
+ 39%|███▉      | 704/1813 [1:17:22<2:00:25,  6.52s/it]
+ 39%|███▉      | 705/1813 [1:17:29<2:02:44,  6.65s/it]
+ 39%|███▉      | 706/1813 [1:17:35<2:01:42,  6.60s/it]
+ 39%|███▉      | 707/1813 [1:17:45<2:18:08,  7.49s/it]
+ 39%|███▉      | 708/1813 [1:17:52<2:17:31,  7.47s/it]
+ 39%|███▉      | 709/1813 [1:17:59<2:13:10,  7.24s/it]
+ 39%|███▉      | 710/1813 [1:18:05<2:05:36,  6.83s/it]
+                                                      
+{'loss': '0.9821', 'grad_norm': '1.278', 'learning_rate': '1.513e-05', 'epoch': '0.3917', 'num_input_tokens_seen': 4732080, 'train_runtime': '4685', 'train_tokens_per_second': '1010'}
+
+ 39%|███▉      | 710/1813 [1:18:05<2:05:36,  6.83s/it]
+ 39%|███▉      | 711/1813 [1:18:12<2:06:06,  6.87s/it]
+ 39%|███▉      | 712/1813 [1:18:18<2:04:54,  6.81s/it]
+ 39%|███▉      | 713/1813 [1:18:24<1:59:01,  6.49s/it]
+ 39%|███▉      | 714/1813 [1:18:30<1:57:18,  6.40s/it]
+ 39%|███▉      | 715/1813 [1:18:37<1:59:11,  6.51s/it]
+ 39%|███▉      | 716/1813 [1:18:44<2:01:40,  6.65s/it]
+ 40%|███▉      | 717/1813 [1:18:50<1:59:09,  6.52s/it]
+ 40%|███▉      | 718/1813 [1:18:57<1:59:20,  6.54s/it]
+ 40%|███▉      | 719/1813 [1:19:02<1:52:11,  6.15s/it]
+ 40%|███▉      | 720/1813 [1:19:08<1:52:21,  6.17s/it]
+                                                      
+{'loss': '1.039', 'grad_norm': '1.578', 'learning_rate': '1.496e-05', 'epoch': '0.3972', 'num_input_tokens_seen': 4799552, 'train_runtime': '4749', 'train_tokens_per_second': '1011'}
+
+ 40%|███▉      | 720/1813 [1:19:08<1:52:21,  6.17s/it]
+ 40%|███▉      | 721/1813 [1:19:15<1:54:07,  6.27s/it]
+ 40%|███▉      | 722/1813 [1:19:20<1:47:26,  5.91s/it]
+ 40%|███▉      | 723/1813 [1:19:25<1:43:20,  5.69s/it]
+ 40%|███▉      | 724/1813 [1:19:33<1:55:45,  6.38s/it]
+ 40%|███▉      | 725/1813 [1:19:42<2:07:42,  7.04s/it]
+ 40%|████      | 726/1813 [1:19:48<2:05:39,  6.94s/it]
+ 40%|████      | 727/1813 [1:19:54<2:01:03,  6.69s/it]
+ 40%|████      | 728/1813 [1:19:59<1:51:55,  6.19s/it]
+ 40%|████      | 729/1813 [1:20:05<1:48:54,  6.03s/it]
+ 40%|████      | 730/1813 [1:20:11<1:45:49,  5.86s/it]
+                                                      
+{'loss': '1.008', 'grad_norm': '1.976', 'learning_rate': '1.479e-05', 'epoch': '0.4028', 'num_input_tokens_seen': 4866864, 'train_runtime': '4811', 'train_tokens_per_second': '1012'}
+
+ 40%|████      | 730/1813 [1:20:11<1:45:49,  5.86s/it]
+ 40%|████      | 731/1813 [1:20:17<1:51:33,  6.19s/it]
+ 40%|████      | 732/1813 [1:20:24<1:51:24,  6.18s/it]
+ 40%|████      | 733/1813 [1:20:31<1:57:34,  6.53s/it]
+ 40%|████      | 734/1813 [1:20:38<1:59:33,  6.65s/it]
+ 41%|████      | 735/1813 [1:20:45<1:59:26,  6.65s/it]
+ 41%|████      | 736/1813 [1:20:50<1:54:37,  6.39s/it]
+ 41%|████      | 737/1813 [1:20:56<1:51:00,  6.19s/it]
+ 41%|████      | 738/1813 [1:21:01<1:46:02,  5.92s/it]
+ 41%|████      | 739/1813 [1:21:07<1:46:34,  5.95s/it]
+ 41%|████      | 740/1813 [1:21:14<1:51:02,  6.21s/it]
+                                                      
+{'loss': '0.9408', 'grad_norm': '1.178', 'learning_rate': '1.462e-05', 'epoch': '0.4083', 'num_input_tokens_seen': 4936352, 'train_runtime': '4875', 'train_tokens_per_second': '1013'}
+
+ 41%|████      | 740/1813 [1:21:14<1:51:02,  6.21s/it]
+ 41%|████      | 741/1813 [1:21:21<1:53:41,  6.36s/it]
+ 41%|████      | 742/1813 [1:21:28<1:59:56,  6.72s/it]
+ 41%|████      | 743/1813 [1:21:36<2:03:46,  6.94s/it]
+ 41%|████      | 744/1813 [1:21:43<2:02:40,  6.89s/it]
+ 41%|████      | 745/1813 [1:21:50<2:03:14,  6.92s/it]
+ 41%|████      | 746/1813 [1:21:56<1:58:14,  6.65s/it]
+ 41%|████      | 747/1813 [1:22:04<2:05:01,  7.04s/it]
+ 41%|████▏     | 748/1813 [1:22:09<1:56:30,  6.56s/it]
+ 41%|████▏     | 749/1813 [1:22:15<1:55:07,  6.49s/it]
+ 41%|████▏     | 750/1813 [1:22:21<1:51:16,  6.28s/it]
+                                                      
+{'loss': '1.02', 'grad_norm': '1.266', 'learning_rate': '1.445e-05', 'epoch': '0.4138', 'num_input_tokens_seen': 5011064, 'train_runtime': '4942', 'train_tokens_per_second': '1014'}
+
+ 41%|████▏     | 750/1813 [1:22:21<1:51:16,  6.28s/it]
+ 41%|████▏     | 751/1813 [1:22:27<1:49:15,  6.17s/it]
+ 41%|████▏     | 752/1813 [1:22:33<1:47:18,  6.07s/it]
+ 42%|████▏     | 753/1813 [1:22:39<1:49:36,  6.20s/it]
+ 42%|████▏     | 754/1813 [1:22:48<2:01:30,  6.88s/it]
+ 42%|████▏     | 755/1813 [1:22:53<1:50:44,  6.28s/it]
+ 42%|████▏     | 756/1813 [1:22:58<1:46:16,  6.03s/it]
+ 42%|████▏     | 757/1813 [1:23:04<1:46:08,  6.03s/it]
+ 42%|████▏     | 758/1813 [1:23:11<1:49:11,  6.21s/it]
+ 42%|████▏     | 759/1813 [1:23:18<1:53:17,  6.45s/it]
+ 42%|████▏     | 760/1813 [1:23:24<1:51:14,  6.34s/it]
+                                                      
+{'loss': '1.021', 'grad_norm': '1.743', 'learning_rate': '1.428e-05', 'epoch': '0.4193', 'num_input_tokens_seen': 5078960, 'train_runtime': '5005', 'train_tokens_per_second': '1015'}
+
+ 42%|████▏     | 760/1813 [1:23:24<1:51:14,  6.34s/it]
+ 42%|████▏     | 761/1813 [1:23:31<1:52:40,  6.43s/it]
+ 42%|████▏     | 762/1813 [1:23:37<1:53:12,  6.46s/it]
+ 42%|████▏     | 763/1813 [1:23:44<1:56:15,  6.64s/it]
+ 42%|████▏     | 764/1813 [1:23:51<1:56:54,  6.69s/it]
+ 42%|████▏     | 765/1813 [1:23:58<2:00:24,  6.89s/it]
+ 42%|████▏     | 766/1813 [1:24:05<1:57:51,  6.75s/it]
+ 42%|████▏     | 767/1813 [1:24:10<1:51:05,  6.37s/it]
+ 42%|████▏     | 768/1813 [1:24:15<1:40:56,  5.80s/it]
+ 42%|████▏     | 769/1813 [1:24:22<1:46:26,  6.12s/it]
+ 42%|████▏     | 770/1813 [1:24:28<1:46:32,  6.13s/it]
+                                                      
+{'loss': '0.9688', 'grad_norm': '1.458', 'learning_rate': '1.41e-05', 'epoch': '0.4248', 'num_input_tokens_seen': 5148504, 'train_runtime': '5068', 'train_tokens_per_second': '1016'}
+
+ 42%|████▏     | 770/1813 [1:24:28<1:46:32,  6.13s/it]
+ 43%|████▎     | 771/1813 [1:24:34<1:47:57,  6.22s/it]
+ 43%|████▎     | 772/1813 [1:24:41<1:53:14,  6.53s/it]
+ 43%|████▎     | 773/1813 [1:24:47<1:49:57,  6.34s/it]
+ 43%|████▎     | 774/1813 [1:24:53<1:44:24,  6.03s/it]
+ 43%|████▎     | 775/1813 [1:24:59<1:45:31,  6.10s/it]
+ 43%|████▎     | 776/1813 [1:25:06<1:50:02,  6.37s/it]
+ 43%|████▎     | 777/1813 [1:25:14<1:57:13,  6.79s/it]
+ 43%|████▎     | 778/1813 [1:25:19<1:47:52,  6.25s/it]
+ 43%|████▎     | 779/1813 [1:25:24<1:41:41,  5.90s/it]
+ 43%|████▎     | 780/1813 [1:25:30<1:45:21,  6.12s/it]
+                                                      
+{'loss': '0.9133', 'grad_norm': '1.6', 'learning_rate': '1.392e-05', 'epoch': '0.4303', 'num_input_tokens_seen': 5215784, 'train_runtime': '5131', 'train_tokens_per_second': '1017'}
+
+ 43%|████▎     | 780/1813 [1:25:30<1:45:21,  6.12s/it]
+ 43%|████▎     | 781/1813 [1:25:37<1:49:18,  6.36s/it]
+ 43%|████▎     | 782/1813 [1:25:46<1:58:56,  6.92s/it]
+ 43%|████▎     | 783/1813 [1:25:53<2:00:10,  7.00s/it]
+ 43%|████▎     | 784/1813 [1:25:59<1:55:01,  6.71s/it]
+ 43%|████▎     | 785/1813 [1:26:06<1:56:05,  6.78s/it]
+ 43%|████▎     | 786/1813 [1:26:13<1:57:22,  6.86s/it]
+ 43%|████▎     | 787/1813 [1:26:18<1:48:20,  6.34s/it]
+ 43%|████▎     | 788/1813 [1:26:23<1:43:08,  6.04s/it]
+ 44%|████▎     | 789/1813 [1:26:28<1:39:01,  5.80s/it]
+ 44%|████▎     | 790/1813 [1:26:34<1:39:24,  5.83s/it]
+                                                      
+{'loss': '1.037', 'grad_norm': '2.184', 'learning_rate': '1.375e-05', 'epoch': '0.4359', 'num_input_tokens_seen': 5285160, 'train_runtime': '5195', 'train_tokens_per_second': '1017'}
+
+ 44%|████▎     | 790/1813 [1:26:34<1:39:24,  5.83s/it]
+ 44%|████▎     | 791/1813 [1:26:41<1:41:03,  5.93s/it]
+ 44%|████▎     | 792/1813 [1:26:48<1:47:04,  6.29s/it]
+ 44%|████▎     | 793/1813 [1:26:55<1:51:40,  6.57s/it]
+ 44%|████▍     | 794/1813 [1:27:01<1:50:58,  6.53s/it]
+ 44%|████▍     | 795/1813 [1:27:07<1:47:17,  6.32s/it]
+ 44%|████▍     | 796/1813 [1:27:14<1:49:31,  6.46s/it]
+ 44%|████▍     | 797/1813 [1:27:22<1:55:07,  6.80s/it]
+ 44%|████▍     | 798/1813 [1:27:28<1:51:24,  6.59s/it]
+ 44%|████▍     | 799/1813 [1:27:34<1:47:55,  6.39s/it]
+ 44%|████▍     | 800/1813 [1:27:39<1:45:23,  6.24s/it]
+                                                      
+{'loss': '0.9735', 'grad_norm': '1.417', 'learning_rate': '1.357e-05', 'epoch': '0.4414', 'num_input_tokens_seen': 5357720, 'train_runtime': '5260', 'train_tokens_per_second': '1019'}
+
+ 44%|████▍     | 800/1813 [1:27:39<1:45:23,  6.24s/it]
+ 44%|████▍     | 801/1813 [1:27:45<1:42:18,  6.07s/it]
+ 44%|████▍     | 802/1813 [1:27:51<1:43:21,  6.13s/it]
+ 44%|████▍     | 803/1813 [1:27:59<1:52:55,  6.71s/it]
+ 44%|████▍     | 804/1813 [1:28:06<1:51:15,  6.62s/it]
+ 44%|████▍     | 805/1813 [1:28:12<1:48:18,  6.45s/it]
+ 44%|████▍     | 806/1813 [1:28:18<1:47:12,  6.39s/it]
+ 45%|████▍     | 807/1813 [1:28:24<1:42:49,  6.13s/it]
+ 45%|████▍     | 808/1813 [1:28:32<1:51:23,  6.65s/it]
+ 45%|████▍     | 809/1813 [1:28:38<1:51:40,  6.67s/it]
+ 45%|████▍     | 810/1813 [1:28:46<1:56:28,  6.97s/it]
+                                                      
+{'loss': '0.9848', 'grad_norm': '10.47', 'learning_rate': '1.338e-05', 'epoch': '0.4469', 'num_input_tokens_seen': 5430288, 'train_runtime': '5326', 'train_tokens_per_second': '1019'}
+
+ 45%|████▍     | 810/1813 [1:28:46<1:56:28,  6.97s/it]
+ 45%|████▍     | 811/1813 [1:28:52<1:53:50,  6.82s/it]
+ 45%|████▍     | 812/1813 [1:28:58<1:47:29,  6.44s/it]
+ 45%|████▍     | 813/1813 [1:29:03<1:42:18,  6.14s/it]
+ 45%|████▍     | 814/1813 [1:29:09<1:41:19,  6.09s/it]
+ 45%|████▍     | 815/1813 [1:29:14<1:34:24,  5.68s/it]
+ 45%|████▌     | 816/1813 [1:29:21<1:40:06,  6.02s/it]
+ 45%|████▌     | 817/1813 [1:29:27<1:38:32,  5.94s/it]
+ 45%|████▌     | 818/1813 [1:29:34<1:45:15,  6.35s/it]
+ 45%|████▌     | 819/1813 [1:29:41<1:46:18,  6.42s/it]
+ 45%|████▌     | 820/1813 [1:29:48<1:51:14,  6.72s/it]
+                                                      
+{'loss': '0.9449', 'grad_norm': '1.706', 'learning_rate': '1.32e-05', 'epoch': '0.4524', 'num_input_tokens_seen': 5497552, 'train_runtime': '5388', 'train_tokens_per_second': '1020'}
+
+ 45%|████▌     | 820/1813 [1:29:48<1:51:14,  6.72s/it]
+ 45%|████▌     | 821/1813 [1:29:54<1:49:47,  6.64s/it]
+ 45%|████▌     | 822/1813 [1:30:00<1:42:04,  6.18s/it]
+ 45%|████▌     | 823/1813 [1:30:04<1:35:08,  5.77s/it]
+ 45%|████▌     | 824/1813 [1:30:11<1:37:27,  5.91s/it]
+ 46%|████▌     | 825/1813 [1:30:17<1:39:12,  6.02s/it]
+ 46%|████▌     | 826/1813 [1:30:24<1:43:14,  6.28s/it]
+ 46%|████▌     | 827/1813 [1:30:30<1:40:38,  6.12s/it]
+ 46%|████▌     | 828/1813 [1:30:35<1:39:26,  6.06s/it]
+ 46%|████▌     | 829/1813 [1:30:41<1:37:00,  5.92s/it]
+ 46%|████▌     | 830/1813 [1:30:48<1:44:07,  6.36s/it]
+                                                      
+{'loss': '0.8369', 'grad_norm': '1.43', 'learning_rate': '1.302e-05', 'epoch': '0.4579', 'num_input_tokens_seen': 5563144, 'train_runtime': '5449', 'train_tokens_per_second': '1021'}
+
+ 46%|████▌     | 830/1813 [1:30:48<1:44:07,  6.36s/it]
+ 46%|████▌     | 831/1813 [1:30:55<1:43:36,  6.33s/it]
+ 46%|████▌     | 832/1813 [1:31:02<1:49:41,  6.71s/it]
+ 46%|████▌     | 833/1813 [1:31:09<1:47:41,  6.59s/it]
+ 46%|████▌     | 834/1813 [1:31:15<1:47:59,  6.62s/it]
+ 46%|████▌     | 835/1813 [1:31:21<1:45:17,  6.46s/it]
+ 46%|████▌     | 836/1813 [1:31:28<1:46:49,  6.56s/it]
+ 46%|████▌     | 837/1813 [1:31:36<1:52:21,  6.91s/it]
+ 46%|████▌     | 838/1813 [1:31:42<1:46:11,  6.53s/it]
+ 46%|████▋     | 839/1813 [1:31:48<1:43:50,  6.40s/it]
+ 46%|████▋     | 840/1813 [1:31:53<1:40:24,  6.19s/it]
+                                                      
+{'loss': '0.9962', 'grad_norm': '1.677', 'learning_rate': '1.284e-05', 'epoch': '0.4634', 'num_input_tokens_seen': 5634104, 'train_runtime': '5514', 'train_tokens_per_second': '1022'}
+
+ 46%|████▋     | 840/1813 [1:31:53<1:40:24,  6.19s/it]
+ 46%|████▋     | 841/1813 [1:31:59<1:39:19,  6.13s/it]
+ 46%|████▋     | 842/1813 [1:32:06<1:42:40,  6.34s/it]
+ 46%|████▋     | 843/1813 [1:32:12<1:41:24,  6.27s/it]
+ 47%|████▋     | 844/1813 [1:32:18<1:38:43,  6.11s/it]
+ 47%|████▋     | 845/1813 [1:32:24<1:40:08,  6.21s/it]
+ 47%|████▋     | 846/1813 [1:32:31<1:42:24,  6.35s/it]
+ 47%|████▋     | 847/1813 [1:32:38<1:45:11,  6.53s/it]
+ 47%|████▋     | 848/1813 [1:32:45<1:48:11,  6.73s/it]
+ 47%|████▋     | 849/1813 [1:32:50<1:40:00,  6.23s/it]
+ 47%|████▋     | 850/1813 [1:32:59<1:49:38,  6.83s/it]
+                                                      
+{'loss': '0.9583', 'grad_norm': '1.351', 'learning_rate': '1.265e-05', 'epoch': '0.469', 'num_input_tokens_seen': 5704744, 'train_runtime': '5579', 'train_tokens_per_second': '1023'}
+
+ 47%|████▋     | 850/1813 [1:32:59<1:49:38,  6.83s/it]
+ 47%|████▋     | 851/1813 [1:33:05<1:48:58,  6.80s/it]
+ 47%|████▋     | 852/1813 [1:33:10<1:39:26,  6.21s/it]
+ 47%|████▋     | 853/1813 [1:33:16<1:39:43,  6.23s/it]
+ 47%|████▋     | 854/1813 [1:33:23<1:42:02,  6.38s/it]
+ 47%|████▋     | 855/1813 [1:33:29<1:40:04,  6.27s/it]
+ 47%|████▋     | 856/1813 [1:33:35<1:38:48,  6.19s/it]
+ 47%|████▋     | 857/1813 [1:33:40<1:34:04,  5.90s/it]
+ 47%|████▋     | 858/1813 [1:33:44<1:24:06,  5.28s/it]
+ 47%|████▋     | 859/1813 [1:33:50<1:28:01,  5.54s/it]
+ 47%|████▋     | 860/1813 [1:33:57<1:32:18,  5.81s/it]
+                                                      
+{'loss': '0.9785', 'grad_norm': '1.439', 'learning_rate': '1.246e-05', 'epoch': '0.4745', 'num_input_tokens_seen': 5766424, 'train_runtime': '5637', 'train_tokens_per_second': '1023'}
+
+ 47%|████▋     | 860/1813 [1:33:57<1:32:18,  5.81s/it]
+ 47%|████▋     | 861/1813 [1:34:02<1:28:07,  5.55s/it]
+ 48%|████▊     | 862/1813 [1:34:08<1:33:41,  5.91s/it]
+ 48%|████▊     | 863/1813 [1:34:15<1:35:03,  6.00s/it]
+ 48%|████▊     | 864/1813 [1:34:19<1:26:18,  5.46s/it]
+ 48%|████▊     | 865/1813 [1:34:26<1:32:49,  5.87s/it]
+ 48%|████▊     | 866/1813 [1:34:32<1:32:29,  5.86s/it]
+ 48%|████▊     | 867/1813 [1:34:37<1:30:16,  5.73s/it]
+ 48%|████▊     | 868/1813 [1:34:42<1:28:22,  5.61s/it]
+ 48%|████▊     | 869/1813 [1:34:50<1:35:50,  6.09s/it]
+ 48%|████▊     | 870/1813 [1:34:57<1:43:13,  6.57s/it]
+                                                      
+{'loss': '0.9743', 'grad_norm': '1.203', 'learning_rate': '1.228e-05', 'epoch': '0.48', 'num_input_tokens_seen': 5831352, 'train_runtime': '5698', 'train_tokens_per_second': '1023'}
+
+ 48%|████▊     | 870/1813 [1:34:57<1:43:13,  6.57s/it]
+ 48%|████▊     | 871/1813 [1:35:02<1:37:07,  6.19s/it]
+ 48%|████▊     | 872/1813 [1:35:10<1:41:52,  6.50s/it]
+ 48%|████▊     | 873/1813 [1:35:16<1:42:56,  6.57s/it]
+ 48%|████▊     | 874/1813 [1:35:24<1:47:25,  6.86s/it]
+ 48%|████▊     | 875/1813 [1:35:32<1:52:12,  7.18s/it]
+ 48%|████▊     | 876/1813 [1:35:39<1:50:48,  7.10s/it]
+ 48%|████▊     | 877/1813 [1:35:47<1:55:09,  7.38s/it]
+ 48%|████▊     | 878/1813 [1:35:53<1:51:06,  7.13s/it]
+ 48%|████▊     | 879/1813 [1:36:01<1:53:32,  7.29s/it]
+ 49%|████▊     | 880/1813 [1:36:06<1:41:05,  6.50s/it]
+                                                      
+{'loss': '1', 'grad_norm': '2.331', 'learning_rate': '1.209e-05', 'epoch': '0.4855', 'num_input_tokens_seen': 5907208, 'train_runtime': '5766', 'train_tokens_per_second': '1024'}
+
+ 49%|████▊     | 880/1813 [1:36:06<1:41:05,  6.50s/it]
+ 49%|████▊     | 881/1813 [1:36:11<1:35:31,  6.15s/it]
+ 49%|████▊     | 882/1813 [1:36:17<1:35:52,  6.18s/it]
+ 49%|████▊     | 883/1813 [1:36:24<1:38:29,  6.35s/it]
+ 49%|████▉     | 884/1813 [1:36:30<1:36:48,  6.25s/it]
+ 49%|████▉     | 885/1813 [1:36:36<1:35:11,  6.15s/it]
+ 49%|████▉     | 886/1813 [1:36:42<1:35:43,  6.20s/it]
+ 49%|████▉     | 887/1813 [1:36:50<1:43:21,  6.70s/it]
+ 49%|████▉     | 888/1813 [1:36:57<1:43:04,  6.69s/it]
+ 49%|████▉     | 889/1813 [1:37:04<1:45:15,  6.84s/it]
+ 49%|████▉     | 890/1813 [1:37:10<1:43:14,  6.71s/it]
+                                                      
+{'loss': '0.9674', 'grad_norm': '1.28', 'learning_rate': '1.19e-05', 'epoch': '0.491', 'num_input_tokens_seen': 5978048, 'train_runtime': '5831', 'train_tokens_per_second': '1025'}
+
+ 49%|████▉     | 890/1813 [1:37:10<1:43:14,  6.71s/it]
+ 49%|████▉     | 891/1813 [1:37:17<1:40:45,  6.56s/it]
+ 49%|████▉     | 892/1813 [1:37:24<1:45:17,  6.86s/it]
+ 49%|████▉     | 893/1813 [1:37:30<1:42:32,  6.69s/it]
+ 49%|████▉     | 894/1813 [1:37:36<1:38:35,  6.44s/it]
+ 49%|████▉     | 895/1813 [1:37:44<1:42:52,  6.72s/it]
+ 49%|████▉     | 896/1813 [1:37:49<1:37:39,  6.39s/it]
+ 49%|████▉     | 897/1813 [1:37:55<1:35:43,  6.27s/it]
+ 50%|████▉     | 898/1813 [1:38:03<1:39:58,  6.56s/it]
+ 50%|████▉     | 899/1813 [1:38:10<1:43:42,  6.81s/it]
+ 50%|████▉     | 900/1813 [1:38:18<1:47:27,  7.06s/it]
+                                                      
+{'loss': '1.014', 'grad_norm': '2.989', 'learning_rate': '1.171e-05', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856, 'train_runtime': '5898', 'train_tokens_per_second': '1026'}
+
+ 50%|████▉     | 900/1813 [1:38:18<1:47:27,  7.06s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.80it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.22it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.32it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.48it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.08it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.99it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.82it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.96it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  6.00it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.98it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:06,  7.66it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.34it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.04it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.94it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.74it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.30it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.35it/s]
+
+  6%|▋         | 63/1000 [00:08<01:35,  9.77it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.27it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.19it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.03it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.37it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  6.99it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:37,  5.77it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.51it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.73it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.80it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.30it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.43it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.20it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.04it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.53it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.80it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.21it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.79it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.13it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.91it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.07it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.06it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.41it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.98it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.63it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.68it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.77it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.82it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.15it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.73it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.44it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.80it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.57it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.51it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.71it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.79it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.40it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.12it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.39it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.76it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.62it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.76it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.62it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.91it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.50it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.67it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.73it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.96it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.36it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.01it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.46it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.26it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.68it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:25,  8.73it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.05it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.85it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.75it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.78it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.01it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.20it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:37,  7.14it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.77it/s]
+
+ 30%|███       | 304/1000 [00:40<01:25,  8.09it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.15it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.96it/s]
+
+ 31%|███       | 307/1000 [00:41<01:41,  6.80it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.80it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.17it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.24it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:07, 10.22it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.55it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.05it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.63it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.54it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.94it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.56it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.41it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.77it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.90it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.18it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.17it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 389/1000 [00:52<01:21,  7.53it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.66it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:23,  7.32it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  8.99it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:23,  7.24it/s]
+
+ 40%|███▉      | 396/1000 [00:53<01:18,  7.67it/s]
+
+ 40%|███▉      | 398/1000 [00:53<01:25,  7.03it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:42,  5.87it/s]
+
+ 40%|████      | 400/1000 [00:53<01:52,  5.34it/s]
+
+ 40%|████      | 402/1000 [00:54<01:44,  5.71it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.19it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.55it/s]
+
+ 41%|████      | 407/1000 [00:54<01:23,  7.07it/s]
+
+ 41%|████      | 408/1000 [00:55<01:36,  6.14it/s]
+
+ 41%|████      | 410/1000 [00:55<01:33,  6.29it/s]
+
+ 41%|████      | 411/1000 [00:55<01:44,  5.64it/s]
+
+ 41%|████      | 412/1000 [00:55<01:53,  5.19it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.79it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.31it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.49it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.52it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.68it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.92it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.96it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:58<01:12,  7.83it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.52it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 10.99it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.77it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.51it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.26it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.19it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.83it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.61it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.74it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.30it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.84it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.38it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.31it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.01it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.97it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.42it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.95it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.97it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.27it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.72it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.92it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.93it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.05it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:10,  6.48it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.69it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.50it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.85it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.04it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.74it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.55it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.58it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.17it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.81it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.40it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.40it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:49,  7.86it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.46it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.72it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.79it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.62it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.75it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.10it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.20it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.45it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.58it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.57it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.33it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.55it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.70it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.99it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.92it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.87it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.80it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.83it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.33it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.93it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.71it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.36it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.15it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.26it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.12it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.59it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.53it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.91it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.76it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.35it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.60it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.44it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.69it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.70it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.78it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.14it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.61it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  8.98it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.21it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.96it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:24,  7.84it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.18it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.98it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.22it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.24it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.31it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.15it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.21it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.04it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.81it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.79it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.73it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.76it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.52it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.15it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.07it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.21it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.76it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.61it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.66it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.30it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.26it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.27it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.79it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.64it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.88it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.61it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.79it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.85it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.71it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.87it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.05it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.76it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.36it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.16it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.60it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  7.90it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.02it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.28it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.03it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.34it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.61it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.79it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.38it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.16it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.40it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.43it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.19it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.79it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.81it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.05it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.37it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.8208', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.386', 'eval_steps_per_second': '7.386', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856}
+
+ 50%|████▉     | 900/1813 [1:40:33<1:47:27,  7.06s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+
+                                                   
+ 50%|████▉     | 901/1813 [1:40:39<12:02:07, 47.51s/it]
+ 50%|████▉     | 902/1813 [1:40:45<8:51:59, 35.04s/it] 
+ 50%|████▉     | 903/1813 [1:40:51<6:39:12, 26.32s/it]
+ 50%|████▉     | 904/1813 [1:40:57<5:06:18, 20.22s/it]
+ 50%|████▉     | 905/1813 [1:41:05<4:09:31, 16.49s/it]
+ 50%|████▉     | 906/1813 [1:41:10<3:17:47, 13.08s/it]
+ 50%|█████     | 907/1813 [1:41:15<2:38:11, 10.48s/it]
+ 50%|█████     | 908/1813 [1:41:20<2:16:35,  9.06s/it]
+ 50%|█████     | 909/1813 [1:41:27<2:05:45,  8.35s/it]
+ 50%|█████     | 910/1813 [1:41:33<1:56:26,  7.74s/it]
+                                                      
+{'loss': '0.9509', 'grad_norm': '1.882', 'learning_rate': '1.152e-05', 'epoch': '0.5021', 'num_input_tokens_seen': 6115160, 'train_runtime': '6094', 'train_tokens_per_second': '1003'}
+
+ 50%|█████     | 910/1813 [1:41:33<1:56:26,  7.74s/it]
+ 50%|█████     | 911/1813 [1:41:39<1:46:59,  7.12s/it]
+ 50%|█████     | 912/1813 [1:41:45<1:40:02,  6.66s/it]
+ 50%|█████     | 913/1813 [1:41:50<1:33:14,  6.22s/it]
+ 50%|█████     | 914/1813 [1:41:55<1:25:56,  5.74s/it]
+ 50%|█████     | 915/1813 [1:42:02<1:34:28,  6.31s/it]
+ 51%|█████     | 916/1813 [1:42:09<1:38:36,  6.60s/it]
+ 51%|█████     | 917/1813 [1:42:16<1:39:26,  6.66s/it]
+ 51%|█████     | 918/1813 [1:42:22<1:34:23,  6.33s/it]
+ 51%|█████     | 919/1813 [1:42:28<1:34:40,  6.35s/it]
+ 51%|█████     | 920/1813 [1:42:34<1:31:32,  6.15s/it]
+                                                      
+{'loss': '0.9941', 'grad_norm': '1.247', 'learning_rate': '1.133e-05', 'epoch': '0.5076', 'num_input_tokens_seen': 6178904, 'train_runtime': '6154', 'train_tokens_per_second': '1004'}
+
+ 51%|█████     | 920/1813 [1:42:34<1:31:32,  6.15s/it]
+ 51%|█████     | 921/1813 [1:42:40<1:29:34,  6.02s/it]
+ 51%|█████     | 922/1813 [1:42:45<1:27:20,  5.88s/it]
+ 51%|█████     | 923/1813 [1:42:51<1:26:09,  5.81s/it]
+ 51%|█████     | 924/1813 [1:42:58<1:30:44,  6.12s/it]
+ 51%|█████     | 925/1813 [1:43:03<1:29:08,  6.02s/it]
+ 51%|█████     | 926/1813 [1:43:10<1:31:36,  6.20s/it]
+ 51%|█████     | 927/1813 [1:43:16<1:30:30,  6.13s/it]
+ 51%|█████     | 928/1813 [1:43:22<1:27:54,  5.96s/it]
+ 51%|█████     | 929/1813 [1:43:28<1:27:52,  5.96s/it]
+ 51%|█████▏    | 930/1813 [1:43:34<1:31:47,  6.24s/it]
+                                                      
+{'loss': '0.9517', 'grad_norm': '1.573', 'learning_rate': '1.114e-05', 'epoch': '0.5131', 'num_input_tokens_seen': 6244360, 'train_runtime': '6215', 'train_tokens_per_second': '1005'}
+
+ 51%|█████▏    | 930/1813 [1:43:34<1:31:47,  6.24s/it]
+ 51%|█████▏    | 931/1813 [1:43:41<1:34:27,  6.43s/it]
+ 51%|█████▏    | 932/1813 [1:43:47<1:30:52,  6.19s/it]
+ 51%|█████▏    | 933/1813 [1:43:53<1:29:36,  6.11s/it]
+ 52%|█████▏    | 934/1813 [1:44:00<1:33:54,  6.41s/it]
+ 52%|█████▏    | 935/1813 [1:44:09<1:44:38,  7.15s/it]
+ 52%|█████▏    | 936/1813 [1:44:15<1:41:16,  6.93s/it]
+ 52%|█████▏    | 937/1813 [1:44:21<1:37:25,  6.67s/it]
+ 52%|█████▏    | 938/1813 [1:44:28<1:38:59,  6.79s/it]
+ 52%|█████▏    | 939/1813 [1:44:34<1:34:43,  6.50s/it]
+ 52%|█████▏    | 940/1813 [1:44:41<1:36:57,  6.66s/it]
+                                                      
+{'loss': '0.937', 'grad_norm': '1.949', 'learning_rate': '1.095e-05', 'epoch': '0.5186', 'num_input_tokens_seen': 6318968, 'train_runtime': '6282', 'train_tokens_per_second': '1006'}
+
+ 52%|█████▏    | 940/1813 [1:44:41<1:36:57,  6.66s/it]
+ 52%|█████▏    | 941/1813 [1:44:46<1:29:57,  6.19s/it]
+ 52%|█████▏    | 942/1813 [1:44:52<1:26:53,  5.99s/it]
+ 52%|█████▏    | 943/1813 [1:44:57<1:25:10,  5.87s/it]
+ 52%|█████▏    | 944/1813 [1:45:04<1:26:30,  5.97s/it]
+ 52%|█████▏    | 945/1813 [1:45:09<1:24:28,  5.84s/it]
+ 52%|█████▏    | 946/1813 [1:45:16<1:26:46,  6.01s/it]
+ 52%|█████▏    | 947/1813 [1:45:22<1:27:17,  6.05s/it]
+ 52%|█████▏    | 948/1813 [1:45:28<1:28:07,  6.11s/it]
+ 52%|█████▏    | 949/1813 [1:45:33<1:23:35,  5.81s/it]
+ 52%|█████▏    | 950/1813 [1:45:39<1:25:29,  5.94s/it]
+                                                      
+{'loss': '0.9808', 'grad_norm': '1.663', 'learning_rate': '1.076e-05', 'epoch': '0.5241', 'num_input_tokens_seen': 6379704, 'train_runtime': '6340', 'train_tokens_per_second': '1006'}
+
+ 52%|█████▏    | 950/1813 [1:45:39<1:25:29,  5.94s/it]
+ 52%|█████▏    | 951/1813 [1:45:45<1:22:57,  5.77s/it]
+ 53%|█████▎    | 952/1813 [1:45:50<1:21:59,  5.71s/it]
+ 53%|█████▎    | 953/1813 [1:45:56<1:23:15,  5.81s/it]
+ 53%|█████▎    | 954/1813 [1:46:02<1:23:44,  5.85s/it]
+ 53%|█████▎    | 955/1813 [1:46:08<1:24:19,  5.90s/it]
+ 53%|█████▎    | 956/1813 [1:46:14<1:24:24,  5.91s/it]
+ 53%|█████▎    | 957/1813 [1:46:20<1:24:08,  5.90s/it]
+ 53%|█████▎    | 958/1813 [1:46:25<1:20:13,  5.63s/it]
+ 53%|█████▎    | 959/1813 [1:46:31<1:21:37,  5.73s/it]
+ 53%|█████▎    | 960/1813 [1:46:39<1:32:21,  6.50s/it]
+                                                      
+{'loss': '0.9973', 'grad_norm': '1.302', 'learning_rate': '1.057e-05', 'epoch': '0.5297', 'num_input_tokens_seen': 6441656, 'train_runtime': '6400', 'train_tokens_per_second': '1007'}
+
+ 53%|█████▎    | 960/1813 [1:46:39<1:32:21,  6.50s/it]
+ 53%|█████▎    | 961/1813 [1:46:46<1:32:16,  6.50s/it]
+ 53%|█████▎    | 962/1813 [1:46:54<1:37:33,  6.88s/it]
+ 53%|█████▎    | 963/1813 [1:47:00<1:37:11,  6.86s/it]
+ 53%|█████▎    | 964/1813 [1:47:06<1:32:25,  6.53s/it]
+ 53%|█████▎    | 965/1813 [1:47:12<1:29:28,  6.33s/it]
+ 53%|█████▎    | 966/1813 [1:47:20<1:34:41,  6.71s/it]
+ 53%|█████▎    | 967/1813 [1:47:27<1:38:10,  6.96s/it]
+ 53%|█████▎    | 968/1813 [1:47:33<1:32:39,  6.58s/it]
+ 53%|█████▎    | 969/1813 [1:47:39<1:29:50,  6.39s/it]
+ 54%|█████▎    | 970/1813 [1:47:44<1:22:34,  5.88s/it]
+                                                      
+{'loss': '1.054', 'grad_norm': '2.214', 'learning_rate': '1.038e-05', 'epoch': '0.5352', 'num_input_tokens_seen': 6512320, 'train_runtime': '6464', 'train_tokens_per_second': '1007'}
+
+ 54%|█████▎    | 970/1813 [1:47:44<1:22:34,  5.88s/it]
+ 54%|█████▎    | 971/1813 [1:47:50<1:25:53,  6.12s/it]
+ 54%|█████▎    | 972/1813 [1:47:57<1:26:56,  6.20s/it]
+ 54%|█████▎    | 973/1813 [1:48:03<1:25:53,  6.14s/it]
+ 54%|█████▎    | 974/1813 [1:48:11<1:35:45,  6.85s/it]
+ 54%|█████▍    | 975/1813 [1:48:18<1:34:23,  6.76s/it]
+ 54%|█████▍    | 976/1813 [1:48:23<1:30:02,  6.46s/it]
+ 54%|█████▍    | 977/1813 [1:48:31<1:35:38,  6.86s/it]
+ 54%|█████▍    | 978/1813 [1:48:38<1:33:24,  6.71s/it]
+ 54%|█████▍    | 979/1813 [1:48:45<1:35:32,  6.87s/it]
+ 54%|█████▍    | 980/1813 [1:48:51<1:33:58,  6.77s/it]
+                                                      
+{'loss': '0.9629', 'grad_norm': '3.044', 'learning_rate': '1.019e-05', 'epoch': '0.5407', 'num_input_tokens_seen': 6586672, 'train_runtime': '6532', 'train_tokens_per_second': '1008'}
+
+ 54%|█████▍    | 980/1813 [1:48:51<1:33:58,  6.77s/it]
+ 54%|█████▍    | 981/1813 [1:48:58<1:33:24,  6.74s/it]
+ 54%|█████▍    | 982/1813 [1:49:03<1:26:39,  6.26s/it]
+ 54%|█████▍    | 983/1813 [1:49:08<1:22:18,  5.95s/it]
+ 54%|█████▍    | 984/1813 [1:49:14<1:20:31,  5.83s/it]
+ 54%|█████▍    | 985/1813 [1:49:21<1:24:17,  6.11s/it]
+ 54%|█████▍    | 986/1813 [1:49:28<1:29:41,  6.51s/it]
+ 54%|█████▍    | 987/1813 [1:49:34<1:27:44,  6.37s/it]
+ 54%|█████▍    | 988/1813 [1:49:40<1:23:55,  6.10s/it]
+ 55%|█████▍    | 989/1813 [1:49:45<1:19:17,  5.77s/it]
+ 55%|█████▍    | 990/1813 [1:49:53<1:28:03,  6.42s/it]
+                                                      
+{'loss': '0.952', 'grad_norm': '1.629', 'learning_rate': '9.995e-06', 'epoch': '0.5462', 'num_input_tokens_seen': 6653096, 'train_runtime': '6593', 'train_tokens_per_second': '1009'}
+
+ 55%|█████▍    | 990/1813 [1:49:53<1:28:03,  6.42s/it]
+ 55%|█████▍    | 991/1813 [1:49:58<1:22:42,  6.04s/it]
+ 55%|█████▍    | 992/1813 [1:50:05<1:28:01,  6.43s/it]
+ 55%|█████▍    | 993/1813 [1:50:11<1:26:09,  6.30s/it]
+ 55%|█████▍    | 994/1813 [1:50:20<1:34:53,  6.95s/it]
+ 55%|█████▍    | 995/1813 [1:50:26<1:30:49,  6.66s/it]
+ 55%|█████▍    | 996/1813 [1:50:32<1:29:37,  6.58s/it]
+ 55%|█████▍    | 997/1813 [1:50:38<1:26:08,  6.33s/it]
+ 55%|█████▌    | 998/1813 [1:50:43<1:21:22,  5.99s/it]
+ 55%|█████▌    | 999/1813 [1:50:49<1:20:29,  5.93s/it]
+ 55%|█████▌    | 1000/1813 [1:50:54<1:17:42,  5.73s/it]
+                                                       
+{'loss': '1.037', 'grad_norm': '1.386', 'learning_rate': '9.804e-06', 'epoch': '0.5517', 'num_input_tokens_seen': 6716784, 'train_runtime': '6654', 'train_tokens_per_second': '1009'}
+
+ 55%|█████▌    | 1000/1813 [1:50:54<1:17:42,  5.73s/it]
+ 55%|█████▌    | 1001/1813 [1:51:01<1:21:10,  6.00s/it]
+ 55%|█████▌    | 1002/1813 [1:51:07<1:22:37,  6.11s/it]
+ 55%|█████▌    | 1003/1813 [1:51:14<1:26:04,  6.38s/it]
+ 55%|█████▌    | 1004/1813 [1:51:19<1:21:15,  6.03s/it]
+ 55%|█████▌    | 1005/1813 [1:51:26<1:25:55,  6.38s/it]
+ 55%|█████▌    | 1006/1813 [1:51:33<1:27:13,  6.49s/it]
+ 56%|█████▌    | 1007/1813 [1:51:39<1:24:09,  6.27s/it]
+ 56%|█████▌    | 1008/1813 [1:51:45<1:21:56,  6.11s/it]
+ 56%|█████▌    | 1009/1813 [1:51:51<1:21:27,  6.08s/it]
+ 56%|█████▌    | 1010/1813 [1:51:56<1:19:25,  5.93s/it]
+                                                       
+{'loss': '0.988', 'grad_norm': '2.217', 'learning_rate': '9.613e-06', 'epoch': '0.5572', 'num_input_tokens_seen': 6783768, 'train_runtime': '6717', 'train_tokens_per_second': '1010'}
+
+ 56%|█████▌    | 1010/1813 [1:51:56<1:19:25,  5.93s/it]
+ 56%|█████▌    | 1011/1813 [1:52:03<1:23:01,  6.21s/it]
+ 56%|█████▌    | 1012/1813 [1:52:08<1:17:49,  5.83s/it]
+ 56%|█████▌    | 1013/1813 [1:52:13<1:13:25,  5.51s/it]
+ 56%|█████▌    | 1014/1813 [1:52:21<1:25:14,  6.40s/it]
+ 56%|█████▌    | 1015/1813 [1:52:27<1:21:05,  6.10s/it]
+ 56%|█████▌    | 1016/1813 [1:52:32<1:18:54,  5.94s/it]
+ 56%|█████▌    | 1017/1813 [1:52:39<1:23:22,  6.28s/it]
+ 56%|█████▌    | 1018/1813 [1:52:46<1:23:57,  6.34s/it]
+ 56%|█████▌    | 1019/1813 [1:52:51<1:19:30,  6.01s/it]
+ 56%|█████▋    | 1020/1813 [1:52:56<1:16:13,  5.77s/it]
+                                                       
+{'loss': '0.9085', 'grad_norm': '1.747', 'learning_rate': '9.421e-06', 'epoch': '0.5628', 'num_input_tokens_seen': 6844992, 'train_runtime': '6777', 'train_tokens_per_second': '1010'}
+
+ 56%|█████▋    | 1020/1813 [1:52:56<1:16:13,  5.77s/it]
+ 56%|█████▋    | 1021/1813 [1:53:03<1:21:23,  6.17s/it]
+ 56%|█████▋    | 1022/1813 [1:53:09<1:21:02,  6.15s/it]
+ 56%|█████▋    | 1023/1813 [1:53:17<1:26:06,  6.54s/it]
+ 56%|█████▋    | 1024/1813 [1:53:23<1:24:39,  6.44s/it]
+ 57%|█████▋    | 1025/1813 [1:53:32<1:35:01,  7.23s/it]
+ 57%|█████▋    | 1026/1813 [1:53:37<1:27:15,  6.65s/it]
+ 57%|█████▋    | 1027/1813 [1:53:44<1:24:54,  6.48s/it]
+ 57%|█████▋    | 1028/1813 [1:53:51<1:28:20,  6.75s/it]
+ 57%|█████▋    | 1029/1813 [1:53:57<1:27:16,  6.68s/it]
+ 57%|█████▋    | 1030/1813 [1:54:04<1:28:01,  6.75s/it]
+                                                       
+{'loss': '1.039', 'grad_norm': '1.212', 'learning_rate': '9.23e-06', 'epoch': '0.5683', 'num_input_tokens_seen': 6920856, 'train_runtime': '6845', 'train_tokens_per_second': '1011'}
+
+ 57%|█████▋    | 1030/1813 [1:54:04<1:28:01,  6.75s/it]
+ 57%|█████▋    | 1031/1813 [1:54:10<1:23:35,  6.41s/it]
+ 57%|█████▋    | 1032/1813 [1:54:17<1:25:03,  6.53s/it]
+ 57%|█████▋    | 1033/1813 [1:54:23<1:23:49,  6.45s/it]
+ 57%|█████▋    | 1034/1813 [1:54:29<1:21:27,  6.27s/it]
+ 57%|█████▋    | 1035/1813 [1:54:35<1:21:19,  6.27s/it]
+ 57%|█████▋    | 1036/1813 [1:54:42<1:22:43,  6.39s/it]
+ 57%|█████▋    | 1037/1813 [1:54:48<1:21:12,  6.28s/it]
+ 57%|█████▋    | 1038/1813 [1:54:54<1:20:25,  6.23s/it]
+ 57%|█████▋    | 1039/1813 [1:55:01<1:22:57,  6.43s/it]
+ 57%|█████▋    | 1040/1813 [1:55:08<1:24:41,  6.57s/it]
+                                                       
+{'loss': '0.9778', 'grad_norm': '1.28', 'learning_rate': '9.04e-06', 'epoch': '0.5738', 'num_input_tokens_seen': 6988568, 'train_runtime': '6908', 'train_tokens_per_second': '1012'}
+
+ 57%|█████▋    | 1040/1813 [1:55:08<1:24:41,  6.57s/it]
+ 57%|█████▋    | 1041/1813 [1:55:14<1:21:59,  6.37s/it]
+ 57%|█████▋    | 1042/1813 [1:55:20<1:22:24,  6.41s/it]
+ 58%|█████▊    | 1043/1813 [1:55:26<1:19:16,  6.18s/it]
+ 58%|█████▊    | 1044/1813 [1:55:31<1:15:57,  5.93s/it]
+ 58%|█████▊    | 1045/1813 [1:55:37<1:15:24,  5.89s/it]
+ 58%|█████▊    | 1046/1813 [1:55:43<1:15:55,  5.94s/it]
+ 58%|█████▊    | 1047/1813 [1:55:49<1:17:30,  6.07s/it]
+ 58%|█████▊    | 1048/1813 [1:55:56<1:21:20,  6.38s/it]
+ 58%|█████▊    | 1049/1813 [1:56:02<1:18:22,  6.15s/it]
+ 58%|█████▊    | 1050/1813 [1:56:08<1:16:59,  6.05s/it]
+                                                       
+{'loss': '0.9287', 'grad_norm': '1.954', 'learning_rate': '8.85e-06', 'epoch': '0.5793', 'num_input_tokens_seen': 7051808, 'train_runtime': '6968', 'train_tokens_per_second': '1012'}
+
+ 58%|█████▊    | 1050/1813 [1:56:08<1:16:59,  6.05s/it]
+ 58%|█████▊    | 1051/1813 [1:56:13<1:12:42,  5.73s/it]
+ 58%|█████▊    | 1052/1813 [1:56:19<1:12:25,  5.71s/it]
+ 58%|█████▊    | 1053/1813 [1:56:24<1:12:30,  5.72s/it]
+ 58%|█████▊    | 1054/1813 [1:56:30<1:12:12,  5.71s/it]
+ 58%|█████▊    | 1055/1813 [1:56:35<1:10:24,  5.57s/it]
+ 58%|█████▊    | 1056/1813 [1:56:41<1:11:04,  5.63s/it]
+ 58%|█████▊    | 1057/1813 [1:56:48<1:14:39,  5.92s/it]
+ 58%|█████▊    | 1058/1813 [1:56:53<1:10:47,  5.63s/it]
+ 58%|█████▊    | 1059/1813 [1:56:57<1:07:24,  5.36s/it]
+ 58%|█████▊    | 1060/1813 [1:57:04<1:13:00,  5.82s/it]
+                                                       
+{'loss': '0.9609', 'grad_norm': '1.211', 'learning_rate': '8.66e-06', 'epoch': '0.5848', 'num_input_tokens_seen': 7110624, 'train_runtime': '7025', 'train_tokens_per_second': '1012'}
+
+ 58%|█████▊    | 1060/1813 [1:57:04<1:13:00,  5.82s/it]
+ 59%|█████▊    | 1061/1813 [1:57:09<1:09:14,  5.52s/it]
+ 59%|█████▊    | 1062/1813 [1:57:14<1:07:07,  5.36s/it]
+ 59%|█████▊    | 1063/1813 [1:57:21<1:11:52,  5.75s/it]
+ 59%|█████▊    | 1064/1813 [1:57:28<1:18:15,  6.27s/it]
+ 59%|█████▊    | 1065/1813 [1:57:35<1:19:06,  6.34s/it]
+ 59%|█████▉    | 1066/1813 [1:57:42<1:23:05,  6.67s/it]
+ 59%|█████▉    | 1067/1813 [1:57:49<1:22:18,  6.62s/it]
+ 59%|█████▉    | 1068/1813 [1:57:54<1:16:16,  6.14s/it]
+ 59%|█████▉    | 1069/1813 [1:58:02<1:22:50,  6.68s/it]
+ 59%|█████▉    | 1070/1813 [1:58:07<1:18:19,  6.33s/it]
+                                                       
+{'loss': '0.9569', 'grad_norm': '1.719', 'learning_rate': '8.471e-06', 'epoch': '0.5903', 'num_input_tokens_seen': 7179728, 'train_runtime': '7088', 'train_tokens_per_second': '1013'}
+
+ 59%|█████▉    | 1070/1813 [1:58:07<1:18:19,  6.33s/it]
+ 59%|█████▉    | 1071/1813 [1:58:12<1:13:39,  5.96s/it]
+ 59%|█████▉    | 1072/1813 [1:58:18<1:15:00,  6.07s/it]
+ 59%|█████▉    | 1073/1813 [1:58:24<1:14:22,  6.03s/it]
+ 59%|█████▉    | 1074/1813 [1:58:30<1:12:44,  5.91s/it]
+ 59%|█████▉    | 1075/1813 [1:58:36<1:14:23,  6.05s/it]
+ 59%|█████▉    | 1076/1813 [1:58:43<1:15:43,  6.16s/it]
+ 59%|█████▉    | 1077/1813 [1:58:47<1:09:32,  5.67s/it]
+ 59%|█████▉    | 1078/1813 [1:58:52<1:07:16,  5.49s/it]
+ 60%|█████▉    | 1079/1813 [1:58:59<1:11:15,  5.82s/it]
+ 60%|█████▉    | 1080/1813 [1:59:06<1:14:04,  6.06s/it]
+                                                       
+{'loss': '0.9476', 'grad_norm': '1.592', 'learning_rate': '8.282e-06', 'epoch': '0.5959', 'num_input_tokens_seen': 7240984, 'train_runtime': '7146', 'train_tokens_per_second': '1013'}
+
+ 60%|█████▉    | 1080/1813 [1:59:06<1:14:04,  6.06s/it]
+ 60%|█████▉    | 1081/1813 [1:59:12<1:14:11,  6.08s/it]
+ 60%|█████▉    | 1082/1813 [1:59:18<1:12:59,  5.99s/it]
+ 60%|█████▉    | 1083/1813 [1:59:24<1:13:41,  6.06s/it]
+ 60%|█████▉    | 1084/1813 [1:59:30<1:13:48,  6.08s/it]
+ 60%|█████▉    | 1085/1813 [1:59:36<1:13:44,  6.08s/it]
+ 60%|█████▉    | 1086/1813 [1:59:40<1:07:10,  5.54s/it]
+ 60%|█████▉    | 1087/1813 [1:59:46<1:08:10,  5.63s/it]
+ 60%|██████    | 1088/1813 [1:59:52<1:09:08,  5.72s/it]
+ 60%|██████    | 1089/1813 [1:59:59<1:12:33,  6.01s/it]
+ 60%|██████    | 1090/1813 [2:00:05<1:12:52,  6.05s/it]
+                                                       
+{'loss': '0.9005', 'grad_norm': '2.157', 'learning_rate': '8.094e-06', 'epoch': '0.6014', 'num_input_tokens_seen': 7301824, 'train_runtime': '7205', 'train_tokens_per_second': '1013'}
+
+ 60%|██████    | 1090/1813 [2:00:05<1:12:52,  6.05s/it]
+ 60%|██████    | 1091/1813 [2:00:11<1:12:33,  6.03s/it]
+ 60%|██████    | 1092/1813 [2:00:17<1:12:29,  6.03s/it]
+ 60%|██████    | 1093/1813 [2:00:23<1:11:15,  5.94s/it]
+ 60%|██████    | 1094/1813 [2:00:29<1:13:13,  6.11s/it]
+ 60%|██████    | 1095/1813 [2:00:35<1:13:40,  6.16s/it]
+ 60%|██████    | 1096/1813 [2:00:41<1:11:27,  5.98s/it]
+ 61%|██████    | 1097/1813 [2:00:47<1:11:17,  5.97s/it]
+ 61%|██████    | 1098/1813 [2:00:54<1:14:54,  6.29s/it]
+ 61%|██████    | 1099/1813 [2:01:00<1:14:05,  6.23s/it]
+ 61%|██████    | 1100/1813 [2:01:08<1:19:44,  6.71s/it]
+                                                       
+{'loss': '0.9341', 'grad_norm': '1.789', 'learning_rate': '7.907e-06', 'epoch': '0.6069', 'num_input_tokens_seen': 7369528, 'train_runtime': '7268', 'train_tokens_per_second': '1014'}
+
+ 61%|██████    | 1100/1813 [2:01:08<1:19:44,  6.71s/it]
+ 61%|██████    | 1101/1813 [2:01:15<1:19:20,  6.69s/it]
+ 61%|██████    | 1102/1813 [2:01:22<1:20:46,  6.82s/it]
+ 61%|██████    | 1103/1813 [2:01:27<1:16:58,  6.50s/it]
+ 61%|██████    | 1104/1813 [2:01:35<1:20:38,  6.82s/it]
+ 61%|██████    | 1105/1813 [2:01:40<1:14:15,  6.29s/it]
+ 61%|██████    | 1106/1813 [2:01:46<1:12:32,  6.16s/it]
+ 61%|██████    | 1107/1813 [2:01:54<1:20:50,  6.87s/it]
+ 61%|██████    | 1108/1813 [2:02:00<1:16:04,  6.47s/it]
+ 61%|██████    | 1109/1813 [2:02:06<1:15:58,  6.47s/it]
+ 61%|██████    | 1110/1813 [2:02:14<1:20:14,  6.85s/it]
+                                                       
+{'loss': '0.9734', 'grad_norm': '1.8', 'learning_rate': '7.72e-06', 'epoch': '0.6124', 'num_input_tokens_seen': 7441040, 'train_runtime': '7335', 'train_tokens_per_second': '1015'}
+
+ 61%|██████    | 1110/1813 [2:02:14<1:20:14,  6.85s/it]
+ 61%|██████▏   | 1111/1813 [2:02:19<1:12:52,  6.23s/it]
+ 61%|██████▏   | 1112/1813 [2:02:25<1:11:33,  6.12s/it]
+ 61%|██████▏   | 1113/1813 [2:02:31<1:12:39,  6.23s/it]
+ 61%|██████▏   | 1114/1813 [2:02:38<1:16:00,  6.52s/it]
+ 62%|██████▏   | 1115/1813 [2:02:45<1:16:52,  6.61s/it]
+ 62%|██████▏   | 1116/1813 [2:02:51<1:15:21,  6.49s/it]
+ 62%|██████▏   | 1117/1813 [2:02:57<1:12:35,  6.26s/it]
+ 62%|██████▏   | 1118/1813 [2:03:02<1:08:41,  5.93s/it]
+ 62%|██████▏   | 1119/1813 [2:03:09<1:10:11,  6.07s/it]
+ 62%|██████▏   | 1120/1813 [2:03:16<1:12:21,  6.27s/it]
+                                                       
+{'loss': '0.9502', 'grad_norm': '16.13', 'learning_rate': '7.535e-06', 'epoch': '0.6179', 'num_input_tokens_seen': 7504976, 'train_runtime': '7396', 'train_tokens_per_second': '1015'}
+
+ 62%|██████▏   | 1120/1813 [2:03:16<1:12:21,  6.27s/it]
+ 62%|██████▏   | 1121/1813 [2:03:23<1:18:03,  6.77s/it]
+ 62%|██████▏   | 1122/1813 [2:03:28<1:11:44,  6.23s/it]
+ 62%|██████▏   | 1123/1813 [2:03:36<1:16:38,  6.66s/it]
+ 62%|██████▏   | 1124/1813 [2:03:42<1:12:56,  6.35s/it]
+ 62%|██████▏   | 1125/1813 [2:03:49<1:14:39,  6.51s/it]
+ 62%|██████▏   | 1126/1813 [2:03:55<1:13:30,  6.42s/it]
+ 62%|██████▏   | 1127/1813 [2:04:00<1:10:22,  6.16s/it]
+ 62%|██████▏   | 1128/1813 [2:04:06<1:07:14,  5.89s/it]
+ 62%|██████▏   | 1129/1813 [2:04:11<1:05:00,  5.70s/it]
+ 62%|██████▏   | 1130/1813 [2:04:16<1:03:17,  5.56s/it]
+                                                       
+{'loss': '0.9412', 'grad_norm': '1.27', 'learning_rate': '7.35e-06', 'epoch': '0.6234', 'num_input_tokens_seen': 7568576, 'train_runtime': '7457', 'train_tokens_per_second': '1015'}
+
+ 62%|██████▏   | 1130/1813 [2:04:16<1:03:17,  5.56s/it]
+ 62%|██████▏   | 1131/1813 [2:04:22<1:04:12,  5.65s/it]
+ 62%|██████▏   | 1132/1813 [2:04:28<1:05:42,  5.79s/it]
+ 62%|██████▏   | 1133/1813 [2:04:35<1:07:56,  5.99s/it]
+ 63%|██████▎   | 1134/1813 [2:04:42<1:14:16,  6.56s/it]
+ 63%|██████▎   | 1135/1813 [2:04:49<1:14:56,  6.63s/it]
+ 63%|██████▎   | 1136/1813 [2:04:56<1:16:28,  6.78s/it]
+ 63%|██████▎   | 1137/1813 [2:05:03<1:16:18,  6.77s/it]
+ 63%|██████▎   | 1138/1813 [2:05:10<1:17:34,  6.89s/it]
+ 63%|██████▎   | 1139/1813 [2:05:16<1:12:50,  6.48s/it]
+ 63%|██████▎   | 1140/1813 [2:05:22<1:10:01,  6.24s/it]
+                                                       
+{'loss': '1.002', 'grad_norm': '1.568', 'learning_rate': '7.167e-06', 'epoch': '0.629', 'num_input_tokens_seen': 7639168, 'train_runtime': '7522', 'train_tokens_per_second': '1016'}
+
+ 63%|██████▎   | 1140/1813 [2:05:22<1:10:01,  6.24s/it]
+ 63%|██████▎   | 1141/1813 [2:05:28<1:09:45,  6.23s/it]
+ 63%|██████▎   | 1142/1813 [2:05:34<1:08:20,  6.11s/it]
+ 63%|██████▎   | 1143/1813 [2:05:39<1:07:39,  6.06s/it]
+ 63%|██████▎   | 1144/1813 [2:05:46<1:10:14,  6.30s/it]
+ 63%|██████▎   | 1145/1813 [2:05:53<1:10:16,  6.31s/it]
+ 63%|██████▎   | 1146/1813 [2:05:58<1:06:08,  5.95s/it]
+ 63%|██████▎   | 1147/1813 [2:06:05<1:11:02,  6.40s/it]
+ 63%|██████▎   | 1148/1813 [2:06:12<1:12:47,  6.57s/it]
+ 63%|██████▎   | 1149/1813 [2:06:18<1:09:58,  6.32s/it]
+ 63%|██████▎   | 1150/1813 [2:06:25<1:13:53,  6.69s/it]
+                                                       
+{'loss': '0.9547', 'grad_norm': '1.426', 'learning_rate': '6.984e-06', 'epoch': '0.6345', 'num_input_tokens_seen': 7707832, 'train_runtime': '7586', 'train_tokens_per_second': '1016'}
+
+ 63%|██████▎   | 1150/1813 [2:06:25<1:13:53,  6.69s/it]
+ 63%|██████▎   | 1151/1813 [2:06:29<1:04:54,  5.88s/it]
+ 64%|██████▎   | 1152/1813 [2:06:35<1:03:09,  5.73s/it]
+ 64%|██████▎   | 1153/1813 [2:06:42<1:07:35,  6.14s/it]
+ 64%|██████▎   | 1154/1813 [2:06:48<1:07:19,  6.13s/it]
+ 64%|██████▎   | 1155/1813 [2:06:55<1:08:50,  6.28s/it]
+ 64%|██████▍   | 1156/1813 [2:07:01<1:09:13,  6.32s/it]
+ 64%|██████▍   | 1157/1813 [2:07:09<1:13:50,  6.75s/it]
+ 64%|██████▍   | 1158/1813 [2:07:16<1:13:44,  6.76s/it]
+ 64%|██████▍   | 1159/1813 [2:07:23<1:15:37,  6.94s/it]
+ 64%|██████▍   | 1160/1813 [2:07:29<1:13:13,  6.73s/it]
+                                                       
+{'loss': '0.9617', 'grad_norm': '1.089', 'learning_rate': '6.803e-06', 'epoch': '0.64', 'num_input_tokens_seen': 7775872, 'train_runtime': '7650', 'train_tokens_per_second': '1016'}
+
+ 64%|██████▍   | 1160/1813 [2:07:29<1:13:13,  6.73s/it]
+ 64%|██████▍   | 1161/1813 [2:07:37<1:15:25,  6.94s/it]
+ 64%|██████▍   | 1162/1813 [2:07:43<1:14:55,  6.90s/it]
+ 64%|██████▍   | 1163/1813 [2:07:50<1:13:26,  6.78s/it]
+ 64%|██████▍   | 1164/1813 [2:07:56<1:12:13,  6.68s/it]
+ 64%|██████▍   | 1165/1813 [2:08:03<1:11:25,  6.61s/it]
+ 64%|██████▍   | 1166/1813 [2:08:09<1:09:05,  6.41s/it]
+ 64%|██████▍   | 1167/1813 [2:08:16<1:11:34,  6.65s/it]
+ 64%|██████▍   | 1168/1813 [2:08:23<1:10:59,  6.60s/it]
+ 64%|██████▍   | 1169/1813 [2:08:29<1:09:53,  6.51s/it]
+ 65%|██████▍   | 1170/1813 [2:08:35<1:07:15,  6.28s/it]
+                                                       
+{'loss': '0.9555', 'grad_norm': '1.415', 'learning_rate': '6.622e-06', 'epoch': '0.6455', 'num_input_tokens_seen': 7846688, 'train_runtime': '7715', 'train_tokens_per_second': '1017'}
+
+ 65%|██████▍   | 1170/1813 [2:08:35<1:07:15,  6.28s/it]
+ 65%|██████▍   | 1171/1813 [2:08:42<1:11:14,  6.66s/it]
+ 65%|██████▍   | 1172/1813 [2:08:49<1:13:12,  6.85s/it]
+ 65%|██████▍   | 1173/1813 [2:08:55<1:09:08,  6.48s/it]
+ 65%|██████▍   | 1174/1813 [2:09:01<1:06:01,  6.20s/it]
+ 65%|██████▍   | 1175/1813 [2:09:07<1:05:16,  6.14s/it]
+ 65%|██████▍   | 1176/1813 [2:09:13<1:05:53,  6.21s/it]
+ 65%|██████▍   | 1177/1813 [2:09:19<1:04:44,  6.11s/it]
+ 65%|██████▍   | 1178/1813 [2:09:25<1:04:02,  6.05s/it]
+ 65%|██████▌   | 1179/1813 [2:09:31<1:04:18,  6.09s/it]
+ 65%|██████▌   | 1180/1813 [2:09:38<1:06:16,  6.28s/it]
+                                                       
+{'loss': '0.9431', 'grad_norm': '1.57', 'learning_rate': '6.443e-06', 'epoch': '0.651', 'num_input_tokens_seen': 7914072, 'train_runtime': '7778', 'train_tokens_per_second': '1017'}
+
+ 65%|██████▌   | 1180/1813 [2:09:38<1:06:16,  6.28s/it]
+ 65%|██████▌   | 1181/1813 [2:09:46<1:11:25,  6.78s/it]
+ 65%|██████▌   | 1182/1813 [2:09:53<1:14:27,  7.08s/it]
+ 65%|██████▌   | 1183/1813 [2:10:00<1:14:19,  7.08s/it]
+ 65%|██████▌   | 1184/1813 [2:10:06<1:11:01,  6.77s/it]
+ 65%|██████▌   | 1185/1813 [2:10:14<1:14:25,  7.11s/it]
+ 65%|██████▌   | 1186/1813 [2:10:22<1:15:25,  7.22s/it]
+ 65%|██████▌   | 1187/1813 [2:10:29<1:15:10,  7.20s/it]
+ 66%|██████▌   | 1188/1813 [2:10:35<1:10:12,  6.74s/it]
+ 66%|██████▌   | 1189/1813 [2:10:40<1:05:53,  6.34s/it]
+ 66%|██████▌   | 1190/1813 [2:10:47<1:06:32,  6.41s/it]
+                                                       
+{'loss': '0.9905', 'grad_norm': '1.691', 'learning_rate': '6.266e-06', 'epoch': '0.6566', 'num_input_tokens_seen': 7989064, 'train_runtime': '7847', 'train_tokens_per_second': '1018'}
+
+ 66%|██████▌   | 1190/1813 [2:10:47<1:06:32,  6.41s/it]
+ 66%|██████▌   | 1191/1813 [2:10:53<1:05:09,  6.29s/it]
+ 66%|██████▌   | 1192/1813 [2:10:59<1:05:55,  6.37s/it]
+ 66%|██████▌   | 1193/1813 [2:11:05<1:05:12,  6.31s/it]
+ 66%|██████▌   | 1194/1813 [2:11:12<1:06:03,  6.40s/it]
+ 66%|██████▌   | 1195/1813 [2:11:17<1:01:34,  5.98s/it]
+ 66%|██████▌   | 1196/1813 [2:11:23<1:02:28,  6.08s/it]
+ 66%|██████▌   | 1197/1813 [2:11:30<1:03:34,  6.19s/it]
+ 66%|██████▌   | 1198/1813 [2:11:36<1:02:39,  6.11s/it]
+ 66%|██████▌   | 1199/1813 [2:11:42<1:02:22,  6.09s/it]
+ 66%|██████▌   | 1200/1813 [2:11:49<1:06:52,  6.55s/it]
+                                                       
+{'loss': '0.8947', 'grad_norm': '1.67', 'learning_rate': '6.089e-06', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352, 'train_runtime': '7910', 'train_tokens_per_second': '1018'}
+
+ 66%|██████▌   | 1200/1813 [2:11:49<1:06:52,  6.55s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.76it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.22it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.32it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.06it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  7.00it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.83it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.59it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.35it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.34it/s]
+
+  3%|▎         | 32/1000 [00:05<02:42,  5.97it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.74it/s]
+
+  4%|▎         | 35/1000 [00:05<02:07,  7.57it/s]
+
+  4%|▎         | 37/1000 [00:05<01:46,  9.03it/s]
+
+  4%|▍         | 39/1000 [00:05<02:07,  7.56it/s]
+
+  4%|▍         | 40/1000 [00:06<02:04,  7.73it/s]
+
+  4%|▍         | 42/1000 [00:06<01:47,  8.90it/s]
+
+  4%|▍         | 44/1000 [00:06<01:34, 10.16it/s]
+
+  5%|▍         | 46/1000 [00:06<01:46,  8.98it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.34it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.73it/s]
+
+  5%|▌         | 51/1000 [00:07<01:36,  9.86it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.89it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+
+  6%|▌         | 57/1000 [00:07<01:42,  9.19it/s]
+
+  6%|▌         | 59/1000 [00:08<01:55,  8.15it/s]
+
+  6%|▌         | 61/1000 [00:08<01:41,  9.29it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.73it/s]
+
+  6%|▋         | 65/1000 [00:08<01:52,  8.33it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.24it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.98it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.11it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.85it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:58,  7.78it/s]
+
+  8%|▊         | 82/1000 [00:10<01:40,  9.13it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  7.99it/s]
+
+  8%|▊         | 85/1000 [00:11<02:15,  6.77it/s]
+
+  9%|▊         | 86/1000 [00:11<02:34,  5.93it/s]
+
+  9%|▊         | 87/1000 [00:12<02:50,  5.37it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  6.99it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.53it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:12,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.38it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.32it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.45it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.21it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.02it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.49it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.79it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.21it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:16<01:48,  8.08it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:17<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.11it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.72it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.16it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.89it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.90it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.07it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.49it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.91it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.05it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.39it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.96it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.60it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.49it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.77it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.83it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.15it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.51it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.37it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.09it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.62it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.74it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.66it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.77it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.64it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.94it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.69it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.41it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.81it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.89it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:01,  6.35it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.01it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.95it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.96it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:31,  8.34it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.01it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:06,  5.95it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.74it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:57,  6.28it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:52,  6.55it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.87it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.50it/s]
+
+ 27%|██▋       | 270/1000 [00:36<01:44,  7.01it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.06it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.63it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:37,  7.33it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:22,  8.65it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.73it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  6.97it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.06it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.18it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.88it/s]
+
+ 30%|██▉       | 298/1000 [00:40<01:25,  8.17it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.46it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.84it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.11it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.75it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:40,  6.94it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.78it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.19it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.25it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:07, 10.22it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.56it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.88it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.59it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.03it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.81it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.02it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.86it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.21it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.40it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.54it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.95it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.59it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.44it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.93it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.54it/s]
+
+ 37%|███▋      | 372/1000 [00:50<01:21,  7.71it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.18it/s]
+
+ 38%|███▊      | 379/1000 [00:51<01:16,  8.12it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.15it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.53it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.07it/s]
+
+ 39%|███▊      | 387/1000 [00:52<01:21,  7.55it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.30it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.84it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.49it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  9.01it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.52it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.84it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.24it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.52it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.26it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.94it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.96it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.55it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.54it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.02it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.79it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.53it/s]
+
+ 44%|████▍     | 439/1000 [00:59<01:09,  8.09it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.31it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.51it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:59,  9.13it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:50, 10.71it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.59it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.72it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:29,  6.00it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.30it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.84it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.19it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.37it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:25,  6.18it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.18it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.29it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.09it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.51it/s]
+
+ 48%|████▊     | 482/1000 [01:05<01:04,  8.04it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.29it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.28it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.43it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.65it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.41it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.94it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.96it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.27it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.10it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.06it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.37it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.02it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.67it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.80it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.53it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.39it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.69it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.67it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:13,  6.35it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.65it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.75it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.25it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<01:01,  7.36it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:02,  7.16it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.13it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.92it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.74it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  6.99it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:49,  8.83it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.03it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.55it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.43it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.72it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.39it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:01,  6.70it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:50,  8.14it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<01:00,  6.77it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.36it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:59,  6.84it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.65it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.70it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.09it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:50,  7.83it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.45it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.72it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.80it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.75it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.09it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.45it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.63it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.18it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:26<00:38,  9.45it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.96it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.55it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.31it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.69it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.90it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.11it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.47it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.32it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.69it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.99it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.89it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.85it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.15it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.49it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.70it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.38it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.07it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.15it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.30it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.11it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.58it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.31it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.51it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.87it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.52it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.75it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.28it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.35it/s]
+
+ 74%|███████▍  | 745/1000 [01:41<00:26,  9.60it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.67it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.16it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.71it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.66it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.46it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.78it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.38it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.61it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:25,  8.38it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  9.00it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.74it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.98it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.20it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.73it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.66it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:24,  7.85it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.19it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.21it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.06it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.30it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.13it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.19it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.98it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.78it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.24it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.48it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.05it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.19it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.51it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.62it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.71it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.26it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.29it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.79it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.64it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.89it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.88it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.61it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.76it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.74it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.83it/s]
+
+ 90%|████████▉ | 898/1000 [02:01<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.69it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.85it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.05it/s]
+
+ 91%|█████████ | 906/1000 [02:02<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.58it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.35it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.59it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.71it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.47it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.82it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.61it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.84it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.42it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.41it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.79it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.49it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.44it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.01it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.40it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.01it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.19it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.78it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.80it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.42it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.93it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.56it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.54it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.76it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                       
+
+                                                   
+{'eval_loss': '0.8076', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.385', 'eval_steps_per_second': '7.385', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352}
+
+ 66%|██████▌   | 1200/1813 [2:14:05<1:06:52,  6.55s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 66%|██████▌   | 1201/1813 [2:14:10<7:58:05, 46.87s/it]
+ 66%|██████▋   | 1202/1813 [2:14:17<5:54:35, 34.82s/it]
+ 66%|██████▋   | 1203/1813 [2:14:24<4:27:36, 26.32s/it]
+ 66%|██████▋   | 1204/1813 [2:14:30<3:27:11, 20.41s/it]
+ 66%|██████▋   | 1205/1813 [2:14:37<2:44:49, 16.27s/it]
+ 67%|██████▋   | 1206/1813 [2:14:43<2:14:11, 13.26s/it]
+ 67%|██████▋   | 1207/1813 [2:14:49<1:53:21, 11.22s/it]
+ 67%|██████▋   | 1208/1813 [2:14:57<1:42:11, 10.14s/it]
+ 67%|██████▋   | 1209/1813 [2:15:04<1:31:44,  9.11s/it]
+ 67%|██████▋   | 1210/1813 [2:15:10<1:21:39,  8.12s/it]
+                                                       
+{'loss': '0.969', 'grad_norm': '1.902', 'learning_rate': '5.914e-06', 'epoch': '0.6676', 'num_input_tokens_seen': 8125224, 'train_runtime': '8110', 'train_tokens_per_second': '1002'}
+
+ 67%|██████▋   | 1210/1813 [2:15:10<1:21:39,  8.12s/it]
+ 67%|██████▋   | 1211/1813 [2:15:16<1:17:45,  7.75s/it]
+ 67%|██████▋   | 1212/1813 [2:15:21<1:07:25,  6.73s/it]
+ 67%|██████▋   | 1213/1813 [2:15:27<1:06:34,  6.66s/it]
+ 67%|██████▋   | 1214/1813 [2:15:36<1:11:36,  7.17s/it]
+ 67%|██████▋   | 1215/1813 [2:15:42<1:08:48,  6.90s/it]
+ 67%|██████▋   | 1216/1813 [2:15:49<1:09:46,  7.01s/it]
+ 67%|██████▋   | 1217/1813 [2:15:55<1:07:27,  6.79s/it]
+ 67%|██████▋   | 1218/1813 [2:16:02<1:05:49,  6.64s/it]
+ 67%|██████▋   | 1219/1813 [2:16:07<1:02:01,  6.27s/it]
+ 67%|██████▋   | 1220/1813 [2:16:15<1:07:19,  6.81s/it]
+                                                       
+{'loss': '1.009', 'grad_norm': '14.39', 'learning_rate': '5.741e-06', 'epoch': '0.6731', 'num_input_tokens_seen': 8197488, 'train_runtime': '8176', 'train_tokens_per_second': '1003'}
+
+ 67%|██████▋   | 1220/1813 [2:16:15<1:07:19,  6.81s/it]
+ 67%|██████▋   | 1221/1813 [2:16:21<1:03:12,  6.41s/it]
+ 67%|██████▋   | 1222/1813 [2:16:26<59:00,  5.99s/it]  
+ 67%|██████▋   | 1223/1813 [2:16:31<58:02,  5.90s/it]
+ 68%|██████▊   | 1224/1813 [2:16:36<55:08,  5.62s/it]
+ 68%|██████▊   | 1225/1813 [2:16:43<56:34,  5.77s/it]
+ 68%|██████▊   | 1226/1813 [2:16:49<57:15,  5.85s/it]
+ 68%|██████▊   | 1227/1813 [2:16:55<59:09,  6.06s/it]
+ 68%|██████▊   | 1228/1813 [2:17:00<56:39,  5.81s/it]
+ 68%|██████▊   | 1229/1813 [2:17:07<58:15,  5.99s/it]
+ 68%|██████▊   | 1230/1813 [2:17:13<57:55,  5.96s/it]
+                                                     
+{'loss': '0.8638', 'grad_norm': '1.41', 'learning_rate': '5.569e-06', 'epoch': '0.6786', 'num_input_tokens_seen': 8257288, 'train_runtime': '8233', 'train_tokens_per_second': '1003'}
+
+ 68%|██████▊   | 1230/1813 [2:17:13<57:55,  5.96s/it]
+ 68%|██████▊   | 1231/1813 [2:17:20<1:02:33,  6.45s/it]
+ 68%|██████▊   | 1232/1813 [2:17:27<1:04:47,  6.69s/it]
+ 68%|██████▊   | 1233/1813 [2:17:34<1:03:02,  6.52s/it]
+ 68%|██████▊   | 1234/1813 [2:17:40<1:01:50,  6.41s/it]
+ 68%|██████▊   | 1235/1813 [2:17:46<1:00:51,  6.32s/it]
+ 68%|██████▊   | 1236/1813 [2:17:54<1:06:57,  6.96s/it]
+ 68%|██████▊   | 1237/1813 [2:18:01<1:04:53,  6.76s/it]
+ 68%|██████▊   | 1238/1813 [2:18:06<1:00:50,  6.35s/it]
+ 68%|██████▊   | 1239/1813 [2:18:11<57:12,  5.98s/it]  
+ 68%|██████▊   | 1240/1813 [2:18:16<54:55,  5.75s/it]
+                                                     
+{'loss': '0.8744', 'grad_norm': '1.528', 'learning_rate': '5.399e-06', 'epoch': '0.6841', 'num_input_tokens_seen': 8327600, 'train_runtime': '8297', 'train_tokens_per_second': '1004'}
+
+ 68%|██████▊   | 1240/1813 [2:18:16<54:55,  5.75s/it]
+ 68%|██████▊   | 1241/1813 [2:18:22<54:34,  5.72s/it]
+ 69%|██████▊   | 1242/1813 [2:18:28<54:56,  5.77s/it]
+ 69%|██████▊   | 1243/1813 [2:18:34<55:47,  5.87s/it]
+ 69%|██████▊   | 1244/1813 [2:18:41<58:18,  6.15s/it]
+ 69%|██████▊   | 1245/1813 [2:18:48<1:01:11,  6.46s/it]
+ 69%|██████▊   | 1246/1813 [2:18:54<59:10,  6.26s/it]  
+ 69%|██████▉   | 1247/1813 [2:19:01<1:00:34,  6.42s/it]
+ 69%|██████▉   | 1248/1813 [2:19:07<1:01:24,  6.52s/it]
+ 69%|██████▉   | 1249/1813 [2:19:14<1:00:30,  6.44s/it]
+ 69%|██████▉   | 1250/1813 [2:19:18<55:52,  5.95s/it]  
+                                                     
+{'loss': '1.006', 'grad_norm': '1.949', 'learning_rate': '5.231e-06', 'epoch': '0.6897', 'num_input_tokens_seen': 8394824, 'train_runtime': '8359', 'train_tokens_per_second': '1004'}
+
+ 69%|██████▉   | 1250/1813 [2:19:18<55:52,  5.95s/it]
+ 69%|██████▉   | 1251/1813 [2:19:25<56:25,  6.02s/it]
+ 69%|██████▉   | 1252/1813 [2:19:30<55:39,  5.95s/it]
+ 69%|██████▉   | 1253/1813 [2:19:36<55:58,  6.00s/it]
+ 69%|██████▉   | 1254/1813 [2:19:43<57:27,  6.17s/it]
+ 69%|██████▉   | 1255/1813 [2:19:49<57:41,  6.20s/it]
+ 69%|██████▉   | 1256/1813 [2:19:55<56:42,  6.11s/it]
+ 69%|██████▉   | 1257/1813 [2:20:01<54:50,  5.92s/it]
+ 69%|██████▉   | 1258/1813 [2:20:07<56:43,  6.13s/it]
+ 69%|██████▉   | 1259/1813 [2:20:14<57:41,  6.25s/it]
+ 69%|██████▉   | 1260/1813 [2:20:21<59:31,  6.46s/it]
+                                                     
+{'loss': '0.9475', 'grad_norm': '2.112', 'learning_rate': '5.064e-06', 'epoch': '0.6952', 'num_input_tokens_seen': 8461096, 'train_runtime': '8421', 'train_tokens_per_second': '1005'}
+
+ 69%|██████▉   | 1260/1813 [2:20:21<59:31,  6.46s/it]
+ 70%|██████▉   | 1261/1813 [2:20:29<1:03:44,  6.93s/it]
+ 70%|██████▉   | 1262/1813 [2:20:36<1:05:02,  7.08s/it]
+ 70%|██████▉   | 1263/1813 [2:20:43<1:05:20,  7.13s/it]
+ 70%|██████▉   | 1264/1813 [2:20:49<1:01:16,  6.70s/it]
+ 70%|██████▉   | 1265/1813 [2:20:54<56:48,  6.22s/it]  
+ 70%|██████▉   | 1266/1813 [2:21:02<1:00:31,  6.64s/it]
+ 70%|██████▉   | 1267/1813 [2:21:07<55:55,  6.15s/it]  
+ 70%|██████▉   | 1268/1813 [2:21:12<52:17,  5.76s/it]
+ 70%|██████▉   | 1269/1813 [2:21:17<51:02,  5.63s/it]
+ 70%|███████   | 1270/1813 [2:21:23<51:45,  5.72s/it]
+                                                     
+{'loss': '0.9557', 'grad_norm': '1.834', 'learning_rate': '4.899e-06', 'epoch': '0.7007', 'num_input_tokens_seen': 8529224, 'train_runtime': '8483', 'train_tokens_per_second': '1005'}
+
+ 70%|███████   | 1270/1813 [2:21:23<51:45,  5.72s/it]
+ 70%|███████   | 1271/1813 [2:21:28<48:44,  5.40s/it]
+ 70%|███████   | 1272/1813 [2:21:34<52:37,  5.84s/it]
+ 70%|███████   | 1273/1813 [2:21:41<55:05,  6.12s/it]
+ 70%|███████   | 1274/1813 [2:21:49<58:06,  6.47s/it]
+ 70%|███████   | 1275/1813 [2:21:55<58:38,  6.54s/it]
+ 70%|███████   | 1276/1813 [2:22:01<55:56,  6.25s/it]
+ 70%|███████   | 1277/1813 [2:22:08<57:55,  6.48s/it]
+ 70%|███████   | 1278/1813 [2:22:13<54:59,  6.17s/it]
+ 71%|███████   | 1279/1813 [2:22:18<50:55,  5.72s/it]
+ 71%|███████   | 1280/1813 [2:22:25<54:02,  6.08s/it]
+                                                     
+{'loss': '1.01', 'grad_norm': '1.758', 'learning_rate': '4.736e-06', 'epoch': '0.7062', 'num_input_tokens_seen': 8596456, 'train_runtime': '8545', 'train_tokens_per_second': '1006'}
+
+ 71%|███████   | 1280/1813 [2:22:25<54:02,  6.08s/it]
+ 71%|███████   | 1281/1813 [2:22:30<51:26,  5.80s/it]
+ 71%|███████   | 1282/1813 [2:22:36<52:34,  5.94s/it]
+ 71%|███████   | 1283/1813 [2:22:43<54:22,  6.16s/it]
+ 71%|███████   | 1284/1813 [2:22:49<53:31,  6.07s/it]
+ 71%|███████   | 1285/1813 [2:22:54<51:28,  5.85s/it]
+ 71%|███████   | 1286/1813 [2:23:01<54:52,  6.25s/it]
+ 71%|███████   | 1287/1813 [2:23:08<54:37,  6.23s/it]
+ 71%|███████   | 1288/1813 [2:23:13<52:06,  5.96s/it]
+ 71%|███████   | 1289/1813 [2:23:20<54:14,  6.21s/it]
+ 71%|███████   | 1290/1813 [2:23:27<56:26,  6.47s/it]
+                                                     
+{'loss': '1.026', 'grad_norm': '1.138', 'learning_rate': '4.575e-06', 'epoch': '0.7117', 'num_input_tokens_seen': 8662472, 'train_runtime': '8607', 'train_tokens_per_second': '1006'}
+
+ 71%|███████   | 1290/1813 [2:23:27<56:26,  6.47s/it]
+ 71%|███████   | 1291/1813 [2:23:33<55:21,  6.36s/it]
+ 71%|███████▏  | 1292/1813 [2:23:40<56:26,  6.50s/it]
+ 71%|███████▏  | 1293/1813 [2:23:47<57:33,  6.64s/it]
+ 71%|███████▏  | 1294/1813 [2:23:53<56:25,  6.52s/it]
+ 71%|███████▏  | 1295/1813 [2:23:59<54:20,  6.29s/it]
+ 71%|███████▏  | 1296/1813 [2:24:04<51:26,  5.97s/it]
+ 72%|███████▏  | 1297/1813 [2:24:09<49:24,  5.74s/it]
+ 72%|███████▏  | 1298/1813 [2:24:16<51:24,  5.99s/it]
+ 72%|███████▏  | 1299/1813 [2:24:23<53:41,  6.27s/it]
+ 72%|███████▏  | 1300/1813 [2:24:29<53:04,  6.21s/it]
+                                                     
+{'loss': '0.9216', 'grad_norm': '1.859', 'learning_rate': '4.416e-06', 'epoch': '0.7172', 'num_input_tokens_seen': 8730600, 'train_runtime': '8669', 'train_tokens_per_second': '1007'}
+
+ 72%|███████▏  | 1300/1813 [2:24:29<53:04,  6.21s/it]
+ 72%|███████▏  | 1301/1813 [2:24:34<51:41,  6.06s/it]
+ 72%|███████▏  | 1302/1813 [2:24:41<53:01,  6.23s/it]
+ 72%|███████▏  | 1303/1813 [2:24:49<56:31,  6.65s/it]
+ 72%|███████▏  | 1304/1813 [2:24:54<53:19,  6.29s/it]
+ 72%|███████▏  | 1305/1813 [2:25:01<55:51,  6.60s/it]
+ 72%|███████▏  | 1306/1813 [2:25:07<53:24,  6.32s/it]
+ 72%|███████▏  | 1307/1813 [2:25:13<53:01,  6.29s/it]
+ 72%|███████▏  | 1308/1813 [2:25:20<53:16,  6.33s/it]
+ 72%|███████▏  | 1309/1813 [2:25:26<52:57,  6.30s/it]
+ 72%|███████▏  | 1310/1813 [2:25:33<54:27,  6.50s/it]
+                                                     
+{'loss': '1.005', 'grad_norm': '1.306', 'learning_rate': '4.259e-06', 'epoch': '0.7228', 'num_input_tokens_seen': 8802000, 'train_runtime': '8733', 'train_tokens_per_second': '1008'}
+
+ 72%|███████▏  | 1310/1813 [2:25:33<54:27,  6.50s/it]
+ 72%|███████▏  | 1311/1813 [2:25:39<53:29,  6.39s/it]
+ 72%|███████▏  | 1312/1813 [2:25:45<51:14,  6.14s/it]
+ 72%|███████▏  | 1313/1813 [2:25:50<50:39,  6.08s/it]
+ 72%|███████▏  | 1314/1813 [2:25:57<50:57,  6.13s/it]
+ 73%|███████▎  | 1315/1813 [2:26:02<49:35,  5.97s/it]
+ 73%|███████▎  | 1316/1813 [2:26:08<48:56,  5.91s/it]
+ 73%|███████▎  | 1317/1813 [2:26:14<49:00,  5.93s/it]
+ 73%|███████▎  | 1318/1813 [2:26:20<47:43,  5.78s/it]
+ 73%|███████▎  | 1319/1813 [2:26:26<49:22,  6.00s/it]
+ 73%|███████▎  | 1320/1813 [2:26:33<50:53,  6.19s/it]
+                                                     
+{'loss': '1.005', 'grad_norm': '1.501', 'learning_rate': '4.104e-06', 'epoch': '0.7283', 'num_input_tokens_seen': 8866824, 'train_runtime': '8793', 'train_tokens_per_second': '1008'}
+
+ 73%|███████▎  | 1320/1813 [2:26:33<50:53,  6.19s/it]
+ 73%|███████▎  | 1321/1813 [2:26:39<51:44,  6.31s/it]
+ 73%|███████▎  | 1322/1813 [2:26:45<50:46,  6.20s/it]
+ 73%|███████▎  | 1323/1813 [2:26:52<51:45,  6.34s/it]
+ 73%|███████▎  | 1324/1813 [2:26:58<50:19,  6.18s/it]
+ 73%|███████▎  | 1325/1813 [2:27:03<49:08,  6.04s/it]
+ 73%|███████▎  | 1326/1813 [2:27:10<49:29,  6.10s/it]
+ 73%|███████▎  | 1327/1813 [2:27:17<53:10,  6.57s/it]
+ 73%|███████▎  | 1328/1813 [2:27:24<53:24,  6.61s/it]
+ 73%|███████▎  | 1329/1813 [2:27:30<51:53,  6.43s/it]
+ 73%|███████▎  | 1330/1813 [2:27:36<49:38,  6.17s/it]
+                                                     
+{'loss': '0.9384', 'grad_norm': '1.723', 'learning_rate': '3.951e-06', 'epoch': '0.7338', 'num_input_tokens_seen': 8935584, 'train_runtime': '8856', 'train_tokens_per_second': '1009'}
+
+ 73%|███████▎  | 1330/1813 [2:27:36<49:38,  6.17s/it]
+ 73%|███████▎  | 1331/1813 [2:27:41<48:17,  6.01s/it]
+ 73%|███████▎  | 1332/1813 [2:27:47<47:30,  5.93s/it]
+ 74%|███████▎  | 1333/1813 [2:27:53<46:43,  5.84s/it]
+ 74%|███████▎  | 1334/1813 [2:27:58<45:31,  5.70s/it]
+ 74%|███████▎  | 1335/1813 [2:28:03<44:34,  5.60s/it]
+ 74%|███████▎  | 1336/1813 [2:28:10<46:06,  5.80s/it]
+ 74%|███████▎  | 1337/1813 [2:28:17<50:11,  6.33s/it]
+ 74%|███████▍  | 1338/1813 [2:28:24<52:00,  6.57s/it]
+ 74%|███████▍  | 1339/1813 [2:28:29<48:16,  6.11s/it]
+ 74%|███████▍  | 1340/1813 [2:28:36<50:23,  6.39s/it]
+                                                     
+{'loss': '1.022', 'grad_norm': '1.414', 'learning_rate': '3.8e-06', 'epoch': '0.7393', 'num_input_tokens_seen': 9000416, 'train_runtime': '8917', 'train_tokens_per_second': '1009'}
+
+ 74%|███████▍  | 1340/1813 [2:28:36<50:23,  6.39s/it]
+ 74%|███████▍  | 1341/1813 [2:28:43<50:40,  6.44s/it]
+ 74%|███████▍  | 1342/1813 [2:28:48<47:40,  6.07s/it]
+ 74%|███████▍  | 1343/1813 [2:28:54<46:11,  5.90s/it]
+ 74%|███████▍  | 1344/1813 [2:29:00<46:36,  5.96s/it]
+ 74%|███████▍  | 1345/1813 [2:29:07<50:12,  6.44s/it]
+ 74%|███████▍  | 1346/1813 [2:29:14<49:58,  6.42s/it]
+ 74%|███████▍  | 1347/1813 [2:29:19<48:11,  6.20s/it]
+ 74%|███████▍  | 1348/1813 [2:29:26<48:11,  6.22s/it]
+ 74%|███████▍  | 1349/1813 [2:29:32<49:22,  6.38s/it]
+ 74%|███████▍  | 1350/1813 [2:29:38<47:32,  6.16s/it]
+                                                     
+{'loss': '0.981', 'grad_norm': '2.191', 'learning_rate': '3.652e-06', 'epoch': '0.7448', 'num_input_tokens_seen': 9066864, 'train_runtime': '8979', 'train_tokens_per_second': '1010'}
+
+ 74%|███████▍  | 1350/1813 [2:29:38<47:32,  6.16s/it]
+ 75%|███████▍  | 1351/1813 [2:29:46<51:21,  6.67s/it]
+ 75%|███████▍  | 1352/1813 [2:29:52<50:46,  6.61s/it]
+ 75%|███████▍  | 1353/1813 [2:29:59<51:07,  6.67s/it]
+ 75%|███████▍  | 1354/1813 [2:30:06<51:17,  6.71s/it]
+ 75%|███████▍  | 1355/1813 [2:30:14<55:13,  7.23s/it]
+ 75%|███████▍  | 1356/1813 [2:30:20<51:37,  6.78s/it]
+ 75%|███████▍  | 1357/1813 [2:30:26<49:43,  6.54s/it]
+ 75%|███████▍  | 1358/1813 [2:30:33<49:19,  6.50s/it]
+ 75%|███████▍  | 1359/1813 [2:30:38<47:55,  6.33s/it]
+ 75%|███████▌  | 1360/1813 [2:30:45<47:26,  6.28s/it]
+                                                     
+{'loss': '0.9594', 'grad_norm': '2.229', 'learning_rate': '3.506e-06', 'epoch': '0.7503', 'num_input_tokens_seen': 9140520, 'train_runtime': '9045', 'train_tokens_per_second': '1011'}
+
+ 75%|███████▌  | 1360/1813 [2:30:45<47:26,  6.28s/it]
+ 75%|███████▌  | 1361/1813 [2:30:51<48:26,  6.43s/it]
+ 75%|███████▌  | 1362/1813 [2:30:58<48:02,  6.39s/it]
+ 75%|███████▌  | 1363/1813 [2:31:05<50:09,  6.69s/it]
+ 75%|███████▌  | 1364/1813 [2:31:11<48:30,  6.48s/it]
+ 75%|███████▌  | 1365/1813 [2:31:19<50:43,  6.79s/it]
+ 75%|███████▌  | 1366/1813 [2:31:25<48:40,  6.53s/it]
+ 75%|███████▌  | 1367/1813 [2:31:31<48:36,  6.54s/it]
+ 75%|███████▌  | 1368/1813 [2:31:36<45:10,  6.09s/it]
+ 76%|███████▌  | 1369/1813 [2:31:44<49:20,  6.67s/it]
+ 76%|███████▌  | 1370/1813 [2:31:50<47:45,  6.47s/it]
+                                                     
+{'loss': '0.9488', 'grad_norm': '1.699', 'learning_rate': '3.362e-06', 'epoch': '0.7559', 'num_input_tokens_seen': 9213048, 'train_runtime': '9111', 'train_tokens_per_second': '1011'}
+
+ 76%|███████▌  | 1370/1813 [2:31:50<47:45,  6.47s/it]
+ 76%|███████▌  | 1371/1813 [2:31:58<51:04,  6.93s/it]
+ 76%|███████▌  | 1372/1813 [2:32:03<47:08,  6.41s/it]
+ 76%|███████▌  | 1373/1813 [2:32:09<46:12,  6.30s/it]
+ 76%|███████▌  | 1374/1813 [2:32:15<45:10,  6.17s/it]
+ 76%|███████▌  | 1375/1813 [2:32:21<43:56,  6.02s/it]
+ 76%|███████▌  | 1376/1813 [2:32:26<42:17,  5.81s/it]
+ 76%|███████▌  | 1377/1813 [2:32:32<42:50,  5.90s/it]
+ 76%|███████▌  | 1378/1813 [2:32:38<41:53,  5.78s/it]
+ 76%|███████▌  | 1379/1813 [2:32:44<41:46,  5.78s/it]
+ 76%|███████▌  | 1380/1813 [2:32:51<44:08,  6.12s/it]
+                                                     
+{'loss': '0.933', 'grad_norm': '1.838', 'learning_rate': '3.221e-06', 'epoch': '0.7614', 'num_input_tokens_seen': 9279088, 'train_runtime': '9171', 'train_tokens_per_second': '1012'}
+
+ 76%|███████▌  | 1380/1813 [2:32:51<44:08,  6.12s/it]
+ 76%|███████▌  | 1381/1813 [2:32:57<45:45,  6.36s/it]
+ 76%|███████▌  | 1382/1813 [2:33:03<43:27,  6.05s/it]
+ 76%|███████▋  | 1383/1813 [2:33:09<42:56,  5.99s/it]
+ 76%|███████▋  | 1384/1813 [2:33:15<42:42,  5.97s/it]
+ 76%|███████▋  | 1385/1813 [2:33:21<44:15,  6.20s/it]
+ 76%|███████▋  | 1386/1813 [2:33:28<44:39,  6.28s/it]
+ 77%|███████▋  | 1387/1813 [2:33:35<46:09,  6.50s/it]
+ 77%|███████▋  | 1388/1813 [2:33:40<43:36,  6.16s/it]
+ 77%|███████▋  | 1389/1813 [2:33:44<39:39,  5.61s/it]
+ 77%|███████▋  | 1390/1813 [2:33:51<42:28,  6.02s/it]
+                                                     
+{'loss': '0.9886', 'grad_norm': '1.886', 'learning_rate': '3.082e-06', 'epoch': '0.7669', 'num_input_tokens_seen': 9344880, 'train_runtime': '9232', 'train_tokens_per_second': '1012'}
+
+ 77%|███████▋  | 1390/1813 [2:33:51<42:28,  6.02s/it]
+ 77%|███████▋  | 1391/1813 [2:33:57<41:22,  5.88s/it]
+ 77%|███████▋  | 1392/1813 [2:34:04<43:11,  6.16s/it]
+ 77%|███████▋  | 1393/1813 [2:34:09<40:54,  5.85s/it]
+ 77%|███████▋  | 1394/1813 [2:34:16<42:20,  6.06s/it]
+ 77%|███████▋  | 1395/1813 [2:34:22<42:41,  6.13s/it]
+ 77%|███████▋  | 1396/1813 [2:34:28<42:17,  6.09s/it]
+ 77%|███████▋  | 1397/1813 [2:34:35<43:36,  6.29s/it]
+ 77%|███████▋  | 1398/1813 [2:34:39<40:09,  5.81s/it]
+ 77%|███████▋  | 1399/1813 [2:34:44<38:20,  5.56s/it]
+ 77%|███████▋  | 1400/1813 [2:34:51<40:50,  5.93s/it]
+                                                     
+{'loss': '0.9127', 'grad_norm': '1.176', 'learning_rate': '2.946e-06', 'epoch': '0.7724', 'num_input_tokens_seen': 9408280, 'train_runtime': '9292', 'train_tokens_per_second': '1013'}
+
+ 77%|███████▋  | 1400/1813 [2:34:51<40:50,  5.93s/it]
+ 77%|███████▋  | 1401/1813 [2:34:56<39:31,  5.76s/it]
+ 77%|███████▋  | 1402/1813 [2:35:02<39:58,  5.84s/it]
+ 77%|███████▋  | 1403/1813 [2:35:09<41:06,  6.02s/it]
+ 77%|███████▋  | 1404/1813 [2:35:15<41:16,  6.05s/it]
+ 77%|███████▋  | 1405/1813 [2:35:20<38:23,  5.65s/it]
+ 78%|███████▊  | 1406/1813 [2:35:25<38:08,  5.62s/it]
+ 78%|███████▊  | 1407/1813 [2:35:32<40:03,  5.92s/it]
+ 78%|███████▊  | 1408/1813 [2:35:37<38:53,  5.76s/it]
+ 78%|███████▊  | 1409/1813 [2:35:42<36:41,  5.45s/it]
+ 78%|███████▊  | 1410/1813 [2:35:48<37:06,  5.52s/it]
+                                                     
+{'loss': '0.9876', 'grad_norm': '2.037', 'learning_rate': '2.812e-06', 'epoch': '0.7779', 'num_input_tokens_seen': 9467408, 'train_runtime': '9348', 'train_tokens_per_second': '1013'}
+
+ 78%|███████▊  | 1410/1813 [2:35:48<37:06,  5.52s/it]
+ 78%|███████▊  | 1411/1813 [2:35:53<37:06,  5.54s/it]
+ 78%|███████▊  | 1412/1813 [2:36:00<38:50,  5.81s/it]
+ 78%|███████▊  | 1413/1813 [2:36:05<38:35,  5.79s/it]
+ 78%|███████▊  | 1414/1813 [2:36:12<39:31,  5.94s/it]
+ 78%|███████▊  | 1415/1813 [2:36:19<41:57,  6.33s/it]
+ 78%|███████▊  | 1416/1813 [2:36:25<41:59,  6.35s/it]
+ 78%|███████▊  | 1417/1813 [2:36:31<40:19,  6.11s/it]
+ 78%|███████▊  | 1418/1813 [2:36:37<39:59,  6.07s/it]
+ 78%|███████▊  | 1419/1813 [2:36:46<45:10,  6.88s/it]
+ 78%|███████▊  | 1420/1813 [2:36:50<40:15,  6.15s/it]
+                                                     
+{'loss': '0.9277', 'grad_norm': '1.681', 'learning_rate': '2.681e-06', 'epoch': '0.7834', 'num_input_tokens_seen': 9532720, 'train_runtime': '9411', 'train_tokens_per_second': '1013'}
+
+ 78%|███████▊  | 1420/1813 [2:36:50<40:15,  6.15s/it]
+ 78%|███████▊  | 1421/1813 [2:36:57<41:24,  6.34s/it]
+ 78%|███████▊  | 1422/1813 [2:37:02<39:46,  6.10s/it]
+ 78%|███████▊  | 1423/1813 [2:37:08<38:46,  5.97s/it]
+ 79%|███████▊  | 1424/1813 [2:37:15<41:32,  6.41s/it]
+ 79%|███████▊  | 1425/1813 [2:37:20<37:45,  5.84s/it]
+ 79%|███████▊  | 1426/1813 [2:37:29<42:55,  6.66s/it]
+ 79%|███████▊  | 1427/1813 [2:37:34<40:32,  6.30s/it]
+ 79%|███████▉  | 1428/1813 [2:37:41<41:19,  6.44s/it]
+ 79%|███████▉  | 1429/1813 [2:37:47<41:07,  6.43s/it]
+ 79%|███████▉  | 1430/1813 [2:37:54<42:10,  6.61s/it]
+                                                     
+{'loss': '0.9376', 'grad_norm': '1.774', 'learning_rate': '2.553e-06', 'epoch': '0.789', 'num_input_tokens_seen': 9602112, 'train_runtime': '9475', 'train_tokens_per_second': '1013'}
+
+ 79%|███████▉  | 1430/1813 [2:37:54<42:10,  6.61s/it]
+ 79%|███████▉  | 1431/1813 [2:38:01<41:34,  6.53s/it]
+ 79%|███████▉  | 1432/1813 [2:38:06<40:12,  6.33s/it]
+ 79%|███████▉  | 1433/1813 [2:38:13<40:15,  6.36s/it]
+ 79%|███████▉  | 1434/1813 [2:38:19<40:27,  6.41s/it]
+ 79%|███████▉  | 1435/1813 [2:38:25<39:04,  6.20s/it]
+ 79%|███████▉  | 1436/1813 [2:38:32<40:28,  6.44s/it]
+ 79%|███████▉  | 1437/1813 [2:38:38<39:18,  6.27s/it]
+ 79%|███████▉  | 1438/1813 [2:38:45<41:13,  6.60s/it]
+ 79%|███████▉  | 1439/1813 [2:38:51<38:53,  6.24s/it]
+ 79%|███████▉  | 1440/1813 [2:38:56<36:26,  5.86s/it]
+                                                     
+{'loss': '0.9417', 'grad_norm': '1.924', 'learning_rate': '2.427e-06', 'epoch': '0.7945', 'num_input_tokens_seen': 9667112, 'train_runtime': '9536', 'train_tokens_per_second': '1014'}
+
+ 79%|███████▉  | 1440/1813 [2:38:56<36:26,  5.86s/it]
+ 79%|███████▉  | 1441/1813 [2:39:02<37:15,  6.01s/it]
+ 80%|███████▉  | 1442/1813 [2:39:07<35:53,  5.80s/it]
+ 80%|███████▉  | 1443/1813 [2:39:13<34:53,  5.66s/it]
+ 80%|███████▉  | 1444/1813 [2:39:20<37:08,  6.04s/it]
+ 80%|███████▉  | 1445/1813 [2:39:25<36:23,  5.93s/it]
+ 80%|███████▉  | 1446/1813 [2:39:31<35:37,  5.83s/it]
+ 80%|███████▉  | 1447/1813 [2:39:39<40:06,  6.58s/it]
+ 80%|███████▉  | 1448/1813 [2:39:47<41:33,  6.83s/it]
+ 80%|███████▉  | 1449/1813 [2:39:53<41:02,  6.77s/it]
+ 80%|███████▉  | 1450/1813 [2:40:00<40:36,  6.71s/it]
+                                                     
+{'loss': '1.025', 'grad_norm': '1.66', 'learning_rate': '2.304e-06', 'epoch': '0.8', 'num_input_tokens_seen': 9735792, 'train_runtime': '9600', 'train_tokens_per_second': '1014'}
+
+ 80%|███████▉  | 1450/1813 [2:40:00<40:36,  6.71s/it]
+ 80%|████████  | 1451/1813 [2:40:07<41:02,  6.80s/it]
+ 80%|████████  | 1452/1813 [2:40:11<36:46,  6.11s/it]
+ 80%|████████  | 1453/1813 [2:40:20<40:27,  6.74s/it]
+ 80%|████████  | 1454/1813 [2:40:25<37:46,  6.31s/it]
+ 80%|████████  | 1455/1813 [2:40:31<37:27,  6.28s/it]
+ 80%|████████  | 1456/1813 [2:40:38<38:51,  6.53s/it]
+ 80%|████████  | 1457/1813 [2:40:42<34:18,  5.78s/it]
+ 80%|████████  | 1458/1813 [2:40:48<34:21,  5.81s/it]
+ 80%|████████  | 1459/1813 [2:40:55<36:36,  6.21s/it]
+ 81%|████████  | 1460/1813 [2:41:03<38:25,  6.53s/it]
+                                                     
+{'loss': '0.961', 'grad_norm': '1.568', 'learning_rate': '2.184e-06', 'epoch': '0.8055', 'num_input_tokens_seen': 9802968, 'train_runtime': '9663', 'train_tokens_per_second': '1014'}
+
+ 81%|████████  | 1460/1813 [2:41:03<38:25,  6.53s/it]
+ 81%|████████  | 1461/1813 [2:41:09<37:19,  6.36s/it]
+ 81%|████████  | 1462/1813 [2:41:14<35:07,  6.00s/it]
+ 81%|████████  | 1463/1813 [2:41:21<37:26,  6.42s/it]
+ 81%|████████  | 1464/1813 [2:41:28<37:38,  6.47s/it]
+ 81%|████████  | 1465/1813 [2:41:36<40:42,  7.02s/it]
+ 81%|████████  | 1466/1813 [2:41:42<38:19,  6.63s/it]
+ 81%|████████  | 1467/1813 [2:41:49<39:37,  6.87s/it]
+ 81%|████████  | 1468/1813 [2:41:56<39:08,  6.81s/it]
+ 81%|████████  | 1469/1813 [2:42:05<42:21,  7.39s/it]
+ 81%|████████  | 1470/1813 [2:42:11<41:09,  7.20s/it]
+                                                     
+{'loss': '0.9532', 'grad_norm': '1.584', 'learning_rate': '2.067e-06', 'epoch': '0.811', 'num_input_tokens_seen': 9879232, 'train_runtime': '9732', 'train_tokens_per_second': '1015'}
+
+ 81%|████████  | 1470/1813 [2:42:11<41:09,  7.20s/it]
+ 81%|████████  | 1471/1813 [2:42:18<39:34,  6.94s/it]
+ 81%|████████  | 1472/1813 [2:42:24<39:16,  6.91s/it]
+ 81%|████████  | 1473/1813 [2:42:30<37:27,  6.61s/it]
+ 81%|████████▏ | 1474/1813 [2:42:37<37:27,  6.63s/it]
+ 81%|████████▏ | 1475/1813 [2:42:43<37:02,  6.58s/it]
+ 81%|████████▏ | 1476/1813 [2:42:50<36:20,  6.47s/it]
+ 81%|████████▏ | 1477/1813 [2:42:56<35:53,  6.41s/it]
+ 82%|████████▏ | 1478/1813 [2:43:01<34:13,  6.13s/it]
+ 82%|████████▏ | 1479/1813 [2:43:08<34:14,  6.15s/it]
+ 82%|████████▏ | 1480/1813 [2:43:15<35:45,  6.44s/it]
+                                                     
+{'loss': '0.9337', 'grad_norm': '1.208', 'learning_rate': '1.952e-06', 'epoch': '0.8166', 'num_input_tokens_seen': 9948032, 'train_runtime': '9795', 'train_tokens_per_second': '1016'}
+
+ 82%|████████▏ | 1480/1813 [2:43:15<35:45,  6.44s/it]
+ 82%|████████▏ | 1481/1813 [2:43:21<36:03,  6.52s/it]
+ 82%|████████▏ | 1482/1813 [2:43:27<34:33,  6.26s/it]
+ 82%|████████▏ | 1483/1813 [2:43:34<35:16,  6.41s/it]
+ 82%|████████▏ | 1484/1813 [2:43:40<35:09,  6.41s/it]
+ 82%|████████▏ | 1485/1813 [2:43:47<35:21,  6.47s/it]
+ 82%|████████▏ | 1486/1813 [2:43:52<33:11,  6.09s/it]
+ 82%|████████▏ | 1487/1813 [2:44:00<35:56,  6.62s/it]
+ 82%|████████▏ | 1488/1813 [2:44:06<35:22,  6.53s/it]
+ 82%|████████▏ | 1489/1813 [2:44:14<36:49,  6.82s/it]
+ 82%|████████▏ | 1490/1813 [2:44:21<36:38,  6.81s/it]
+                                                     
+{'loss': '1.014', 'grad_norm': '2.084', 'learning_rate': '1.841e-06', 'epoch': '0.8221', 'num_input_tokens_seen': 10021400, 'train_runtime': '9861', 'train_tokens_per_second': '1016'}
+
+ 82%|████████▏ | 1490/1813 [2:44:21<36:38,  6.81s/it]
+ 82%|████████▏ | 1491/1813 [2:44:27<35:09,  6.55s/it]
+ 82%|████████▏ | 1492/1813 [2:44:35<37:44,  7.05s/it]
+ 82%|████████▏ | 1493/1813 [2:44:41<35:42,  6.70s/it]
+ 82%|████████▏ | 1494/1813 [2:44:47<34:33,  6.50s/it]
+ 82%|████████▏ | 1495/1813 [2:44:55<36:56,  6.97s/it]
+ 83%|████████▎ | 1496/1813 [2:45:00<33:49,  6.40s/it]
+ 83%|████████▎ | 1497/1813 [2:45:05<31:39,  6.01s/it]
+ 83%|████████▎ | 1498/1813 [2:45:11<32:08,  6.12s/it]
+ 83%|████████▎ | 1499/1813 [2:45:16<29:33,  5.65s/it]
+ 83%|████████▎ | 1500/1813 [2:45:23<32:19,  6.20s/it]
+                                                     
+{'loss': '0.9051', 'grad_norm': '1.697', 'learning_rate': '1.732e-06', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904, 'train_runtime': '9924', 'train_tokens_per_second': '1017'}
+
+ 83%|████████▎ | 1500/1813 [2:45:23<32:19,  6.20s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.81it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.23it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.33it/s]
+
+  1%|          | 9/1000 [00:01<02:33,  6.47it/s]
+
+  1%|          | 11/1000 [00:01<02:03,  8.03it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  6.97it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.80it/s]
+
+  2%|▏         | 15/1000 [00:02<02:45,  5.95it/s]
+
+  2%|▏         | 16/1000 [00:02<02:52,  5.72it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.14it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.37it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.98it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:06<02:03,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.11it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.34it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.05it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.35it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.75it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.90it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.29it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.38it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.27it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+
+  7%|▋         | 72/1000 [00:09<02:17,  6.73it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.13it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.88it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.32it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.18it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.38it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.18it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.64it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.32it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.45it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.21it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.05it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.54it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.81it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.21it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.07it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:33,  5.71it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.14it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.69it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.13it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.89it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.88it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.06it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.51it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.06it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.40it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.98it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:41,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.82it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.52it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:22,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.76it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.12it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.13it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.58it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.72it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.60it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.90it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.50it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.48it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.66it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:30,  8.72it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:25,  9.13it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.49it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.78it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.82it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.91it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.44it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:25,  8.95it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.35it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  6.01it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.46it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.65it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.25it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.85it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.70it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.51it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.03it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.05it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.01it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.73it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.85it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.12it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:40,  6.93it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.78it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.79it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.25it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.23it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.55it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.23it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.43it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.74it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.97it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.89it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.71it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.94it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.55it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.42it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.74it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.90it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.19it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.18it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.13it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.65it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.38it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.79it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:21,  7.45it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:09,  8.73it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.50it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.26it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.51it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.83it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.51it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 10.98it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:48, 11.76it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.51it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.26it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.20it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.51it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.27it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.84it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.62it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.85it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.20it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.38it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:25,  6.18it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.19it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.29it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.09it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.51it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.05it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.98it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.44it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.92it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.54it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.81it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.80it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.42it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.94it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:49,  9.96it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.38it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:21,  5.82it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.57it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:12,  6.58it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.13it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.73it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.69it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.95it/s]
+
+ 55%|█████▍    | 545/1000 [01:14<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.18it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.67it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  6.97it/s]
+
+ 56%|█████▌    | 559/1000 [01:16<00:52,  8.39it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:46,  9.49it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:54,  8.09it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:55,  7.82it/s]
+
+ 57%|█████▋    | 566/1000 [01:17<01:05,  6.64it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:06,  6.49it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:07,  6.40it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.41it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:08,  6.28it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.33it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.82it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:19<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.54it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.58it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.53it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.81it/s]
+
+ 60%|█████▉    | 595/1000 [01:21<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.67it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.40it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.72it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:10,  5.65it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.07it/s]
+
+ 61%|██████    | 608/1000 [01:23<00:50,  7.82it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.43it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.70it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.78it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:24<00:49,  7.74it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:42,  9.00it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  7.99it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.18it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.42it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.61it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.18it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:39,  9.40it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.33it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.72it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.93it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.77it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:32<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.95it/s]
+
+ 68%|██████▊   | 683/1000 [01:33<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.89it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.82it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.85it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.27it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.35it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.94it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:37<00:44,  6.51it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.72it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.87it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.37it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:38<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.17it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.26it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:39<00:26, 10.16it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.63it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.34it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:27,  9.54it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.90it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.54it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.79it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.35it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.59it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.44it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.69it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.23it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:43<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.71it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.66it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.62it/s]
+
+ 77%|███████▋  | 766/1000 [01:44<00:36,  6.48it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.57it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:27,  8.14it/s]
+
+ 77%|███████▋  | 773/1000 [01:45<00:35,  6.40it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.80it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.59it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.94it/s]
+
+ 78%|███████▊  | 782/1000 [01:46<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:47<00:23,  9.00it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.73it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+
+ 80%|████████  | 804/1000 [01:49<00:25,  7.84it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.18it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.22it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.24it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.32it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.15it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.21it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.78it/s]
+
+ 85%|████████▍ | 847/1000 [01:55<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.56it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.50it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.88it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.05it/s]
+
+ 86%|████████▌ | 857/1000 [01:56<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.76it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.61it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.66it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.31it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.28it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.28it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.77it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.63it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.97it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.20it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.37it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.62it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.86it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.53it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.88it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.09it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.05it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.59it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.78it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.38it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.11it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.62it/s]
+
+ 93%|█████████▎| 934/1000 [02:06<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.38it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.70it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.82it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.40it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.41it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.03it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.43it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.19it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.80it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.37it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.73it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:15<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.8014', 'eval_runtime': '135.4', 'eval_samples_per_second': '7.388', 'eval_steps_per_second': '7.388', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904}
+
+ 83%|████████▎ | 1500/1813 [2:47:39<32:19,  6.20s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+
+                                                   
+ 83%|████████▎ | 1501/1813 [2:47:45<4:03:40, 46.86s/it]
+ 83%|████████▎ | 1502/1813 [2:47:49<2:56:56, 34.14s/it]
+ 83%|████████▎ | 1503/1813 [2:47:56<2:13:15, 25.79s/it]
+ 83%|████████▎ | 1504/1813 [2:48:01<1:41:01, 19.62s/it]
+ 83%|████████▎ | 1505/1813 [2:48:06<1:18:37, 15.31s/it]
+ 83%|████████▎ | 1506/1813 [2:48:13<1:05:45, 12.85s/it]
+ 83%|████████▎ | 1507/1813 [2:48:18<53:40, 10.53s/it]  
+ 83%|████████▎ | 1508/1813 [2:48:25<48:08,  9.47s/it]
+ 83%|████████▎ | 1509/1813 [2:48:31<42:29,  8.39s/it]
+ 83%|████████▎ | 1510/1813 [2:48:37<38:45,  7.68s/it]
+                                                     
+{'loss': '0.9194', 'grad_norm': '1.926', 'learning_rate': '1.627e-06', 'epoch': '0.8331', 'num_input_tokens_seen': 10151928, 'train_runtime': '1.012e+04', 'train_tokens_per_second': '1003'}
+
+ 83%|████████▎ | 1510/1813 [2:48:37<38:45,  7.68s/it]
+ 83%|████████▎ | 1511/1813 [2:48:44<36:40,  7.29s/it]
+ 83%|████████▎ | 1512/1813 [2:48:49<34:10,  6.81s/it]
+ 83%|████████▎ | 1513/1813 [2:48:55<32:06,  6.42s/it]
+ 84%|████████▎ | 1514/1813 [2:49:02<32:25,  6.51s/it]
+ 84%|████████▎ | 1515/1813 [2:49:09<33:53,  6.83s/it]
+ 84%|████████▎ | 1516/1813 [2:49:16<33:56,  6.86s/it]
+ 84%|████████▎ | 1517/1813 [2:49:23<33:20,  6.76s/it]
+ 84%|████████▎ | 1518/1813 [2:49:30<34:23,  6.99s/it]
+ 84%|████████▍ | 1519/1813 [2:49:37<34:26,  7.03s/it]
+ 84%|████████▍ | 1520/1813 [2:49:44<33:21,  6.83s/it]
+                                                     
+{'loss': '0.9457', 'grad_norm': '2.296', 'learning_rate': '1.524e-06', 'epoch': '0.8386', 'num_input_tokens_seen': 10225024, 'train_runtime': '1.018e+04', 'train_tokens_per_second': '1004'}
+
+ 84%|████████▍ | 1520/1813 [2:49:44<33:21,  6.83s/it]
+ 84%|████████▍ | 1521/1813 [2:49:51<33:15,  6.84s/it]
+ 84%|████████▍ | 1522/1813 [2:49:57<32:25,  6.69s/it]
+ 84%|████████▍ | 1523/1813 [2:50:04<33:34,  6.95s/it]
+ 84%|████████▍ | 1524/1813 [2:50:11<32:56,  6.84s/it]
+ 84%|████████▍ | 1525/1813 [2:50:16<30:03,  6.26s/it]
+ 84%|████████▍ | 1526/1813 [2:50:22<30:20,  6.34s/it]
+ 84%|████████▍ | 1527/1813 [2:50:29<30:47,  6.46s/it]
+ 84%|████████▍ | 1528/1813 [2:50:36<30:36,  6.44s/it]
+ 84%|████████▍ | 1529/1813 [2:50:43<31:23,  6.63s/it]
+ 84%|████████▍ | 1530/1813 [2:50:48<29:47,  6.32s/it]
+                                                     
+{'loss': '0.9765', 'grad_norm': '1.285', 'learning_rate': '1.425e-06', 'epoch': '0.8441', 'num_input_tokens_seen': 10294368, 'train_runtime': '1.025e+04', 'train_tokens_per_second': '1004'}
+
+ 84%|████████▍ | 1530/1813 [2:50:48<29:47,  6.32s/it]
+ 84%|████████▍ | 1531/1813 [2:50:54<28:13,  6.01s/it]
+ 85%|████████▍ | 1532/1813 [2:51:00<29:25,  6.28s/it]
+ 85%|████████▍ | 1533/1813 [2:51:05<27:30,  5.90s/it]
+ 85%|████████▍ | 1534/1813 [2:51:14<30:26,  6.55s/it]
+ 85%|████████▍ | 1535/1813 [2:51:19<28:41,  6.19s/it]
+ 85%|████████▍ | 1536/1813 [2:51:26<30:22,  6.58s/it]
+ 85%|████████▍ | 1537/1813 [2:51:33<30:21,  6.60s/it]
+ 85%|████████▍ | 1538/1813 [2:51:38<27:27,  5.99s/it]
+ 85%|████████▍ | 1539/1813 [2:51:43<26:44,  5.86s/it]
+ 85%|████████▍ | 1540/1813 [2:51:49<26:07,  5.74s/it]
+                                                     
+{'loss': '0.956', 'grad_norm': '2.069', 'learning_rate': '1.328e-06', 'epoch': '0.8497', 'num_input_tokens_seen': 10359416, 'train_runtime': '1.031e+04', 'train_tokens_per_second': '1005'}
+
+ 85%|████████▍ | 1540/1813 [2:51:49<26:07,  5.74s/it]
+ 85%|████████▍ | 1541/1813 [2:51:55<27:07,  5.98s/it]
+ 85%|████████▌ | 1542/1813 [2:52:01<27:19,  6.05s/it]
+ 85%|████████▌ | 1543/1813 [2:52:06<25:40,  5.71s/it]
+ 85%|████████▌ | 1544/1813 [2:52:11<24:31,  5.47s/it]
+ 85%|████████▌ | 1545/1813 [2:52:19<27:38,  6.19s/it]
+ 85%|████████▌ | 1546/1813 [2:52:25<27:28,  6.17s/it]
+ 85%|████████▌ | 1547/1813 [2:52:30<26:03,  5.88s/it]
+ 85%|████████▌ | 1548/1813 [2:52:39<28:58,  6.56s/it]
+ 85%|████████▌ | 1549/1813 [2:52:44<27:09,  6.17s/it]
+ 85%|████████▌ | 1550/1813 [2:52:51<28:46,  6.57s/it]
+                                                     
+{'loss': '1.062', 'grad_norm': '1.646', 'learning_rate': '1.235e-06', 'epoch': '0.8552', 'num_input_tokens_seen': 10427368, 'train_runtime': '1.037e+04', 'train_tokens_per_second': '1005'}
+
+ 85%|████████▌ | 1550/1813 [2:52:51<28:46,  6.57s/it]
+ 86%|████████▌ | 1551/1813 [2:52:59<30:10,  6.91s/it]
+ 86%|████████▌ | 1552/1813 [2:53:05<29:00,  6.67s/it]
+ 86%|████████▌ | 1553/1813 [2:53:11<27:47,  6.41s/it]
+ 86%|████████▌ | 1554/1813 [2:53:18<28:04,  6.50s/it]
+ 86%|████████▌ | 1555/1813 [2:53:24<28:07,  6.54s/it]
+ 86%|████████▌ | 1556/1813 [2:53:31<27:47,  6.49s/it]
+ 86%|████████▌ | 1557/1813 [2:53:38<29:25,  6.90s/it]
+ 86%|████████▌ | 1558/1813 [2:53:45<28:23,  6.68s/it]
+ 86%|████████▌ | 1559/1813 [2:53:51<27:52,  6.59s/it]
+ 86%|████████▌ | 1560/1813 [2:53:57<26:24,  6.26s/it]
+                                                     
+{'loss': '1.014', 'grad_norm': '1.478', 'learning_rate': '1.145e-06', 'epoch': '0.8607', 'num_input_tokens_seen': 10497472, 'train_runtime': '1.044e+04', 'train_tokens_per_second': '1006'}
+
+ 86%|████████▌ | 1560/1813 [2:53:57<26:24,  6.26s/it]
+ 86%|████████▌ | 1561/1813 [2:54:04<27:16,  6.50s/it]
+ 86%|████████▌ | 1562/1813 [2:54:10<27:11,  6.50s/it]
+ 86%|████████▌ | 1563/1813 [2:54:16<26:46,  6.42s/it]
+ 86%|████████▋ | 1564/1813 [2:54:23<26:36,  6.41s/it]
+ 86%|████████▋ | 1565/1813 [2:54:27<24:18,  5.88s/it]
+ 86%|████████▋ | 1566/1813 [2:54:33<24:28,  5.95s/it]
+ 86%|████████▋ | 1567/1813 [2:54:39<24:28,  5.97s/it]
+ 86%|████████▋ | 1568/1813 [2:54:44<22:51,  5.60s/it]
+ 87%|████████▋ | 1569/1813 [2:54:52<25:16,  6.21s/it]
+ 87%|████████▋ | 1570/1813 [2:54:59<26:26,  6.53s/it]
+                                                     
+{'loss': '0.8988', 'grad_norm': '1.788', 'learning_rate': '1.058e-06', 'epoch': '0.8662', 'num_input_tokens_seen': 10564336, 'train_runtime': '1.05e+04', 'train_tokens_per_second': '1006'}
+
+ 87%|████████▋ | 1570/1813 [2:54:59<26:26,  6.53s/it]
+ 87%|████████▋ | 1571/1813 [2:55:05<25:05,  6.22s/it]
+ 87%|████████▋ | 1572/1813 [2:55:10<24:12,  6.02s/it]
+ 87%|████████▋ | 1573/1813 [2:55:16<23:29,  5.87s/it]
+ 87%|████████▋ | 1574/1813 [2:55:22<23:57,  6.01s/it]
+ 87%|████████▋ | 1575/1813 [2:55:28<23:43,  5.98s/it]
+ 87%|████████▋ | 1576/1813 [2:55:35<25:15,  6.40s/it]
+ 87%|████████▋ | 1577/1813 [2:55:42<25:41,  6.53s/it]
+ 87%|████████▋ | 1578/1813 [2:55:48<25:19,  6.47s/it]
+ 87%|████████▋ | 1579/1813 [2:55:55<24:48,  6.36s/it]
+ 87%|████████▋ | 1580/1813 [2:56:00<23:03,  5.94s/it]
+                                                     
+{'loss': '0.9684', 'grad_norm': '2.604', 'learning_rate': '9.746e-07', 'epoch': '0.8717', 'num_input_tokens_seen': 10628400, 'train_runtime': '1.056e+04', 'train_tokens_per_second': '1006'}
+
+ 87%|████████▋ | 1580/1813 [2:56:00<23:03,  5.94s/it]
+ 87%|████████▋ | 1581/1813 [2:56:05<22:44,  5.88s/it]
+ 87%|████████▋ | 1582/1813 [2:56:09<20:40,  5.37s/it]
+ 87%|████████▋ | 1583/1813 [2:56:16<21:32,  5.62s/it]
+ 87%|████████▋ | 1584/1813 [2:56:22<22:11,  5.81s/it]
+ 87%|████████▋ | 1585/1813 [2:56:29<23:39,  6.22s/it]
+ 87%|████████▋ | 1586/1813 [2:56:36<24:48,  6.56s/it]
+ 88%|████████▊ | 1587/1813 [2:56:44<25:48,  6.85s/it]
+ 88%|████████▊ | 1588/1813 [2:56:50<24:18,  6.48s/it]
+ 88%|████████▊ | 1589/1813 [2:56:55<22:31,  6.03s/it]
+ 88%|████████▊ | 1590/1813 [2:57:01<23:00,  6.19s/it]
+                                                     
+{'loss': '0.9383', 'grad_norm': '1.495', 'learning_rate': '8.943e-07', 'epoch': '0.8772', 'num_input_tokens_seen': 10695288, 'train_runtime': '1.062e+04', 'train_tokens_per_second': '1007'}
+
+ 88%|████████▊ | 1590/1813 [2:57:01<23:00,  6.19s/it]
+ 88%|████████▊ | 1591/1813 [2:57:07<22:43,  6.14s/it]
+ 88%|████████▊ | 1592/1813 [2:57:14<22:50,  6.20s/it]
+ 88%|████████▊ | 1593/1813 [2:57:20<23:10,  6.32s/it]
+ 88%|████████▊ | 1594/1813 [2:57:27<23:23,  6.41s/it]
+ 88%|████████▊ | 1595/1813 [2:57:34<24:13,  6.67s/it]
+ 88%|████████▊ | 1596/1813 [2:57:43<26:22,  7.29s/it]
+ 88%|████████▊ | 1597/1813 [2:57:49<25:34,  7.10s/it]
+ 88%|████████▊ | 1598/1813 [2:57:56<24:43,  6.90s/it]
+ 88%|████████▊ | 1599/1813 [2:58:01<23:07,  6.48s/it]
+ 88%|████████▊ | 1600/1813 [2:58:06<21:06,  5.95s/it]
+                                                     
+{'loss': '0.9869', 'grad_norm': '1.624', 'learning_rate': '8.173e-07', 'epoch': '0.8828', 'num_input_tokens_seen': 10767512, 'train_runtime': '1.069e+04', 'train_tokens_per_second': '1008'}
+
+ 88%|████████▊ | 1600/1813 [2:58:06<21:06,  5.95s/it]
+ 88%|████████▊ | 1601/1813 [2:58:14<23:04,  6.53s/it]
+ 88%|████████▊ | 1602/1813 [2:58:20<22:19,  6.35s/it]
+ 88%|████████▊ | 1603/1813 [2:58:27<22:38,  6.47s/it]
+ 88%|████████▊ | 1604/1813 [2:58:33<22:46,  6.54s/it]
+ 89%|████████▊ | 1605/1813 [2:58:42<24:26,  7.05s/it]
+ 89%|████████▊ | 1606/1813 [2:58:47<22:16,  6.45s/it]
+ 89%|████████▊ | 1607/1813 [2:58:53<22:21,  6.51s/it]
+ 89%|████████▊ | 1608/1813 [2:59:01<23:02,  6.74s/it]
+ 89%|████████▊ | 1609/1813 [2:59:06<21:26,  6.31s/it]
+ 89%|████████▉ | 1610/1813 [2:59:12<20:54,  6.18s/it]
+                                                     
+{'loss': '0.9558', 'grad_norm': '2.11', 'learning_rate': '7.436e-07', 'epoch': '0.8883', 'num_input_tokens_seen': 10840800, 'train_runtime': '1.075e+04', 'train_tokens_per_second': '1008'}
+
+ 89%|████████▉ | 1610/1813 [2:59:12<20:54,  6.18s/it]
+ 89%|████████▉ | 1611/1813 [2:59:17<19:30,  5.80s/it]
+ 89%|████████▉ | 1612/1813 [2:59:22<19:26,  5.80s/it]
+ 89%|████████▉ | 1613/1813 [2:59:27<17:57,  5.39s/it]
+ 89%|████████▉ | 1614/1813 [2:59:33<18:26,  5.56s/it]
+ 89%|████████▉ | 1615/1813 [2:59:40<19:28,  5.90s/it]
+ 89%|████████▉ | 1616/1813 [2:59:46<20:06,  6.12s/it]
+ 89%|████████▉ | 1617/1813 [2:59:53<20:32,  6.29s/it]
+ 89%|████████▉ | 1618/1813 [2:59:58<19:21,  5.95s/it]
+ 89%|████████▉ | 1619/1813 [3:00:04<18:57,  5.86s/it]
+ 89%|████████▉ | 1620/1813 [3:00:09<17:52,  5.56s/it]
+                                                     
+{'loss': '0.9626', 'grad_norm': '1.843', 'learning_rate': '6.733e-07', 'epoch': '0.8938', 'num_input_tokens_seen': 10899640, 'train_runtime': '1.081e+04', 'train_tokens_per_second': '1008'}
+
+ 89%|████████▉ | 1620/1813 [3:00:09<17:52,  5.56s/it]
+ 89%|████████▉ | 1621/1813 [3:00:14<17:31,  5.48s/it]
+ 89%|████████▉ | 1622/1813 [3:00:21<19:21,  6.08s/it]
+ 90%|████████▉ | 1623/1813 [3:00:28<19:58,  6.31s/it]
+ 90%|████████▉ | 1624/1813 [3:00:34<19:30,  6.20s/it]
+ 90%|████████▉ | 1625/1813 [3:00:41<20:26,  6.52s/it]
+ 90%|████████▉ | 1626/1813 [3:00:47<19:49,  6.36s/it]
+ 90%|████████▉ | 1627/1813 [3:00:53<19:12,  6.19s/it]
+ 90%|████████▉ | 1628/1813 [3:00:59<18:32,  6.01s/it]
+ 90%|████████▉ | 1629/1813 [3:01:06<19:28,  6.35s/it]
+ 90%|████████▉ | 1630/1813 [3:01:11<18:20,  6.02s/it]
+                                                     
+{'loss': '0.9749', 'grad_norm': '2.952', 'learning_rate': '6.064e-07', 'epoch': '0.8993', 'num_input_tokens_seen': 10968528, 'train_runtime': '1.087e+04', 'train_tokens_per_second': '1009'}
+
+ 90%|████████▉ | 1630/1813 [3:01:11<18:20,  6.02s/it]
+ 90%|████████▉ | 1631/1813 [3:01:17<17:39,  5.82s/it]
+ 90%|█████████ | 1632/1813 [3:01:21<16:36,  5.50s/it]
+ 90%|█████████ | 1633/1813 [3:01:29<18:10,  6.06s/it]
+ 90%|█████████ | 1634/1813 [3:01:33<16:59,  5.70s/it]
+ 90%|█████████ | 1635/1813 [3:01:40<17:47,  6.00s/it]
+ 90%|█████████ | 1636/1813 [3:01:46<17:12,  5.83s/it]
+ 90%|█████████ | 1637/1813 [3:01:52<17:32,  5.98s/it]
+ 90%|█████████ | 1638/1813 [3:01:57<17:03,  5.85s/it]
+ 90%|█████████ | 1639/1813 [3:02:04<17:46,  6.13s/it]
+ 90%|█████████ | 1640/1813 [3:02:09<16:49,  5.84s/it]
+                                                     
+{'loss': '0.9791', 'grad_norm': '1.325', 'learning_rate': '5.428e-07', 'epoch': '0.9048', 'num_input_tokens_seen': 11030048, 'train_runtime': '1.093e+04', 'train_tokens_per_second': '1009'}
+
+ 90%|█████████ | 1640/1813 [3:02:09<16:49,  5.84s/it]
+ 91%|█████████ | 1641/1813 [3:02:15<16:39,  5.81s/it]
+ 91%|█████████ | 1642/1813 [3:02:22<17:01,  5.98s/it]
+ 91%|█████████ | 1643/1813 [3:02:28<16:59,  6.00s/it]
+ 91%|█████████ | 1644/1813 [3:02:34<17:28,  6.21s/it]
+ 91%|█████████ | 1645/1813 [3:02:44<20:04,  7.17s/it]
+ 91%|█████████ | 1646/1813 [3:02:50<19:30,  7.01s/it]
+ 91%|█████████ | 1647/1813 [3:02:56<18:32,  6.70s/it]
+ 91%|█████████ | 1648/1813 [3:03:03<18:14,  6.63s/it]
+ 91%|█████████ | 1649/1813 [3:03:08<16:49,  6.16s/it]
+ 91%|█████████ | 1650/1813 [3:03:13<15:55,  5.86s/it]
+                                                     
+{'loss': '0.9422', 'grad_norm': '1.703', 'learning_rate': '4.827e-07', 'epoch': '0.9103', 'num_input_tokens_seen': 11098768, 'train_runtime': '1.099e+04', 'train_tokens_per_second': '1010'}
+
+ 91%|█████████ | 1650/1813 [3:03:13<15:55,  5.86s/it]
+ 91%|█████████ | 1651/1813 [3:03:19<16:14,  6.01s/it]
+ 91%|█████████ | 1652/1813 [3:03:26<16:24,  6.11s/it]
+ 91%|█████████ | 1653/1813 [3:03:32<16:39,  6.25s/it]
+ 91%|█████████ | 1654/1813 [3:03:38<16:26,  6.20s/it]
+ 91%|█████████▏| 1655/1813 [3:03:45<16:36,  6.30s/it]
+ 91%|█████████▏| 1656/1813 [3:03:51<16:08,  6.17s/it]
+ 91%|█████████▏| 1657/1813 [3:03:58<17:11,  6.61s/it]
+ 91%|█████████▏| 1658/1813 [3:04:05<16:53,  6.54s/it]
+ 92%|█████████▏| 1659/1813 [3:04:12<17:06,  6.66s/it]
+ 92%|█████████▏| 1660/1813 [3:04:18<16:42,  6.55s/it]
+                                                     
+{'loss': '0.9932', 'grad_norm': '2.114', 'learning_rate': '4.26e-07', 'epoch': '0.9159', 'num_input_tokens_seen': 11168848, 'train_runtime': '1.106e+04', 'train_tokens_per_second': '1010'}
+
+ 92%|█████████▏| 1660/1813 [3:04:18<16:42,  6.55s/it]
+ 92%|█████████▏| 1661/1813 [3:04:25<16:55,  6.68s/it]
+ 92%|█████████▏| 1662/1813 [3:04:32<16:41,  6.63s/it]
+ 92%|█████████▏| 1663/1813 [3:04:39<16:54,  6.76s/it]
+ 92%|█████████▏| 1664/1813 [3:04:46<16:53,  6.80s/it]
+ 92%|█████████▏| 1665/1813 [3:04:53<17:22,  7.04s/it]
+ 92%|█████████▏| 1666/1813 [3:04:58<15:19,  6.25s/it]
+ 92%|█████████▏| 1667/1813 [3:05:04<15:28,  6.36s/it]
+ 92%|█████████▏| 1668/1813 [3:05:10<15:05,  6.24s/it]
+ 92%|█████████▏| 1669/1813 [3:05:20<17:37,  7.34s/it]
+ 92%|█████████▏| 1670/1813 [3:05:25<15:58,  6.70s/it]
+                                                     
+{'loss': '0.9552', 'grad_norm': '1.965', 'learning_rate': '3.728e-07', 'epoch': '0.9214', 'num_input_tokens_seen': 11244288, 'train_runtime': '1.113e+04', 'train_tokens_per_second': '1011'}
+
+ 92%|█████████▏| 1670/1813 [3:05:25<15:58,  6.70s/it]
+ 92%|█████████▏| 1671/1813 [3:05:30<14:41,  6.21s/it]
+ 92%|█████████▏| 1672/1813 [3:05:36<14:13,  6.05s/it]
+ 92%|█████████▏| 1673/1813 [3:05:42<13:55,  5.97s/it]
+ 92%|█████████▏| 1674/1813 [3:05:49<14:27,  6.24s/it]
+ 92%|█████████▏| 1675/1813 [3:05:56<14:57,  6.50s/it]
+ 92%|█████████▏| 1676/1813 [3:06:03<15:23,  6.74s/it]
+ 92%|█████████▏| 1677/1813 [3:06:09<15:00,  6.62s/it]
+ 93%|█████████▎| 1678/1813 [3:06:16<14:40,  6.52s/it]
+ 93%|█████████▎| 1679/1813 [3:06:21<14:04,  6.30s/it]
+ 93%|█████████▎| 1680/1813 [3:06:30<15:32,  7.01s/it]
+                                                     
+{'loss': '1.012', 'grad_norm': '1.748', 'learning_rate': '3.231e-07', 'epoch': '0.9269', 'num_input_tokens_seen': 11317056, 'train_runtime': '1.119e+04', 'train_tokens_per_second': '1011'}
+
+ 93%|█████████▎| 1680/1813 [3:06:30<15:32,  7.01s/it]
+ 93%|█████████▎| 1681/1813 [3:06:35<13:55,  6.33s/it]
+ 93%|█████████▎| 1682/1813 [3:06:40<13:01,  5.96s/it]
+ 93%|█████████▎| 1683/1813 [3:06:45<12:01,  5.55s/it]
+ 93%|█████████▎| 1684/1813 [3:06:51<12:43,  5.92s/it]
+ 93%|█████████▎| 1685/1813 [3:06:58<13:04,  6.13s/it]
+ 93%|█████████▎| 1686/1813 [3:07:05<13:26,  6.35s/it]
+ 93%|█████████▎| 1687/1813 [3:07:11<13:25,  6.39s/it]
+ 93%|█████████▎| 1688/1813 [3:07:18<13:34,  6.51s/it]
+ 93%|█████████▎| 1689/1813 [3:07:23<12:19,  5.96s/it]
+ 93%|█████████▎| 1690/1813 [3:07:29<12:32,  6.12s/it]
+                                                     
+{'loss': '0.935', 'grad_norm': '1.34', 'learning_rate': '2.769e-07', 'epoch': '0.9324', 'num_input_tokens_seen': 11380560, 'train_runtime': '1.125e+04', 'train_tokens_per_second': '1012'}
+
+ 93%|█████████▎| 1690/1813 [3:07:29<12:32,  6.12s/it]
+ 93%|█████████▎| 1691/1813 [3:07:35<12:27,  6.12s/it]
+ 93%|█████████▎| 1692/1813 [3:07:43<13:12,  6.55s/it]
+ 93%|█████████▎| 1693/1813 [3:07:49<12:49,  6.41s/it]
+ 93%|█████████▎| 1694/1813 [3:07:57<13:47,  6.95s/it]
+ 93%|█████████▎| 1695/1813 [3:08:03<13:04,  6.65s/it]
+ 94%|█████████▎| 1696/1813 [3:08:09<12:23,  6.35s/it]
+ 94%|█████████▎| 1697/1813 [3:08:15<12:15,  6.34s/it]
+ 94%|█████████▎| 1698/1813 [3:08:20<11:27,  5.98s/it]
+ 94%|█████████▎| 1699/1813 [3:08:27<11:34,  6.09s/it]
+ 94%|█████████▍| 1700/1813 [3:08:33<11:33,  6.14s/it]
+                                                     
+{'loss': '0.9701', 'grad_norm': '2.14', 'learning_rate': '2.342e-07', 'epoch': '0.9379', 'num_input_tokens_seen': 11450712, 'train_runtime': '1.131e+04', 'train_tokens_per_second': '1012'}
+
+ 94%|█████████▍| 1700/1813 [3:08:33<11:33,  6.14s/it]
+ 94%|█████████▍| 1701/1813 [3:08:37<10:31,  5.64s/it]
+ 94%|█████████▍| 1702/1813 [3:08:42<10:08,  5.48s/it]
+ 94%|█████████▍| 1703/1813 [3:08:48<10:08,  5.53s/it]
+ 94%|█████████▍| 1704/1813 [3:08:56<11:09,  6.14s/it]
+ 94%|█████████▍| 1705/1813 [3:09:02<11:12,  6.23s/it]
+ 94%|█████████▍| 1706/1813 [3:09:08<11:07,  6.23s/it]
+ 94%|█████████▍| 1707/1813 [3:09:14<10:40,  6.04s/it]
+ 94%|█████████▍| 1708/1813 [3:09:19<09:56,  5.68s/it]
+ 94%|█████████▍| 1709/1813 [3:09:25<10:04,  5.81s/it]
+ 94%|█████████▍| 1710/1813 [3:09:31<09:57,  5.80s/it]
+                                                     
+{'loss': '0.9712', 'grad_norm': '2.14', 'learning_rate': '1.951e-07', 'epoch': '0.9434', 'num_input_tokens_seen': 11511984, 'train_runtime': '1.137e+04', 'train_tokens_per_second': '1012'}
+
+ 94%|█████████▍| 1710/1813 [3:09:31<09:57,  5.80s/it]
+ 94%|█████████▍| 1711/1813 [3:09:36<09:42,  5.71s/it]
+ 94%|█████████▍| 1712/1813 [3:09:42<09:34,  5.68s/it]
+ 94%|█████████▍| 1713/1813 [3:09:48<09:31,  5.71s/it]
+ 95%|█████████▍| 1714/1813 [3:09:53<09:28,  5.74s/it]
+ 95%|█████████▍| 1715/1813 [3:09:59<09:17,  5.69s/it]
+ 95%|█████████▍| 1716/1813 [3:10:04<08:56,  5.54s/it]
+ 95%|█████████▍| 1717/1813 [3:10:09<08:41,  5.43s/it]
+ 95%|█████████▍| 1718/1813 [3:10:15<08:56,  5.65s/it]
+ 95%|█████████▍| 1719/1813 [3:10:22<09:09,  5.85s/it]
+ 95%|█████████▍| 1720/1813 [3:10:28<09:11,  5.93s/it]
+                                                     
+{'loss': '0.9653', 'grad_norm': '1.565', 'learning_rate': '1.594e-07', 'epoch': '0.949', 'num_input_tokens_seen': 11571368, 'train_runtime': '1.143e+04', 'train_tokens_per_second': '1013'}
+
+ 95%|█████████▍| 1720/1813 [3:10:28<09:11,  5.93s/it]
+ 95%|█████████▍| 1721/1813 [3:10:35<09:45,  6.37s/it]
+ 95%|█████████▍| 1722/1813 [3:10:41<09:29,  6.26s/it]
+ 95%|█████████▌| 1723/1813 [3:10:47<09:16,  6.19s/it]
+ 95%|█████████▌| 1724/1813 [3:10:53<09:04,  6.12s/it]
+ 95%|█████████▌| 1725/1813 [3:11:01<09:29,  6.48s/it]
+ 95%|█████████▌| 1726/1813 [3:11:08<09:46,  6.74s/it]
+ 95%|█████████▌| 1727/1813 [3:11:14<09:22,  6.54s/it]
+ 95%|█████████▌| 1728/1813 [3:11:19<08:45,  6.18s/it]
+ 95%|█████████▌| 1729/1813 [3:11:25<08:21,  5.97s/it]
+ 95%|█████████▌| 1730/1813 [3:11:32<08:34,  6.20s/it]
+                                                     
+{'loss': '0.9773', 'grad_norm': '1.129', 'learning_rate': '1.274e-07', 'epoch': '0.9545', 'num_input_tokens_seen': 11640392, 'train_runtime': '1.149e+04', 'train_tokens_per_second': '1013'}
+
+ 95%|█████████▌| 1730/1813 [3:11:32<08:34,  6.20s/it]
+ 95%|█████████▌| 1731/1813 [3:11:38<08:26,  6.17s/it]
+ 96%|█████████▌| 1732/1813 [3:11:45<08:37,  6.39s/it]
+ 96%|█████████▌| 1733/1813 [3:11:50<08:03,  6.05s/it]
+ 96%|█████████▌| 1734/1813 [3:11:56<07:52,  5.98s/it]
+ 96%|█████████▌| 1735/1813 [3:12:02<07:57,  6.12s/it]
+ 96%|█████████▌| 1736/1813 [3:12:09<08:12,  6.40s/it]
+ 96%|█████████▌| 1737/1813 [3:12:15<08:04,  6.37s/it]
+ 96%|█████████▌| 1738/1813 [3:12:21<07:29,  5.99s/it]
+ 96%|█████████▌| 1739/1813 [3:12:29<08:08,  6.60s/it]
+ 96%|█████████▌| 1740/1813 [3:12:35<08:01,  6.60s/it]
+                                                     
+{'loss': '0.9618', 'grad_norm': '1.94', 'learning_rate': '9.891e-08', 'epoch': '0.96', 'num_input_tokens_seen': 11709120, 'train_runtime': '1.156e+04', 'train_tokens_per_second': '1013'}
+
+ 96%|█████████▌| 1740/1813 [3:12:35<08:01,  6.60s/it]
+ 96%|█████████▌| 1741/1813 [3:12:41<07:36,  6.34s/it]
+ 96%|█████████▌| 1742/1813 [3:12:48<07:41,  6.50s/it]
+ 96%|█████████▌| 1743/1813 [3:12:55<07:44,  6.64s/it]
+ 96%|█████████▌| 1744/1813 [3:13:02<07:50,  6.81s/it]
+ 96%|█████████▌| 1745/1813 [3:13:08<07:27,  6.58s/it]
+ 96%|█████████▋| 1746/1813 [3:13:16<07:39,  6.85s/it]
+ 96%|█████████▋| 1747/1813 [3:13:21<06:56,  6.31s/it]
+ 96%|█████████▋| 1748/1813 [3:13:28<07:12,  6.65s/it]
+ 96%|█████████▋| 1749/1813 [3:13:33<06:41,  6.27s/it]
+ 97%|█████████▋| 1750/1813 [3:13:39<06:17,  6.00s/it]
+                                                     
+{'loss': '0.9789', 'grad_norm': '1.341', 'learning_rate': '7.401e-08', 'epoch': '0.9655', 'num_input_tokens_seen': 11777768, 'train_runtime': '1.162e+04', 'train_tokens_per_second': '1014'}
+
+ 97%|█████████▋| 1750/1813 [3:13:39<06:17,  6.00s/it]
+ 97%|█████████▋| 1751/1813 [3:13:45<06:12,  6.01s/it]
+ 97%|█████████▋| 1752/1813 [3:13:51<06:05,  5.98s/it]
+ 97%|█████████▋| 1753/1813 [3:13:58<06:28,  6.47s/it]
+ 97%|█████████▋| 1754/1813 [3:14:06<06:36,  6.72s/it]
+ 97%|█████████▋| 1755/1813 [3:14:12<06:24,  6.62s/it]
+ 97%|█████████▋| 1756/1813 [3:14:17<05:54,  6.22s/it]
+ 97%|█████████▋| 1757/1813 [3:14:23<05:36,  6.01s/it]
+ 97%|█████████▋| 1758/1813 [3:14:28<05:24,  5.89s/it]
+ 97%|█████████▋| 1759/1813 [3:14:35<05:22,  5.97s/it]
+ 97%|█████████▋| 1760/1813 [3:14:40<05:09,  5.84s/it]
+                                                     
+{'loss': '0.9583', 'grad_norm': '1.95', 'learning_rate': '5.271e-08', 'epoch': '0.971', 'num_input_tokens_seen': 11843192, 'train_runtime': '1.168e+04', 'train_tokens_per_second': '1014'}
+
+ 97%|█████████▋| 1760/1813 [3:14:40<05:09,  5.84s/it]
+ 97%|█████████▋| 1761/1813 [3:14:46<04:57,  5.72s/it]
+ 97%|█████████▋| 1762/1813 [3:14:50<04:34,  5.38s/it]
+ 97%|█████████▋| 1763/1813 [3:14:58<05:02,  6.04s/it]
+ 97%|█████████▋| 1764/1813 [3:15:03<04:41,  5.75s/it]
+ 97%|█████████▋| 1765/1813 [3:15:08<04:28,  5.60s/it]
+ 97%|█████████▋| 1766/1813 [3:15:14<04:22,  5.58s/it]
+ 97%|█████████▋| 1767/1813 [3:15:19<04:15,  5.56s/it]
+ 98%|█████████▊| 1768/1813 [3:15:25<04:12,  5.62s/it]
+ 98%|█████████▊| 1769/1813 [3:15:31<04:08,  5.65s/it]
+ 98%|█████████▊| 1770/1813 [3:15:37<04:15,  5.95s/it]
+                                                     
+{'loss': '0.9702', 'grad_norm': '1.576', 'learning_rate': '3.501e-08', 'epoch': '0.9766', 'num_input_tokens_seen': 11903728, 'train_runtime': '1.174e+04', 'train_tokens_per_second': '1014'}
+
+ 98%|█████████▊| 1770/1813 [3:15:37<04:15,  5.95s/it]
+ 98%|█████████▊| 1771/1813 [3:15:44<04:13,  6.05s/it]
+ 98%|█████████▊| 1772/1813 [3:15:50<04:11,  6.13s/it]
+ 98%|█████████▊| 1773/1813 [3:15:57<04:15,  6.39s/it]
+ 98%|█████████▊| 1774/1813 [3:16:01<03:47,  5.84s/it]
+ 98%|█████████▊| 1775/1813 [3:16:09<04:01,  6.36s/it]
+ 98%|█████████▊| 1776/1813 [3:16:16<04:07,  6.69s/it]
+ 98%|█████████▊| 1777/1813 [3:16:24<04:10,  6.96s/it]
+ 98%|█████████▊| 1778/1813 [3:16:30<03:48,  6.53s/it]
+ 98%|█████████▊| 1779/1813 [3:16:35<03:30,  6.20s/it]
+ 98%|█████████▊| 1780/1813 [3:16:41<03:20,  6.08s/it]
+                                                     
+{'loss': '0.9799', 'grad_norm': '1.22', 'learning_rate': '2.091e-08', 'epoch': '0.9821', 'num_input_tokens_seen': 11972864, 'train_runtime': '1.18e+04', 'train_tokens_per_second': '1015'}
+
+ 98%|█████████▊| 1780/1813 [3:16:41<03:20,  6.08s/it]
+ 98%|█████████▊| 1781/1813 [3:16:47<03:16,  6.13s/it]
+ 98%|█████████▊| 1782/1813 [3:16:53<03:08,  6.07s/it]
+ 98%|█████████▊| 1783/1813 [3:16:59<03:02,  6.09s/it]
+ 98%|█████████▊| 1784/1813 [3:17:06<03:04,  6.35s/it]
+ 98%|█████████▊| 1785/1813 [3:17:12<02:56,  6.32s/it]
+ 99%|█████████▊| 1786/1813 [3:17:19<02:50,  6.30s/it]
+ 99%|█████████▊| 1787/1813 [3:17:24<02:37,  6.05s/it]
+ 99%|█████████▊| 1788/1813 [3:17:31<02:35,  6.21s/it]
+ 99%|█████████▊| 1789/1813 [3:17:39<02:45,  6.92s/it]
+ 99%|█████████▊| 1790/1813 [3:17:46<02:41,  7.02s/it]
+                                                     
+{'loss': '0.9627', 'grad_norm': '1.617', 'learning_rate': '1.042e-08', 'epoch': '0.9876', 'num_input_tokens_seen': 12043496, 'train_runtime': '1.187e+04', 'train_tokens_per_second': '1015'}
+
+ 99%|█████████▊| 1790/1813 [3:17:46<02:41,  7.02s/it]
+ 99%|█████████▉| 1791/1813 [3:17:52<02:24,  6.55s/it]
+ 99%|█████████▉| 1792/1813 [3:17:58<02:16,  6.48s/it]
+ 99%|█████████▉| 1793/1813 [3:18:05<02:12,  6.62s/it]
+ 99%|█████████▉| 1794/1813 [3:18:12<02:08,  6.74s/it]
+ 99%|█████████▉| 1795/1813 [3:18:19<02:01,  6.74s/it]
+ 99%|█████████▉| 1796/1813 [3:18:26<01:55,  6.79s/it]
+ 99%|█████████▉| 1797/1813 [3:18:32<01:46,  6.64s/it]
+ 99%|█████████▉| 1798/1813 [3:18:41<01:51,  7.42s/it]
+ 99%|█████████▉| 1799/1813 [3:18:48<01:39,  7.11s/it]
+ 99%|█████████▉| 1800/1813 [3:18:52<01:22,  6.32s/it]
+                                                     
+{'loss': '0.9408', 'grad_norm': '1.455', 'learning_rate': '3.546e-09', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448, 'train_runtime': '1.193e+04', 'train_tokens_per_second': '1015'}
+
+ 99%|█████████▉| 1800/1813 [3:18:52<01:22,  6.32s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:02, 15.84it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.23it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.34it/s]
+
+  1%|          | 9/1000 [00:01<02:32,  6.48it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.06it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.00it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.83it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.06it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.61it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.26it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.37it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.27it/s]
+
+  3%|▎         | 28/1000 [00:04<02:41,  6.00it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  6.00it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.77it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.69it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.13it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.59it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.12it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.35it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.74it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.90it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.75it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.31it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:35,  9.77it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.27it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.01it/s]
+
+  7%|▋         | 72/1000 [00:09<02:17,  6.73it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.13it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.88it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.69it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.34it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.88it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.21it/s]
+
+  8%|▊         | 84/1000 [00:11<01:53,  8.04it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.96it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:10,  7.00it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:26,  6.18it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.53it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:12,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:49,  8.23it/s]
+
+ 10%|█         | 101/1000 [00:13<01:34,  9.51it/s]
+
+ 10%|█         | 103/1000 [00:14<01:56,  7.72it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.80it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.31it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.43it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.20it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.04it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.52it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.82it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.25it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.09it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.44it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.09it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:42,  8.50it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:27,  9.87it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:31,  9.49it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.82it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.92it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  8.98it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.84it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25, 10.04it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.51it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.92it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.02it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.38it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.42it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.19it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:23,  5.79it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.17it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.16it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.23it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.74it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:40,  8.20it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:02,  6.69it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:36,  8.48it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:37,  8.45it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.72it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.69it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.78it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:42,  7.91it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.40it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.12it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.50it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.39it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.20it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.68it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.79it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.65it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.96it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.53it/s]
+
+ 21%|██        | 210/1000 [00:27<01:32,  8.50it/s]
+
+ 21%|██        | 212/1000 [00:27<01:23,  9.39it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:31,  8.61it/s]
+
+ 22%|██▏       | 216/1000 [00:28<01:26,  9.05it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:45,  7.44it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:29,  8.71it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:04,  6.28it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:04,  6.28it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:20,  5.54it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.03it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.09it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.22it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.94it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.47it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.99it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.38it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.03it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.27it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:33<02:05,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.74it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.70it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.53it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.44it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.61it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:45,  6.93it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:36,  7.56it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.05it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:13,  5.45it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.71it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:47,  6.72it/s]
+
+ 28%|██▊       | 277/1000 [00:37<01:29,  8.07it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.82it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.63it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.87it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.72it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.72it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.13it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.77it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.09it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.16it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.96it/s]
+
+ 31%|███       | 307/1000 [00:41<01:41,  6.80it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.82it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.21it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.28it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.26it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.57it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.76it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.59it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.97it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.39it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.05it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.84it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.05it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.91it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.74it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.23it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.43it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:33,  6.93it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:48,  5.97it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.56it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.37it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.60it/s]
+
+ 36%|███▌      | 361/1000 [00:48<01:07,  9.44it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.20it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.29it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.93it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:18,  7.96it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:41,  6.15it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:54,  5.45it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:30,  6.89it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.74it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.90it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.20it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.20it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.15it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.66it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.38it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:08,  8.80it/s]
+
+ 40%|███▉      | 395/1000 [00:53<01:20,  7.48it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.76it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.25it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+
+ 41%|████      | 408/1000 [00:55<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:50,  5.35it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.04it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.25it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.50it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.53it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.55it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.02it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.79it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.50it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.08it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.31it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.71it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.01it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:16,  7.18it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.50it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.83it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.58it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.73it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.01it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.58it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.52it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.39it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.31it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.01it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:16,  6.64it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.42it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.45it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:12,  6.81it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.57it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.55it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:07,  7.13it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.09it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.40it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.05it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.60it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.55it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.42it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.14it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:47,  9.75it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.71it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.38it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.67it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.77it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:26,  5.28it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.95it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.71it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.93it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.52it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.28it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.96it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.86it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.05it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.99it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.46it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.84it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.26it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.57it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.61it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.93it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.17it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.55it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.77it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.18it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.81it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.35it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.86it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.48it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.74it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.82it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.62it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.11it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  7.93it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.14it/s]
+
+ 62%|██████▎   | 625/1000 [01:25<00:50,  7.42it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.60it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.18it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.86it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.42it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.08it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.95it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.57it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.33it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.71it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.57it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.50it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.79it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.34it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.64it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.70it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.97it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.89it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.88it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.82it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.85it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.28it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.44it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.35it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:40,  7.23it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.04it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.69it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.85it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.37it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.31it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.14it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.61it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.53it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.90it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.54it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:27,  9.30it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.19it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.56it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.64it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.13it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.20it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.39it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.59it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:33,  7.23it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.64it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.63it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.59it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.46it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.78it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.57it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:27,  8.14it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:25,  8.38it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.17it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.55it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.38it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.01it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.73it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.98it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.01it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:24,  7.85it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.18it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.19it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.22it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.07it/s]
+
+ 82%|████████▏ | 815/1000 [01:50<00:19,  9.33it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.15it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.21it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.06it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.83it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.81it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.25it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.23it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.12it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.46it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.17it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.43it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.78it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.57it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.50it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.88it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.48it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:13, 10.22it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.53it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.79it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.65it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.71it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.33it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.27it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.28it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.79it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.64it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.78it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.63it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.86it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.71it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.87it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.09it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.06it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.60it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.79it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.40it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.08it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.38it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.61it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.58it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.19it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.68it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:08,  8.11it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.64it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.84it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.63it/s]
+
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.84it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.07it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.17it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.35it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.08it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.84it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.37it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.38it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.77it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.81it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.04it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.79it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.43it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.07it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.44it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.01it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.54it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.8', 'eval_runtime': '135.3', 'eval_samples_per_second': '7.391', 'eval_steps_per_second': '7.391', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448}
+
+ 99%|█████████▉| 1800/1813 [3:21:07<01:22,  6.32s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_007_lr_2.028e-05/checkpoint-1800
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+ 99%|█████████▉| 1801/1813 [3:21:16<09:31, 47.64s/it]
+ 99%|█████████▉| 1802/1813 [3:21:21<06:24, 34.92s/it]
+ 99%|█████████▉| 1803/1813 [3:21:27<04:21, 26.16s/it]
+100%|█████████▉| 1804/1813 [3:21:35<03:05, 20.63s/it]
+100%|█████████▉| 1805/1813 [3:21:41<02:09, 16.25s/it]
+100%|█████████▉| 1806/1813 [3:21:47<01:33, 13.30s/it]
+100%|█████████▉| 1807/1813 [3:21:54<01:08, 11.40s/it]
+100%|█████████▉| 1808/1813 [3:22:02<00:51, 10.22s/it]
+100%|█████████▉| 1809/1813 [3:22:08<00:36,  9.06s/it]
+100%|█████████▉| 1810/1813 [3:22:14<00:23,  7.97s/it]
+                                                     
+{'loss': '0.9075', 'grad_norm': '1.887', 'learning_rate': '2.895e-10', 'epoch': '0.9986', 'num_input_tokens_seen': 12187432, 'train_runtime': '1.213e+04', 'train_tokens_per_second': '1004'}
+
+100%|█████████▉| 1810/1813 [3:22:14<00:23,  7.97s/it]
+100%|█████████▉| 1811/1813 [3:22:19<00:14,  7.26s/it]
+100%|█████████▉| 1812/1813 [3:22:24<00:06,  6.65s/it]
+100%|██████████| 1813/1813 [3:22:27<00:00,  5.54s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.82it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.25it/s]
+
+  1%|          | 7/1000 [00:00<01:58,  8.35it/s]
+
+  1%|          | 9/1000 [00:01<02:32,  6.49it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.10it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.01it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.97it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.06it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.34it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.25it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  6.00it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.12it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.59it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.77it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.12it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.37it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.75it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.90it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.92it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.30it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.35it/s]
+
+  6%|▋         | 63/1000 [00:08<01:35,  9.77it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.38it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.13it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.88it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.88it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.22it/s]
+
+  8%|▊         | 84/1000 [00:11<01:53,  8.04it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.53it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.76it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.30it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.42it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.20it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.04it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.52it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.82it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.24it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.10it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.70it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.14it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.64it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:26,  9.91it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.05it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.90it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.08it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.05it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.36it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.61it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.81it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.68it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.18it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.15it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.75it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.46it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.51it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:22,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:40,  8.11it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.36it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.73it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.67it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.77it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.65it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.95it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.68it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.75it/s]
+
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.17it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.52it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.80it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.56it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.92it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.37it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.21it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.96it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.48it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  9.01it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.39it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.03it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.48it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.68it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:30,  8.28it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.87it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:05,  5.97it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.88it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.04it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.70it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.76it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.70it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.53it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.44it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.61it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.92it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.53it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.02it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.43it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.70it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.05it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:46,  6.80it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.76it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.79it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.02it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.09it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.20it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.90it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.20it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.48it/s]
+
+ 30%|███       | 300/1000 [00:40<01:41,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:37,  7.13it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.76it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.08it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.14it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:41,  6.80it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.79it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.20it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.26it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.24it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.58it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:33,  7.25it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.76it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.71it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.42it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.18it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.59it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.97it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.39it/s]
+
+ 34%|███▎      | 336/1000 [00:44<01:22,  8.06it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.83it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.62it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.41it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.56it/s]
+
+ 36%|███▌      | 361/1000 [00:47<01:07,  9.41it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.10it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.23it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.19it/s]
+
+ 38%|███▊      | 379/1000 [00:50<01:15,  8.19it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.21it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.56it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.09it/s]
+
+ 39%|███▊      | 387/1000 [00:51<01:20,  7.57it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.31it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.85it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.49it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  9.00it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.53it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.85it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.52it/s]
+
+ 41%|████      | 408/1000 [00:54<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.07it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.83it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.28it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.45it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.51it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.68it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.96it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.55it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.85it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.54it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.02it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.79it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.51it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.09it/s]
+
+ 44%|████▍     | 441/1000 [00:59<00:59,  9.32it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.11it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.26it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.74it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.03it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.19it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.20it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.52it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.83it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.29it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.85it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.02it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.82it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.58it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.40it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.07it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.32it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.33it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.01it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:16,  6.64it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.42it/s]
+
+ 50%|████▉     | 498/1000 [01:06<00:59,  8.45it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:06,  7.55it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:13,  6.80it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.97it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.99it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.55it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.12it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.09it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.40it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.04it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.14it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.72it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.36it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.93it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.26it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.52it/s]
+
+ 55%|█████▌    | 552/1000 [01:14<01:01,  7.28it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.96it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.04it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.87it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.39it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.99it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.76it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.30it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.34it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.83it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.25it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.97it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.61it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:19<00:57,  7.15it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.76it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.19it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.81it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.49it/s]
+
+ 60%|██████    | 601/1000 [01:21<01:03,  6.33it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.65it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.67it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.08it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:50,  7.82it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.45it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.71it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.77it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.61it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.74it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.09it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.04it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.22it/s]
+
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.47it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.65it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.20it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.74it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:39,  9.33it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.70it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.06it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.92it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.15it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.86it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.53it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:50,  7.08it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.57it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:54,  6.50it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.55it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:55,  6.31it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:45,  7.69it/s]
+
+ 66%|██████▌   | 656/1000 [01:28<00:38,  8.90it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:03,  5.31it/s]
+
+ 67%|██████▋   | 667/1000 [01:30<00:59,  5.61it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.19it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.88it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.26it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.65it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.97it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.92it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.90it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:33<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.83it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:35<00:40,  7.23it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.51it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.70it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.37it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.80it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.48it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.59it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.53it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:39<00:27,  9.52it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.78it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.32it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.62it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.46it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.64it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.70it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.24it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.22it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.39it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.71it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.60it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.48it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.80it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.79it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.56it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.93it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.38it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.00it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.74it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.21it/s]
+
+ 80%|███████▉  | 797/1000 [01:47<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.03it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.73it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:24,  7.84it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.18it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.98it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.23it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.25it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.06it/s]
+
+ 82%|████████▏ | 815/1000 [01:49<00:19,  9.31it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.00it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.81it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.76it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:52<00:27,  6.03it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.12it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.46it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:53<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.78it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.24it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.42it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.43it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.82it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.42it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14,  9.83it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:14, 10.09it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.46it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.73it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.61it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.68it/s]
+
+ 87%|████████▋ | 867/1000 [01:56<00:14,  9.29it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.24it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.28it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.79it/s]
+
+ 88%|████████▊ | 875/1000 [01:57<00:16,  7.64it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.88it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.37it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.79it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.78it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.62it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.68it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.48it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:13,  7.60it/s]
+
+ 90%|█████████ | 901/1000 [02:01<00:12,  7.62it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.85it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.35it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  6.96it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.26it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.52it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.35it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.06it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.35it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.58it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.70it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.57it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.67it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.11it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.66it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.49it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.84it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.62it/s]
+
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.73it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.85it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.43it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.18it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.41it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.41it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.83it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:09<00:04,  7.51it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.46it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.03it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.44it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.19it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.80it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.83it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.45it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.07it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.45it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.39it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.75it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.54it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.07it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.78it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.8001', 'eval_runtime': '135.3', 'eval_samples_per_second': '7.393', 'eval_steps_per_second': '7.393', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:24:43<00:00,  5.54s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_007_lr_2.028e-05/checkpoint-1813
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+
+Training completed. Do not forget to share your model on huggingface.co/models =)
+
+
+
+                                                     
+{'train_runtime': '1.228e+04', 'train_samples_per_second': '2.361', 'train_steps_per_second': '0.148', 'train_loss': '1.019', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:24:44<00:00,  5.54s/it]
+100%|██████████| 1813/1813 [3:24:44<00:00,  6.78s/it]
+Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_007_lr_2.028e-05
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+***** train metrics *****
+  epoch                    =         1.0
+  num_input_tokens_seen    =    12201816
+  total_flos               = 487685654GF
+  train_loss               =      1.0186
+  train_runtime            =  3:24:44.09
+  train_samples_per_second =       2.361
+  train_steps_per_second   =       0.148
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_007_lr_2.028e-05/training_loss.png
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_007_lr_2.028e-05/training_eval_loss.png
+[WARNING|2026-10-16 01:32:51] llamafactory.extras.ploting:149 >> No metric eval_accuracy to plot.
+
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+  0%|          | 3/1000 [00:00<01:02, 15.98it/s]
+  0%|          | 5/1000 [00:00<02:16,  7.27it/s]
+  1%|          | 7/1000 [00:00<01:58,  8.37it/s]
+  1%|          | 9/1000 [00:01<02:32,  6.50it/s]
+  1%|          | 11/1000 [00:01<02:01,  8.12it/s]
+  1%|▏         | 13/1000 [00:01<02:20,  7.03it/s]
+  1%|▏         | 14/1000 [00:01<02:23,  6.86it/s]
+  2%|▏         | 15/1000 [00:02<02:44,  5.99it/s]
+  2%|▏         | 16/1000 [00:02<02:51,  5.75it/s]
+  2%|▏         | 18/1000 [00:02<02:41,  6.07it/s]
+  2%|▏         | 19/1000 [00:02<03:10,  5.16it/s]
+  2%|▏         | 20/1000 [00:03<03:32,  4.61it/s]
+  2%|▏         | 22/1000 [00:03<02:36,  6.26it/s]
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+  2%|▎         | 25/1000 [00:03<02:12,  7.37it/s]
+  3%|▎         | 26/1000 [00:03<02:34,  6.29it/s]
+  3%|▎         | 28/1000 [00:04<02:41,  6.01it/s]
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+  3%|▎         | 31/1000 [00:04<03:00,  5.36it/s]
+  3%|▎         | 32/1000 [00:05<02:41,  6.00it/s]
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+  4%|▎         | 37/1000 [00:05<01:45,  9.12it/s]
+  4%|▍         | 39/1000 [00:05<02:06,  7.60it/s]
+  4%|▍         | 40/1000 [00:05<02:03,  7.76it/s]
+  4%|▍         | 42/1000 [00:06<01:45,  9.12it/s]
+  4%|▍         | 44/1000 [00:06<01:32, 10.34it/s]
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+  5%|▍         | 47/1000 [00:06<02:09,  7.38it/s]
+  5%|▍         | 49/1000 [00:06<01:48,  8.77it/s]
+  5%|▌         | 51/1000 [00:07<01:35,  9.93it/s]
+  5%|▌         | 53/1000 [00:07<01:26, 10.96it/s]
+  6%|▌         | 55/1000 [00:07<01:27, 10.77it/s]
+  6%|▌         | 57/1000 [00:07<01:41,  9.32it/s]
+  6%|▌         | 59/1000 [00:08<01:54,  8.22it/s]
+  6%|▌         | 61/1000 [00:08<01:40,  9.37it/s]
+  6%|▋         | 63/1000 [00:08<01:35,  9.79it/s]
+  6%|▋         | 65/1000 [00:08<01:51,  8.39it/s]
+  7%|▋         | 67/1000 [00:08<02:02,  7.60it/s]
+  7%|▋         | 69/1000 [00:09<01:52,  8.28it/s]
+  7%|▋         | 70/1000 [00:09<02:12,  7.01it/s]
+  7%|▋         | 72/1000 [00:09<02:17,  6.74it/s]
+  7%|▋         | 74/1000 [00:09<01:53,  8.14it/s]
+  8%|▊         | 75/1000 [00:10<02:14,  6.88it/s]
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+  8%|▊         | 82/1000 [00:10<01:39,  9.22it/s]
+  8%|▊         | 84/1000 [00:11<01:53,  8.04it/s]
+  8%|▊         | 85/1000 [00:11<02:15,  6.77it/s]
+  9%|▊         | 86/1000 [00:11<02:33,  5.94it/s]
+  9%|▊         | 87/1000 [00:11<02:49,  5.38it/s]
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+  9%|▉         | 90/1000 [00:12<02:05,  7.24it/s]
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+  9%|▉         | 92/1000 [00:12<02:26,  6.18it/s]
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+ 10%|▉         | 95/1000 [00:13<02:43,  5.53it/s]
+ 10%|▉         | 97/1000 [00:13<02:12,  6.80it/s]
+ 10%|▉         | 99/1000 [00:13<01:47,  8.38it/s]
+ 10%|█         | 101/1000 [00:13<01:33,  9.65it/s]
+ 10%|█         | 103/1000 [00:14<01:55,  7.77it/s]
+ 10%|█         | 104/1000 [00:14<01:54,  7.84it/s]
+ 11%|█         | 106/1000 [00:14<01:35,  9.35it/s]
+ 11%|█         | 108/1000 [00:14<01:25, 10.49it/s]
+ 11%|█         | 110/1000 [00:14<01:19, 11.24it/s]
+ 11%|█         | 112/1000 [00:14<01:13, 12.06it/s]
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.55it/s]
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.81it/s]
+ 12%|█▏        | 118/1000 [00:15<01:07, 13.15it/s]
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.91it/s]
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.08it/s]
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+ 13%|█▎        | 128/1000 [00:17<02:45,  5.28it/s]
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.72it/s]
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.17it/s]
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.66it/s]
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.91it/s]
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.98it/s]
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.06it/s]
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.92it/s]
+ 14%|█▍        | 145/1000 [00:18<01:27,  9.78it/s]
+ 15%|█▍        | 147/1000 [00:19<01:42,  8.31it/s]
+ 15%|█▍        | 148/1000 [00:19<01:41,  8.40it/s]
+ 15%|█▌        | 150/1000 [00:19<01:26,  9.85it/s]
+ 15%|█▌        | 152/1000 [00:19<01:17, 10.99it/s]
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.33it/s]
+ 16%|█▌        | 156/1000 [00:20<02:07,  6.64it/s]
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.97it/s]
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.59it/s]
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.80it/s]
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.50it/s]
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.42it/s]
+ 16%|█▋        | 165/1000 [00:21<02:26,  5.68it/s]
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.18it/s]
+ 17%|█▋        | 167/1000 [00:21<02:23,  5.79it/s]
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.86it/s]
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.18it/s]
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.17it/s]
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.23it/s]
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.75it/s]
+ 18%|█▊        | 176/1000 [00:22<01:37,  8.45it/s]
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.83it/s]
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.61it/s]
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.53it/s]
+ 18%|█▊        | 181/1000 [00:23<02:00,  6.77it/s]
+ 18%|█▊        | 182/1000 [00:23<02:22,  5.72it/s]
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.81it/s]
+ 18%|█▊        | 185/1000 [00:24<01:44,  7.77it/s]
+ 19%|█▊        | 186/1000 [00:24<01:42,  7.92it/s]
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.40it/s]
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.12it/s]
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.63it/s]
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.50it/s]
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.39it/s]
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+ 20%|█▉        | 198/1000 [00:25<01:27,  9.20it/s]
+ 20%|██        | 200/1000 [00:26<01:15, 10.65it/s]
+ 20%|██        | 202/1000 [00:26<01:30,  8.77it/s]
+ 20%|██        | 204/1000 [00:26<01:32,  8.63it/s]
+ 21%|██        | 206/1000 [00:26<01:20,  9.92it/s]
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+ 21%|██        | 210/1000 [00:27<01:32,  8.50it/s]
+ 21%|██        | 212/1000 [00:27<01:21,  9.70it/s]
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.75it/s]
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.17it/s]
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.52it/s]
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.81it/s]
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.32it/s]
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.31it/s]
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.56it/s]
+ 22%|██▏       | 224/1000 [00:29<02:07,  6.07it/s]
+ 22%|██▎       | 225/1000 [00:29<02:06,  6.11it/s]
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.43it/s]
+ 23%|██▎       | 228/1000 [00:29<01:53,  6.83it/s]
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.92it/s]
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.39it/s]
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.06it/s]
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.23it/s]
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.96it/s]
+ 24%|██▎       | 237/1000 [00:30<01:20,  9.47it/s]
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.98it/s]
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.37it/s]
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.48it/s]
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.67it/s]
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.27it/s]
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+ 25%|██▍       | 249/1000 [00:32<02:05,  5.97it/s]
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.89it/s]
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.04it/s]
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+ 26%|██▌       | 255/1000 [00:33<01:40,  7.38it/s]
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.71it/s]
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.79it/s]
+ 26%|██▌       | 260/1000 [00:34<01:35,  7.72it/s]
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.53it/s]
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.44it/s]
+ 26%|██▋       | 264/1000 [00:35<01:55,  6.35it/s]
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.62it/s]
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.92it/s]
+ 27%|██▋       | 268/1000 [00:35<01:36,  7.55it/s]
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.05it/s]
+ 27%|██▋       | 271/1000 [00:36<02:13,  5.45it/s]
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.69it/s]
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.06it/s]
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.81it/s]
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.63it/s]
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.04it/s]
+ 28%|██▊       | 283/1000 [00:37<02:02,  5.87it/s]
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.72it/s]
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.45it/s]
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.76it/s]
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.79it/s]
+ 29%|██▉       | 291/1000 [00:38<01:41,  7.01it/s]
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+ 29%|██▉       | 294/1000 [00:39<01:37,  7.22it/s]
+ 30%|██▉       | 296/1000 [00:39<01:41,  6.91it/s]
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.20it/s]
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+ 30%|███       | 301/1000 [00:40<01:37,  7.15it/s]
+ 30%|███       | 303/1000 [00:40<01:19,  8.79it/s]
+ 30%|███       | 304/1000 [00:40<01:25,  8.11it/s]
+ 30%|███       | 305/1000 [00:40<01:25,  8.16it/s]
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+ 31%|███       | 307/1000 [00:41<01:41,  6.80it/s]
+ 31%|███       | 309/1000 [00:41<01:18,  8.82it/s]
+ 31%|███       | 311/1000 [00:41<01:07, 10.22it/s]
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.27it/s]
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.59it/s]
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.45it/s]
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.77it/s]
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.42it/s]
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.19it/s]
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.81it/s]
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.60it/s]
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.98it/s]
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.39it/s]
+ 34%|███▎      | 336/1000 [00:44<01:22,  8.06it/s]
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.84it/s]
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.04it/s]
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.89it/s]
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+ 34%|███▍      | 345/1000 [00:45<01:24,  7.73it/s]
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.23it/s]
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.43it/s]
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+ 35%|███▌      | 352/1000 [00:46<01:24,  7.64it/s]
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.56it/s]
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.59it/s]
+ 36%|███▌      | 361/1000 [00:47<01:07,  9.43it/s]
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.20it/s]
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.29it/s]
+ 36%|███▋      | 365/1000 [00:48<02:08,  4.93it/s]
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.38it/s]
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.51it/s]
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.69it/s]
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.07it/s]
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.19it/s]
+ 38%|███▊      | 379/1000 [00:50<01:16,  8.17it/s]
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.19it/s]
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.57it/s]
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.10it/s]
+ 39%|███▊      | 387/1000 [00:51<01:20,  7.57it/s]
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.31it/s]
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.86it/s]
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.51it/s]
+ 39%|███▉      | 393/1000 [00:52<01:07,  9.01it/s]
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.50it/s]
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.82it/s]
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+ 40%|████      | 404/1000 [00:54<01:22,  7.26it/s]
+ 41%|████      | 406/1000 [00:54<01:09,  8.52it/s]
+ 41%|████      | 408/1000 [00:54<01:32,  6.40it/s]
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+ 41%|████      | 411/1000 [00:55<01:40,  5.83it/s]
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.06it/s]
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.27it/s]
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.46it/s]
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.51it/s]
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.94it/s]
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.38it/s]
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.96it/s]
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.84it/s]
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.55it/s]
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.03it/s]
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.79it/s]
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.52it/s]
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.10it/s]
+ 44%|████▍     | 441/1000 [00:59<00:59,  9.32it/s]
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.11it/s]
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.26it/s]
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.74it/s]
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.50it/s]
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.19it/s]
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+ 45%|████▌     | 454/1000 [01:00<01:15,  7.20it/s]
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.52it/s]
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.82it/s]
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.28it/s]
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.85it/s]
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.64it/s]
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.76it/s]
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.02it/s]
+ 47%|████▋     | 467/1000 [01:02<01:50,  4.83it/s]
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.32it/s]
+ 47%|████▋     | 471/1000 [01:03<01:08,  7.74it/s]
+ 47%|████▋     | 473/1000 [01:03<01:10,  7.51it/s]
+ 47%|████▋     | 474/1000 [01:03<01:21,  6.47it/s]
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.37it/s]
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.49it/s]
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.31it/s]
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.00it/s]
+ 49%|████▉     | 489/1000 [01:06<01:16,  6.64it/s]
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.23it/s]
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.96it/s]
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+ 50%|████▉     | 498/1000 [01:06<00:59,  8.45it/s]
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+ 50%|█████     | 503/1000 [01:07<01:12,  6.82it/s]
+ 50%|█████     | 505/1000 [01:08<01:15,  6.57it/s]
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+ 51%|█████     | 510/1000 [01:08<00:54,  8.96it/s]
+ 51%|█████     | 512/1000 [01:08<00:48,  9.99it/s]
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+ 52%|█████▏    | 516/1000 [01:09<01:07,  7.12it/s]
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.04it/s]
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.60it/s]
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.51it/s]
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.38it/s]
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.11it/s]
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.72it/s]
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.67it/s]
+ 54%|█████▍    | 539/1000 [01:13<01:19,  5.78it/s]
+ 54%|█████▍    | 540/1000 [01:13<01:21,  5.62it/s]
+ 54%|█████▍    | 541/1000 [01:13<01:28,  5.17it/s]
+ 54%|█████▍    | 543/1000 [01:13<01:06,  6.84it/s]
+ 55%|█████▍    | 545/1000 [01:13<01:05,  7.00it/s]
+ 55%|█████▍    | 546/1000 [01:14<01:10,  6.46it/s]
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.68it/s]
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.90it/s]
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.49it/s]
+ 55%|█████▌    | 552/1000 [01:14<01:01,  7.25it/s]
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.17it/s]
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.94it/s]
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.77it/s]
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.02it/s]
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.85it/s]
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.07it/s]
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.39it/s]
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.76it/s]
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.46it/s]
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.32it/s]
+ 57%|█████▋    | 574/1000 [01:18<01:06,  6.36it/s]
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.85it/s]
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.26it/s]
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+ 58%|█████▊    | 585/1000 [01:19<01:00,  6.91it/s]
+ 59%|█████▊    | 587/1000 [01:19<00:57,  7.15it/s]
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.54it/s]
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.75it/s]
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.19it/s]
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.80it/s]
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.40it/s]
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+ 60%|██████    | 601/1000 [01:21<01:02,  6.40it/s]
+ 60%|██████    | 602/1000 [01:22<00:59,  6.72it/s]
+ 60%|██████    | 604/1000 [01:22<01:09,  5.70it/s]
+ 61%|██████    | 606/1000 [01:22<00:55,  7.12it/s]
+ 61%|██████    | 608/1000 [01:22<00:49,  7.87it/s]
+ 61%|██████    | 610/1000 [01:23<00:45,  8.49it/s]
+ 61%|██████    | 612/1000 [01:23<00:39,  9.75it/s]
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.82it/s]
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.76it/s]
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.09it/s]
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.04it/s]
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.46it/s]
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.65it/s]
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.20it/s]
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.85it/s]
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.43it/s]
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.73it/s]
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.94it/s]
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.97it/s]
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.61it/s]
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.12it/s]
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.58it/s]
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.34it/s]
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.71it/s]
+ 66%|██████▌   | 656/1000 [01:28<00:38,  8.92it/s]
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.10it/s]
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.48it/s]
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+ 67%|██████▋   | 667/1000 [01:30<00:59,  5.62it/s]
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.90it/s]
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.65it/s]
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.67it/s]
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.72it/s]
+ 68%|██████▊   | 681/1000 [01:32<00:29, 11.00it/s]
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.95it/s]
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.92it/s]
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.88it/s]
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.18it/s]
+ 69%|██████▉   | 690/1000 [01:33<00:45,  6.83it/s]
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.85it/s]
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.27it/s]
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.44it/s]
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.35it/s]
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+ 70%|███████   | 704/1000 [01:35<00:40,  7.22it/s]
+ 70%|███████   | 705/1000 [01:36<00:54,  5.43it/s]
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+ 71%|███████   | 710/1000 [01:36<00:44,  6.50it/s]
+ 71%|███████   | 712/1000 [01:37<00:37,  7.72it/s]
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.88it/s]
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.39it/s]
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.08it/s]
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.81it/s]
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.95it/s]
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.48it/s]
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.17it/s]
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.25it/s]
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.26it/s]
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.33it/s]
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.15it/s]
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.63it/s]
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.35it/s]
+ 73%|███████▎  | 733/1000 [01:39<00:27,  9.55it/s]
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.90it/s]
+ 74%|███████▎  | 737/1000 [01:39<00:27,  9.54it/s]
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.76it/s]
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.62it/s]
+ 75%|███████▍  | 747/1000 [01:40<00:24, 10.46it/s]
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.70it/s]
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.18it/s]
+ 75%|███████▌  | 753/1000 [01:41<00:34,  7.25it/s]
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.24it/s]
+ 76%|███████▌  | 756/1000 [01:42<00:37,  6.42it/s]
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.63it/s]
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.34it/s]
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.73it/s]
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.67it/s]
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.62it/s]
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.48it/s]
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.14it/s]
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.40it/s]
+ 77%|███████▋  | 774/1000 [01:44<00:33,  6.79it/s]
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.58it/s]
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.94it/s]
+ 78%|███████▊  | 782/1000 [01:45<00:26,  8.36it/s]
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.00it/s]
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.74it/s]
+ 79%|███████▉  | 793/1000 [01:47<00:25,  8.00it/s]
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.22it/s]
+ 80%|███████▉  | 797/1000 [01:47<00:29,  6.88it/s]
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.03it/s]
+ 80%|████████  | 800/1000 [01:48<00:29,  6.74it/s]
+ 80%|████████  | 802/1000 [01:48<00:25,  7.67it/s]
+ 80%|████████  | 804/1000 [01:48<00:24,  7.86it/s]
+ 81%|████████  | 806/1000 [01:48<00:21,  9.19it/s]
+ 81%|████████  | 807/1000 [01:49<00:21,  9.00it/s]
+ 81%|████████  | 809/1000 [01:49<00:18, 10.23it/s]
+ 81%|████████  | 811/1000 [01:49<00:16, 11.25it/s]
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+ 82%|████████▏ | 815/1000 [01:49<00:19,  9.31it/s]
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.21it/s]
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+ 82%|████████▏ | 823/1000 [01:50<00:22,  7.82it/s]
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.00it/s]
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.80it/s]
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.76it/s]
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+ 83%|████████▎ | 834/1000 [01:52<00:27,  6.03it/s]
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.55it/s]
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.12it/s]
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.48it/s]
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.64it/s]
+ 84%|████████▍ | 842/1000 [01:53<00:19,  8.17it/s]
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.78it/s]
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.26it/s]
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.55it/s]
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.05it/s]
+ 86%|████████▌ | 857/1000 [01:55<00:14, 10.20it/s]
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.51it/s]
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.75it/s]
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.61it/s]
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.68it/s]
+ 87%|████████▋ | 867/1000 [01:56<00:14,  9.32it/s]
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.28it/s]
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.29it/s]
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.79it/s]
+ 88%|████████▊ | 875/1000 [01:57<00:16,  7.65it/s]
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.89it/s]
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.88it/s]
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.62it/s]
+ 88%|████████▊ | 882/1000 [01:58<00:17,  6.56it/s]
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.98it/s]
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.20it/s]
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.37it/s]
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.78it/s]
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.78it/s]
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.63it/s]
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.86it/s]
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.54it/s]
+ 90%|█████████ | 900/1000 [02:00<00:12,  7.71it/s]
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.87it/s]
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.07it/s]
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.05it/s]
+ 91%|█████████ | 908/1000 [02:01<00:11,  8.32it/s]
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.60it/s]
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.79it/s]
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.40it/s]
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.10it/s]
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.39it/s]
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.61it/s]
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.73it/s]
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.95it/s]
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.11it/s]
+ 93%|█████████▎| 928/1000 [02:04<00:10,  6.66it/s]
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.49it/s]
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.62it/s]
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.18it/s]
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.32it/s]
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.84it/s]
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+ 94%|█████████▍| 942/1000 [02:06<00:09,  6.34it/s]
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.41it/s]
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.73it/s]
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.84it/s]
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.43it/s]
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.18it/s]
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.41it/s]
+ 96%|█████████▌| 957/1000 [02:08<00:06,  6.41it/s]
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.79it/s]
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.83it/s]
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.05it/s]
+ 96%|█████████▋| 964/1000 [02:09<00:04,  7.50it/s]
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.04it/s]
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.44it/s]
+ 97%|█████████▋| 972/1000 [02:10<00:03,  8.03it/s]
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.82it/s]
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.85it/s]
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.44it/s]
+ 98%|█████████▊| 981/1000 [02:11<00:02,  7.07it/s]
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.45it/s]
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.75it/s]
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.01it/s]
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+ 99%|█████████▉| 992/1000 [02:13<00:01,  4.99it/s]
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.56it/s]
+100%|█████████▉| 995/1000 [02:14<00:00,  5.54it/s]
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+100%|█████████▉| 999/1000 [02:14<00:00,  6.77it/s]
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+100%|██████████| 1000/1000 [02:15<00:00,  7.41it/s]
+***** eval metrics *****
+  epoch                   =        1.0
+  eval_loss               =     0.8001
+  eval_runtime            = 0:02:15.17
+  eval_samples_per_second =      7.398
+  eval_steps_per_second   =      7.398
+  num_input_tokens_seen   =   12201816
+Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
diff --git a/optuna_30k_experiments/logs/trial_008_lr_1.865e-04.log b/optuna_30k_experiments/logs/trial_008_lr_1.865e-04.log
new file mode 100644
index 0000000..fb06553
--- /dev/null
+++ b/optuna_30k_experiments/logs/trial_008_lr_1.865e-04.log
@@ -0,0 +1,12401 @@
+[INFO|2026-10-16 01:35:13] llamafactory.hparams.parser:651 >> Process rank: 0, world size: 1, device: cuda:0, distributed training: False, compute dtype: torch.bfloat16
+[INFO|configuration_utils.py:765] 2026-10-16 01:35:13,769 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-16 01:35:13,779 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
+[INFO|configuration_utils.py:765] 2026-10-16 01:35:15,396 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-16 01:35:15,399 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|configuration_utils.py:765] 2026-10-16 01:35:15,546 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-16 01:35:15,549 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-16 01:35:16] llamafactory.data.loader:144 >> Loading dataset aya_slovak_hpo_30k.jsonl...
+training example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28828, 2176, 28711, 28863, 29114, 10268, 28709, 6612, 28714, 2892, 493, 363, 6294, 10719, 363, 3811, 28723, 7379, 626, 28889, 307, 14637, 311, 28768, 2218, 22885, 3748, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 284, 8341, 6729, 345, 28738, 28739, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28723, 524, 4634, 28711, 570, 1879, 261, 28863, 532, 613, 14483, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 345, 15962, 28739, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 1879, 15173, 537, 284, 8341, 705, 28723, 367, 280, 28879, 570, 4415, 28716, 28797, 9335, 3175, 29826, 626, 323, 412, 8514, 16986, 29826, 28708, 345, 15962, 28739, 1879, 15173, 537, 1221, 4566, 28718, 325, 28707, 28723, 461, 28723, 724, 494, 8891, 24160, 28725, 8582, 28862, 24160, 28725, 20532, 28764, 271, 28725, 438, 29365, 15745, 1319, 12035, 28862, 8269, 284, 28827, 2762, 28709, 264, 4798, 8359, 1450, 8325, 12121, 1798, 420, 762, 1089, 290, 28906, 18765, 724, 2331, 29114, 1221, 4304, 1780, 363, 13881, 29826, 316, 2085, 28708, 28723, 22627, 28873, 263, 335, 20208, 284, 28827, 2762, 28709, 264, 4798, 5424, 10056, 1286, 290, 28906, 18765, 724, 2331, 29114, 11323, 10719, 12490, 363, 13881, 29826, 316, 284, 8341, 2390, 28723, 22127, 280, 28879, 21022, 1150, 424, 724, 28827, 1144, 28873, 340, 17066, 270, 21022, 485, 284, 8341, 705, 28725, 446, 6909, 28797, 637, 28705, 29114, 28708, 28879, 4798, 6612, 28827, 1100, 28768, 28863, 28723, 320, 412, 28879, 637, 15612, 28716, 28889, 28726, 1150, 424, 19523, 412, 28873, 276, 28797, 2390, 284, 8341, 2390, 363, 23407, 11277, 9538, 2085, 28708, 28723, 9532, 2144, 686, 7865, 28718, 341, 14475, 1687, 28706, 18037, 452, 270, 8269, 284, 8341, 705, 28725, 290, 4827, 486, 2710, 2144, 332, 1800, 16344, 29114, 6588, 269, 28862, 28711, 28863, 686, 1653, 5388, 28718, 710, 261, 1296, 28709, 284, 8341, 705, 28723, 351, 28711, 1371, 28797, 1776, 29826, 8269, 284, 8341, 705, 268, 28863, 1776, 29826, 8269, 1879, 3497, 598, 8269, 8631, 28879, 279, 412, 28725, 12121, 7988, 268, 28863, 24270, 12115, 28797, 1879, 5626, 263, 28862, 8269, 8631, 28879, 279, 412, 28723, 318, 15612, 1473, 276, 15690, 2245, 626, 15690, 307, 14637, 311, 28768, 300, 15469, 28711, 570, 446, 450, 4798, 29826, 333, 28729, 1879, 319, 3976, 294, 405, 28768, 4882, 405, 28725, 534, 28724, 2710, 724, 313, 4827, 2245, 626, 28889, 20532, 28764, 271, 28723, 10650, 300, 12235, 28827, 28873, 424, 268, 27172, 2245, 28723, 1054, 15504, 28879, 424, 724, 313, 8768, 4545, 504, 270, 8269, 2124, 2245, 626, 12689, 4584, 10906, 28718, 710, 18782, 28879, 28715, 28797, 1135, 10356, 28723, 2645, 28764, 12275, 28820, 28873, 29114, 13067, 549, 2892, 5872, 3811, 8631, 25991, 3026, 29114, 408, 28906, 28764, 485, 2082, 1403, 12121, 1798, 1773, 29826, 28729, 22160, 710, 408, 28906, 28764, 485, 1135, 10874, 1776, 4536, 28873, 300, 23407, 11277, 9538, 2085, 28708, 28723, 22127, 3361, 13470, 1150, 317, 28873, 424, 2082, 7330, 511, 2085, 28708, 28723, 401, 492, 13591, 28820, 261, 2220, 28862, 10766, 381, 28827, 486, 29114, 363, 28879, 4689, 3418, 28879, 21775, 29114, 280, 28723, 418, 28820, 11277, 28716, 6294, 26177, 903, 12689, 6612, 20122, 28733, 12868, 291, 2124, 2085, 28708, 1879, 6670, 28862, 9348, 493, 686, 375, 5620, 386, 28862, 28827, 28725, 23374, 4536, 25306, 16388, 957, 450, 15612, 28764, 2126, 29114, 511, 2152, 363, 25769, 14110, 17413, 26177, 903, 28873, 15808, 19396, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 7379, 1879, 277, 7939, 1879, 724, 313, 8768, 2245, 28718, 363, 446, 551, 9538, 12121, 1798, 24473, 21694, 300, 261, 5869, 28723, 384, 28728, 491, 28712, 4566, 15469, 28711, 570, 1879, 2245, 28725, 534, 28724, 2710, 7463, 15612, 13335, 28728, 4371, 1144, 28723, 10650, 300, 15469, 28711, 570, 1879, 1160, 28709, 28879, 3907, 13288, 491, 363, 3292, 10719, 6612, 529, 28710, 818, 5237, 626, 4845, 28725, 534, 28724, 637, 686, 598, 5237, 5945, 17984, 28718, 2117, 18904, 3146, 515, 13288, 25208, 28723, 367, 280, 28879, 570, 12472, 1006, 5872, 28863, 358, 446, 311, 4845, 1879, 6149, 783, 494, 28718, 2245, 28718, 363, 17984, 28718, 405, 13288, 491, 28723, 13343, 29826, 28729, 385, 29114, 284, 8341, 705, 28747, 399, 3711, 3175, 29826, 626, 12990, 17984, 28718, 2117, 268, 6125, 15690, 345, 28738, 28739, 16986, 29826, 28708, 1773, 29826, 10641, 2124, 345, 28738, 28739, 582, 20524, 12214, 1773, 29826, 28729, 385, 29114, 284, 8341, 705, 28723, 351, 28906, 28879, 3811, 14628, 28879, 8631, 28879, 28710, 29114, 307, 14637, 311, 28768, 15173, 537, 28718, 325, 849, 3748, 28725, 446, 6909, 28820, 3061, 28720, 300, 28827, 1520, 6612, 749, 1780, 20532, 28764, 271, 586, 28873, 28710, 363, 8357, 28710, 307, 14637, 311, 28768, 626, 363, 29826, 17339, 28731, 1879, 686, 6729, 1773, 29826, 28729, 22160, 284, 8341, 705, 28723, 550, 11245, 412, 28747, 550, 11245, 412, 1683, 28827, 16032, 271, 1342, 4295, 3191, 316, 28729, 6449, 2245, 28718, 28723, 367, 280, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 28725, 446, 6909, 28820, 637, 3531, 598, 28820, 345, 28741, 28739, 1879, 22954, 338, 28718, 297, 12689, 345, 28741, 28739, 686, 2129, 28710, 29114, 16986, 28863, 358, 28723, 17563, 971, 28747, 17563, 971, 582, 20524, 12214, 1342, 28764, 1193, 1342, 4295, 5765, 6175, 2959, 15416, 28710, 13288, 491, 3589, 28723, 9532, 484, 28717, 3811, 18904, 17101, 29114, 22399, 971, 28725, 15612, 13335, 28728, 4371, 28711, 570, 13288, 20996, 28725, 446, 6909, 28797, 484, 28717, 3811, 18904, 17101, 29114, 28725, 264, 2513, 300, 8631, 28879, 570, 9335, 3175, 29826, 626, 323, 7509, 17984, 2950, 28718, 16986, 29826, 28708, 22885, 2557, 268, 345, 28741, 28739, 264, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Zhrnúť tento článok v jednej vete. Textový nástroj je ikona, ktorá sa podobá písmenu "T" v paneli nástrojov vľavo. Kliknite na túto ikonu. Použite rozbaľovaciu ponuku vedľa "Character" v hornej časti obrazovky na výber písma. Použite druhé rozbaľovacie menu vedľa "Character" na výber formátu (t. j. pravidelný, tučný, kurzor, atď.). Tradičné písmo ako Times New Roman alebo Garamond môže pridať formálny vzhľad loga. Bezšerifové písmo ako Helvetica môže pridať modernejší vzhľad písmu. Nepoužívajte príliš dekoratívne písma, ktoré sa ťažko čítajú. Tiež sa vyhýbajte zmiešanému písmu v návrhu loga. Ak si z internetu stiahnete bezplatné písma, mali by ste si uvedomiť licenčnú zmluvu pre tieto písma. Mnohé voľné písma sú voľné na osobné použitie, ale nie sú povolené na komerčné použitie. S vybraným textovým nástrojom kliknite kdekoľvek na grafickej doske, aby ste pridali textový kurzor. Potom napíšte svoj text. Zvážte pridanie samostatného textového objektu pre každé slovo. Obzvlášť ak plánujete používať rôzne farby alebo veľkosti pre rôzne slová vo vašom návrhu loga. Nepridávaj ešte farbu do loga. Farebná tlač nemusí byť vždy možnosťou. Návrh jednoduchého čierno-bieleho loga na začiatok zabezpečí, že vaše logo bude vyzerať dobre v jeho najjednoduchšej forme. Môžete tiež použiť nástroj Text na ceste na pridanie textu v kruhu alebo zakrivenom tvare. Dvakrát kliknite na text, aby ste ho vyzdvihli. Potom kliknite na položku znak v hornej časti obrazovky, aby sa zobrazila ponuka nastavenia znakov. Použite nasledujúce kroky na úpravu textu v ponuke znak. Veľkosť písma: Rozbaľovacia ponuka s malým "T" vedľa veľkého "T" upravuje veľkosť písma. Môžete tiež použiť nástroj výberu (ikona, ktorá pripomína čierny kurzor myši v paneli nástrojov vľavo) na zmenu veľkosti písma. Vedenie: Vedenie mení priestor medzi riadkami textu. Použite rozbaľovaciu ponuku vedľa ikony, ktorá sa podobá "A" na vrchu iného "A" zmeniť vedúce. Kerning: Kerning upravuje medzery medzi konkrétnymi znakmi. Ak chcete nastaviť kerning, vyzdvihnite znaky, ktoré chcete nastaviť, a potom použite rozbaľovaciu ponuku vedľa ikony s "A" aVyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 28790, 28724, 537, 424, 2245, 626, 28889, 307, 14637, 311, 28768, 28723, 550, 28724, 537, 424, 284, 28827, 2762, 28709, 28723, 1921, 313, 1150, 424, 268, 27172, 2245, 28723, 500, 783, 494, 424, 1773, 29826, 28729, 385, 29114, 264, 4321, 303, 715, 2245, 28718, 28723, 550, 28724, 28764, 1520, 28862, 424, 1773, 22605, 284, 8341, 705, 28723, 2, 13, 13]
+labels:
+Vyberte textový nástroj. Vyberte písmo. Pridajte svoj text. Upravte veľkosť a odstup textu. Vyznačte vektor písma.</s>
+
+
+eval example:
+input_ids:
+[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28965, 28709, 637, 8631, 25991, 3026, 28725, 13067, 10531, 28729, 350, 2591, 686, 3023, 28862, 28827, 284, 28873, 269, 13096, 28711, 28863, 6149, 3399, 28804, 550, 13881, 29826, 316, 300, 1879, 7871, 338, 28820, 6936, 1150, 28863, 7340, 6957, 8905, 3907, 12235, 28827, 28873, 424, 5765, 772, 28725, 446, 6909, 28889, 14848, 912, 12214, 4321, 28720, 751, 29365, 28723, 351, 28906, 18765, 298, 486, 29114, 28705, 28740, 387, 28705, 28750, 28734, 22116, 28827, 28723, 8294, 772, 28747, 13, 13, 27332, 12107, 28747, 13, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+inputs:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+Čo sa používa, ak škodca zničí pšeničnú úrodu? Vzhľadom na predchádzajúcu otázku napíšte kontext, ktorý obsahuje odpoveď. Môže to byť 1 - 20 vetí. Kontext:
+
+### Response:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+label_ids:
+[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 15774, 4295, 28873, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 549, 350, 5905, 2218, 6612, 28710, 26041, 28862, 485, 686, 350, 28720, 2947, 20946, 6670, 298, 28725, 23374, 17984, 28863, 28729, 433, 408, 28906, 7745, 271, 350, 385, 29114, 363, 1879, 28873, 300, 6294, 291, 28723, 560, 12423, 841, 335, 681, 28820, 9335, 1294, 17122, 29114, 28725, 9335, 1294, 17122, 29114, 12121, 28714, 363, 408, 10422, 1189, 6294, 8269, 2124, 4415, 9538, 28725, 307, 10422, 14628, 28879, 17984, 28863, 2117, 1776, 29826, 7330, 363, 1879, 28873, 15808, 1117, 458, 28723, 9532, 686, 346, 28716, 28820, 6149, 311, 1705, 363, 1326, 493, 517, 28863, 267, 28725, 668, 328, 412, 28716, 433, 637, 1879, 1627, 29826, 28711, 1371, 3382, 350, 5856, 1252, 28718, 9335, 1294, 17122, 29114, 28725, 534, 28724, 991, 28706, 13288, 626, 28718, 686, 293, 316, 2689, 284, 28906, 670, 7988, 28862, 9619, 4147, 15690, 28723, 9532, 284, 374, 284, 374, 626, 380, 29826, 686, 3023, 28862, 28827, 6149, 3399, 284, 28873, 269, 535, 28725, 290, 28906, 28879, 5037, 363, 12472, 1006, 5872, 28863, 675, 6739, 686, 293, 12035, 29114, 4321, 328, 10719, 28873, 7509, 4321, 3399, 284, 28873, 269, 535, 28725, 3061, 28862, 300, 637, 668, 328, 412, 28716, 433, 1879, 363, 28711, 24043, 2737, 841, 335, 681, 28863, 9335, 1294, 17122, 29114, 28723, 351, 28906, 28879, 5037, 637, 363, 28764, 1705, 29114, 1219, 28729, 9041, 284, 28873, 269, 535, 363, 711, 28768, 532, 818, 4081, 28710, 264, 363, 846, 12035, 29114, 6149, 452, 485, 297, 28889, 4415, 28716, 28725, 668, 328, 412, 28716, 1150, 28863, 28717, 637, 1879, 1342, 28764, 313, 551, 28716, 626, 28863, 9335, 1294, 17122, 29114, 28723, 384, 493, 266, 2591, 23371, 1627, 29826, 28711, 1371, 3382, 350, 5856, 9883, 668, 5053, 28862, 21775, 29114, 28725, 446, 6909, 28820, 2413, 5856, 485, 284, 374, 12214, 1326, 493, 517, 28863, 643, 28725, 637, 363, 4273, 28862, 279, 300, 3614, 314, 4371, 28718, 668, 328, 412, 2799, 1879, 287, 2563, 2023, 28764, 6454, 28723, 2, 13, 13]
+labels:
+Medzišpecifická rozmanitosť plodín je čiastočne zodpovedná za to, že ponúkame rôznorodosť v našom jedle. Intraspecifická rozmanitosť, rozmanitosť alel v rámci jedného druhu, nám tiež ponúka voľbu v našej strave. Ak zlyhá úroda v monokultúre, spoliehame sa na poľnohospodársku rozmanitosť, aby sme znovu zasadili pôdu niečím novým. Ak pest pestovateľ zničí úrodu pšenice, môžeme v nasledujúcom roku zasadiť odolnejšiu odrodu pšenice, pričom sa spoliehame na vnútrospecifickú rozmanitosť. Môžeme sa vzdať produkcie pšenice v tejto oblasti a vysadiť úplne iný druh, spoliehajúc sa na medzidruhovú rozmanitosť. Dokonca aj poľnohospodárska spoločnosť, ktorá primárne pestuje monokultúry, sa v určitom okamihu spolieha na biodiverzitu.</s>
+
+
+[INFO|configuration_utils.py:765] 2026-10-16 01:35:17,036 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+[INFO|configuration_utils.py:841] 2026-10-16 01:35:17,039 >> Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+[INFO|2026-10-16 01:35:17] llamafactory.model.model_utils.kv_cache:144 >> KV cache is disabled during training.
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/unsloth/__init__.py:165: UserWarning: WARNING: Unsloth should be imported before [trl, transformers, peft] to ensure all optimizations are applied. Your code may run slower or encounter memory issues without these optimizations.
+
+Please restructure your imports with 'import unsloth' at the top of your file.
+  from ._gpu_init import *
+🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
+🦥 Unsloth Zoo will now patch everything to make training faster!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: WARNING `trust_remote_code` is True.
+Are you certain you want to do remote code execution?
+==((====))==  Unsloth 2026.6.1: Fast Mistral patching. Transformers: 5.5.0.
+   \\   /|    NVIDIA RTX A4500. Num GPUs = 1. Max memory: 19.573 GB. Platform: Linux.
+O^O/ \_/ \    Torch: 2.6.0+cu124. CUDA: 8.6. CUDA Toolkit: 12.4. Triton: 3.2.0
+\        /    Bfloat16 = TRUE. FA [Xformers = 0.0.29.post3. FA2 = False]
+ "-____-"     Free license: http://github.com/unslothai/unsloth
+Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading weights file model.safetensors from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/model.safetensors.index.json
+Will use dtype=torch.bfloat16 as defined in model's config object
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 2,
+  "use_cache": true
+}
+
+
+Loading weights:   0%|          | 0/291 [00:00<?, ?it/s]
+Loading weights:   4%|▍         | 13/291 [00:00<00:02, 113.58it/s]
+Loading weights:   9%|▊         | 25/291 [00:00<00:02, 93.07it/s] 
+Loading weights:  14%|█▎        | 40/291 [00:00<00:02, 111.28it/s]
+Loading weights:  18%|█▊        | 52/291 [00:00<00:02, 111.16it/s]
+Loading weights:  22%|██▏       | 65/291 [00:00<00:01, 116.72it/s]
+Loading weights:  26%|██▋       | 77/291 [00:00<00:02, 101.65it/s]
+Loading weights:  30%|███       | 88/291 [00:00<00:02, 98.34it/s] 
+Loading weights:  34%|███▍      | 99/291 [00:00<00:01, 99.12it/s]
+Loading weights:  38%|███▊      | 110/291 [00:01<00:01, 99.68it/s]
+Loading weights:  42%|████▏     | 121/291 [00:01<00:01, 98.60it/s]
+Loading weights:  45%|████▌     | 131/291 [00:01<00:01, 91.71it/s]
+Loading weights:  48%|████▊     | 141/291 [00:01<00:01, 90.80it/s]
+Loading weights:  52%|█████▏    | 151/291 [00:01<00:01, 90.88it/s]
+Loading weights:  56%|█████▌    | 163/291 [00:01<00:01, 97.79it/s]
+Loading weights:  60%|██████    | 176/291 [00:01<00:01, 100.99it/s]
+Loading weights:  65%|██████▍   | 189/291 [00:01<00:00, 108.52it/s]
+Loading weights:  69%|██████▉   | 202/291 [00:01<00:00, 110.87it/s]
+Loading weights:  74%|███████▎  | 214/291 [00:02<00:00, 98.70it/s] 
+Loading weights:  78%|███████▊  | 226/291 [00:02<00:00, 103.92it/s]
+Loading weights:  82%|████████▏ | 238/291 [00:02<00:00, 101.95it/s]
+Loading weights:  86%|████████▌ | 249/291 [00:02<00:00, 90.96it/s] 
+Loading weights:  89%|████████▉ | 259/291 [00:02<00:00, 92.92it/s]
+Loading weights:  93%|█████████▎| 271/291 [00:02<00:00, 99.40it/s]
+Loading weights:  98%|█████████▊| 284/291 [00:02<00:00, 104.31it/s]
+Loading weights: 100%|██████████| 291/291 [00:02<00:00, 101.70it/s]
+loading configuration file generation_config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/generation_config.json
+Generate config GenerationConfig {
+  "bos_token_id": 1,
+  "eos_token_id": 2
+}
+
+Could not locate the custom_generate/generate.py inside slovak-nlp/mistral-sk-7b.
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+Unsloth: slovak-nlp/mistral-sk-7b had a bad pad_token (</s>). Using pad_token = <unk>.
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:68: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_fwd
+/home/schwarc/LLaMA-Factory/src/llamafactory/model/model_utils/checkpointing.py:85: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
+  @torch.cuda.amp.custom_bwd
+[INFO|2026-10-16 01:35:34] llamafactory.model.model_utils.checkpointing:144 >> Gradient checkpointing enabled.
+[INFO|2026-10-16 01:35:34] llamafactory.model.adapter:144 >> Upcasting trainable params to float32.
+[INFO|2026-10-16 01:35:34] llamafactory.model.adapter:144 >> Fine-tuning method: LoRA
+Unsloth 2026.6.1 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers.
+[INFO|2026-10-16 01:35:37] llamafactory.model.loader:144 >> trainable params: 41,943,040 || all params: 7,283,675,136 || trainable%: 0.5758
+The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'pad_token_id': 2}.
+==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 1
+   \\   /|    Num examples = 29,000 | Num Epochs = 1 | Total steps = 1,813
+O^O/ \_/ \    Batch size per device = 1 | Gradient accumulation steps = 16
+\        /    Data Parallel GPUs = 1 | Total batch size (1 x 16 x 1) = 16
+ "-____-"     Trainable parameters = 41,943,040 of 7,283,675,136 (0.58% trained)
+
+  0%|          | 0/1813 [00:00<?, ?it/s]`use_return_dict` is deprecated! Use `return_dict` instead!
+Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
+Unsloth: Will smartly offload gradients to save VRAM!
+
+  0%|          | 1/1813 [00:06<3:26:19,  6.83s/it]
+  0%|          | 2/1813 [00:13<3:21:33,  6.68s/it]
+  0%|          | 3/1813 [00:18<3:05:22,  6.15s/it]
+  0%|          | 4/1813 [00:22<2:40:35,  5.33s/it]
+  0%|          | 5/1813 [00:29<2:53:34,  5.76s/it]
+  0%|          | 6/1813 [00:36<3:10:34,  6.33s/it]
+  0%|          | 7/1813 [00:43<3:11:56,  6.38s/it]
+  0%|          | 8/1813 [00:48<3:00:03,  5.99s/it]
+  0%|          | 9/1813 [00:56<3:14:05,  6.46s/it]
+  1%|          | 10/1813 [01:01<3:04:48,  6.15s/it]
+                                                   
+{'loss': '1.78', 'grad_norm': '3.371', 'learning_rate': '1.119e-05', 'epoch': '0.005517', 'num_input_tokens_seen': 65928, 'train_runtime': '61.53', 'train_tokens_per_second': '1072'}
+
+  1%|          | 10/1813 [01:01<3:04:48,  6.15s/it]
+  1%|          | 11/1813 [01:07<3:06:26,  6.21s/it]
+  1%|          | 12/1813 [01:15<3:20:38,  6.68s/it]
+  1%|          | 13/1813 [01:19<2:57:23,  5.91s/it]
+  1%|          | 14/1813 [01:25<2:58:33,  5.96s/it]
+  1%|          | 15/1813 [01:30<2:49:55,  5.67s/it]
+  1%|          | 16/1813 [01:36<2:53:00,  5.78s/it]
+  1%|          | 17/1813 [01:43<2:58:22,  5.96s/it]
+  1%|          | 18/1813 [01:49<3:00:29,  6.03s/it]
+  1%|          | 19/1813 [01:56<3:08:16,  6.30s/it]
+  1%|          | 20/1813 [02:02<3:06:52,  6.25s/it]
+                                                   
+{'loss': '1.462', 'grad_norm': '1.297', 'learning_rate': '2.362e-05', 'epoch': '0.01103', 'num_input_tokens_seen': 131352, 'train_runtime': '122.5', 'train_tokens_per_second': '1072'}
+
+  1%|          | 20/1813 [02:02<3:06:52,  6.25s/it]
+  1%|          | 21/1813 [02:08<3:04:43,  6.18s/it]
+  1%|          | 22/1813 [02:13<2:55:21,  5.87s/it]
+  1%|▏         | 23/1813 [02:19<2:51:19,  5.74s/it]
+  1%|▏         | 24/1813 [02:24<2:52:16,  5.78s/it]
+  1%|▏         | 25/1813 [02:32<3:05:21,  6.22s/it]
+  1%|▏         | 26/1813 [02:39<3:15:29,  6.56s/it]
+  1%|▏         | 27/1813 [02:45<3:10:31,  6.40s/it]
+  2%|▏         | 28/1813 [02:52<3:15:34,  6.57s/it]
+  2%|▏         | 29/1813 [03:00<3:28:24,  7.01s/it]
+  2%|▏         | 30/1813 [03:05<3:09:22,  6.37s/it]
+                                                   
+{'loss': '1.336', 'grad_norm': '1.312', 'learning_rate': '3.605e-05', 'epoch': '0.01655', 'num_input_tokens_seen': 197376, 'train_runtime': '185.5', 'train_tokens_per_second': '1064'}
+
+  2%|▏         | 30/1813 [03:05<3:09:22,  6.37s/it]
+  2%|▏         | 31/1813 [03:11<3:09:27,  6.38s/it]
+  2%|▏         | 32/1813 [03:17<2:59:04,  6.03s/it]
+  2%|▏         | 33/1813 [03:23<2:58:07,  6.00s/it]
+  2%|▏         | 34/1813 [03:30<3:11:28,  6.46s/it]
+  2%|▏         | 35/1813 [03:37<3:11:08,  6.45s/it]
+  2%|▏         | 36/1813 [03:43<3:10:26,  6.43s/it]
+  2%|▏         | 37/1813 [03:48<2:58:36,  6.03s/it]
+  2%|▏         | 38/1813 [03:53<2:48:57,  5.71s/it]
+  2%|▏         | 39/1813 [03:59<2:48:01,  5.68s/it]
+  2%|▏         | 40/1813 [04:04<2:47:52,  5.68s/it]
+                                                   
+{'loss': '1.215', 'grad_norm': '1.211', 'learning_rate': '4.848e-05', 'epoch': '0.02207', 'num_input_tokens_seen': 261168, 'train_runtime': '244.8', 'train_tokens_per_second': '1067'}
+
+  2%|▏         | 40/1813 [04:04<2:47:52,  5.68s/it]
+  2%|▏         | 41/1813 [04:10<2:44:45,  5.58s/it]
+  2%|▏         | 42/1813 [04:16<2:49:10,  5.73s/it]
+  2%|▏         | 43/1813 [04:23<3:05:27,  6.29s/it]
+  2%|▏         | 44/1813 [04:30<3:06:56,  6.34s/it]
+  2%|▏         | 45/1813 [04:37<3:16:01,  6.65s/it]
+  3%|▎         | 46/1813 [04:43<3:08:44,  6.41s/it]
+  3%|▎         | 47/1813 [04:49<3:01:04,  6.15s/it]
+  3%|▎         | 48/1813 [04:55<3:06:40,  6.35s/it]
+  3%|▎         | 49/1813 [05:02<3:07:02,  6.36s/it]
+  3%|▎         | 50/1813 [05:08<3:05:29,  6.31s/it]
+                                                   
+{'loss': '1.281', 'grad_norm': '1.519', 'learning_rate': '6.091e-05', 'epoch': '0.02759', 'num_input_tokens_seen': 330104, 'train_runtime': '308.4', 'train_tokens_per_second': '1070'}
+
+  3%|▎         | 50/1813 [05:08<3:05:29,  6.31s/it]
+  3%|▎         | 51/1813 [05:15<3:15:27,  6.66s/it]
+  3%|▎         | 52/1813 [05:25<3:38:42,  7.45s/it]
+  3%|▎         | 53/1813 [05:31<3:24:57,  6.99s/it]
+  3%|▎         | 54/1813 [05:38<3:28:29,  7.11s/it]
+  3%|▎         | 55/1813 [05:44<3:21:16,  6.87s/it]
+  3%|▎         | 56/1813 [05:51<3:20:00,  6.83s/it]
+  3%|▎         | 57/1813 [05:56<3:06:48,  6.38s/it]
+  3%|▎         | 58/1813 [06:05<3:24:24,  6.99s/it]
+  3%|▎         | 59/1813 [06:12<3:24:28,  6.99s/it]
+  3%|▎         | 60/1813 [06:17<3:11:56,  6.57s/it]
+                                                   
+{'loss': '1.174', 'grad_norm': '1.418', 'learning_rate': '7.334e-05', 'epoch': '0.0331', 'num_input_tokens_seen': 407792, 'train_runtime': '377.9', 'train_tokens_per_second': '1079'}
+
+  3%|▎         | 60/1813 [06:17<3:11:56,  6.57s/it]
+  3%|▎         | 61/1813 [06:23<2:59:56,  6.16s/it]
+  3%|▎         | 62/1813 [06:28<2:54:35,  5.98s/it]
+  3%|▎         | 63/1813 [06:34<2:52:54,  5.93s/it]
+  4%|▎         | 64/1813 [06:40<2:50:49,  5.86s/it]
+  4%|▎         | 65/1813 [06:47<3:02:38,  6.27s/it]
+  4%|▎         | 66/1813 [06:53<3:03:24,  6.30s/it]
+  4%|▎         | 67/1813 [06:59<2:58:35,  6.14s/it]
+  4%|▍         | 68/1813 [07:05<2:59:20,  6.17s/it]
+  4%|▍         | 69/1813 [07:12<3:06:57,  6.43s/it]
+  4%|▍         | 70/1813 [07:20<3:15:30,  6.73s/it]
+                                                   
+{'loss': '1.069', 'grad_norm': '1.09', 'learning_rate': '8.577e-05', 'epoch': '0.03862', 'num_input_tokens_seen': 474368, 'train_runtime': '440.2', 'train_tokens_per_second': '1078'}
+
+  4%|▍         | 70/1813 [07:20<3:15:30,  6.73s/it]
+  4%|▍         | 71/1813 [07:25<3:03:40,  6.33s/it]
+  4%|▍         | 72/1813 [07:34<3:23:19,  7.01s/it]
+  4%|▍         | 73/1813 [07:39<3:09:34,  6.54s/it]
+  4%|▍         | 74/1813 [07:45<3:06:15,  6.43s/it]
+  4%|▍         | 75/1813 [07:52<3:07:18,  6.47s/it]
+  4%|▍         | 76/1813 [07:58<3:04:31,  6.37s/it]
+  4%|▍         | 77/1813 [08:05<3:07:33,  6.48s/it]
+  4%|▍         | 78/1813 [08:10<2:56:52,  6.12s/it]
+  4%|▍         | 79/1813 [08:15<2:51:12,  5.92s/it]
+  4%|▍         | 80/1813 [08:22<2:52:28,  5.97s/it]
+                                                   
+{'loss': '1.092', 'grad_norm': '1.23', 'learning_rate': '9.82e-05', 'epoch': '0.04414', 'num_input_tokens_seen': 541616, 'train_runtime': '502.1', 'train_tokens_per_second': '1079'}
+
+  4%|▍         | 80/1813 [08:22<2:52:28,  5.97s/it]
+  4%|▍         | 81/1813 [08:27<2:51:07,  5.93s/it]
+  5%|▍         | 82/1813 [08:34<2:59:53,  6.24s/it]
+  5%|▍         | 83/1813 [08:41<3:07:13,  6.49s/it]
+  5%|▍         | 84/1813 [08:48<3:10:25,  6.61s/it]
+  5%|▍         | 85/1813 [08:54<3:06:39,  6.48s/it]
+  5%|▍         | 86/1813 [09:01<3:05:14,  6.44s/it]
+  5%|▍         | 87/1813 [09:06<2:52:53,  6.01s/it]
+  5%|▍         | 88/1813 [09:12<2:53:56,  6.05s/it]
+  5%|▍         | 89/1813 [09:18<2:52:56,  6.02s/it]
+  5%|▍         | 90/1813 [09:24<2:55:35,  6.11s/it]
+                                                   
+{'loss': '1.026', 'grad_norm': '0.8934', 'learning_rate': '0.0001106', 'epoch': '0.04966', 'num_input_tokens_seen': 608792, 'train_runtime': '564.8', 'train_tokens_per_second': '1078'}
+
+  5%|▍         | 90/1813 [09:24<2:55:35,  6.11s/it]
+  5%|▌         | 91/1813 [09:30<2:51:12,  5.97s/it]
+  5%|▌         | 92/1813 [09:35<2:41:14,  5.62s/it]
+  5%|▌         | 93/1813 [09:41<2:43:18,  5.70s/it]
+  5%|▌         | 94/1813 [09:47<2:47:43,  5.85s/it]
+  5%|▌         | 95/1813 [09:53<2:51:52,  6.00s/it]
+  5%|▌         | 96/1813 [09:59<2:51:06,  5.98s/it]
+  5%|▌         | 97/1813 [10:06<2:57:03,  6.19s/it]
+  5%|▌         | 98/1813 [10:11<2:49:47,  5.94s/it]
+  5%|▌         | 99/1813 [10:17<2:48:35,  5.90s/it]
+  6%|▌         | 100/1813 [10:23<2:53:03,  6.06s/it]
+                                                    
+{'loss': '1.067', 'grad_norm': '0.8758', 'learning_rate': '0.0001231', 'epoch': '0.05517', 'num_input_tokens_seen': 670936, 'train_runtime': '623.9', 'train_tokens_per_second': '1075'}
+
+  6%|▌         | 100/1813 [10:23<2:53:03,  6.06s/it]
+  6%|▌         | 101/1813 [10:30<2:58:45,  6.26s/it]
+  6%|▌         | 102/1813 [10:37<3:02:41,  6.41s/it]
+  6%|▌         | 103/1813 [10:43<3:00:30,  6.33s/it]
+  6%|▌         | 104/1813 [10:49<3:00:01,  6.32s/it]
+  6%|▌         | 105/1813 [10:55<2:54:01,  6.11s/it]
+  6%|▌         | 106/1813 [11:01<2:52:58,  6.08s/it]
+  6%|▌         | 107/1813 [11:07<2:55:29,  6.17s/it]
+  6%|▌         | 108/1813 [11:14<2:57:38,  6.25s/it]
+  6%|▌         | 109/1813 [11:20<2:56:00,  6.20s/it]
+  6%|▌         | 110/1813 [11:27<3:00:52,  6.37s/it]
+                                                    
+{'loss': '1.096', 'grad_norm': '0.8131', 'learning_rate': '0.0001355', 'epoch': '0.06069', 'num_input_tokens_seen': 738632, 'train_runtime': '687.1', 'train_tokens_per_second': '1075'}
+
+  6%|▌         | 110/1813 [11:27<3:00:52,  6.37s/it]
+  6%|▌         | 111/1813 [11:33<3:04:30,  6.50s/it]
+  6%|▌         | 112/1813 [11:40<3:06:01,  6.56s/it]
+  6%|▌         | 113/1813 [11:46<3:03:16,  6.47s/it]
+  6%|▋         | 114/1813 [11:54<3:11:21,  6.76s/it]
+  6%|▋         | 115/1813 [12:00<3:03:00,  6.47s/it]
+  6%|▋         | 116/1813 [12:06<3:06:36,  6.60s/it]
+  6%|▋         | 117/1813 [12:12<2:56:41,  6.25s/it]
+  7%|▋         | 118/1813 [12:18<2:52:43,  6.11s/it]
+  7%|▋         | 119/1813 [12:24<2:53:51,  6.16s/it]
+  7%|▋         | 120/1813 [12:30<2:54:39,  6.19s/it]
+                                                    
+{'loss': '1.105', 'grad_norm': '1.156', 'learning_rate': '0.0001479', 'epoch': '0.06621', 'num_input_tokens_seen': 808824, 'train_runtime': '750.7', 'train_tokens_per_second': '1077'}
+
+  7%|▋         | 120/1813 [12:30<2:54:39,  6.19s/it]
+  7%|▋         | 121/1813 [12:35<2:39:49,  5.67s/it]
+  7%|▋         | 122/1813 [12:40<2:34:53,  5.50s/it]
+  7%|▋         | 123/1813 [12:46<2:42:18,  5.76s/it]
+  7%|▋         | 124/1813 [12:51<2:37:30,  5.60s/it]
+  7%|▋         | 125/1813 [12:58<2:45:27,  5.88s/it]
+  7%|▋         | 126/1813 [13:04<2:47:29,  5.96s/it]
+  7%|▋         | 127/1813 [13:09<2:37:31,  5.61s/it]
+  7%|▋         | 128/1813 [13:13<2:28:49,  5.30s/it]
+  7%|▋         | 129/1813 [13:21<2:50:51,  6.09s/it]
+  7%|▋         | 130/1813 [13:28<2:55:02,  6.24s/it]
+                                                    
+{'loss': '1.045', 'grad_norm': '0.7321', 'learning_rate': '0.0001603', 'epoch': '0.07172', 'num_input_tokens_seen': 869720, 'train_runtime': '808.4', 'train_tokens_per_second': '1076'}
+
+  7%|▋         | 130/1813 [13:28<2:55:02,  6.24s/it]
+  7%|▋         | 131/1813 [13:34<2:51:36,  6.12s/it]
+  7%|▋         | 132/1813 [13:39<2:43:27,  5.83s/it]
+  7%|▋         | 133/1813 [13:44<2:39:48,  5.71s/it]
+  7%|▋         | 134/1813 [13:49<2:33:36,  5.49s/it]
+  7%|▋         | 135/1813 [13:54<2:28:41,  5.32s/it]
+  8%|▊         | 136/1813 [14:00<2:30:40,  5.39s/it]
+  8%|▊         | 137/1813 [14:06<2:34:05,  5.52s/it]
+  8%|▊         | 138/1813 [14:13<2:45:42,  5.94s/it]
+  8%|▊         | 139/1813 [14:19<2:49:53,  6.09s/it]
+  8%|▊         | 140/1813 [14:26<2:56:21,  6.32s/it]
+                                                    
+{'loss': '1.044', 'grad_norm': '1.057', 'learning_rate': '0.0001728', 'epoch': '0.07724', 'num_input_tokens_seen': 932000, 'train_runtime': '866.4', 'train_tokens_per_second': '1076'}
+
+  8%|▊         | 140/1813 [14:26<2:56:21,  6.32s/it]
+  8%|▊         | 141/1813 [14:31<2:49:28,  6.08s/it]
+  8%|▊         | 142/1813 [14:39<3:04:21,  6.62s/it]
+  8%|▊         | 143/1813 [14:45<2:53:25,  6.23s/it]
+  8%|▊         | 144/1813 [14:50<2:43:55,  5.89s/it]
+  8%|▊         | 145/1813 [14:57<2:54:14,  6.27s/it]
+  8%|▊         | 146/1813 [15:03<2:54:04,  6.27s/it]
+  8%|▊         | 147/1813 [15:08<2:46:08,  5.98s/it]
+  8%|▊         | 148/1813 [15:14<2:45:41,  5.97s/it]
+  8%|▊         | 149/1813 [15:20<2:45:00,  5.95s/it]
+  8%|▊         | 150/1813 [15:26<2:47:05,  6.03s/it]
+                                                    
+{'loss': '1.1', 'grad_norm': '0.8846', 'learning_rate': '0.0001852', 'epoch': '0.08276', 'num_input_tokens_seen': 997296, 'train_runtime': '927', 'train_tokens_per_second': '1076'}
+
+  8%|▊         | 150/1813 [15:26<2:47:05,  6.03s/it]
+  8%|▊         | 151/1813 [15:32<2:41:02,  5.81s/it]
+  8%|▊         | 152/1813 [15:37<2:37:08,  5.68s/it]
+  8%|▊         | 153/1813 [15:43<2:39:03,  5.75s/it]
+  8%|▊         | 154/1813 [15:50<2:44:50,  5.96s/it]
+  9%|▊         | 155/1813 [15:57<2:55:05,  6.34s/it]
+  9%|▊         | 156/1813 [16:03<2:51:36,  6.21s/it]
+  9%|▊         | 157/1813 [16:09<2:50:12,  6.17s/it]
+  9%|▊         | 158/1813 [16:14<2:43:04,  5.91s/it]
+  9%|▉         | 159/1813 [16:20<2:43:41,  5.94s/it]
+  9%|▉         | 160/1813 [16:26<2:41:26,  5.86s/it]
+                                                    
+{'loss': '1.026', 'grad_norm': '1.427', 'learning_rate': '0.0001864', 'epoch': '0.08828', 'num_input_tokens_seen': 1060096, 'train_runtime': '986.2', 'train_tokens_per_second': '1075'}
+
+  9%|▉         | 160/1813 [16:26<2:41:26,  5.86s/it]
+  9%|▉         | 161/1813 [16:32<2:41:55,  5.88s/it]
+  9%|▉         | 162/1813 [16:36<2:28:43,  5.40s/it]
+  9%|▉         | 163/1813 [16:43<2:38:46,  5.77s/it]
+  9%|▉         | 164/1813 [16:50<2:51:09,  6.23s/it]
+  9%|▉         | 165/1813 [16:56<2:50:00,  6.19s/it]
+  9%|▉         | 166/1813 [17:04<3:02:24,  6.65s/it]
+  9%|▉         | 167/1813 [17:10<3:01:54,  6.63s/it]
+  9%|▉         | 168/1813 [17:17<3:06:48,  6.81s/it]
+  9%|▉         | 169/1813 [17:23<2:57:03,  6.46s/it]
+  9%|▉         | 170/1813 [17:30<2:58:50,  6.53s/it]
+                                                    
+{'loss': '1.055', 'grad_norm': '0.6037', 'learning_rate': '0.0001864', 'epoch': '0.09379', 'num_input_tokens_seen': 1130368, 'train_runtime': '1050', 'train_tokens_per_second': '1076'}
+
+  9%|▉         | 170/1813 [17:30<2:58:50,  6.53s/it]
+  9%|▉         | 171/1813 [17:35<2:50:17,  6.22s/it]
+  9%|▉         | 172/1813 [17:41<2:45:26,  6.05s/it]
+ 10%|▉         | 173/1813 [17:47<2:48:07,  6.15s/it]
+ 10%|▉         | 174/1813 [17:53<2:46:15,  6.09s/it]
+ 10%|▉         | 175/1813 [18:00<2:49:16,  6.20s/it]
+ 10%|▉         | 176/1813 [18:05<2:40:24,  5.88s/it]
+ 10%|▉         | 177/1813 [18:12<2:53:24,  6.36s/it]
+ 10%|▉         | 178/1813 [18:17<2:41:16,  5.92s/it]
+ 10%|▉         | 179/1813 [18:23<2:41:13,  5.92s/it]
+ 10%|▉         | 180/1813 [18:29<2:42:13,  5.96s/it]
+                                                    
+{'loss': '1.036', 'grad_norm': '0.9465', 'learning_rate': '0.0001863', 'epoch': '0.09931', 'num_input_tokens_seen': 1193192, 'train_runtime': '1110', 'train_tokens_per_second': '1075'}
+
+ 10%|▉         | 180/1813 [18:29<2:42:13,  5.96s/it]
+ 10%|▉         | 181/1813 [18:35<2:38:22,  5.82s/it]
+ 10%|█         | 182/1813 [18:40<2:34:35,  5.69s/it]
+ 10%|█         | 183/1813 [18:46<2:33:15,  5.64s/it]
+ 10%|█         | 184/1813 [18:51<2:31:20,  5.57s/it]
+ 10%|█         | 185/1813 [18:59<2:51:10,  6.31s/it]
+ 10%|█         | 186/1813 [19:06<2:53:05,  6.38s/it]
+ 10%|█         | 187/1813 [19:12<2:52:40,  6.37s/it]
+ 10%|█         | 188/1813 [19:17<2:44:39,  6.08s/it]
+ 10%|█         | 189/1813 [19:24<2:50:42,  6.31s/it]
+ 10%|█         | 190/1813 [19:31<2:54:20,  6.45s/it]
+                                                    
+{'loss': '0.9755', 'grad_norm': '0.9605', 'learning_rate': '0.0001862', 'epoch': '0.1048', 'num_input_tokens_seen': 1259336, 'train_runtime': '1171', 'train_tokens_per_second': '1075'}
+
+ 10%|█         | 190/1813 [19:31<2:54:20,  6.45s/it]
+ 11%|█         | 191/1813 [19:37<2:47:35,  6.20s/it]
+ 11%|█         | 192/1813 [19:43<2:51:19,  6.34s/it]
+ 11%|█         | 193/1813 [19:50<2:52:52,  6.40s/it]
+ 11%|█         | 194/1813 [19:56<2:48:24,  6.24s/it]
+ 11%|█         | 195/1813 [20:03<2:59:03,  6.64s/it]
+ 11%|█         | 196/1813 [20:09<2:49:13,  6.28s/it]
+ 11%|█         | 197/1813 [20:16<2:56:35,  6.56s/it]
+ 11%|█         | 198/1813 [20:23<3:03:20,  6.81s/it]
+ 11%|█         | 199/1813 [20:30<3:02:50,  6.80s/it]
+ 11%|█         | 200/1813 [20:37<3:01:52,  6.77s/it]
+                                                    
+{'loss': '1.087', 'grad_norm': '1.033', 'learning_rate': '0.0001861', 'epoch': '0.1103', 'num_input_tokens_seen': 1331224, 'train_runtime': '1237', 'train_tokens_per_second': '1076'}
+
+ 11%|█         | 200/1813 [20:37<3:01:52,  6.77s/it]
+ 11%|█         | 201/1813 [20:42<2:53:09,  6.45s/it]
+ 11%|█         | 202/1813 [20:47<2:41:23,  6.01s/it]
+ 11%|█         | 203/1813 [20:52<2:31:34,  5.65s/it]
+ 11%|█▏        | 204/1813 [20:59<2:40:22,  5.98s/it]
+ 11%|█▏        | 205/1813 [21:04<2:34:33,  5.77s/it]
+ 11%|█▏        | 206/1813 [21:10<2:36:30,  5.84s/it]
+ 11%|█▏        | 207/1813 [21:16<2:36:44,  5.86s/it]
+ 11%|█▏        | 208/1813 [21:24<2:52:07,  6.43s/it]
+ 12%|█▏        | 209/1813 [21:29<2:41:24,  6.04s/it]
+ 12%|█▏        | 210/1813 [21:35<2:41:24,  6.04s/it]
+                                                    
+{'loss': '0.9685', 'grad_norm': '1.055', 'learning_rate': '0.0001859', 'epoch': '0.1159', 'num_input_tokens_seen': 1392472, 'train_runtime': '1296', 'train_tokens_per_second': '1075'}
+
+ 12%|█▏        | 210/1813 [21:35<2:41:24,  6.04s/it]
+ 12%|█▏        | 211/1813 [21:44<3:02:37,  6.84s/it]
+ 12%|█▏        | 212/1813 [21:50<2:54:06,  6.52s/it]
+ 12%|█▏        | 213/1813 [21:55<2:45:57,  6.22s/it]
+ 12%|█▏        | 214/1813 [22:01<2:39:19,  5.98s/it]
+ 12%|█▏        | 215/1813 [22:06<2:34:56,  5.82s/it]
+ 12%|█▏        | 216/1813 [22:11<2:24:26,  5.43s/it]
+ 12%|█▏        | 217/1813 [22:16<2:21:59,  5.34s/it]
+ 12%|█▏        | 218/1813 [22:22<2:27:52,  5.56s/it]
+ 12%|█▏        | 219/1813 [22:28<2:30:18,  5.66s/it]
+ 12%|█▏        | 220/1813 [22:34<2:32:37,  5.75s/it]
+                                                    
+{'loss': '1.042', 'grad_norm': '0.8649', 'learning_rate': '0.0001857', 'epoch': '0.1214', 'num_input_tokens_seen': 1454040, 'train_runtime': '1354', 'train_tokens_per_second': '1074'}
+
+ 12%|█▏        | 220/1813 [22:34<2:32:37,  5.75s/it]
+ 12%|█▏        | 221/1813 [22:40<2:38:03,  5.96s/it]
+ 12%|█▏        | 222/1813 [22:45<2:31:36,  5.72s/it]
+ 12%|█▏        | 223/1813 [22:53<2:46:25,  6.28s/it]
+ 12%|█▏        | 224/1813 [22:58<2:37:19,  5.94s/it]
+ 12%|█▏        | 225/1813 [23:05<2:46:15,  6.28s/it]
+ 12%|█▏        | 226/1813 [23:12<2:51:49,  6.50s/it]
+ 13%|█▎        | 227/1813 [23:18<2:46:25,  6.30s/it]
+ 13%|█▎        | 228/1813 [23:24<2:48:08,  6.36s/it]
+ 13%|█▎        | 229/1813 [23:31<2:52:59,  6.55s/it]
+ 13%|█▎        | 230/1813 [23:36<2:39:23,  6.04s/it]
+                                                    
+{'loss': '1.007', 'grad_norm': '1.198', 'learning_rate': '0.0001854', 'epoch': '0.1269', 'num_input_tokens_seen': 1522056, 'train_runtime': '1417', 'train_tokens_per_second': '1074'}
+
+ 13%|█▎        | 230/1813 [23:36<2:39:23,  6.04s/it]
+ 13%|█▎        | 231/1813 [23:41<2:29:29,  5.67s/it]
+ 13%|█▎        | 232/1813 [23:48<2:42:02,  6.15s/it]
+ 13%|█▎        | 233/1813 [23:53<2:32:27,  5.79s/it]
+ 13%|█▎        | 234/1813 [23:58<2:27:12,  5.59s/it]
+ 13%|█▎        | 235/1813 [24:05<2:36:21,  5.95s/it]
+ 13%|█▎        | 236/1813 [24:13<2:49:14,  6.44s/it]
+ 13%|█▎        | 237/1813 [24:20<2:52:14,  6.56s/it]
+ 13%|█▎        | 238/1813 [24:25<2:44:48,  6.28s/it]
+ 13%|█▎        | 239/1813 [24:33<2:54:46,  6.66s/it]
+ 13%|█▎        | 240/1813 [24:39<2:51:17,  6.53s/it]
+                                                    
+{'loss': '1.005', 'grad_norm': '0.8324', 'learning_rate': '0.0001851', 'epoch': '0.1324', 'num_input_tokens_seen': 1590200, 'train_runtime': '1479', 'train_tokens_per_second': '1075'}
+
+ 13%|█▎        | 240/1813 [24:39<2:51:17,  6.53s/it]
+ 13%|█▎        | 241/1813 [24:45<2:50:04,  6.49s/it]
+ 13%|█▎        | 242/1813 [24:52<2:48:43,  6.44s/it]
+ 13%|█▎        | 243/1813 [24:57<2:42:43,  6.22s/it]
+ 13%|█▎        | 244/1813 [25:04<2:49:20,  6.48s/it]
+ 14%|█▎        | 245/1813 [25:11<2:53:25,  6.64s/it]
+ 14%|█▎        | 246/1813 [25:17<2:43:13,  6.25s/it]
+ 14%|█▎        | 247/1813 [25:24<2:47:49,  6.43s/it]
+ 14%|█▎        | 248/1813 [25:31<2:58:05,  6.83s/it]
+ 14%|█▎        | 249/1813 [25:37<2:48:27,  6.46s/it]
+ 14%|█▍        | 250/1813 [25:43<2:41:40,  6.21s/it]
+                                                    
+{'loss': '1.083', 'grad_norm': '0.8815', 'learning_rate': '0.0001848', 'epoch': '0.1379', 'num_input_tokens_seen': 1658648, 'train_runtime': '1543', 'train_tokens_per_second': '1075'}
+
+ 14%|█▍        | 250/1813 [25:43<2:41:40,  6.21s/it]
+ 14%|█▍        | 251/1813 [25:47<2:28:31,  5.71s/it]
+ 14%|█▍        | 252/1813 [25:52<2:25:19,  5.59s/it]
+ 14%|█▍        | 253/1813 [25:59<2:29:45,  5.76s/it]
+ 14%|█▍        | 254/1813 [26:06<2:42:22,  6.25s/it]
+ 14%|█▍        | 255/1813 [26:13<2:46:57,  6.43s/it]
+ 14%|█▍        | 256/1813 [26:18<2:40:12,  6.17s/it]
+ 14%|█▍        | 257/1813 [26:25<2:44:42,  6.35s/it]
+ 14%|█▍        | 258/1813 [26:32<2:47:35,  6.47s/it]
+ 14%|█▍        | 259/1813 [26:37<2:34:05,  5.95s/it]
+ 14%|█▍        | 260/1813 [26:43<2:36:02,  6.03s/it]
+                                                    
+{'loss': '0.8762', 'grad_norm': '1.196', 'learning_rate': '0.0001845', 'epoch': '0.1434', 'num_input_tokens_seen': 1722792, 'train_runtime': '1603', 'train_tokens_per_second': '1074'}
+
+ 14%|█▍        | 260/1813 [26:43<2:36:02,  6.03s/it]
+ 14%|█▍        | 261/1813 [26:50<2:45:41,  6.41s/it]
+ 14%|█▍        | 262/1813 [26:57<2:48:20,  6.51s/it]
+ 15%|█▍        | 263/1813 [27:04<2:49:41,  6.57s/it]
+ 15%|█▍        | 264/1813 [27:10<2:47:20,  6.48s/it]
+ 15%|█▍        | 265/1813 [27:16<2:45:02,  6.40s/it]
+ 15%|█▍        | 266/1813 [27:21<2:32:49,  5.93s/it]
+ 15%|█▍        | 267/1813 [27:26<2:27:51,  5.74s/it]
+ 15%|█▍        | 268/1813 [27:31<2:22:33,  5.54s/it]
+ 15%|█▍        | 269/1813 [27:38<2:27:17,  5.72s/it]
+ 15%|█▍        | 270/1813 [27:43<2:27:51,  5.75s/it]
+                                                    
+{'loss': '0.9321', 'grad_norm': '0.8938', 'learning_rate': '0.0001841', 'epoch': '0.149', 'num_input_tokens_seen': 1786808, 'train_runtime': '1664', 'train_tokens_per_second': '1074'}
+
+ 15%|█▍        | 270/1813 [27:43<2:27:51,  5.75s/it]
+ 15%|█▍        | 271/1813 [27:51<2:44:45,  6.41s/it]
+ 15%|█▌        | 272/1813 [27:57<2:35:44,  6.06s/it]
+ 15%|█▌        | 273/1813 [28:05<2:52:14,  6.71s/it]
+ 15%|█▌        | 274/1813 [28:11<2:51:21,  6.68s/it]
+ 15%|█▌        | 275/1813 [28:17<2:40:29,  6.26s/it]
+ 15%|█▌        | 276/1813 [28:22<2:33:46,  6.00s/it]
+ 15%|█▌        | 277/1813 [28:28<2:36:24,  6.11s/it]
+ 15%|█▌        | 278/1813 [28:35<2:37:39,  6.16s/it]
+ 15%|█▌        | 279/1813 [28:42<2:49:44,  6.64s/it]
+ 15%|█▌        | 280/1813 [28:49<2:49:39,  6.64s/it]
+                                                    
+{'loss': '1.005', 'grad_norm': '0.8635', 'learning_rate': '0.0001837', 'epoch': '0.1545', 'num_input_tokens_seen': 1858112, 'train_runtime': '1730', 'train_tokens_per_second': '1074'}
+
+ 15%|█▌        | 280/1813 [28:49<2:49:39,  6.64s/it]
+ 15%|█▌        | 281/1813 [28:55<2:44:24,  6.44s/it]
+ 16%|█▌        | 282/1813 [29:01<2:38:09,  6.20s/it]
+ 16%|█▌        | 283/1813 [29:06<2:32:10,  5.97s/it]
+ 16%|█▌        | 284/1813 [29:12<2:30:45,  5.92s/it]
+ 16%|█▌        | 285/1813 [29:19<2:36:01,  6.13s/it]
+ 16%|█▌        | 286/1813 [29:24<2:34:26,  6.07s/it]
+ 16%|█▌        | 287/1813 [29:31<2:36:01,  6.13s/it]
+ 16%|█▌        | 288/1813 [29:36<2:31:47,  5.97s/it]
+ 16%|█▌        | 289/1813 [29:43<2:36:16,  6.15s/it]
+ 16%|█▌        | 290/1813 [29:50<2:39:49,  6.30s/it]
+                                                    
+{'loss': '0.9707', 'grad_norm': '0.9038', 'learning_rate': '0.0001833', 'epoch': '0.16', 'num_input_tokens_seen': 1920600, 'train_runtime': '1790', 'train_tokens_per_second': '1073'}
+
+ 16%|█▌        | 290/1813 [29:50<2:39:49,  6.30s/it]
+ 16%|█▌        | 291/1813 [29:57<2:45:55,  6.54s/it]
+ 16%|█▌        | 292/1813 [30:04<2:52:34,  6.81s/it]
+ 16%|█▌        | 293/1813 [30:10<2:46:06,  6.56s/it]
+ 16%|█▌        | 294/1813 [30:17<2:51:07,  6.76s/it]
+ 16%|█▋        | 295/1813 [30:25<2:56:20,  6.97s/it]
+ 16%|█▋        | 296/1813 [30:31<2:46:57,  6.60s/it]
+ 16%|█▋        | 297/1813 [30:37<2:44:21,  6.50s/it]
+ 16%|█▋        | 298/1813 [30:42<2:33:32,  6.08s/it]
+ 16%|█▋        | 299/1813 [30:48<2:31:23,  6.00s/it]
+ 17%|█▋        | 300/1813 [30:53<2:29:39,  5.93s/it]
+                                                    
+{'loss': '1.041', 'grad_norm': '1.413', 'learning_rate': '0.0001828', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312, 'train_runtime': '1854', 'train_tokens_per_second': '1072'}
+
+ 17%|█▋        | 300/1813 [30:53<2:29:39,  5.93s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.81it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.26it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.34it/s]
+
+  1%|          | 9/1000 [00:01<02:32,  6.48it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.11it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.02it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.98it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.74it/s]
+
+  2%|▏         | 18/1000 [00:02<02:41,  6.06it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.61it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.26it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.61it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.38it/s]
+
+  3%|▎         | 26/1000 [00:03<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.68it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.12it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.10it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.34it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.37it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.77it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.92it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.96it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.78it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.32it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.21it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.35it/s]
+
+  6%|▋         | 63/1000 [00:08<01:35,  9.77it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.38it/s]
+
+  7%|▋         | 67/1000 [00:08<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.13it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.88it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.22it/s]
+
+  8%|▊         | 84/1000 [00:11<01:53,  8.04it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.96it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.40it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.03it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.27it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.09it/s]
+
+  9%|▉         | 92/1000 [00:12<02:26,  6.19it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.53it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:12,  6.80it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.76it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.83it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.34it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.47it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.22it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.05it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.52it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.79it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.21it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.09it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.44it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.81it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.11it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.73it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.17it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.65it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:26,  9.91it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.98it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.05it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.91it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.08it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.94it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.07it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:20, 10.46it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.67it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.99it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.63it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.82it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.42it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:26,  5.69it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.19it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:23,  5.79it/s]
+
+ 17%|█▋        | 169/1000 [00:21<01:45,  7.85it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.17it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.16it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.23it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.75it/s]
+
+ 18%|█▊        | 176/1000 [00:22<01:37,  8.48it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.83it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.60it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.52it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:23<02:22,  5.73it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.81it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.76it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:42,  7.91it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.12it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:01,  6.65it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.50it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.74it/s]
+
+ 20%|█▉        | 198/1000 [00:25<01:27,  9.19it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.70it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.81it/s]
+
+ 20%|██        | 204/1000 [00:26<01:31,  8.66it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.97it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.54it/s]
+
+ 21%|██        | 210/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.72it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.76it/s]
+
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.16it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.51it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.80it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.32it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.56it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.06it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:06,  6.12it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.43it/s]
+
+ 23%|██▎       | 228/1000 [00:29<01:52,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.92it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.39it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.06it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.23it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.95it/s]
+
+ 24%|██▎       | 237/1000 [00:30<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.99it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.37it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.03it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.48it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.67it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.27it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:05,  5.97it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.88it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.04it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.38it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.72it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.77it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.70it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.53it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.44it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:55,  6.35it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.61it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:45,  6.93it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:44,  7.00it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.42it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.68it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.69it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.04it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.81it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.62it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.03it/s]
+
+ 28%|██▊       | 283/1000 [00:37<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.72it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.46it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.74it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:38<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.20it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.48it/s]
+
+ 30%|███       | 300/1000 [00:40<01:41,  6.87it/s]
+
+ 30%|███       | 301/1000 [00:40<01:37,  7.14it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.79it/s]
+
+ 30%|███       | 304/1000 [00:40<01:25,  8.10it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.15it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.79it/s]
+
+ 31%|███       | 311/1000 [00:41<01:09,  9.95it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:01, 11.10it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:07, 10.17it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.53it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.22it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.89it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.42it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.95it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:44<01:22,  8.05it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.83it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.05it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.91it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:45<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.23it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.43it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.64it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.57it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.37it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.98it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.59it/s]
+
+ 36%|███▌      | 361/1000 [00:47<01:07,  9.44it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.20it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.29it/s]
+
+ 36%|███▋      | 365/1000 [00:48<02:08,  4.93it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.10it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.23it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.93it/s]
+
+ 38%|███▊      | 380/1000 [00:50<01:10,  8.78it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.21it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.20it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.15it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.65it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.38it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.78it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:09,  8.69it/s]
+
+ 40%|███▉      | 398/1000 [00:53<01:23,  7.20it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:41,  5.95it/s]
+
+ 40%|████      | 400/1000 [00:53<01:51,  5.39it/s]
+
+ 40%|████      | 402/1000 [00:54<01:46,  5.61it/s]
+
+ 40%|████      | 404/1000 [00:54<01:24,  7.09it/s]
+
+ 41%|████      | 406/1000 [00:54<01:10,  8.42it/s]
+
+ 41%|████      | 407/1000 [00:54<01:24,  6.99it/s]
+
+ 41%|████      | 408/1000 [00:54<01:37,  6.09it/s]
+
+ 41%|████      | 410/1000 [00:55<01:34,  6.27it/s]
+
+ 41%|████      | 411/1000 [00:55<01:44,  5.63it/s]
+
+ 41%|████      | 412/1000 [00:55<01:53,  5.16it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.03it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:26,  6.79it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.30it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.50it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.51it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.68it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.92it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.35it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.83it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.55it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.03it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:50, 11.27it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:54, 10.27it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:10,  7.92it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:01,  9.15it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:09,  8.00it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:17,  7.20it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.68it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:14,  7.46it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.00it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.51it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.27it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.84it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.64it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.76it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.02it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.31it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.57it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.52it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.41it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.15it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.07it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.33it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  7.99it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.23it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.29it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.99it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.41it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.45it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.94it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.57it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:12,  6.81it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.44it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.97it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.29it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.05it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.69it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.58it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.60it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.55it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.14it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:47,  9.74it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.71it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.38it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.67it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.77it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.93it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.52it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.96it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.04it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.86it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.06it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  8.00it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.76it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.57it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.46it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.46it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.33it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:06,  6.36it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.85it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.26it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.99it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:19<00:57,  7.17it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.55it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.77it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.20it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.82it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.41it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.90it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.69it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.49it/s]
+
+ 60%|██████    | 601/1000 [01:21<01:02,  6.41it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.73it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.70it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.12it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.86it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:45,  8.49it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.76it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.83it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:42,  8.96it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  7.97it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:46,  8.16it/s]
+
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.43it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.61it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.21it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.85it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:40,  9.15it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.64it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.01it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.90it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.13it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.94it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.59it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.58it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.34it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.72it/s]
+
+ 66%|██████▌   | 656/1000 [01:28<00:38,  8.93it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.10it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.57it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.49it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.21it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.65it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.68it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.72it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:28, 11.02it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.95it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.92it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.88it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.18it/s]
+
+ 69%|██████▉   | 690/1000 [01:33<00:45,  6.82it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.85it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.27it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.62it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:35<00:40,  7.23it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.43it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.04it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.64it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.51it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.72it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.88it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.40it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.08it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.82it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.95it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.49it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:33,  8.18it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.26it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.27it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.33it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.15it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.62it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.32it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.52it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.88it/s]
+
+ 74%|███████▎  | 737/1000 [01:39<00:27,  9.51it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.77it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.30it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.62it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.46it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.70it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.18it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.25it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.24it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:37,  6.43it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.63it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.34it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.73it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.68it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.63it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.48it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.80it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.89it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:26,  8.38it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.18it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.55it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.39it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.02it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.24it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.74it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.99it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.21it/s]
+
+ 80%|███████▉  | 797/1000 [01:47<00:29,  6.88it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.03it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.73it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.66it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:24,  7.86it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.19it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.99it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.23it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.25it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.07it/s]
+
+ 82%|████████▏ | 815/1000 [01:49<00:19,  9.33it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.15it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:50<00:22,  7.83it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.75it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:25,  6.96it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:30,  5.78it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.72it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.01it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:52<00:27,  6.03it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.42it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:27,  6.04it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.41it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.61it/s]
+
+ 84%|████████▍ | 842/1000 [01:53<00:19,  8.13it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.55it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.88it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.04it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.64it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.73it/s]
+
+ 87%|████████▋ | 867/1000 [01:56<00:14,  9.34it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.30it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.31it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.80it/s]
+
+ 88%|████████▊ | 875/1000 [01:57<00:16,  7.65it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.90it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.89it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.62it/s]
+
+ 88%|████████▊ | 882/1000 [01:58<00:17,  6.56it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.97it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.37it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.78it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.78it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.62it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.86it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.87it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.08it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.06it/s]
+
+ 91%|█████████ | 908/1000 [02:01<00:11,  8.33it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.60it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.79it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.40it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.10it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.38it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.61it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.95it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.18it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:04<00:10,  6.66it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.49it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.62it/s]
+
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.32it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.82it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.04it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:06<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.84it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.42it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.18it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.42it/s]
+
+ 96%|█████████▌| 957/1000 [02:08<00:06,  6.42it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.06it/s]
+
+ 96%|█████████▋| 964/1000 [02:09<00:04,  7.50it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.43it/s]
+
+ 97%|█████████▋| 972/1000 [02:10<00:03,  8.04it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.22it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.81it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.44it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.07it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.45it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.75it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.01it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:13<00:01,  4.99it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.54it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+                                                    
+
+                                                   
+{'eval_loss': '0.8416', 'eval_runtime': '135.3', 'eval_samples_per_second': '7.392', 'eval_steps_per_second': '7.392', 'epoch': '0.1655', 'num_input_tokens_seen': 1988312}
+
+ 17%|█▋        | 300/1813 [33:09<2:29:39,  5.93s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.67it/s]
+
+                                                   
+ 17%|█▋        | 301/1813 [33:14<19:27:03, 46.31s/it]
+ 17%|█▋        | 302/1813 [33:19<14:13:33, 33.89s/it]
+ 17%|█▋        | 303/1813 [33:26<10:49:54, 25.82s/it]
+ 17%|█▋        | 304/1813 [33:33<8:31:54, 20.35s/it] 
+ 17%|█▋        | 305/1813 [33:40<6:44:45, 16.10s/it]
+ 17%|█▋        | 306/1813 [33:46<5:28:52, 13.09s/it]
+ 17%|█▋        | 307/1813 [33:51<4:33:16, 10.89s/it]
+ 17%|█▋        | 308/1813 [33:57<3:54:05,  9.33s/it]
+ 17%|█▋        | 309/1813 [34:04<3:32:55,  8.49s/it]
+ 17%|█▋        | 310/1813 [34:09<3:07:31,  7.49s/it]
+                                                    
+{'loss': '0.9684', 'grad_norm': '1.203', 'learning_rate': '0.0001823', 'epoch': '0.171', 'num_input_tokens_seen': 2050672, 'train_runtime': '2049', 'train_tokens_per_second': '1001'}
+
+ 17%|█▋        | 310/1813 [34:09<3:07:31,  7.49s/it]
+ 17%|█▋        | 311/1813 [34:16<3:04:13,  7.36s/it]
+ 17%|█▋        | 312/1813 [34:23<3:03:38,  7.34s/it]
+ 17%|█▋        | 313/1813 [34:30<2:59:08,  7.17s/it]
+ 17%|█▋        | 314/1813 [34:35<2:46:06,  6.65s/it]
+ 17%|█▋        | 315/1813 [34:42<2:47:33,  6.71s/it]
+ 17%|█▋        | 316/1813 [34:49<2:45:48,  6.65s/it]
+ 17%|█▋        | 317/1813 [34:55<2:40:52,  6.45s/it]
+ 18%|█▊        | 318/1813 [35:00<2:34:16,  6.19s/it]
+ 18%|█▊        | 319/1813 [35:05<2:25:37,  5.85s/it]
+ 18%|█▊        | 320/1813 [35:11<2:26:40,  5.89s/it]
+                                                    
+{'loss': '1.043', 'grad_norm': '0.6369', 'learning_rate': '0.0001817', 'epoch': '0.1766', 'num_input_tokens_seen': 2118000, 'train_runtime': '2112', 'train_tokens_per_second': '1003'}
+
+ 18%|█▊        | 320/1813 [35:11<2:26:40,  5.89s/it]
+ 18%|█▊        | 321/1813 [35:17<2:27:05,  5.92s/it]
+ 18%|█▊        | 322/1813 [35:23<2:25:43,  5.86s/it]
+ 18%|█▊        | 323/1813 [35:29<2:28:15,  5.97s/it]
+ 18%|█▊        | 324/1813 [35:37<2:37:26,  6.34s/it]
+ 18%|█▊        | 325/1813 [35:44<2:43:06,  6.58s/it]
+ 18%|█▊        | 326/1813 [35:50<2:39:13,  6.42s/it]
+ 18%|█▊        | 327/1813 [35:56<2:35:27,  6.28s/it]
+ 18%|█▊        | 328/1813 [36:02<2:36:15,  6.31s/it]
+ 18%|█▊        | 329/1813 [36:08<2:31:13,  6.11s/it]
+ 18%|█▊        | 330/1813 [36:13<2:28:25,  6.01s/it]
+                                                    
+{'loss': '0.9582', 'grad_norm': '1.013', 'learning_rate': '0.0001812', 'epoch': '0.1821', 'num_input_tokens_seen': 2184640, 'train_runtime': '2174', 'train_tokens_per_second': '1005'}
+
+ 18%|█▊        | 330/1813 [36:13<2:28:25,  6.01s/it]
+ 18%|█▊        | 331/1813 [36:20<2:30:29,  6.09s/it]
+ 18%|█▊        | 332/1813 [36:25<2:22:00,  5.75s/it]
+ 18%|█▊        | 333/1813 [36:31<2:23:01,  5.80s/it]
+ 18%|█▊        | 334/1813 [36:36<2:22:22,  5.78s/it]
+ 18%|█▊        | 335/1813 [36:44<2:37:08,  6.38s/it]
+ 19%|█▊        | 336/1813 [36:51<2:41:46,  6.57s/it]
+ 19%|█▊        | 337/1813 [36:57<2:35:41,  6.33s/it]
+ 19%|█▊        | 338/1813 [37:03<2:35:29,  6.33s/it]
+ 19%|█▊        | 339/1813 [37:09<2:31:25,  6.16s/it]
+ 19%|█▉        | 340/1813 [37:16<2:37:17,  6.41s/it]
+                                                    
+{'loss': '0.9408', 'grad_norm': '1.174', 'learning_rate': '0.0001806', 'epoch': '0.1876', 'num_input_tokens_seen': 2251392, 'train_runtime': '2237', 'train_tokens_per_second': '1007'}
+
+ 19%|█▉        | 340/1813 [37:16<2:37:17,  6.41s/it]
+ 19%|█▉        | 341/1813 [37:22<2:37:03,  6.40s/it]
+ 19%|█▉        | 342/1813 [37:30<2:44:33,  6.71s/it]
+ 19%|█▉        | 343/1813 [37:35<2:36:08,  6.37s/it]
+ 19%|█▉        | 344/1813 [37:41<2:30:10,  6.13s/it]
+ 19%|█▉        | 345/1813 [37:48<2:37:35,  6.44s/it]
+ 19%|█▉        | 346/1813 [37:55<2:36:58,  6.42s/it]
+ 19%|█▉        | 347/1813 [37:59<2:25:54,  5.97s/it]
+ 19%|█▉        | 348/1813 [38:06<2:29:27,  6.12s/it]
+ 19%|█▉        | 349/1813 [38:14<2:46:07,  6.81s/it]
+ 19%|█▉        | 350/1813 [38:22<2:52:11,  7.06s/it]
+                                                    
+{'loss': '1.057', 'grad_norm': '0.8784', 'learning_rate': '0.0001799', 'epoch': '0.1931', 'num_input_tokens_seen': 2324712, 'train_runtime': '2302', 'train_tokens_per_second': '1010'}
+
+ 19%|█▉        | 350/1813 [38:22<2:52:11,  7.06s/it]
+ 19%|█▉        | 351/1813 [38:29<2:51:27,  7.04s/it]
+ 19%|█▉        | 352/1813 [38:35<2:44:20,  6.75s/it]
+ 19%|█▉        | 353/1813 [38:41<2:36:13,  6.42s/it]
+ 20%|█▉        | 354/1813 [38:47<2:31:40,  6.24s/it]
+ 20%|█▉        | 355/1813 [38:53<2:29:57,  6.17s/it]
+ 20%|█▉        | 356/1813 [39:00<2:36:14,  6.43s/it]
+ 20%|█▉        | 357/1813 [39:05<2:31:08,  6.23s/it]
+ 20%|█▉        | 358/1813 [39:12<2:33:02,  6.31s/it]
+ 20%|█▉        | 359/1813 [39:18<2:30:33,  6.21s/it]
+ 20%|█▉        | 360/1813 [39:25<2:35:09,  6.41s/it]
+                                                    
+{'loss': '0.9963', 'grad_norm': '0.9282', 'learning_rate': '0.0001793', 'epoch': '0.1986', 'num_input_tokens_seen': 2390776, 'train_runtime': '2365', 'train_tokens_per_second': '1011'}
+
+ 20%|█▉        | 360/1813 [39:25<2:35:09,  6.41s/it]
+ 20%|█▉        | 361/1813 [39:30<2:25:20,  6.01s/it]
+ 20%|█▉        | 362/1813 [39:35<2:22:40,  5.90s/it]
+ 20%|██        | 363/1813 [39:42<2:29:39,  6.19s/it]
+ 20%|██        | 364/1813 [39:49<2:32:31,  6.32s/it]
+ 20%|██        | 365/1813 [39:55<2:30:23,  6.23s/it]
+ 20%|██        | 366/1813 [40:02<2:35:51,  6.46s/it]
+ 20%|██        | 367/1813 [40:08<2:32:18,  6.32s/it]
+ 20%|██        | 368/1813 [40:13<2:25:21,  6.04s/it]
+ 20%|██        | 369/1813 [40:18<2:16:01,  5.65s/it]
+ 20%|██        | 370/1813 [40:24<2:17:17,  5.71s/it]
+                                                    
+{'loss': '0.9945', 'grad_norm': '1.109', 'learning_rate': '0.0001786', 'epoch': '0.2041', 'num_input_tokens_seen': 2454808, 'train_runtime': '2424', 'train_tokens_per_second': '1013'}
+
+ 20%|██        | 370/1813 [40:24<2:17:17,  5.71s/it]
+ 20%|██        | 371/1813 [40:29<2:12:13,  5.50s/it]
+ 21%|██        | 372/1813 [40:37<2:31:27,  6.31s/it]
+ 21%|██        | 373/1813 [40:43<2:30:50,  6.28s/it]
+ 21%|██        | 374/1813 [40:51<2:38:39,  6.62s/it]
+ 21%|██        | 375/1813 [40:58<2:41:02,  6.72s/it]
+ 21%|██        | 376/1813 [41:03<2:33:01,  6.39s/it]
+ 21%|██        | 377/1813 [41:10<2:31:56,  6.35s/it]
+ 21%|██        | 378/1813 [41:16<2:30:59,  6.31s/it]
+ 21%|██        | 379/1813 [41:21<2:21:40,  5.93s/it]
+ 21%|██        | 380/1813 [41:26<2:15:17,  5.66s/it]
+                                                    
+{'loss': '0.9595', 'grad_norm': '1.123', 'learning_rate': '0.0001779', 'epoch': '0.2097', 'num_input_tokens_seen': 2520472, 'train_runtime': '2486', 'train_tokens_per_second': '1014'}
+
+ 21%|██        | 380/1813 [41:26<2:15:17,  5.66s/it]
+ 21%|██        | 381/1813 [41:33<2:25:40,  6.10s/it]
+ 21%|██        | 382/1813 [41:40<2:31:23,  6.35s/it]
+ 21%|██        | 383/1813 [41:46<2:29:45,  6.28s/it]
+ 21%|██        | 384/1813 [41:52<2:29:24,  6.27s/it]
+ 21%|██        | 385/1813 [41:58<2:28:13,  6.23s/it]
+ 21%|██▏       | 386/1813 [42:03<2:17:04,  5.76s/it]
+ 21%|██▏       | 387/1813 [42:09<2:20:21,  5.91s/it]
+ 21%|██▏       | 388/1813 [42:16<2:28:54,  6.27s/it]
+ 21%|██▏       | 389/1813 [42:24<2:39:35,  6.72s/it]
+ 22%|██▏       | 390/1813 [42:31<2:39:53,  6.74s/it]
+                                                    
+{'loss': '0.9978', 'grad_norm': '0.8193', 'learning_rate': '0.0001771', 'epoch': '0.2152', 'num_input_tokens_seen': 2591512, 'train_runtime': '2551', 'train_tokens_per_second': '1016'}
+
+ 22%|██▏       | 390/1813 [42:31<2:39:53,  6.74s/it]
+ 22%|██▏       | 391/1813 [42:37<2:35:10,  6.55s/it]
+ 22%|██▏       | 392/1813 [42:43<2:33:18,  6.47s/it]
+ 22%|██▏       | 393/1813 [42:50<2:34:24,  6.52s/it]
+ 22%|██▏       | 394/1813 [42:55<2:23:28,  6.07s/it]
+ 22%|██▏       | 395/1813 [43:01<2:20:34,  5.95s/it]
+ 22%|██▏       | 396/1813 [43:07<2:24:20,  6.11s/it]
+ 22%|██▏       | 397/1813 [43:12<2:18:19,  5.86s/it]
+ 22%|██▏       | 398/1813 [43:19<2:22:29,  6.04s/it]
+ 22%|██▏       | 399/1813 [43:27<2:34:24,  6.55s/it]
+ 22%|██▏       | 400/1813 [43:35<2:46:23,  7.07s/it]
+                                                    
+{'loss': '0.9761', 'grad_norm': '0.9673', 'learning_rate': '0.0001763', 'epoch': '0.2207', 'num_input_tokens_seen': 2660336, 'train_runtime': '2615', 'train_tokens_per_second': '1017'}
+
+ 22%|██▏       | 400/1813 [43:35<2:46:23,  7.07s/it]
+ 22%|██▏       | 401/1813 [43:42<2:48:06,  7.14s/it]
+ 22%|██▏       | 402/1813 [43:47<2:31:15,  6.43s/it]
+ 22%|██▏       | 403/1813 [43:53<2:28:36,  6.32s/it]
+ 22%|██▏       | 404/1813 [44:00<2:31:43,  6.46s/it]
+ 22%|██▏       | 405/1813 [44:07<2:35:40,  6.63s/it]
+ 22%|██▏       | 406/1813 [44:12<2:23:33,  6.12s/it]
+ 22%|██▏       | 407/1813 [44:16<2:11:49,  5.63s/it]
+ 23%|██▎       | 408/1813 [44:22<2:12:54,  5.68s/it]
+ 23%|██▎       | 409/1813 [44:28<2:15:08,  5.78s/it]
+ 23%|██▎       | 410/1813 [44:33<2:11:29,  5.62s/it]
+                                                    
+{'loss': '1.011', 'grad_norm': '1.109', 'learning_rate': '0.0001755', 'epoch': '0.2262', 'num_input_tokens_seen': 2722056, 'train_runtime': '2674', 'train_tokens_per_second': '1018'}
+
+ 23%|██▎       | 410/1813 [44:33<2:11:29,  5.62s/it]
+ 23%|██▎       | 411/1813 [44:39<2:14:34,  5.76s/it]
+ 23%|██▎       | 412/1813 [44:46<2:18:22,  5.93s/it]
+ 23%|██▎       | 413/1813 [44:53<2:26:16,  6.27s/it]
+ 23%|██▎       | 414/1813 [44:58<2:18:41,  5.95s/it]
+ 23%|██▎       | 415/1813 [45:04<2:20:13,  6.02s/it]
+ 23%|██▎       | 416/1813 [45:11<2:22:33,  6.12s/it]
+ 23%|██▎       | 417/1813 [45:18<2:33:22,  6.59s/it]
+ 23%|██▎       | 418/1813 [45:24<2:27:53,  6.36s/it]
+ 23%|██▎       | 419/1813 [45:31<2:28:36,  6.40s/it]
+ 23%|██▎       | 420/1813 [45:36<2:22:51,  6.15s/it]
+                                                    
+{'loss': '1.012', 'grad_norm': '0.7823', 'learning_rate': '0.0001747', 'epoch': '0.2317', 'num_input_tokens_seen': 2790960, 'train_runtime': '2737', 'train_tokens_per_second': '1020'}
+
+ 23%|██▎       | 420/1813 [45:36<2:22:51,  6.15s/it]
+ 23%|██▎       | 421/1813 [45:42<2:22:07,  6.13s/it]
+ 23%|██▎       | 422/1813 [45:49<2:29:23,  6.44s/it]
+ 23%|██▎       | 423/1813 [45:55<2:25:02,  6.26s/it]
+ 23%|██▎       | 424/1813 [46:01<2:22:50,  6.17s/it]
+ 23%|██▎       | 425/1813 [46:07<2:18:31,  5.99s/it]
+ 23%|██▎       | 426/1813 [46:14<2:27:53,  6.40s/it]
+ 24%|██▎       | 427/1813 [46:20<2:24:12,  6.24s/it]
+ 24%|██▎       | 428/1813 [46:26<2:23:44,  6.23s/it]
+ 24%|██▎       | 429/1813 [46:32<2:18:33,  6.01s/it]
+ 24%|██▎       | 430/1813 [46:39<2:29:24,  6.48s/it]
+                                                    
+{'loss': '0.9905', 'grad_norm': '0.7096', 'learning_rate': '0.0001738', 'epoch': '0.2372', 'num_input_tokens_seen': 2860096, 'train_runtime': '2800', 'train_tokens_per_second': '1022'}
+
+ 24%|██▎       | 430/1813 [46:39<2:29:24,  6.48s/it]
+ 24%|██▍       | 431/1813 [46:45<2:26:23,  6.36s/it]
+ 24%|██▍       | 432/1813 [46:51<2:23:00,  6.21s/it]
+ 24%|██▍       | 433/1813 [46:57<2:18:17,  6.01s/it]
+ 24%|██▍       | 434/1813 [47:03<2:21:30,  6.16s/it]
+ 24%|██▍       | 435/1813 [47:09<2:19:23,  6.07s/it]
+ 24%|██▍       | 436/1813 [47:15<2:21:18,  6.16s/it]
+ 24%|██▍       | 437/1813 [47:21<2:18:02,  6.02s/it]
+ 24%|██▍       | 438/1813 [47:27<2:19:47,  6.10s/it]
+ 24%|██▍       | 439/1813 [47:33<2:18:35,  6.05s/it]
+ 24%|██▍       | 440/1813 [47:40<2:19:24,  6.09s/it]
+                                                    
+{'loss': '0.8976', 'grad_norm': '1.254', 'learning_rate': '0.0001729', 'epoch': '0.2428', 'num_input_tokens_seen': 2924856, 'train_runtime': '2860', 'train_tokens_per_second': '1023'}
+
+ 24%|██▍       | 440/1813 [47:40<2:19:24,  6.09s/it]
+ 24%|██▍       | 441/1813 [47:45<2:17:26,  6.01s/it]
+ 24%|██▍       | 442/1813 [47:51<2:17:08,  6.00s/it]
+ 24%|██▍       | 443/1813 [47:58<2:22:44,  6.25s/it]
+ 24%|██▍       | 444/1813 [48:04<2:18:42,  6.08s/it]
+ 25%|██▍       | 445/1813 [48:11<2:26:23,  6.42s/it]
+ 25%|██▍       | 446/1813 [48:18<2:27:29,  6.47s/it]
+ 25%|██▍       | 447/1813 [48:24<2:26:08,  6.42s/it]
+ 25%|██▍       | 448/1813 [48:30<2:26:03,  6.42s/it]
+ 25%|██▍       | 449/1813 [48:36<2:18:33,  6.10s/it]
+ 25%|██▍       | 450/1813 [48:41<2:14:10,  5.91s/it]
+                                                    
+{'loss': '1.023', 'grad_norm': '1.056', 'learning_rate': '0.000172', 'epoch': '0.2483', 'num_input_tokens_seen': 2990040, 'train_runtime': '2922', 'train_tokens_per_second': '1023'}
+
+ 25%|██▍       | 450/1813 [48:41<2:14:10,  5.91s/it]
+ 25%|██▍       | 451/1813 [48:49<2:24:50,  6.38s/it]
+ 25%|██▍       | 452/1813 [48:55<2:20:50,  6.21s/it]
+ 25%|██▍       | 453/1813 [49:00<2:13:18,  5.88s/it]
+ 25%|██▌       | 454/1813 [49:06<2:18:40,  6.12s/it]
+ 25%|██▌       | 455/1813 [49:13<2:21:13,  6.24s/it]
+ 25%|██▌       | 456/1813 [49:17<2:09:47,  5.74s/it]
+ 25%|██▌       | 457/1813 [49:24<2:13:26,  5.90s/it]
+ 25%|██▌       | 458/1813 [49:32<2:28:27,  6.57s/it]
+ 25%|██▌       | 459/1813 [49:39<2:30:31,  6.67s/it]
+ 25%|██▌       | 460/1813 [49:45<2:27:59,  6.56s/it]
+                                                    
+{'loss': '1.088', 'grad_norm': '0.8054', 'learning_rate': '0.000171', 'epoch': '0.2538', 'num_input_tokens_seen': 3058848, 'train_runtime': '2986', 'train_tokens_per_second': '1025'}
+
+ 25%|██▌       | 460/1813 [49:45<2:27:59,  6.56s/it]
+ 25%|██▌       | 461/1813 [49:51<2:23:45,  6.38s/it]
+ 25%|██▌       | 462/1813 [49:58<2:30:21,  6.68s/it]
+ 26%|██▌       | 463/1813 [50:04<2:24:51,  6.44s/it]
+ 26%|██▌       | 464/1813 [50:09<2:15:37,  6.03s/it]
+ 26%|██▌       | 465/1813 [50:15<2:14:37,  5.99s/it]
+ 26%|██▌       | 466/1813 [50:22<2:19:56,  6.23s/it]
+ 26%|██▌       | 467/1813 [50:28<2:17:57,  6.15s/it]
+ 26%|██▌       | 468/1813 [50:34<2:20:04,  6.25s/it]
+ 26%|██▌       | 469/1813 [50:41<2:23:14,  6.39s/it]
+ 26%|██▌       | 470/1813 [50:48<2:25:30,  6.50s/it]
+                                                    
+{'loss': '0.9539', 'grad_norm': '0.7178', 'learning_rate': '0.00017', 'epoch': '0.2593', 'num_input_tokens_seen': 3126680, 'train_runtime': '3048', 'train_tokens_per_second': '1026'}
+
+ 26%|██▌       | 470/1813 [50:48<2:25:30,  6.50s/it]
+ 26%|██▌       | 471/1813 [50:54<2:20:10,  6.27s/it]
+ 26%|██▌       | 472/1813 [51:00<2:22:52,  6.39s/it]
+ 26%|██▌       | 473/1813 [51:07<2:26:01,  6.54s/it]
+ 26%|██▌       | 474/1813 [51:13<2:22:07,  6.37s/it]
+ 26%|██▌       | 475/1813 [51:18<2:13:53,  6.00s/it]
+ 26%|██▋       | 476/1813 [51:24<2:13:06,  5.97s/it]
+ 26%|██▋       | 477/1813 [51:31<2:16:43,  6.14s/it]
+ 26%|██▋       | 478/1813 [51:38<2:23:06,  6.43s/it]
+ 26%|██▋       | 479/1813 [51:43<2:17:19,  6.18s/it]
+ 26%|██▋       | 480/1813 [51:50<2:18:55,  6.25s/it]
+                                                    
+{'loss': '1.064', 'grad_norm': '1.125', 'learning_rate': '0.000169', 'epoch': '0.2648', 'num_input_tokens_seen': 3193704, 'train_runtime': '3110', 'train_tokens_per_second': '1027'}
+
+ 26%|██▋       | 480/1813 [51:50<2:18:55,  6.25s/it]
+ 27%|██▋       | 481/1813 [51:56<2:18:57,  6.26s/it]
+ 27%|██▋       | 482/1813 [52:03<2:22:45,  6.44s/it]
+ 27%|██▋       | 483/1813 [52:11<2:30:37,  6.80s/it]
+ 27%|██▋       | 484/1813 [52:16<2:23:15,  6.47s/it]
+ 27%|██▋       | 485/1813 [52:24<2:31:22,  6.84s/it]
+ 27%|██▋       | 486/1813 [52:30<2:27:30,  6.67s/it]
+ 27%|██▋       | 487/1813 [52:37<2:28:48,  6.73s/it]
+ 27%|██▋       | 488/1813 [52:43<2:22:35,  6.46s/it]
+ 27%|██▋       | 489/1813 [52:50<2:26:45,  6.65s/it]
+ 27%|██▋       | 490/1813 [52:56<2:23:25,  6.50s/it]
+                                                    
+{'loss': '1.011', 'grad_norm': '1.03', 'learning_rate': '0.000168', 'epoch': '0.2703', 'num_input_tokens_seen': 3268064, 'train_runtime': '3177', 'train_tokens_per_second': '1029'}
+
+ 27%|██▋       | 490/1813 [52:56<2:23:25,  6.50s/it]
+ 27%|██▋       | 491/1813 [53:03<2:24:36,  6.56s/it]
+ 27%|██▋       | 492/1813 [53:11<2:31:36,  6.89s/it]
+ 27%|██▋       | 493/1813 [53:16<2:23:02,  6.50s/it]
+ 27%|██▋       | 494/1813 [53:22<2:17:18,  6.25s/it]
+ 27%|██▋       | 495/1813 [53:29<2:22:21,  6.48s/it]
+ 27%|██▋       | 496/1813 [53:35<2:20:36,  6.41s/it]
+ 27%|██▋       | 497/1813 [53:41<2:13:27,  6.08s/it]
+ 27%|██▋       | 498/1813 [53:47<2:19:14,  6.35s/it]
+ 28%|██▊       | 499/1813 [53:53<2:15:49,  6.20s/it]
+ 28%|██▊       | 500/1813 [54:02<2:34:27,  7.06s/it]
+                                                    
+{'loss': '1.085', 'grad_norm': '0.838', 'learning_rate': '0.0001669', 'epoch': '0.2759', 'num_input_tokens_seen': 3341536, 'train_runtime': '3243', 'train_tokens_per_second': '1030'}
+
+ 28%|██▊       | 500/1813 [54:02<2:34:27,  7.06s/it]
+ 28%|██▊       | 501/1813 [54:08<2:27:57,  6.77s/it]
+ 28%|██▊       | 502/1813 [54:14<2:19:23,  6.38s/it]
+ 28%|██▊       | 503/1813 [54:19<2:09:05,  5.91s/it]
+ 28%|██▊       | 504/1813 [54:25<2:09:36,  5.94s/it]
+ 28%|██▊       | 505/1813 [54:32<2:16:51,  6.28s/it]
+ 28%|██▊       | 506/1813 [54:38<2:13:30,  6.13s/it]
+ 28%|██▊       | 507/1813 [54:45<2:20:23,  6.45s/it]
+ 28%|██▊       | 508/1813 [54:50<2:10:08,  5.98s/it]
+ 28%|██▊       | 509/1813 [54:55<2:03:12,  5.67s/it]
+ 28%|██▊       | 510/1813 [55:00<2:00:08,  5.53s/it]
+                                                    
+{'loss': '0.9227', 'grad_norm': '1.162', 'learning_rate': '0.0001658', 'epoch': '0.2814', 'num_input_tokens_seen': 3402712, 'train_runtime': '3300', 'train_tokens_per_second': '1031'}
+
+ 28%|██▊       | 510/1813 [55:00<2:00:08,  5.53s/it]
+ 28%|██▊       | 511/1813 [55:05<1:58:59,  5.48s/it]
+ 28%|██▊       | 512/1813 [55:12<2:07:40,  5.89s/it]
+ 28%|██▊       | 513/1813 [55:19<2:16:45,  6.31s/it]
+ 28%|██▊       | 514/1813 [55:26<2:19:23,  6.44s/it]
+ 28%|██▊       | 515/1813 [55:31<2:10:10,  6.02s/it]
+ 28%|██▊       | 516/1813 [55:37<2:08:49,  5.96s/it]
+ 29%|██▊       | 517/1813 [55:44<2:13:16,  6.17s/it]
+ 29%|██▊       | 518/1813 [55:50<2:13:32,  6.19s/it]
+ 29%|██▊       | 519/1813 [55:56<2:14:09,  6.22s/it]
+ 29%|██▊       | 520/1813 [56:03<2:17:31,  6.38s/it]
+                                                    
+{'loss': '0.9747', 'grad_norm': '0.9882', 'learning_rate': '0.0001647', 'epoch': '0.2869', 'num_input_tokens_seen': 3469624, 'train_runtime': '3363', 'train_tokens_per_second': '1032'}
+
+ 29%|██▊       | 520/1813 [56:03<2:17:31,  6.38s/it]
+ 29%|██▊       | 521/1813 [56:07<2:01:39,  5.65s/it]
+ 29%|██▉       | 522/1813 [56:13<2:02:30,  5.69s/it]
+ 29%|██▉       | 523/1813 [56:19<2:04:58,  5.81s/it]
+ 29%|██▉       | 524/1813 [56:25<2:07:50,  5.95s/it]
+ 29%|██▉       | 525/1813 [56:30<2:01:47,  5.67s/it]
+ 29%|██▉       | 526/1813 [56:37<2:09:29,  6.04s/it]
+ 29%|██▉       | 527/1813 [56:44<2:12:57,  6.20s/it]
+ 29%|██▉       | 528/1813 [56:50<2:17:43,  6.43s/it]
+ 29%|██▉       | 529/1813 [56:57<2:21:10,  6.60s/it]
+ 29%|██▉       | 530/1813 [57:03<2:17:26,  6.43s/it]
+                                                    
+{'loss': '0.9432', 'grad_norm': '1.139', 'learning_rate': '0.0001636', 'epoch': '0.2924', 'num_input_tokens_seen': 3534904, 'train_runtime': '3424', 'train_tokens_per_second': '1032'}
+
+ 29%|██▉       | 530/1813 [57:03<2:17:26,  6.43s/it]
+ 29%|██▉       | 531/1813 [57:12<2:28:30,  6.95s/it]
+ 29%|██▉       | 532/1813 [57:18<2:25:01,  6.79s/it]
+ 29%|██▉       | 533/1813 [57:25<2:27:39,  6.92s/it]
+ 29%|██▉       | 534/1813 [57:33<2:29:20,  7.01s/it]
+ 30%|██▉       | 535/1813 [57:37<2:12:18,  6.21s/it]
+ 30%|██▉       | 536/1813 [57:42<2:05:07,  5.88s/it]
+ 30%|██▉       | 537/1813 [57:49<2:09:20,  6.08s/it]
+ 30%|██▉       | 538/1813 [57:54<2:04:45,  5.87s/it]
+ 30%|██▉       | 539/1813 [58:00<2:06:21,  5.95s/it]
+ 30%|██▉       | 540/1813 [58:06<2:08:40,  6.06s/it]
+                                                    
+{'loss': '0.9697', 'grad_norm': '0.8789', 'learning_rate': '0.0001624', 'epoch': '0.2979', 'num_input_tokens_seen': 3603488, 'train_runtime': '3487', 'train_tokens_per_second': '1033'}
+
+ 30%|██▉       | 540/1813 [58:06<2:08:40,  6.06s/it]
+ 30%|██▉       | 541/1813 [58:12<2:03:54,  5.85s/it]
+ 30%|██▉       | 542/1813 [58:17<2:02:29,  5.78s/it]
+ 30%|██▉       | 543/1813 [58:24<2:10:55,  6.19s/it]
+ 30%|███       | 544/1813 [58:33<2:23:45,  6.80s/it]
+ 30%|███       | 545/1813 [58:39<2:21:17,  6.69s/it]
+ 30%|███       | 546/1813 [58:45<2:19:09,  6.59s/it]
+ 30%|███       | 547/1813 [58:53<2:25:22,  6.89s/it]
+ 30%|███       | 548/1813 [59:01<2:28:36,  7.05s/it]
+ 30%|███       | 549/1813 [59:06<2:18:43,  6.59s/it]
+ 30%|███       | 550/1813 [59:12<2:13:02,  6.32s/it]
+                                                    
+{'loss': '0.9091', 'grad_norm': '1.082', 'learning_rate': '0.0001612', 'epoch': '0.3034', 'num_input_tokens_seen': 3676576, 'train_runtime': '3552', 'train_tokens_per_second': '1035'}
+
+ 30%|███       | 550/1813 [59:12<2:13:02,  6.32s/it]
+ 30%|███       | 551/1813 [59:18<2:10:13,  6.19s/it]
+ 30%|███       | 552/1813 [59:23<2:04:55,  5.94s/it]
+ 31%|███       | 553/1813 [59:28<1:58:53,  5.66s/it]
+ 31%|███       | 554/1813 [59:35<2:05:46,  5.99s/it]
+ 31%|███       | 555/1813 [59:40<1:58:49,  5.67s/it]
+ 31%|███       | 556/1813 [59:46<2:00:29,  5.75s/it]
+ 31%|███       | 557/1813 [59:52<2:06:43,  6.05s/it]
+ 31%|███       | 558/1813 [59:58<2:04:41,  5.96s/it]
+ 31%|███       | 559/1813 [1:00:03<1:59:48,  5.73s/it]
+ 31%|███       | 560/1813 [1:00:10<2:04:17,  5.95s/it]
+                                                      
+{'loss': '0.9754', 'grad_norm': '0.7284', 'learning_rate': '0.00016', 'epoch': '0.309', 'num_input_tokens_seen': 3738056, 'train_runtime': '3610', 'train_tokens_per_second': '1035'}
+
+ 31%|███       | 560/1813 [1:00:10<2:04:17,  5.95s/it]
+ 31%|███       | 561/1813 [1:00:16<2:03:40,  5.93s/it]
+ 31%|███       | 562/1813 [1:00:22<2:07:49,  6.13s/it]
+ 31%|███       | 563/1813 [1:00:29<2:12:58,  6.38s/it]
+ 31%|███       | 564/1813 [1:00:35<2:11:13,  6.30s/it]
+ 31%|███       | 565/1813 [1:00:41<2:07:12,  6.12s/it]
+ 31%|███       | 566/1813 [1:00:48<2:10:51,  6.30s/it]
+ 31%|███▏      | 567/1813 [1:00:53<2:06:24,  6.09s/it]
+ 31%|███▏      | 568/1813 [1:01:00<2:11:34,  6.34s/it]
+ 31%|███▏      | 569/1813 [1:01:08<2:19:07,  6.71s/it]
+ 31%|███▏      | 570/1813 [1:01:15<2:24:08,  6.96s/it]
+                                                      
+{'loss': '0.9789', 'grad_norm': '0.806', 'learning_rate': '0.0001587', 'epoch': '0.3145', 'num_input_tokens_seen': 3808208, 'train_runtime': '3676', 'train_tokens_per_second': '1036'}
+
+ 31%|███▏      | 570/1813 [1:01:15<2:24:08,  6.96s/it]
+ 31%|███▏      | 571/1813 [1:01:22<2:23:10,  6.92s/it]
+ 32%|███▏      | 572/1813 [1:01:28<2:17:07,  6.63s/it]
+ 32%|███▏      | 573/1813 [1:01:33<2:08:36,  6.22s/it]
+ 32%|███▏      | 574/1813 [1:01:39<2:05:42,  6.09s/it]
+ 32%|███▏      | 575/1813 [1:01:46<2:08:28,  6.23s/it]
+ 32%|███▏      | 576/1813 [1:01:50<1:57:33,  5.70s/it]
+ 32%|███▏      | 577/1813 [1:01:57<2:02:50,  5.96s/it]
+ 32%|███▏      | 578/1813 [1:02:03<2:02:59,  5.97s/it]
+ 32%|███▏      | 579/1813 [1:02:09<2:03:30,  6.01s/it]
+ 32%|███▏      | 580/1813 [1:02:15<2:04:14,  6.05s/it]
+                                                      
+{'loss': '0.9818', 'grad_norm': '0.9', 'learning_rate': '0.0001575', 'epoch': '0.32', 'num_input_tokens_seen': 3872328, 'train_runtime': '3736', 'train_tokens_per_second': '1037'}
+
+ 32%|███▏      | 580/1813 [1:02:15<2:04:14,  6.05s/it]
+ 32%|███▏      | 581/1813 [1:02:21<2:01:37,  5.92s/it]
+ 32%|███▏      | 582/1813 [1:02:26<1:58:23,  5.77s/it]
+ 32%|███▏      | 583/1813 [1:02:32<1:58:49,  5.80s/it]
+ 32%|███▏      | 584/1813 [1:02:38<2:03:22,  6.02s/it]
+ 32%|███▏      | 585/1813 [1:02:45<2:03:53,  6.05s/it]
+ 32%|███▏      | 586/1813 [1:02:51<2:06:05,  6.17s/it]
+ 32%|███▏      | 587/1813 [1:02:57<2:05:09,  6.13s/it]
+ 32%|███▏      | 588/1813 [1:03:03<2:04:58,  6.12s/it]
+ 32%|███▏      | 589/1813 [1:03:10<2:06:33,  6.20s/it]
+ 33%|███▎      | 590/1813 [1:03:16<2:05:26,  6.15s/it]
+                                                      
+{'loss': '0.8954', 'grad_norm': '1.096', 'learning_rate': '0.0001562', 'epoch': '0.3255', 'num_input_tokens_seen': 3936024, 'train_runtime': '3796', 'train_tokens_per_second': '1037'}
+
+ 33%|███▎      | 590/1813 [1:03:16<2:05:26,  6.15s/it]
+ 33%|███▎      | 591/1813 [1:03:21<2:00:36,  5.92s/it]
+ 33%|███▎      | 592/1813 [1:03:27<2:01:16,  5.96s/it]
+ 33%|███▎      | 593/1813 [1:03:34<2:05:03,  6.15s/it]
+ 33%|███▎      | 594/1813 [1:03:40<2:03:34,  6.08s/it]
+ 33%|███▎      | 595/1813 [1:03:45<1:59:53,  5.91s/it]
+ 33%|███▎      | 596/1813 [1:03:51<1:58:38,  5.85s/it]
+ 33%|███▎      | 597/1813 [1:03:56<1:57:25,  5.79s/it]
+ 33%|███▎      | 598/1813 [1:04:02<1:57:33,  5.81s/it]
+ 33%|███▎      | 599/1813 [1:04:08<1:59:00,  5.88s/it]
+ 33%|███▎      | 600/1813 [1:04:15<2:05:00,  6.18s/it]
+                                                      
+{'loss': '0.93', 'grad_norm': '0.8832', 'learning_rate': '0.0001549', 'epoch': '0.331', 'num_input_tokens_seen': 3999176, 'train_runtime': '3856', 'train_tokens_per_second': '1037'}
+
+ 33%|███▎      | 600/1813 [1:04:15<2:05:00,  6.18s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.77it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.23it/s]
+
+  1%|          | 7/1000 [00:00<01:58,  8.35it/s]
+
+  1%|          | 9/1000 [00:01<02:32,  6.49it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.11it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.02it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.98it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.05it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:33,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.24it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.59it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.27it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.96it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  6.00it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.68it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.12it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.76it/s]
+
+  4%|▍         | 42/1000 [00:06<01:44,  9.13it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.35it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.36it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.75it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.72it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.30it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.39it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.58it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.27it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+
+  7%|▋         | 72/1000 [00:09<02:17,  6.73it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.14it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.34it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.88it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.22it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.03it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.96it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.25it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:27,  6.17it/s]
+
+  9%|▉         | 93/1000 [00:12<02:37,  5.78it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:44,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.62it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.74it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.32it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.44it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.21it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.05it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.54it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.82it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.22it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.09it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.44it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.81it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.11it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.72it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.14it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.63it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.96it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.91it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.09it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.44it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.51it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.94it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.07it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.42it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.66it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.99it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.63it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.82it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.42it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:26,  5.69it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.19it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:23,  5.79it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.85it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.16it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:13,  6.19it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:03,  6.71it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.41it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:01,  6.79it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.55it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.49it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.75it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:22,  5.72it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:42,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.40it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.13it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:01,  6.65it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.50it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.39it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.77it/s]
+
+ 20%|█▉        | 198/1000 [00:25<01:27,  9.22it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.69it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.79it/s]
+
+ 20%|██        | 204/1000 [00:26<01:31,  8.66it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.96it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.53it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.67it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.16it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.51it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:29,  8.77it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.43it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.92it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.38it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.23it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.98it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.48it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.99it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.38it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.03it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.48it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.67it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.27it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.87it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:05,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.74it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.44it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.61it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.05it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:13,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:47,  6.71it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.08it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.82it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.63it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.04it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.72it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.74it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.77it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.07it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.90it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.48it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.13it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.78it/s]
+
+ 30%|███       | 304/1000 [00:40<01:25,  8.10it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.16it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:41,  6.80it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.82it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.23it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.30it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.28it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.60it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:33,  7.26it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.91it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.76it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.71it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.42it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.18it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:44<01:22,  8.05it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.83it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.05it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.91it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.62it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.73it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.24it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.43it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:33,  6.93it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:48,  5.97it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.65it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.57it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.37it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.58it/s]
+
+ 36%|███▌      | 361/1000 [00:47<01:07,  9.43it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.20it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.30it/s]
+
+ 36%|███▋      | 365/1000 [00:48<02:08,  4.94it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.38it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.04it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.23it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.93it/s]
+
+ 38%|███▊      | 380/1000 [00:50<01:10,  8.78it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.92it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.22it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.19it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.15it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.66it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.39it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:08,  8.80it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.24it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.51it/s]
+
+ 41%|████      | 408/1000 [00:54<01:32,  6.39it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.36it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.07it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.83it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:11,  8.13it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:09,  8.37it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.46it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.67it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.91it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.35it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.94it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.85it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.56it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.03it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.79it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.53it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.02it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.27it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.09it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.24it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.18it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.52it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.82it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.27it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.84it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.64it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.76it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.02it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.32it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.87it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.58it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.40it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:34,  5.51it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.17it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.33it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.33it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.23it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.29it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50, 10.00it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.42it/s]
+
+ 50%|████▉     | 498/1000 [01:06<00:59,  8.45it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.94it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.56it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:12,  6.81it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.56it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.44it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.98it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.99it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:07,  7.12it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.09it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:21,  5.82it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.13it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:47,  9.73it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.71it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.67it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.77it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.71it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.52it/s]
+
+ 55%|█████▌    | 552/1000 [01:14<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.96it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:56,  7.79it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.04it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.87it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.07it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.99it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.57it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.46it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.44it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.36it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.85it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.26it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.57it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.61it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.93it/s]
+
+ 59%|█████▊    | 587/1000 [01:19<00:57,  7.17it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.55it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.77it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:50,  8.05it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<01:00,  6.75it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.34it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.86it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:02,  6.46it/s]
+
+ 60%|██████    | 601/1000 [01:21<01:02,  6.38it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.84it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.82it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.75it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.09it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.22it/s]
+
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.47it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.20it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.23it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.87it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.46it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.74it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.08it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.96it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.17it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.96it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.10it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.60it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.57it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.34it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.72it/s]
+
+ 66%|██████▌   | 656/1000 [01:28<00:38,  8.93it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.58it/s]
+
+ 66%|██████▌   | 662/1000 [01:29<00:52,  6.49it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.34it/s]
+
+ 67%|██████▋   | 667/1000 [01:30<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.21it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.90it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.65it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.68it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.98it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.94it/s]
+
+ 68%|██████▊   | 685/1000 [01:32<00:31,  9.89it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:33<00:45,  6.83it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.85it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.27it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.35it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.96it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.06it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.62it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:52,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:35<00:40,  7.24it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.43it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.04it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.65it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.52it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.73it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.87it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.37it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.06it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.26it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:32,  8.34it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.16it/s]
+
+ 73%|███████▎  | 729/1000 [01:38<00:23, 11.64it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.51it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:39<00:27,  9.55it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.80it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.62it/s]
+
+ 75%|███████▍  | 747/1000 [01:40<00:24, 10.47it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:28,  8.68it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.70it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.18it/s]
+
+ 75%|███████▌  | 753/1000 [01:41<00:34,  7.25it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.25it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.61it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:42<00:27,  8.71it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.67it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.62it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:44<00:33,  6.80it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.92it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.64it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.96it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:26,  8.38it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.17it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.55it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.38it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.01it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.24it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.75it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.99it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.21it/s]
+
+ 80%|███████▉  | 797/1000 [01:47<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.65it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:25,  7.84it/s]
+
+ 81%|████████  | 806/1000 [01:48<00:21,  9.18it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.99it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.24it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.25it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.06it/s]
+
+ 82%|████████▏ | 815/1000 [01:49<00:19,  9.33it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.16it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.22it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:50<00:22,  7.84it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.76it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:52<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.11it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.44it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.62it/s]
+
+ 84%|████████▍ | 842/1000 [01:53<00:19,  8.15it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.56it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.07it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:13, 10.22it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.53it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.65it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.75it/s]
+
+ 87%|████████▋ | 867/1000 [01:56<00:14,  9.33it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.30it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.31it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.78it/s]
+
+ 88%|████████▊ | 875/1000 [01:57<00:16,  7.64it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.89it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.88it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:58<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.97it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.78it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.77it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.62it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.85it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:00<00:12,  7.71it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.88it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.09it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.06it/s]
+
+ 91%|█████████ | 908/1000 [02:01<00:11,  8.32it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.60it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.79it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.40it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.38it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.61it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.18it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.09it/s]
+
+ 93%|█████████▎| 928/1000 [02:04<00:10,  6.66it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.62it/s]
+
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.19it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.33it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.85it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:06<00:09,  6.35it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.42it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.41it/s]
+
+ 96%|█████████▌| 957/1000 [02:08<00:06,  6.41it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.06it/s]
+
+ 96%|█████████▋| 964/1000 [02:09<00:04,  7.51it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.46it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.03it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.43it/s]
+
+ 97%|█████████▋| 972/1000 [02:10<00:03,  8.04it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.23it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.74it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.77it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.41it/s]
+
+ 98%|█████████▊| 981/1000 [02:11<00:02,  7.05it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.43it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.92it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.37it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:13<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.04it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.75it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.8064', 'eval_runtime': '135.2', 'eval_samples_per_second': '7.398', 'eval_steps_per_second': '7.398', 'epoch': '0.331', 'num_input_tokens_seen': 3999176}
+
+ 33%|███▎      | 600/1813 [1:06:30<2:05:00,  6.18s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.66it/s]
+
+                                                   
+ 33%|███▎      | 601/1813 [1:06:35<15:34:40, 46.27s/it]
+ 33%|███▎      | 602/1813 [1:06:40<11:23:34, 33.87s/it]
+ 33%|███▎      | 603/1813 [1:06:45<8:31:03, 25.34s/it] 
+ 33%|███▎      | 604/1813 [1:06:52<6:35:01, 19.60s/it]
+ 33%|███▎      | 605/1813 [1:07:00<5:24:22, 16.11s/it]
+ 33%|███▎      | 606/1813 [1:07:07<4:32:01, 13.52s/it]
+ 33%|███▎      | 607/1813 [1:07:13<3:48:24, 11.36s/it]
+ 34%|███▎      | 608/1813 [1:07:20<3:18:40,  9.89s/it]
+ 34%|███▎      | 609/1813 [1:07:26<2:56:20,  8.79s/it]
+ 34%|███▎      | 610/1813 [1:07:32<2:41:05,  8.03s/it]
+                                                      
+{'loss': '0.9751', 'grad_norm': '0.7893', 'learning_rate': '0.0001535', 'epoch': '0.3366', 'num_input_tokens_seen': 4066376, 'train_runtime': '4053', 'train_tokens_per_second': '1003'}
+
+ 34%|███▎      | 610/1813 [1:07:32<2:41:05,  8.03s/it]
+ 34%|███▎      | 611/1813 [1:07:38<2:24:11,  7.20s/it]
+ 34%|███▍      | 612/1813 [1:07:42<2:05:46,  6.28s/it]
+ 34%|███▍      | 613/1813 [1:07:47<2:02:24,  6.12s/it]
+ 34%|███▍      | 614/1813 [1:07:53<2:01:17,  6.07s/it]
+ 34%|███▍      | 615/1813 [1:07:59<1:56:19,  5.83s/it]
+ 34%|███▍      | 616/1813 [1:08:04<1:51:05,  5.57s/it]
+ 34%|███▍      | 617/1813 [1:08:09<1:49:16,  5.48s/it]
+ 34%|███▍      | 618/1813 [1:08:16<1:58:34,  5.95s/it]
+ 34%|███▍      | 619/1813 [1:08:21<1:51:55,  5.62s/it]
+ 34%|███▍      | 620/1813 [1:08:27<1:57:05,  5.89s/it]
+                                                      
+{'loss': '0.873', 'grad_norm': '0.9814', 'learning_rate': '0.0001522', 'epoch': '0.3421', 'num_input_tokens_seen': 4123976, 'train_runtime': '4108', 'train_tokens_per_second': '1004'}
+
+ 34%|███▍      | 620/1813 [1:08:27<1:57:05,  5.89s/it]
+ 34%|███▍      | 621/1813 [1:08:34<2:03:29,  6.22s/it]
+ 34%|███▍      | 622/1813 [1:08:42<2:11:36,  6.63s/it]
+ 34%|███▍      | 623/1813 [1:08:47<2:01:13,  6.11s/it]
+ 34%|███▍      | 624/1813 [1:08:53<2:02:12,  6.17s/it]
+ 34%|███▍      | 625/1813 [1:08:58<1:54:35,  5.79s/it]
+ 35%|███▍      | 626/1813 [1:09:02<1:44:57,  5.31s/it]
+ 35%|███▍      | 627/1813 [1:09:08<1:49:24,  5.53s/it]
+ 35%|███▍      | 628/1813 [1:09:14<1:52:28,  5.70s/it]
+ 35%|███▍      | 629/1813 [1:09:23<2:07:53,  6.48s/it]
+ 35%|███▍      | 630/1813 [1:09:30<2:10:17,  6.61s/it]
+                                                      
+{'loss': '0.9461', 'grad_norm': '0.7751', 'learning_rate': '0.0001508', 'epoch': '0.3476', 'num_input_tokens_seen': 4191992, 'train_runtime': '4170', 'train_tokens_per_second': '1005'}
+
+ 35%|███▍      | 630/1813 [1:09:30<2:10:17,  6.61s/it]
+ 35%|███▍      | 631/1813 [1:09:36<2:10:13,  6.61s/it]
+ 35%|███▍      | 632/1813 [1:09:42<2:06:41,  6.44s/it]
+ 35%|███▍      | 633/1813 [1:09:48<2:03:21,  6.27s/it]
+ 35%|███▍      | 634/1813 [1:09:55<2:09:58,  6.61s/it]
+ 35%|███▌      | 635/1813 [1:10:02<2:09:55,  6.62s/it]
+ 35%|███▌      | 636/1813 [1:10:07<2:00:48,  6.16s/it]
+ 35%|███▌      | 637/1813 [1:10:14<2:05:34,  6.41s/it]
+ 35%|███▌      | 638/1813 [1:10:20<2:01:28,  6.20s/it]
+ 35%|███▌      | 639/1813 [1:10:25<1:57:31,  6.01s/it]
+ 35%|███▌      | 640/1813 [1:10:32<1:58:04,  6.04s/it]
+                                                      
+{'loss': '0.952', 'grad_norm': '1.102', 'learning_rate': '0.0001494', 'epoch': '0.3531', 'num_input_tokens_seen': 4257760, 'train_runtime': '4232', 'train_tokens_per_second': '1006'}
+
+ 35%|███▌      | 640/1813 [1:10:32<1:58:04,  6.04s/it]
+ 35%|███▌      | 641/1813 [1:10:37<1:52:22,  5.75s/it]
+ 35%|███▌      | 642/1813 [1:10:44<2:00:13,  6.16s/it]
+ 35%|███▌      | 643/1813 [1:10:50<2:02:46,  6.30s/it]
+ 36%|███▌      | 644/1813 [1:10:56<1:58:42,  6.09s/it]
+ 36%|███▌      | 645/1813 [1:11:02<1:58:41,  6.10s/it]
+ 36%|███▌      | 646/1813 [1:11:08<1:58:22,  6.09s/it]
+ 36%|███▌      | 647/1813 [1:11:15<2:00:49,  6.22s/it]
+ 36%|███▌      | 648/1813 [1:11:22<2:06:35,  6.52s/it]
+ 36%|███▌      | 649/1813 [1:11:28<2:01:18,  6.25s/it]
+ 36%|███▌      | 650/1813 [1:11:33<1:55:43,  5.97s/it]
+                                                      
+{'loss': '0.9424', 'grad_norm': '0.9758', 'learning_rate': '0.000148', 'epoch': '0.3586', 'num_input_tokens_seen': 4322136, 'train_runtime': '4293', 'train_tokens_per_second': '1007'}
+
+ 36%|███▌      | 650/1813 [1:11:33<1:55:43,  5.97s/it]
+ 36%|███▌      | 651/1813 [1:11:40<1:59:48,  6.19s/it]
+ 36%|███▌      | 652/1813 [1:11:46<2:03:17,  6.37s/it]
+ 36%|███▌      | 653/1813 [1:11:52<1:58:47,  6.14s/it]
+ 36%|███▌      | 654/1813 [1:11:57<1:54:58,  5.95s/it]
+ 36%|███▌      | 655/1813 [1:12:04<1:56:47,  6.05s/it]
+ 36%|███▌      | 656/1813 [1:12:09<1:52:24,  5.83s/it]
+ 36%|███▌      | 657/1813 [1:12:14<1:49:03,  5.66s/it]
+ 36%|███▋      | 658/1813 [1:12:21<1:54:27,  5.95s/it]
+ 36%|███▋      | 659/1813 [1:12:27<1:57:01,  6.08s/it]
+ 36%|███▋      | 660/1813 [1:12:33<1:54:12,  5.94s/it]
+                                                      
+{'loss': '0.8992', 'grad_norm': '0.9052', 'learning_rate': '0.0001466', 'epoch': '0.3641', 'num_input_tokens_seen': 4385688, 'train_runtime': '4353', 'train_tokens_per_second': '1007'}
+
+ 36%|███▋      | 660/1813 [1:12:33<1:54:12,  5.94s/it]
+ 36%|███▋      | 661/1813 [1:12:39<1:55:27,  6.01s/it]
+ 37%|███▋      | 662/1813 [1:12:45<1:55:49,  6.04s/it]
+ 37%|███▋      | 663/1813 [1:12:52<1:59:44,  6.25s/it]
+ 37%|███▋      | 664/1813 [1:12:58<2:00:02,  6.27s/it]
+ 37%|███▋      | 665/1813 [1:13:03<1:51:58,  5.85s/it]
+ 37%|███▋      | 666/1813 [1:13:08<1:48:34,  5.68s/it]
+ 37%|███▋      | 667/1813 [1:13:15<1:54:07,  5.98s/it]
+ 37%|███▋      | 668/1813 [1:13:21<1:55:50,  6.07s/it]
+ 37%|███▋      | 669/1813 [1:13:27<1:54:27,  6.00s/it]
+ 37%|███▋      | 670/1813 [1:13:33<1:51:10,  5.84s/it]
+                                                      
+{'loss': '0.9756', 'grad_norm': '1.075', 'learning_rate': '0.0001451', 'epoch': '0.3697', 'num_input_tokens_seen': 4449456, 'train_runtime': '4413', 'train_tokens_per_second': '1008'}
+
+ 37%|███▋      | 670/1813 [1:13:33<1:51:10,  5.84s/it]
+ 37%|███▋      | 671/1813 [1:13:39<1:52:34,  5.92s/it]
+ 37%|███▋      | 672/1813 [1:13:45<1:56:40,  6.14s/it]
+ 37%|███▋      | 673/1813 [1:13:52<1:56:51,  6.15s/it]
+ 37%|███▋      | 674/1813 [1:13:58<1:59:12,  6.28s/it]
+ 37%|███▋      | 675/1813 [1:14:04<1:56:44,  6.16s/it]
+ 37%|███▋      | 676/1813 [1:14:10<1:54:09,  6.02s/it]
+ 37%|███▋      | 677/1813 [1:14:18<2:06:17,  6.67s/it]
+ 37%|███▋      | 678/1813 [1:14:23<1:58:22,  6.26s/it]
+ 37%|███▋      | 679/1813 [1:14:29<1:52:56,  5.98s/it]
+ 38%|███▊      | 680/1813 [1:14:34<1:51:42,  5.92s/it]
+                                                      
+{'loss': '0.9278', 'grad_norm': '0.937', 'learning_rate': '0.0001436', 'epoch': '0.3752', 'num_input_tokens_seen': 4514984, 'train_runtime': '4475', 'train_tokens_per_second': '1009'}
+
+ 38%|███▊      | 680/1813 [1:14:34<1:51:42,  5.92s/it]
+ 38%|███▊      | 681/1813 [1:14:41<1:55:22,  6.11s/it]
+ 38%|███▊      | 682/1813 [1:14:46<1:48:35,  5.76s/it]
+ 38%|███▊      | 683/1813 [1:14:52<1:49:08,  5.80s/it]
+ 38%|███▊      | 684/1813 [1:14:58<1:52:23,  5.97s/it]
+ 38%|███▊      | 685/1813 [1:15:06<2:02:18,  6.51s/it]
+ 38%|███▊      | 686/1813 [1:15:12<2:01:15,  6.46s/it]
+ 38%|███▊      | 687/1813 [1:15:20<2:08:27,  6.85s/it]
+ 38%|███▊      | 688/1813 [1:15:25<1:57:13,  6.25s/it]
+ 38%|███▊      | 689/1813 [1:15:34<2:11:27,  7.02s/it]
+ 38%|███▊      | 690/1813 [1:15:41<2:12:02,  7.05s/it]
+                                                      
+{'loss': '0.9785', 'grad_norm': '1.033', 'learning_rate': '0.0001422', 'epoch': '0.3807', 'num_input_tokens_seen': 4587840, 'train_runtime': '4541', 'train_tokens_per_second': '1010'}
+
+ 38%|███▊      | 690/1813 [1:15:41<2:12:02,  7.05s/it]
+ 38%|███▊      | 691/1813 [1:15:46<2:04:18,  6.65s/it]
+ 38%|███▊      | 692/1813 [1:15:53<2:01:42,  6.51s/it]
+ 38%|███▊      | 693/1813 [1:15:58<1:56:28,  6.24s/it]
+ 38%|███▊      | 694/1813 [1:16:05<1:56:28,  6.25s/it]
+ 38%|███▊      | 695/1813 [1:16:11<1:56:09,  6.23s/it]
+ 38%|███▊      | 696/1813 [1:16:19<2:06:11,  6.78s/it]
+ 38%|███▊      | 697/1813 [1:16:24<1:56:03,  6.24s/it]
+ 38%|███▊      | 698/1813 [1:16:29<1:52:25,  6.05s/it]
+ 39%|███▊      | 699/1813 [1:16:36<1:57:38,  6.34s/it]
+ 39%|███▊      | 700/1813 [1:16:43<1:59:15,  6.43s/it]
+                                                      
+{'loss': '1.005', 'grad_norm': '1.241', 'learning_rate': '0.0001407', 'epoch': '0.3862', 'num_input_tokens_seen': 4653312, 'train_runtime': '4604', 'train_tokens_per_second': '1011'}
+
+ 39%|███▊      | 700/1813 [1:16:43<1:59:15,  6.43s/it]
+ 39%|███▊      | 701/1813 [1:16:50<2:03:49,  6.68s/it]
+ 39%|███▊      | 702/1813 [1:16:57<2:01:04,  6.54s/it]
+ 39%|███▉      | 703/1813 [1:17:04<2:08:18,  6.94s/it]
+ 39%|███▉      | 704/1813 [1:17:10<2:01:02,  6.55s/it]
+ 39%|███▉      | 705/1813 [1:17:17<2:03:44,  6.70s/it]
+ 39%|███▉      | 706/1813 [1:17:23<2:01:27,  6.58s/it]
+ 39%|███▉      | 707/1813 [1:17:33<2:18:18,  7.50s/it]
+ 39%|███▉      | 708/1813 [1:17:40<2:17:05,  7.44s/it]
+ 39%|███▉      | 709/1813 [1:17:47<2:12:39,  7.21s/it]
+ 39%|███▉      | 710/1813 [1:17:53<2:03:43,  6.73s/it]
+                                                      
+{'loss': '0.9636', 'grad_norm': '0.7813', 'learning_rate': '0.0001391', 'epoch': '0.3917', 'num_input_tokens_seen': 4732080, 'train_runtime': '4673', 'train_tokens_per_second': '1013'}
+
+ 39%|███▉      | 710/1813 [1:17:53<2:03:43,  6.73s/it]
+ 39%|███▉      | 711/1813 [1:17:59<2:03:30,  6.72s/it]
+ 39%|███▉      | 712/1813 [1:18:06<2:02:22,  6.67s/it]
+ 39%|███▉      | 713/1813 [1:18:11<1:56:03,  6.33s/it]
+ 39%|███▉      | 714/1813 [1:18:18<1:54:38,  6.26s/it]
+ 39%|███▉      | 715/1813 [1:18:24<1:57:03,  6.40s/it]
+ 39%|███▉      | 716/1813 [1:18:31<2:00:09,  6.57s/it]
+ 40%|███▉      | 717/1813 [1:18:37<1:58:08,  6.47s/it]
+ 40%|███▉      | 718/1813 [1:18:44<1:57:55,  6.46s/it]
+ 40%|███▉      | 719/1813 [1:18:49<1:50:33,  6.06s/it]
+ 40%|███▉      | 720/1813 [1:18:55<1:51:24,  6.12s/it]
+                                                      
+{'loss': '0.9832', 'grad_norm': '0.9861', 'learning_rate': '0.0001376', 'epoch': '0.3972', 'num_input_tokens_seen': 4799552, 'train_runtime': '4736', 'train_tokens_per_second': '1013'}
+
+ 40%|███▉      | 720/1813 [1:18:55<1:51:24,  6.12s/it]
+ 40%|███▉      | 721/1813 [1:19:02<1:53:40,  6.25s/it]
+ 40%|███▉      | 722/1813 [1:19:07<1:48:24,  5.96s/it]
+ 40%|███▉      | 723/1813 [1:19:12<1:44:50,  5.77s/it]
+ 40%|███▉      | 724/1813 [1:19:20<1:56:04,  6.39s/it]
+ 40%|███▉      | 725/1813 [1:19:29<2:06:46,  6.99s/it]
+ 40%|████      | 726/1813 [1:19:35<2:04:07,  6.85s/it]
+ 40%|████      | 727/1813 [1:19:41<1:58:49,  6.57s/it]
+ 40%|████      | 728/1813 [1:19:46<1:50:25,  6.11s/it]
+ 40%|████      | 729/1813 [1:19:52<1:48:25,  6.00s/it]
+ 40%|████      | 730/1813 [1:19:57<1:44:48,  5.81s/it]
+                                                      
+{'loss': '0.9566', 'grad_norm': '1.034', 'learning_rate': '0.000136', 'epoch': '0.4028', 'num_input_tokens_seen': 4866864, 'train_runtime': '4798', 'train_tokens_per_second': '1014'}
+
+ 40%|████      | 730/1813 [1:19:57<1:44:48,  5.81s/it]
+ 40%|████      | 731/1813 [1:20:04<1:50:47,  6.14s/it]
+ 40%|████      | 732/1813 [1:20:10<1:51:10,  6.17s/it]
+ 40%|████      | 733/1813 [1:20:18<1:57:24,  6.52s/it]
+ 40%|████      | 734/1813 [1:20:25<1:59:23,  6.64s/it]
+ 41%|████      | 735/1813 [1:20:31<1:59:14,  6.64s/it]
+ 41%|████      | 736/1813 [1:20:37<1:54:47,  6.40s/it]
+ 41%|████      | 737/1813 [1:20:43<1:51:24,  6.21s/it]
+ 41%|████      | 738/1813 [1:20:48<1:45:33,  5.89s/it]
+ 41%|████      | 739/1813 [1:20:54<1:45:03,  5.87s/it]
+ 41%|████      | 740/1813 [1:21:00<1:48:33,  6.07s/it]
+                                                      
+{'loss': '0.8841', 'grad_norm': '0.8534', 'learning_rate': '0.0001345', 'epoch': '0.4083', 'num_input_tokens_seen': 4936352, 'train_runtime': '4861', 'train_tokens_per_second': '1016'}
+
+ 41%|████      | 740/1813 [1:21:00<1:48:33,  6.07s/it]
+ 41%|████      | 741/1813 [1:21:07<1:50:13,  6.17s/it]
+ 41%|████      | 742/1813 [1:21:14<1:56:49,  6.54s/it]
+ 41%|████      | 743/1813 [1:21:22<2:01:22,  6.81s/it]
+ 41%|████      | 744/1813 [1:21:28<2:00:01,  6.74s/it]
+ 41%|████      | 745/1813 [1:21:35<2:01:10,  6.81s/it]
+ 41%|████      | 746/1813 [1:21:41<1:56:42,  6.56s/it]
+ 41%|████      | 747/1813 [1:21:49<2:03:44,  6.97s/it]
+ 41%|████▏     | 748/1813 [1:21:55<1:55:34,  6.51s/it]
+ 41%|████▏     | 749/1813 [1:22:01<1:54:24,  6.45s/it]
+ 41%|████▏     | 750/1813 [1:22:07<1:50:33,  6.24s/it]
+                                                      
+{'loss': '0.9683', 'grad_norm': '0.859', 'learning_rate': '0.0001329', 'epoch': '0.4138', 'num_input_tokens_seen': 5011064, 'train_runtime': '4927', 'train_tokens_per_second': '1017'}
+
+ 41%|████▏     | 750/1813 [1:22:07<1:50:33,  6.24s/it]
+ 41%|████▏     | 751/1813 [1:22:13<1:49:46,  6.20s/it]
+ 41%|████▏     | 752/1813 [1:22:19<1:49:09,  6.17s/it]
+ 42%|████▏     | 753/1813 [1:22:25<1:51:43,  6.32s/it]
+ 42%|████▏     | 754/1813 [1:22:34<2:02:52,  6.96s/it]
+ 42%|████▏     | 755/1813 [1:22:39<1:52:04,  6.36s/it]
+ 42%|████▏     | 756/1813 [1:22:44<1:47:17,  6.09s/it]
+ 42%|████▏     | 757/1813 [1:22:50<1:45:16,  5.98s/it]
+ 42%|████▏     | 758/1813 [1:22:57<1:48:13,  6.15s/it]
+ 42%|████▏     | 759/1813 [1:23:04<1:51:57,  6.37s/it]
+ 42%|████▏     | 760/1813 [1:23:09<1:49:26,  6.24s/it]
+                                                      
+{'loss': '1.001', 'grad_norm': '1.192', 'learning_rate': '0.0001313', 'epoch': '0.4193', 'num_input_tokens_seen': 5078960, 'train_runtime': '4990', 'train_tokens_per_second': '1018'}
+
+ 42%|████▏     | 760/1813 [1:23:09<1:49:26,  6.24s/it]
+ 42%|████▏     | 761/1813 [1:23:16<1:51:17,  6.35s/it]
+ 42%|████▏     | 762/1813 [1:23:23<1:52:50,  6.44s/it]
+ 42%|████▏     | 763/1813 [1:23:30<1:56:49,  6.68s/it]
+ 42%|████▏     | 764/1813 [1:23:37<1:58:37,  6.78s/it]
+ 42%|████▏     | 765/1813 [1:23:44<2:02:12,  7.00s/it]
+ 42%|████▏     | 766/1813 [1:23:51<1:59:31,  6.85s/it]
+ 42%|████▏     | 767/1813 [1:23:57<1:53:08,  6.49s/it]
+ 42%|████▏     | 768/1813 [1:24:01<1:40:39,  5.78s/it]
+ 42%|████▏     | 769/1813 [1:24:07<1:45:06,  6.04s/it]
+ 42%|████▏     | 770/1813 [1:24:14<1:45:54,  6.09s/it]
+                                                      
+{'loss': '0.9124', 'grad_norm': '2.448', 'learning_rate': '0.0001297', 'epoch': '0.4248', 'num_input_tokens_seen': 5148504, 'train_runtime': '5054', 'train_tokens_per_second': '1019'}
+
+ 42%|████▏     | 770/1813 [1:24:14<1:45:54,  6.09s/it]
+ 43%|████▎     | 771/1813 [1:24:20<1:47:44,  6.20s/it]
+ 43%|████▎     | 772/1813 [1:24:27<1:52:49,  6.50s/it]
+ 43%|████▎     | 773/1813 [1:24:33<1:50:10,  6.36s/it]
+ 43%|████▎     | 774/1813 [1:24:38<1:43:32,  5.98s/it]
+ 43%|████▎     | 775/1813 [1:24:44<1:43:51,  6.00s/it]
+ 43%|████▎     | 776/1813 [1:24:51<1:48:11,  6.26s/it]
+ 43%|████▎     | 777/1813 [1:24:59<1:55:55,  6.71s/it]
+ 43%|████▎     | 778/1813 [1:25:04<1:46:56,  6.20s/it]
+ 43%|████▎     | 779/1813 [1:25:09<1:40:57,  5.86s/it]
+ 43%|████▎     | 780/1813 [1:25:16<1:44:42,  6.08s/it]
+                                                      
+{'loss': '0.8835', 'grad_norm': '1.032', 'learning_rate': '0.000128', 'epoch': '0.4303', 'num_input_tokens_seen': 5215784, 'train_runtime': '5116', 'train_tokens_per_second': '1019'}
+
+ 43%|████▎     | 780/1813 [1:25:16<1:44:42,  6.08s/it]
+ 43%|████▎     | 781/1813 [1:25:23<1:48:28,  6.31s/it]
+ 43%|████▎     | 782/1813 [1:25:31<1:57:51,  6.86s/it]
+ 43%|████▎     | 783/1813 [1:25:38<1:59:07,  6.94s/it]
+ 43%|████▎     | 784/1813 [1:25:44<1:54:18,  6.66s/it]
+ 43%|████▎     | 785/1813 [1:25:51<1:56:26,  6.80s/it]
+ 43%|████▎     | 786/1813 [1:25:58<1:58:22,  6.92s/it]
+ 43%|████▎     | 787/1813 [1:26:03<1:49:54,  6.43s/it]
+ 43%|████▎     | 788/1813 [1:26:09<1:45:38,  6.18s/it]
+ 44%|████▎     | 789/1813 [1:26:14<1:41:12,  5.93s/it]
+ 44%|████▎     | 790/1813 [1:26:20<1:41:03,  5.93s/it]
+                                                      
+{'loss': '0.9879', 'grad_norm': '0.9594', 'learning_rate': '0.0001264', 'epoch': '0.4359', 'num_input_tokens_seen': 5285160, 'train_runtime': '5181', 'train_tokens_per_second': '1020'}
+
+ 44%|████▎     | 790/1813 [1:26:20<1:41:03,  5.93s/it]
+ 44%|████▎     | 791/1813 [1:26:26<1:41:54,  5.98s/it]
+ 44%|████▎     | 792/1813 [1:26:34<1:47:45,  6.33s/it]
+ 44%|████▎     | 793/1813 [1:26:41<1:52:15,  6.60s/it]
+ 44%|████▍     | 794/1813 [1:26:47<1:52:32,  6.63s/it]
+ 44%|████▍     | 795/1813 [1:26:54<1:49:22,  6.45s/it]
+ 44%|████▍     | 796/1813 [1:27:00<1:51:45,  6.59s/it]
+ 44%|████▍     | 797/1813 [1:27:08<1:56:32,  6.88s/it]
+ 44%|████▍     | 798/1813 [1:27:14<1:52:19,  6.64s/it]
+ 44%|████▍     | 799/1813 [1:27:20<1:49:36,  6.49s/it]
+ 44%|████▍     | 800/1813 [1:27:26<1:46:48,  6.33s/it]
+                                                      
+{'loss': '0.941', 'grad_norm': '0.9306', 'learning_rate': '0.0001247', 'epoch': '0.4414', 'num_input_tokens_seen': 5357720, 'train_runtime': '5247', 'train_tokens_per_second': '1021'}
+
+ 44%|████▍     | 800/1813 [1:27:26<1:46:48,  6.33s/it]
+ 44%|████▍     | 801/1813 [1:27:32<1:42:04,  6.05s/it]
+ 44%|████▍     | 802/1813 [1:27:38<1:42:16,  6.07s/it]
+ 44%|████▍     | 803/1813 [1:27:46<1:51:16,  6.61s/it]
+ 44%|████▍     | 804/1813 [1:27:52<1:49:13,  6.49s/it]
+ 44%|████▍     | 805/1813 [1:27:58<1:46:44,  6.35s/it]
+ 44%|████▍     | 806/1813 [1:28:04<1:44:30,  6.23s/it]
+ 45%|████▍     | 807/1813 [1:28:09<1:41:00,  6.02s/it]
+ 45%|████▍     | 808/1813 [1:28:17<1:49:45,  6.55s/it]
+ 45%|████▍     | 809/1813 [1:28:24<1:50:45,  6.62s/it]
+ 45%|████▍     | 810/1813 [1:28:32<1:56:49,  6.99s/it]
+                                                      
+{'loss': '0.9508', 'grad_norm': '0.9403', 'learning_rate': '0.0001231', 'epoch': '0.4469', 'num_input_tokens_seen': 5430288, 'train_runtime': '5312', 'train_tokens_per_second': '1022'}
+
+ 45%|████▍     | 810/1813 [1:28:32<1:56:49,  6.99s/it]
+ 45%|████▍     | 811/1813 [1:28:38<1:55:16,  6.90s/it]
+ 45%|████▍     | 812/1813 [1:28:44<1:48:58,  6.53s/it]
+ 45%|████▍     | 813/1813 [1:28:50<1:44:17,  6.26s/it]
+ 45%|████▍     | 814/1813 [1:28:56<1:43:49,  6.24s/it]
+ 45%|████▍     | 815/1813 [1:29:01<1:37:02,  5.83s/it]
+ 45%|████▌     | 816/1813 [1:29:08<1:41:46,  6.12s/it]
+ 45%|████▌     | 817/1813 [1:29:13<1:39:39,  6.00s/it]
+ 45%|████▌     | 818/1813 [1:29:20<1:45:27,  6.36s/it]
+ 45%|████▌     | 819/1813 [1:29:27<1:45:28,  6.37s/it]
+ 45%|████▌     | 820/1813 [1:29:34<1:49:59,  6.65s/it]
+                                                      
+{'loss': '0.8849', 'grad_norm': '0.8486', 'learning_rate': '0.0001214', 'epoch': '0.4524', 'num_input_tokens_seen': 5497552, 'train_runtime': '5375', 'train_tokens_per_second': '1023'}
+
+ 45%|████▌     | 820/1813 [1:29:34<1:49:59,  6.65s/it]
+ 45%|████▌     | 821/1813 [1:29:41<1:49:15,  6.61s/it]
+ 45%|████▌     | 822/1813 [1:29:46<1:42:02,  6.18s/it]
+ 45%|████▌     | 823/1813 [1:29:51<1:35:14,  5.77s/it]
+ 45%|████▌     | 824/1813 [1:29:57<1:37:31,  5.92s/it]
+ 46%|████▌     | 825/1813 [1:30:03<1:39:30,  6.04s/it]
+ 46%|████▌     | 826/1813 [1:30:10<1:43:21,  6.28s/it]
+ 46%|████▌     | 827/1813 [1:30:16<1:41:11,  6.16s/it]
+ 46%|████▌     | 828/1813 [1:30:22<1:40:01,  6.09s/it]
+ 46%|████▌     | 829/1813 [1:30:27<1:37:20,  5.94s/it]
+ 46%|████▌     | 830/1813 [1:30:35<1:45:07,  6.42s/it]
+                                                      
+{'loss': '0.7955', 'grad_norm': '0.8755', 'learning_rate': '0.0001197', 'epoch': '0.4579', 'num_input_tokens_seen': 5563144, 'train_runtime': '5436', 'train_tokens_per_second': '1023'}
+
+ 46%|████▌     | 830/1813 [1:30:35<1:45:07,  6.42s/it]
+ 46%|████▌     | 831/1813 [1:30:41<1:43:36,  6.33s/it]
+ 46%|████▌     | 832/1813 [1:30:48<1:48:24,  6.63s/it]
+ 46%|████▌     | 833/1813 [1:30:55<1:46:05,  6.50s/it]
+ 46%|████▌     | 834/1813 [1:31:01<1:46:36,  6.53s/it]
+ 46%|████▌     | 835/1813 [1:31:07<1:44:28,  6.41s/it]
+ 46%|████▌     | 836/1813 [1:31:14<1:46:56,  6.57s/it]
+ 46%|████▌     | 837/1813 [1:31:22<1:52:54,  6.94s/it]
+ 46%|████▌     | 838/1813 [1:31:28<1:47:51,  6.64s/it]
+ 46%|████▋     | 839/1813 [1:31:34<1:44:43,  6.45s/it]
+ 46%|████▋     | 840/1813 [1:31:40<1:41:24,  6.25s/it]
+                                                      
+{'loss': '0.9622', 'grad_norm': '1.083', 'learning_rate': '0.000118', 'epoch': '0.4634', 'num_input_tokens_seen': 5634104, 'train_runtime': '5500', 'train_tokens_per_second': '1024'}
+
+ 46%|████▋     | 840/1813 [1:31:40<1:41:24,  6.25s/it]
+ 46%|████▋     | 841/1813 [1:31:46<1:38:53,  6.10s/it]
+ 46%|████▋     | 842/1813 [1:31:52<1:41:40,  6.28s/it]
+ 46%|████▋     | 843/1813 [1:31:58<1:40:47,  6.23s/it]
+ 47%|████▋     | 844/1813 [1:32:04<1:38:28,  6.10s/it]
+ 47%|████▋     | 845/1813 [1:32:11<1:39:47,  6.19s/it]
+ 47%|████▋     | 846/1813 [1:32:17<1:41:40,  6.31s/it]
+ 47%|████▋     | 847/1813 [1:32:24<1:44:42,  6.50s/it]
+ 47%|████▋     | 848/1813 [1:32:31<1:47:47,  6.70s/it]
+ 47%|████▋     | 849/1813 [1:32:37<1:40:25,  6.25s/it]
+ 47%|████▋     | 850/1813 [1:32:45<1:49:34,  6.83s/it]
+                                                      
+{'loss': '0.9172', 'grad_norm': '0.795', 'learning_rate': '0.0001163', 'epoch': '0.469', 'num_input_tokens_seen': 5704744, 'train_runtime': '5565', 'train_tokens_per_second': '1025'}
+
+ 47%|████▋     | 850/1813 [1:32:45<1:49:34,  6.83s/it]
+ 47%|████▋     | 851/1813 [1:32:51<1:48:59,  6.80s/it]
+ 47%|████▋     | 852/1813 [1:32:56<1:39:24,  6.21s/it]
+ 47%|████▋     | 853/1813 [1:33:03<1:39:31,  6.22s/it]
+ 47%|████▋     | 854/1813 [1:33:09<1:42:45,  6.43s/it]
+ 47%|████▋     | 855/1813 [1:33:16<1:41:03,  6.33s/it]
+ 47%|████▋     | 856/1813 [1:33:22<1:40:24,  6.30s/it]
+ 47%|████▋     | 857/1813 [1:33:27<1:36:27,  6.05s/it]
+ 47%|████▋     | 858/1813 [1:33:31<1:27:21,  5.49s/it]
+ 47%|████▋     | 859/1813 [1:33:38<1:31:30,  5.76s/it]
+ 47%|████▋     | 860/1813 [1:33:44<1:35:07,  5.99s/it]
+                                                      
+{'loss': '0.9462', 'grad_norm': '1.163', 'learning_rate': '0.0001146', 'epoch': '0.4745', 'num_input_tokens_seen': 5766424, 'train_runtime': '5625', 'train_tokens_per_second': '1025'}
+
+ 47%|████▋     | 860/1813 [1:33:44<1:35:07,  5.99s/it]
+ 47%|████▋     | 861/1813 [1:33:49<1:29:59,  5.67s/it]
+ 48%|████▊     | 862/1813 [1:33:56<1:34:55,  5.99s/it]
+ 48%|████▊     | 863/1813 [1:34:02<1:35:12,  6.01s/it]
+ 48%|████▊     | 864/1813 [1:34:06<1:26:22,  5.46s/it]
+ 48%|████▊     | 865/1813 [1:34:13<1:32:38,  5.86s/it]
+ 48%|████▊     | 866/1813 [1:34:19<1:32:19,  5.85s/it]
+ 48%|████▊     | 867/1813 [1:34:24<1:30:07,  5.72s/it]
+ 48%|████▊     | 868/1813 [1:34:30<1:28:14,  5.60s/it]
+ 48%|████▊     | 869/1813 [1:34:37<1:35:54,  6.10s/it]
+ 48%|████▊     | 870/1813 [1:34:45<1:43:14,  6.57s/it]
+                                                      
+{'loss': '0.9061', 'grad_norm': '0.8601', 'learning_rate': '0.0001129', 'epoch': '0.48', 'num_input_tokens_seen': 5831352, 'train_runtime': '5685', 'train_tokens_per_second': '1026'}
+
+ 48%|████▊     | 870/1813 [1:34:45<1:43:14,  6.57s/it]
+ 48%|████▊     | 871/1813 [1:34:50<1:37:22,  6.20s/it]
+ 48%|████▊     | 872/1813 [1:34:57<1:41:56,  6.50s/it]
+ 48%|████▊     | 873/1813 [1:35:04<1:42:26,  6.54s/it]
+ 48%|████▊     | 874/1813 [1:35:11<1:46:22,  6.80s/it]
+ 48%|████▊     | 875/1813 [1:35:19<1:50:26,  7.06s/it]
+ 48%|████▊     | 876/1813 [1:35:26<1:48:49,  6.97s/it]
+ 48%|████▊     | 877/1813 [1:35:34<1:53:23,  7.27s/it]
+ 48%|████▊     | 878/1813 [1:35:40<1:49:32,  7.03s/it]
+ 48%|████▊     | 879/1813 [1:35:47<1:51:21,  7.15s/it]
+ 49%|████▊     | 880/1813 [1:35:52<1:38:25,  6.33s/it]
+                                                      
+{'loss': '0.936', 'grad_norm': '1.329', 'learning_rate': '0.0001112', 'epoch': '0.4855', 'num_input_tokens_seen': 5907208, 'train_runtime': '5752', 'train_tokens_per_second': '1027'}
+
+ 49%|████▊     | 880/1813 [1:35:52<1:38:25,  6.33s/it]
+ 49%|████▊     | 881/1813 [1:35:57<1:33:54,  6.05s/it]
+ 49%|████▊     | 882/1813 [1:36:03<1:34:08,  6.07s/it]
+ 49%|████▊     | 883/1813 [1:36:10<1:37:06,  6.27s/it]
+ 49%|████▉     | 884/1813 [1:36:16<1:35:53,  6.19s/it]
+ 49%|████▉     | 885/1813 [1:36:22<1:35:02,  6.14s/it]
+ 49%|████▉     | 886/1813 [1:36:28<1:35:16,  6.17s/it]
+ 49%|████▉     | 887/1813 [1:36:36<1:43:08,  6.68s/it]
+ 49%|████▉     | 888/1813 [1:36:43<1:42:52,  6.67s/it]
+ 49%|████▉     | 889/1813 [1:36:50<1:44:58,  6.82s/it]
+ 49%|████▉     | 890/1813 [1:36:56<1:41:30,  6.60s/it]
+                                                      
+{'loss': '0.9029', 'grad_norm': '0.8816', 'learning_rate': '0.0001094', 'epoch': '0.491', 'num_input_tokens_seen': 5978048, 'train_runtime': '5817', 'train_tokens_per_second': '1028'}
+
+ 49%|████▉     | 890/1813 [1:36:56<1:41:30,  6.60s/it]
+ 49%|████▉     | 891/1813 [1:37:02<1:38:17,  6.40s/it]
+ 49%|████▉     | 892/1813 [1:37:09<1:43:02,  6.71s/it]
+ 49%|████▉     | 893/1813 [1:37:16<1:40:26,  6.55s/it]
+ 49%|████▉     | 894/1813 [1:37:21<1:36:39,  6.31s/it]
+ 49%|████▉     | 895/1813 [1:37:29<1:40:59,  6.60s/it]
+ 49%|████▉     | 896/1813 [1:37:34<1:34:53,  6.21s/it]
+ 49%|████▉     | 897/1813 [1:37:40<1:32:36,  6.07s/it]
+ 50%|████▉     | 898/1813 [1:37:47<1:37:46,  6.41s/it]
+ 50%|████▉     | 899/1813 [1:37:54<1:42:07,  6.70s/it]
+ 50%|████▉     | 900/1813 [1:38:02<1:46:33,  7.00s/it]
+                                                      
+{'loss': '0.9841', 'grad_norm': '0.9941', 'learning_rate': '0.0001077', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856, 'train_runtime': '5883', 'train_tokens_per_second': '1029'}
+
+ 50%|████▉     | 900/1813 [1:38:02<1:46:33,  7.00s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.82it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.21it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.32it/s]
+
+  1%|          | 9/1000 [00:01<02:32,  6.48it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.09it/s]
+
+  1%|▏         | 13/1000 [00:01<02:21,  7.00it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.98it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.74it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.06it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.61it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.37it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  6.00it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+
+  3%|▎         | 30/1000 [00:04<02:41,  5.99it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.68it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.12it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:05<02:04,  7.74it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.09it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.32it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.37it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.76it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.90it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.92it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.76it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.32it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.34it/s]
+
+  6%|▋         | 63/1000 [00:08<01:35,  9.77it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.39it/s]
+
+  7%|▋         | 67/1000 [00:09<02:02,  7.59it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.27it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.01it/s]
+
+  7%|▋         | 72/1000 [00:09<02:17,  6.73it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.14it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.88it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.34it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.87it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.21it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.03it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.27it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.10it/s]
+
+  9%|▉         | 92/1000 [00:12<02:26,  6.19it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.80it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.53it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.78it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.36it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.64it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.75it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.82it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.33it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.46it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.23it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.06it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.55it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.84it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.25it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.09it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.80it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.15it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.66it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.90it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.98it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.05it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.91it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.09it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:40,  8.45it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.51it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.06it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:20, 10.46it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.67it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  8.00it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.64it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.82it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:26,  5.68it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.19it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:24,  5.78it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:46,  7.84it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.16it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.24it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.75it/s]
+
+ 18%|█▊        | 176/1000 [00:22<01:37,  8.46it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.52it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:23<02:22,  5.73it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.75it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.13it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:01,  6.65it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.50it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.39it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.77it/s]
+
+ 20%|█▉        | 198/1000 [00:25<01:27,  9.22it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.68it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.79it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.65it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.96it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 210/1000 [00:27<01:32,  8.50it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.69it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.15it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.50it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.80it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.32it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.30it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.56it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:07,  6.06it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:06,  6.11it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.43it/s]
+
+ 23%|██▎       | 228/1000 [00:29<01:52,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.92it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.39it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.05it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.23it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.96it/s]
+
+ 24%|██▎       | 237/1000 [00:30<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.98it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.37it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.48it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.67it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.27it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:05,  5.97it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.88it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.04it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.71it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.77it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.71it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.53it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.45it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:55,  6.35it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.61it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:45,  6.93it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:36,  7.55it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.04it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:13,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.70it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.71it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.07it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.82it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.63it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.04it/s]
+
+ 28%|██▊       | 283/1000 [00:37<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.75it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.80it/s]
+
+ 29%|██▉       | 291/1000 [00:38<01:41,  7.02it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:37,  7.21it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:41,  6.91it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.20it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.48it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:37,  7.14it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.78it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.09it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.15it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.95it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.81it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.22it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.28it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.26it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.59it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:33,  7.25it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.45it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.70it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.97it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.39it/s]
+
+ 34%|███▎      | 336/1000 [00:44<01:22,  8.05it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.84it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.05it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:45<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.42it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.92it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:48,  5.96it/s]
+
+ 35%|███▌      | 352/1000 [00:46<01:24,  7.65it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.56it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.35it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.58it/s]
+
+ 36%|███▌      | 361/1000 [00:47<01:07,  9.43it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.28it/s]
+
+ 36%|███▋      | 365/1000 [00:48<02:08,  4.93it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.38it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.10it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.23it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.48it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.19it/s]
+
+ 38%|███▊      | 379/1000 [00:50<01:16,  8.16it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.19it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:12,  8.55it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.08it/s]
+
+ 39%|███▊      | 387/1000 [00:51<01:20,  7.58it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.31it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.84it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.49it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  9.00it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.54it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.85it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.23it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.51it/s]
+
+ 41%|████      | 408/1000 [00:54<01:32,  6.41it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.46it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.37it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.07it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.26it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.46it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.51it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.70it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.94it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.38it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.97it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.55it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.85it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.55it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.04it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.81it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.54it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.10it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.30it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.11it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.26it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.03it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.20it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.58it/s]
+
+ 45%|████▌     | 454/1000 [01:00<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.52it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.83it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.29it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.85it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.63it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.76it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.02it/s]
+
+ 47%|████▋     | 467/1000 [01:02<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:23,  6.32it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.87it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.58it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.52it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.40it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.32it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:04,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.62it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:02,  8.21it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.27it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.98it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.42it/s]
+
+ 50%|████▉     | 498/1000 [01:06<00:59,  8.45it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.94it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.56it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:12,  6.82it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.57it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.82it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.45it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.99it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48, 10.00it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.30it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.55it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:07,  7.13it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<01:00,  7.95it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.33it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<01:00,  7.97it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.65it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.78it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.39it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:47,  9.75it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.71it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.67it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.77it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.08it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.51it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.71it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.93it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<01:01,  7.28it/s]
+
+ 55%|█████▌    | 552/1000 [01:14<01:03,  7.10it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:13,  6.10it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:15,  5.90it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.71it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  6.97it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:50,  8.80it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.01it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.35it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.97it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.32it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.36it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.84it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.73it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.26it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.99it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.58it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.61it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+
+ 59%|█████▊    | 587/1000 [01:19<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:49,  8.31it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:01,  6.66it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:50,  8.08it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<01:00,  6.75it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.35it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:59,  6.83it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.66it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+
+ 60%|██████    | 601/1000 [01:21<01:02,  6.39it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.70it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.12it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.48it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:42,  9.06it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:37, 10.26it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:51,  7.44it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:50,  7.62it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:42,  8.99it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:48,  7.75it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:47,  8.02it/s]
+
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.37it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.54it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:52,  7.14it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.19it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:47,  7.82it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:39,  9.40it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.71it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.96it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.19it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:45,  7.93it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:47,  7.57it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.52it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.33it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.71it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.93it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.58it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.50it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.34it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.63it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.27it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 11.00it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.95it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.89it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:34<00:45,  6.82it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.86it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.28it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.44it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.35it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.96it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.06it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:36<00:41,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.02it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.62it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.71it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.37it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.07it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.95it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.48it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.17it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.26it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.15it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.63it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.34it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:27,  9.55it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.90it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.54it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.79it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.45it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.69it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.24it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.41it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.62it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.33it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.72it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.67it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.62it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.48it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.80it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.65it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.57it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.79it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:26,  8.38it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.16it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  8.98it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.74it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.98it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.20it/s]
+
+ 80%|███████▉  | 797/1000 [01:48<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.73it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.66it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:24,  7.85it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.19it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.99it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.24it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.25it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.06it/s]
+
+ 82%|████████▏ | 815/1000 [01:49<00:19,  9.33it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.16it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.21it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.06it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.83it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:52<00:27,  6.03it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.55it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.12it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.47it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.64it/s]
+
+ 84%|████████▍ | 842/1000 [01:53<00:19,  8.17it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.43it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.78it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.57it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.50it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.88it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.47it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:13, 10.23it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.53it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.78it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.64it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.71it/s]
+
+ 87%|████████▋ | 867/1000 [01:56<00:14,  9.31it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.23it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.25it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.79it/s]
+
+ 88%|████████▊ | 875/1000 [01:57<00:16,  7.64it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.88it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.88it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.61it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.97it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.78it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.78it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.62it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.85it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.71it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.86it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.07it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.06it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.33it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.60it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.79it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.61it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.18it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+
+ 93%|█████████▎| 928/1000 [02:04<00:10,  6.66it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.62it/s]
+
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.31it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.15it/s]
+
+ 94%|█████████▍| 942/1000 [02:06<00:09,  6.35it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.39it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.84it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.41it/s]
+
+ 96%|█████████▌| 957/1000 [02:08<00:06,  6.41it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.83it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:09<00:04,  7.51it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.46it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.03it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:10<00:03,  8.03it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.21it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.80it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.44it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.08it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.45it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.76it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.15it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.99it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.56it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.54it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.78it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+                                                      
+
+                                                   
+{'eval_loss': '0.7739', 'eval_runtime': '135.2', 'eval_samples_per_second': '7.394', 'eval_steps_per_second': '7.394', 'epoch': '0.4966', 'num_input_tokens_seen': 6050856}
+
+ 50%|████▉     | 900/1813 [1:40:17<1:46:33,  7.00s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+
+                                                   
+ 50%|████▉     | 901/1813 [1:40:24<12:00:37, 47.41s/it]
+ 50%|████▉     | 902/1813 [1:40:30<8:51:21, 35.00s/it] 
+ 50%|████▉     | 903/1813 [1:40:36<6:39:39, 26.35s/it]
+ 50%|████▉     | 904/1813 [1:40:42<5:07:41, 20.31s/it]
+ 50%|████▉     | 905/1813 [1:40:50<4:11:14, 16.60s/it]
+ 50%|████▉     | 906/1813 [1:40:55<3:19:59, 13.23s/it]
+ 50%|█████     | 907/1813 [1:41:00<2:40:44, 10.65s/it]
+ 50%|█████     | 908/1813 [1:41:06<2:18:04,  9.15s/it]
+ 50%|█████     | 909/1813 [1:41:12<2:06:37,  8.40s/it]
+ 50%|█████     | 910/1813 [1:41:19<1:56:48,  7.76s/it]
+                                                      
+{'loss': '0.9121', 'grad_norm': '1.041', 'learning_rate': '0.000106', 'epoch': '0.5021', 'num_input_tokens_seen': 6115160, 'train_runtime': '6079', 'train_tokens_per_second': '1006'}
+
+ 50%|█████     | 910/1813 [1:41:19<1:56:48,  7.76s/it]
+ 50%|█████     | 911/1813 [1:41:24<1:46:44,  7.10s/it]
+ 50%|█████     | 912/1813 [1:41:30<1:40:06,  6.67s/it]
+ 50%|█████     | 913/1813 [1:41:35<1:33:22,  6.22s/it]
+ 50%|█████     | 914/1813 [1:41:40<1:25:56,  5.74s/it]
+ 50%|█████     | 915/1813 [1:41:47<1:34:23,  6.31s/it]
+ 51%|█████     | 916/1813 [1:41:54<1:37:40,  6.53s/it]
+ 51%|█████     | 917/1813 [1:42:01<1:38:11,  6.58s/it]
+ 51%|█████     | 918/1813 [1:42:07<1:33:20,  6.26s/it]
+ 51%|█████     | 919/1813 [1:42:13<1:32:59,  6.24s/it]
+ 51%|█████     | 920/1813 [1:42:18<1:29:25,  6.01s/it]
+                                                      
+{'loss': '0.9388', 'grad_norm': '0.8134', 'learning_rate': '0.0001042', 'epoch': '0.5076', 'num_input_tokens_seen': 6178904, 'train_runtime': '6139', 'train_tokens_per_second': '1007'}
+
+ 51%|█████     | 920/1813 [1:42:18<1:29:25,  6.01s/it]
+ 51%|█████     | 921/1813 [1:42:24<1:27:23,  5.88s/it]
+ 51%|█████     | 922/1813 [1:42:29<1:25:47,  5.78s/it]
+ 51%|█████     | 923/1813 [1:42:35<1:26:06,  5.81s/it]
+ 51%|█████     | 924/1813 [1:42:42<1:30:38,  6.12s/it]
+ 51%|█████     | 925/1813 [1:42:48<1:28:59,  6.01s/it]
+ 51%|█████     | 926/1813 [1:42:54<1:31:28,  6.19s/it]
+ 51%|█████     | 927/1813 [1:43:00<1:30:23,  6.12s/it]
+ 51%|█████     | 928/1813 [1:43:06<1:27:44,  5.95s/it]
+ 51%|█████     | 929/1813 [1:43:12<1:27:44,  5.96s/it]
+ 51%|█████▏    | 930/1813 [1:43:19<1:31:43,  6.23s/it]
+                                                      
+{'loss': '0.8985', 'grad_norm': '1.001', 'learning_rate': '0.0001025', 'epoch': '0.5131', 'num_input_tokens_seen': 6244360, 'train_runtime': '6199', 'train_tokens_per_second': '1007'}
+
+ 51%|█████▏    | 930/1813 [1:43:19<1:31:43,  6.23s/it]
+ 51%|█████▏    | 931/1813 [1:43:26<1:35:06,  6.47s/it]
+ 51%|█████▏    | 932/1813 [1:43:31<1:31:11,  6.21s/it]
+ 51%|█████▏    | 933/1813 [1:43:38<1:30:56,  6.20s/it]
+ 52%|█████▏    | 934/1813 [1:43:44<1:32:29,  6.31s/it]
+ 52%|█████▏    | 935/1813 [1:43:53<1:43:32,  7.08s/it]
+ 52%|█████▏    | 936/1813 [1:44:00<1:41:02,  6.91s/it]
+ 52%|█████▏    | 937/1813 [1:44:06<1:37:54,  6.71s/it]
+ 52%|█████▏    | 938/1813 [1:44:13<1:39:15,  6.81s/it]
+ 52%|█████▏    | 939/1813 [1:44:19<1:35:34,  6.56s/it]
+ 52%|█████▏    | 940/1813 [1:44:26<1:38:14,  6.75s/it]
+                                                      
+{'loss': '0.8857', 'grad_norm': '1.112', 'learning_rate': '0.0001007', 'epoch': '0.5186', 'num_input_tokens_seen': 6318968, 'train_runtime': '6266', 'train_tokens_per_second': '1008'}
+
+ 52%|█████▏    | 940/1813 [1:44:26<1:38:14,  6.75s/it]
+ 52%|█████▏    | 941/1813 [1:44:31<1:31:19,  6.28s/it]
+ 52%|█████▏    | 942/1813 [1:44:37<1:28:00,  6.06s/it]
+ 52%|█████▏    | 943/1813 [1:44:42<1:26:39,  5.98s/it]
+ 52%|█████▏    | 944/1813 [1:44:49<1:27:08,  6.02s/it]
+ 52%|█████▏    | 945/1813 [1:44:54<1:23:39,  5.78s/it]
+ 52%|█████▏    | 946/1813 [1:45:00<1:24:46,  5.87s/it]
+ 52%|█████▏    | 947/1813 [1:45:06<1:25:34,  5.93s/it]
+ 52%|█████▏    | 948/1813 [1:45:12<1:27:19,  6.06s/it]
+ 52%|█████▏    | 949/1813 [1:45:17<1:22:56,  5.76s/it]
+ 52%|█████▏    | 950/1813 [1:45:24<1:24:56,  5.91s/it]
+                                                      
+{'loss': '0.9437', 'grad_norm': '1.156', 'learning_rate': '9.895e-05', 'epoch': '0.5241', 'num_input_tokens_seen': 6379704, 'train_runtime': '6324', 'train_tokens_per_second': '1009'}
+
+ 52%|█████▏    | 950/1813 [1:45:24<1:24:56,  5.91s/it]
+ 52%|█████▏    | 951/1813 [1:45:29<1:22:29,  5.74s/it]
+ 53%|█████▎    | 952/1813 [1:45:35<1:21:24,  5.67s/it]
+ 53%|█████▎    | 953/1813 [1:45:40<1:22:25,  5.75s/it]
+ 53%|█████▎    | 954/1813 [1:45:46<1:22:14,  5.74s/it]
+ 53%|█████▎    | 955/1813 [1:45:52<1:21:46,  5.72s/it]
+ 53%|█████▎    | 956/1813 [1:45:57<1:20:57,  5.67s/it]
+ 53%|█████▎    | 957/1813 [1:46:03<1:20:17,  5.63s/it]
+ 53%|█████▎    | 958/1813 [1:46:07<1:15:18,  5.28s/it]
+ 53%|█████▎    | 959/1813 [1:46:13<1:17:32,  5.45s/it]
+ 53%|█████▎    | 960/1813 [1:46:21<1:28:52,  6.25s/it]
+                                                      
+{'loss': '0.9495', 'grad_norm': '0.7477', 'learning_rate': '9.719e-05', 'epoch': '0.5297', 'num_input_tokens_seen': 6441656, 'train_runtime': '6382', 'train_tokens_per_second': '1009'}
+
+ 53%|█████▎    | 960/1813 [1:46:21<1:28:52,  6.25s/it]
+ 53%|█████▎    | 961/1813 [1:46:28<1:30:28,  6.37s/it]
+ 53%|█████▎    | 962/1813 [1:46:36<1:36:15,  6.79s/it]
+ 53%|█████▎    | 963/1813 [1:46:43<1:36:12,  6.79s/it]
+ 53%|█████▎    | 964/1813 [1:46:48<1:31:42,  6.48s/it]
+ 53%|█████▎    | 965/1813 [1:46:54<1:28:56,  6.29s/it]
+ 53%|█████▎    | 966/1813 [1:47:02<1:34:05,  6.67s/it]
+ 53%|█████▎    | 967/1813 [1:47:09<1:37:20,  6.90s/it]
+ 53%|█████▎    | 968/1813 [1:47:15<1:31:13,  6.48s/it]
+ 53%|█████▎    | 969/1813 [1:47:21<1:28:47,  6.31s/it]
+ 54%|█████▎    | 970/1813 [1:47:25<1:21:42,  5.82s/it]
+                                                      
+{'loss': '0.9724', 'grad_norm': '1.019', 'learning_rate': '9.543e-05', 'epoch': '0.5352', 'num_input_tokens_seen': 6512320, 'train_runtime': '6446', 'train_tokens_per_second': '1010'}
+
+ 54%|█████▎    | 970/1813 [1:47:25<1:21:42,  5.82s/it]
+ 54%|█████▎    | 971/1813 [1:47:32<1:25:12,  6.07s/it]
+ 54%|█████▎    | 972/1813 [1:47:38<1:25:49,  6.12s/it]
+ 54%|█████▎    | 973/1813 [1:47:44<1:24:47,  6.06s/it]
+ 54%|█████▎    | 974/1813 [1:47:53<1:34:50,  6.78s/it]
+ 54%|█████▍    | 975/1813 [1:47:59<1:32:27,  6.62s/it]
+ 54%|█████▍    | 976/1813 [1:48:04<1:27:37,  6.28s/it]
+ 54%|█████▍    | 977/1813 [1:48:12<1:33:34,  6.72s/it]
+ 54%|█████▍    | 978/1813 [1:48:18<1:32:24,  6.64s/it]
+ 54%|█████▍    | 979/1813 [1:48:26<1:34:24,  6.79s/it]
+ 54%|█████▍    | 980/1813 [1:48:32<1:34:06,  6.78s/it]
+                                                      
+{'loss': '0.9324', 'grad_norm': '1.696', 'learning_rate': '9.367e-05', 'epoch': '0.5407', 'num_input_tokens_seen': 6586672, 'train_runtime': '6513', 'train_tokens_per_second': '1011'}
+
+ 54%|█████▍    | 980/1813 [1:48:32<1:34:06,  6.78s/it]
+ 54%|█████▍    | 981/1813 [1:48:39<1:33:46,  6.76s/it]
+ 54%|█████▍    | 982/1813 [1:48:44<1:27:37,  6.33s/it]
+ 54%|█████▍    | 983/1813 [1:48:50<1:23:06,  6.01s/it]
+ 54%|█████▍    | 984/1813 [1:48:55<1:20:40,  5.84s/it]
+ 54%|█████▍    | 985/1813 [1:49:02<1:23:40,  6.06s/it]
+ 54%|█████▍    | 986/1813 [1:49:09<1:29:12,  6.47s/it]
+ 54%|█████▍    | 987/1813 [1:49:15<1:26:53,  6.31s/it]
+ 54%|█████▍    | 988/1813 [1:49:20<1:23:13,  6.05s/it]
+ 55%|█████▍    | 989/1813 [1:49:25<1:18:40,  5.73s/it]
+ 55%|█████▍    | 990/1813 [1:49:33<1:27:32,  6.38s/it]
+                                                      
+{'loss': '0.8823', 'grad_norm': '0.9724', 'learning_rate': '9.191e-05', 'epoch': '0.5462', 'num_input_tokens_seen': 6653096, 'train_runtime': '6574', 'train_tokens_per_second': '1012'}
+
+ 55%|█████▍    | 990/1813 [1:49:33<1:27:32,  6.38s/it]
+ 55%|█████▍    | 991/1813 [1:49:39<1:23:07,  6.07s/it]
+ 55%|█████▍    | 992/1813 [1:49:46<1:28:42,  6.48s/it]
+ 55%|█████▍    | 993/1813 [1:49:52<1:26:28,  6.33s/it]
+ 55%|█████▍    | 994/1813 [1:50:01<1:34:55,  6.95s/it]
+ 55%|█████▍    | 995/1813 [1:50:06<1:29:58,  6.60s/it]
+ 55%|█████▍    | 996/1813 [1:50:12<1:27:55,  6.46s/it]
+ 55%|█████▍    | 997/1813 [1:50:18<1:24:09,  6.19s/it]
+ 55%|█████▌    | 998/1813 [1:50:23<1:18:51,  5.81s/it]
+ 55%|█████▌    | 999/1813 [1:50:28<1:17:40,  5.73s/it]
+ 55%|█████▌    | 1000/1813 [1:50:33<1:14:15,  5.48s/it]
+                                                       
+{'loss': '0.9983', 'grad_norm': '1.066', 'learning_rate': '9.014e-05', 'epoch': '0.5517', 'num_input_tokens_seen': 6716784, 'train_runtime': '6634', 'train_tokens_per_second': '1012'}
+
+ 55%|█████▌    | 1000/1813 [1:50:33<1:14:15,  5.48s/it]
+ 55%|█████▌    | 1001/1813 [1:50:40<1:18:10,  5.78s/it]
+ 55%|█████▌    | 1002/1813 [1:50:46<1:19:36,  5.89s/it]
+ 55%|█████▌    | 1003/1813 [1:50:53<1:24:01,  6.22s/it]
+ 55%|█████▌    | 1004/1813 [1:50:59<1:21:10,  6.02s/it]
+ 55%|█████▌    | 1005/1813 [1:51:06<1:25:34,  6.36s/it]
+ 55%|█████▌    | 1006/1813 [1:51:12<1:25:48,  6.38s/it]
+ 56%|█████▌    | 1007/1813 [1:51:18<1:22:29,  6.14s/it]
+ 56%|█████▌    | 1008/1813 [1:51:23<1:20:56,  6.03s/it]
+ 56%|█████▌    | 1009/1813 [1:51:29<1:20:44,  6.03s/it]
+ 56%|█████▌    | 1010/1813 [1:51:35<1:18:48,  5.89s/it]
+                                                       
+{'loss': '0.9274', 'grad_norm': '1.086', 'learning_rate': '8.839e-05', 'epoch': '0.5572', 'num_input_tokens_seen': 6783768, 'train_runtime': '6696', 'train_tokens_per_second': '1013'}
+
+ 56%|█████▌    | 1010/1813 [1:51:35<1:18:48,  5.89s/it]
+ 56%|█████▌    | 1011/1813 [1:51:42<1:22:31,  6.17s/it]
+ 56%|█████▌    | 1012/1813 [1:51:47<1:16:41,  5.75s/it]
+ 56%|█████▌    | 1013/1813 [1:51:51<1:10:55,  5.32s/it]
+ 56%|█████▌    | 1014/1813 [1:51:59<1:23:01,  6.23s/it]
+ 56%|█████▌    | 1015/1813 [1:52:04<1:18:28,  5.90s/it]
+ 56%|█████▌    | 1016/1813 [1:52:10<1:15:43,  5.70s/it]
+ 56%|█████▌    | 1017/1813 [1:52:16<1:19:56,  6.03s/it]
+ 56%|█████▌    | 1018/1813 [1:52:23<1:20:43,  6.09s/it]
+ 56%|█████▌    | 1019/1813 [1:52:28<1:16:16,  5.76s/it]
+ 56%|█████▋    | 1020/1813 [1:52:33<1:13:18,  5.55s/it]
+                                                       
+{'loss': '0.8471', 'grad_norm': '1.229', 'learning_rate': '8.663e-05', 'epoch': '0.5628', 'num_input_tokens_seen': 6844992, 'train_runtime': '6753', 'train_tokens_per_second': '1014'}
+
+ 56%|█████▋    | 1020/1813 [1:52:33<1:13:18,  5.55s/it]
+ 56%|█████▋    | 1021/1813 [1:52:40<1:18:33,  5.95s/it]
+ 56%|█████▋    | 1022/1813 [1:52:46<1:19:55,  6.06s/it]
+ 56%|█████▋    | 1023/1813 [1:52:54<1:25:56,  6.53s/it]
+ 56%|█████▋    | 1024/1813 [1:53:00<1:25:16,  6.49s/it]
+ 57%|█████▋    | 1025/1813 [1:53:09<1:35:32,  7.27s/it]
+ 57%|█████▋    | 1026/1813 [1:53:14<1:27:43,  6.69s/it]
+ 57%|█████▋    | 1027/1813 [1:53:20<1:25:12,  6.50s/it]
+ 57%|█████▋    | 1028/1813 [1:53:28<1:28:30,  6.76s/it]
+ 57%|█████▋    | 1029/1813 [1:53:34<1:26:54,  6.65s/it]
+ 57%|█████▋    | 1030/1813 [1:53:41<1:26:45,  6.65s/it]
+                                                       
+{'loss': '0.9876', 'grad_norm': '0.7939', 'learning_rate': '8.487e-05', 'epoch': '0.5683', 'num_input_tokens_seen': 6920856, 'train_runtime': '6821', 'train_tokens_per_second': '1015'}
+
+ 57%|█████▋    | 1030/1813 [1:53:41<1:26:45,  6.65s/it]
+ 57%|█████▋    | 1031/1813 [1:53:46<1:21:59,  6.29s/it]
+ 57%|█████▋    | 1032/1813 [1:53:53<1:23:14,  6.39s/it]
+ 57%|█████▋    | 1033/1813 [1:53:59<1:21:48,  6.29s/it]
+ 57%|█████▋    | 1034/1813 [1:54:05<1:19:02,  6.09s/it]
+ 57%|█████▋    | 1035/1813 [1:54:11<1:18:50,  6.08s/it]
+ 57%|█████▋    | 1036/1813 [1:54:17<1:20:14,  6.20s/it]
+ 57%|█████▋    | 1037/1813 [1:54:23<1:18:25,  6.06s/it]
+ 57%|█████▋    | 1038/1813 [1:54:29<1:18:26,  6.07s/it]
+ 57%|█████▋    | 1039/1813 [1:54:36<1:21:27,  6.31s/it]
+ 57%|█████▋    | 1040/1813 [1:54:43<1:23:26,  6.48s/it]
+                                                       
+{'loss': '0.9103', 'grad_norm': '0.8938', 'learning_rate': '8.312e-05', 'epoch': '0.5738', 'num_input_tokens_seen': 6988568, 'train_runtime': '6883', 'train_tokens_per_second': '1015'}
+
+ 57%|█████▋    | 1040/1813 [1:54:43<1:23:26,  6.48s/it]
+ 57%|█████▋    | 1041/1813 [1:54:49<1:21:04,  6.30s/it]
+ 57%|█████▋    | 1042/1813 [1:54:55<1:20:48,  6.29s/it]
+ 58%|█████▊    | 1043/1813 [1:55:00<1:17:27,  6.04s/it]
+ 58%|█████▊    | 1044/1813 [1:55:05<1:13:53,  5.76s/it]
+ 58%|█████▊    | 1045/1813 [1:55:11<1:13:25,  5.74s/it]
+ 58%|█████▊    | 1046/1813 [1:55:17<1:13:06,  5.72s/it]
+ 58%|█████▊    | 1047/1813 [1:55:23<1:15:19,  5.90s/it]
+ 58%|█████▊    | 1048/1813 [1:55:30<1:18:42,  6.17s/it]
+ 58%|█████▊    | 1049/1813 [1:55:35<1:15:39,  5.94s/it]
+ 58%|█████▊    | 1050/1813 [1:55:41<1:15:19,  5.92s/it]
+                                                       
+{'loss': '0.8996', 'grad_norm': '1.192', 'learning_rate': '8.137e-05', 'epoch': '0.5793', 'num_input_tokens_seen': 7051808, 'train_runtime': '6942', 'train_tokens_per_second': '1016'}
+
+ 58%|█████▊    | 1050/1813 [1:55:41<1:15:19,  5.92s/it]
+ 58%|█████▊    | 1051/1813 [1:55:46<1:12:31,  5.71s/it]
+ 58%|█████▊    | 1052/1813 [1:55:52<1:13:33,  5.80s/it]
+ 58%|█████▊    | 1053/1813 [1:55:58<1:14:10,  5.86s/it]
+ 58%|█████▊    | 1054/1813 [1:56:04<1:14:04,  5.86s/it]
+ 58%|█████▊    | 1055/1813 [1:56:10<1:12:48,  5.76s/it]
+ 58%|█████▊    | 1056/1813 [1:56:16<1:12:40,  5.76s/it]
+ 58%|█████▊    | 1057/1813 [1:56:22<1:15:43,  6.01s/it]
+ 58%|█████▊    | 1058/1813 [1:56:27<1:11:59,  5.72s/it]
+ 58%|█████▊    | 1059/1813 [1:56:32<1:07:39,  5.38s/it]
+ 58%|█████▊    | 1060/1813 [1:56:39<1:13:07,  5.83s/it]
+                                                       
+{'loss': '0.9026', 'grad_norm': '0.7492', 'learning_rate': '7.963e-05', 'epoch': '0.5848', 'num_input_tokens_seen': 7110624, 'train_runtime': '6999', 'train_tokens_per_second': '1016'}
+
+ 58%|█████▊    | 1060/1813 [1:56:39<1:13:07,  5.83s/it]
+ 59%|█████▊    | 1061/1813 [1:56:44<1:09:16,  5.53s/it]
+ 59%|█████▊    | 1062/1813 [1:56:49<1:07:20,  5.38s/it]
+ 59%|█████▊    | 1063/1813 [1:56:55<1:11:56,  5.76s/it]
+ 59%|█████▊    | 1064/1813 [1:57:03<1:18:30,  6.29s/it]
+ 59%|█████▊    | 1065/1813 [1:57:10<1:20:12,  6.43s/it]
+ 59%|█████▉    | 1066/1813 [1:57:17<1:23:49,  6.73s/it]
+ 59%|█████▉    | 1067/1813 [1:57:24<1:23:08,  6.69s/it]
+ 59%|█████▉    | 1068/1813 [1:57:29<1:18:10,  6.30s/it]
+ 59%|█████▉    | 1069/1813 [1:57:37<1:24:35,  6.82s/it]
+ 59%|█████▉    | 1070/1813 [1:57:43<1:20:28,  6.50s/it]
+                                                       
+{'loss': '0.9037', 'grad_norm': '1.174', 'learning_rate': '7.789e-05', 'epoch': '0.5903', 'num_input_tokens_seen': 7179728, 'train_runtime': '7063', 'train_tokens_per_second': '1016'}
+
+ 59%|█████▉    | 1070/1813 [1:57:43<1:20:28,  6.50s/it]
+ 59%|█████▉    | 1071/1813 [1:57:48<1:15:06,  6.07s/it]
+ 59%|█████▉    | 1072/1813 [1:57:54<1:15:59,  6.15s/it]
+ 59%|█████▉    | 1073/1813 [1:58:00<1:14:27,  6.04s/it]
+ 59%|█████▉    | 1074/1813 [1:58:06<1:13:14,  5.95s/it]
+ 59%|█████▉    | 1075/1813 [1:58:12<1:15:29,  6.14s/it]
+ 59%|█████▉    | 1076/1813 [1:58:19<1:16:47,  6.25s/it]
+ 59%|█████▉    | 1077/1813 [1:58:23<1:10:54,  5.78s/it]
+ 59%|█████▉    | 1078/1813 [1:58:28<1:08:13,  5.57s/it]
+ 60%|█████▉    | 1079/1813 [1:58:35<1:11:11,  5.82s/it]
+ 60%|█████▉    | 1080/1813 [1:58:41<1:13:38,  6.03s/it]
+                                                       
+{'loss': '0.9005', 'grad_norm': '0.7815', 'learning_rate': '7.615e-05', 'epoch': '0.5959', 'num_input_tokens_seen': 7240984, 'train_runtime': '7122', 'train_tokens_per_second': '1017'}
+
+ 60%|█████▉    | 1080/1813 [1:58:41<1:13:38,  6.03s/it]
+ 60%|█████▉    | 1081/1813 [1:58:48<1:14:41,  6.12s/it]
+ 60%|█████▉    | 1082/1813 [1:58:54<1:13:35,  6.04s/it]
+ 60%|█████▉    | 1083/1813 [1:59:00<1:14:13,  6.10s/it]
+ 60%|█████▉    | 1084/1813 [1:59:06<1:13:51,  6.08s/it]
+ 60%|█████▉    | 1085/1813 [1:59:12<1:13:35,  6.06s/it]
+ 60%|█████▉    | 1086/1813 [1:59:16<1:06:50,  5.52s/it]
+ 60%|█████▉    | 1087/1813 [1:59:22<1:07:51,  5.61s/it]
+ 60%|██████    | 1088/1813 [1:59:28<1:08:54,  5.70s/it]
+ 60%|██████    | 1089/1813 [1:59:35<1:12:22,  6.00s/it]
+ 60%|██████    | 1090/1813 [1:59:41<1:12:06,  5.98s/it]
+                                                       
+{'loss': '0.8087', 'grad_norm': '1.023', 'learning_rate': '7.442e-05', 'epoch': '0.6014', 'num_input_tokens_seen': 7301824, 'train_runtime': '7181', 'train_tokens_per_second': '1017'}
+
+ 60%|██████    | 1090/1813 [1:59:41<1:12:06,  5.98s/it]
+ 60%|██████    | 1091/1813 [1:59:46<1:11:51,  5.97s/it]
+ 60%|██████    | 1092/1813 [1:59:52<1:11:03,  5.91s/it]
+ 60%|██████    | 1093/1813 [1:59:58<1:09:09,  5.76s/it]
+ 60%|██████    | 1094/1813 [2:00:04<1:10:44,  5.90s/it]
+ 60%|██████    | 1095/1813 [2:00:10<1:11:30,  5.98s/it]
+ 60%|██████    | 1096/1813 [2:00:15<1:09:07,  5.78s/it]
+ 61%|██████    | 1097/1813 [2:00:21<1:09:12,  5.80s/it]
+ 61%|██████    | 1098/1813 [2:00:28<1:13:24,  6.16s/it]
+ 61%|██████    | 1099/1813 [2:00:34<1:13:00,  6.14s/it]
+ 61%|██████    | 1100/1813 [2:00:42<1:18:51,  6.64s/it]
+                                                       
+{'loss': '0.8979', 'grad_norm': '0.9885', 'learning_rate': '7.27e-05', 'epoch': '0.6069', 'num_input_tokens_seen': 7369528, 'train_runtime': '7243', 'train_tokens_per_second': '1018'}
+
+ 61%|██████    | 1100/1813 [2:00:42<1:18:51,  6.64s/it]
+ 61%|██████    | 1101/1813 [2:00:48<1:17:45,  6.55s/it]
+ 61%|██████    | 1102/1813 [2:00:55<1:19:07,  6.68s/it]
+ 61%|██████    | 1103/1813 [2:01:01<1:14:16,  6.28s/it]
+ 61%|██████    | 1104/1813 [2:01:08<1:18:21,  6.63s/it]
+ 61%|██████    | 1105/1813 [2:01:13<1:11:30,  6.06s/it]
+ 61%|██████    | 1106/1813 [2:01:18<1:09:40,  5.91s/it]
+ 61%|██████    | 1107/1813 [2:01:27<1:18:05,  6.64s/it]
+ 61%|██████    | 1108/1813 [2:01:32<1:14:07,  6.31s/it]
+ 61%|██████    | 1109/1813 [2:01:39<1:14:20,  6.34s/it]
+ 61%|██████    | 1110/1813 [2:01:46<1:18:32,  6.70s/it]
+                                                       
+{'loss': '0.9361', 'grad_norm': '0.9045', 'learning_rate': '7.099e-05', 'epoch': '0.6124', 'num_input_tokens_seen': 7441040, 'train_runtime': '7307', 'train_tokens_per_second': '1018'}
+
+ 61%|██████    | 1110/1813 [2:01:46<1:18:32,  6.70s/it]
+ 61%|██████▏   | 1111/1813 [2:01:51<1:10:50,  6.05s/it]
+ 61%|██████▏   | 1112/1813 [2:01:57<1:09:17,  5.93s/it]
+ 61%|██████▏   | 1113/1813 [2:02:03<1:10:24,  6.04s/it]
+ 61%|██████▏   | 1114/1813 [2:02:10<1:14:05,  6.36s/it]
+ 62%|██████▏   | 1115/1813 [2:02:17<1:15:03,  6.45s/it]
+ 62%|██████▏   | 1116/1813 [2:02:23<1:14:05,  6.38s/it]
+ 62%|██████▏   | 1117/1813 [2:02:28<1:10:30,  6.08s/it]
+ 62%|██████▏   | 1118/1813 [2:02:33<1:05:53,  5.69s/it]
+ 62%|██████▏   | 1119/1813 [2:02:39<1:07:53,  5.87s/it]
+ 62%|██████▏   | 1120/1813 [2:02:46<1:10:45,  6.13s/it]
+                                                       
+{'loss': '0.8743', 'grad_norm': '2.007', 'learning_rate': '6.928e-05', 'epoch': '0.6179', 'num_input_tokens_seen': 7504976, 'train_runtime': '7366', 'train_tokens_per_second': '1019'}
+
+ 62%|██████▏   | 1120/1813 [2:02:46<1:10:45,  6.13s/it]
+ 62%|██████▏   | 1121/1813 [2:02:54<1:17:10,  6.69s/it]
+ 62%|██████▏   | 1122/1813 [2:02:59<1:09:59,  6.08s/it]
+ 62%|██████▏   | 1123/1813 [2:03:06<1:14:48,  6.50s/it]
+ 62%|██████▏   | 1124/1813 [2:03:12<1:11:26,  6.22s/it]
+ 62%|██████▏   | 1125/1813 [2:03:19<1:13:32,  6.41s/it]
+ 62%|██████▏   | 1126/1813 [2:03:25<1:12:44,  6.35s/it]
+ 62%|██████▏   | 1127/1813 [2:03:30<1:09:58,  6.12s/it]
+ 62%|██████▏   | 1128/1813 [2:03:36<1:07:19,  5.90s/it]
+ 62%|██████▏   | 1129/1813 [2:03:41<1:04:42,  5.68s/it]
+ 62%|██████▏   | 1130/1813 [2:03:46<1:02:36,  5.50s/it]
+                                                       
+{'loss': '0.8543', 'grad_norm': '0.9054', 'learning_rate': '6.758e-05', 'epoch': '0.6234', 'num_input_tokens_seen': 7568576, 'train_runtime': '7426', 'train_tokens_per_second': '1019'}
+
+ 62%|██████▏   | 1130/1813 [2:03:46<1:02:36,  5.50s/it]
+ 62%|██████▏   | 1131/1813 [2:03:52<1:02:57,  5.54s/it]
+ 62%|██████▏   | 1132/1813 [2:03:58<1:04:21,  5.67s/it]
+ 62%|██████▏   | 1133/1813 [2:04:04<1:07:29,  5.96s/it]
+ 63%|██████▎   | 1134/1813 [2:04:12<1:14:17,  6.56s/it]
+ 63%|██████▎   | 1135/1813 [2:04:19<1:14:48,  6.62s/it]
+ 63%|██████▎   | 1136/1813 [2:04:26<1:15:30,  6.69s/it]
+ 63%|██████▎   | 1137/1813 [2:04:32<1:15:27,  6.70s/it]
+ 63%|██████▎   | 1138/1813 [2:04:40<1:16:38,  6.81s/it]
+ 63%|██████▎   | 1139/1813 [2:04:45<1:11:44,  6.39s/it]
+ 63%|██████▎   | 1140/1813 [2:04:51<1:09:18,  6.18s/it]
+                                                       
+{'loss': '0.9344', 'grad_norm': '0.9702', 'learning_rate': '6.59e-05', 'epoch': '0.629', 'num_input_tokens_seen': 7639168, 'train_runtime': '7491', 'train_tokens_per_second': '1020'}
+
+ 63%|██████▎   | 1140/1813 [2:04:51<1:09:18,  6.18s/it]
+ 63%|██████▎   | 1141/1813 [2:04:57<1:09:21,  6.19s/it]
+ 63%|██████▎   | 1142/1813 [2:05:03<1:07:49,  6.07s/it]
+ 63%|██████▎   | 1143/1813 [2:05:09<1:07:04,  6.01s/it]
+ 63%|██████▎   | 1144/1813 [2:05:15<1:09:54,  6.27s/it]
+ 63%|██████▎   | 1145/1813 [2:05:22<1:10:09,  6.30s/it]
+ 63%|██████▎   | 1146/1813 [2:05:27<1:07:12,  6.05s/it]
+ 63%|██████▎   | 1147/1813 [2:05:35<1:11:50,  6.47s/it]
+ 63%|██████▎   | 1148/1813 [2:05:42<1:13:25,  6.63s/it]
+ 63%|██████▎   | 1149/1813 [2:05:48<1:10:57,  6.41s/it]
+ 63%|██████▎   | 1150/1813 [2:05:55<1:14:35,  6.75s/it]
+                                                       
+{'loss': '0.8502', 'grad_norm': '0.8745', 'learning_rate': '6.422e-05', 'epoch': '0.6345', 'num_input_tokens_seen': 7707832, 'train_runtime': '7556', 'train_tokens_per_second': '1020'}
+
+ 63%|██████▎   | 1150/1813 [2:05:55<1:14:35,  6.75s/it]
+ 63%|██████▎   | 1151/1813 [2:05:59<1:05:17,  5.92s/it]
+ 64%|██████▎   | 1152/1813 [2:06:04<1:03:24,  5.76s/it]
+ 64%|██████▎   | 1153/1813 [2:06:12<1:07:31,  6.14s/it]
+ 64%|██████▎   | 1154/1813 [2:06:18<1:07:19,  6.13s/it]
+ 64%|██████▎   | 1155/1813 [2:06:24<1:08:34,  6.25s/it]
+ 64%|██████▍   | 1156/1813 [2:06:30<1:08:39,  6.27s/it]
+ 64%|██████▍   | 1157/1813 [2:06:38<1:12:43,  6.65s/it]
+ 64%|██████▍   | 1158/1813 [2:06:45<1:12:41,  6.66s/it]
+ 64%|██████▍   | 1159/1813 [2:06:52<1:14:08,  6.80s/it]
+ 64%|██████▍   | 1160/1813 [2:06:58<1:11:43,  6.59s/it]
+                                                       
+{'loss': '0.9157', 'grad_norm': '0.7578', 'learning_rate': '6.255e-05', 'epoch': '0.64', 'num_input_tokens_seen': 7775872, 'train_runtime': '7618', 'train_tokens_per_second': '1021'}
+
+ 64%|██████▍   | 1160/1813 [2:06:58<1:11:43,  6.59s/it]
+ 64%|██████▍   | 1161/1813 [2:07:05<1:13:57,  6.81s/it]
+ 64%|██████▍   | 1162/1813 [2:07:12<1:13:44,  6.80s/it]
+ 64%|██████▍   | 1163/1813 [2:07:18<1:12:30,  6.69s/it]
+ 64%|██████▍   | 1164/1813 [2:07:25<1:10:51,  6.55s/it]
+ 64%|██████▍   | 1165/1813 [2:07:31<1:10:18,  6.51s/it]
+ 64%|██████▍   | 1166/1813 [2:07:37<1:08:14,  6.33s/it]
+ 64%|██████▍   | 1167/1813 [2:07:44<1:10:51,  6.58s/it]
+ 64%|██████▍   | 1168/1813 [2:07:51<1:10:50,  6.59s/it]
+ 64%|██████▍   | 1169/1813 [2:07:57<1:09:54,  6.51s/it]
+ 65%|██████▍   | 1170/1813 [2:08:03<1:07:36,  6.31s/it]
+                                                       
+{'loss': '0.8931', 'grad_norm': '1.074', 'learning_rate': '6.089e-05', 'epoch': '0.6455', 'num_input_tokens_seen': 7846688, 'train_runtime': '7683', 'train_tokens_per_second': '1021'}
+
+ 65%|██████▍   | 1170/1813 [2:08:03<1:07:36,  6.31s/it]
+ 65%|██████▍   | 1171/1813 [2:08:11<1:11:34,  6.69s/it]
+ 65%|██████▍   | 1172/1813 [2:08:18<1:13:26,  6.87s/it]
+ 65%|██████▍   | 1173/1813 [2:08:24<1:09:29,  6.51s/it]
+ 65%|██████▍   | 1174/1813 [2:08:29<1:05:10,  6.12s/it]
+ 65%|██████▍   | 1175/1813 [2:08:34<1:04:02,  6.02s/it]
+ 65%|██████▍   | 1176/1813 [2:08:41<1:04:38,  6.09s/it]
+ 65%|██████▍   | 1177/1813 [2:08:46<1:03:08,  5.96s/it]
+ 65%|██████▍   | 1178/1813 [2:08:52<1:02:19,  5.89s/it]
+ 65%|██████▌   | 1179/1813 [2:08:58<1:02:10,  5.88s/it]
+ 65%|██████▌   | 1180/1813 [2:09:05<1:04:23,  6.10s/it]
+                                                       
+{'loss': '0.87', 'grad_norm': '0.9957', 'learning_rate': '5.924e-05', 'epoch': '0.651', 'num_input_tokens_seen': 7914072, 'train_runtime': '7745', 'train_tokens_per_second': '1022'}
+
+ 65%|██████▌   | 1180/1813 [2:09:05<1:04:23,  6.10s/it]
+ 65%|██████▌   | 1181/1813 [2:09:12<1:09:35,  6.61s/it]
+ 65%|██████▌   | 1182/1813 [2:09:20<1:12:55,  6.93s/it]
+ 65%|██████▌   | 1183/1813 [2:09:27<1:12:54,  6.94s/it]
+ 65%|██████▌   | 1184/1813 [2:09:33<1:09:25,  6.62s/it]
+ 65%|██████▌   | 1185/1813 [2:09:41<1:12:48,  6.96s/it]
+ 65%|██████▌   | 1186/1813 [2:09:48<1:13:48,  7.06s/it]
+ 65%|██████▌   | 1187/1813 [2:09:55<1:13:19,  7.03s/it]
+ 66%|██████▌   | 1188/1813 [2:10:00<1:07:52,  6.52s/it]
+ 66%|██████▌   | 1189/1813 [2:10:05<1:03:30,  6.11s/it]
+ 66%|██████▌   | 1190/1813 [2:10:12<1:04:14,  6.19s/it]
+                                                       
+{'loss': '0.934', 'grad_norm': '1.159', 'learning_rate': '5.761e-05', 'epoch': '0.6566', 'num_input_tokens_seen': 7989064, 'train_runtime': '7812', 'train_tokens_per_second': '1023'}
+
+ 66%|██████▌   | 1190/1813 [2:10:12<1:04:14,  6.19s/it]
+ 66%|██████▌   | 1191/1813 [2:10:17<1:02:36,  6.04s/it]
+ 66%|██████▌   | 1192/1813 [2:10:24<1:04:07,  6.20s/it]
+ 66%|██████▌   | 1193/1813 [2:10:30<1:03:40,  6.16s/it]
+ 66%|██████▌   | 1194/1813 [2:10:37<1:05:17,  6.33s/it]
+ 66%|██████▌   | 1195/1813 [2:10:42<1:01:09,  5.94s/it]
+ 66%|██████▌   | 1196/1813 [2:10:48<1:02:44,  6.10s/it]
+ 66%|██████▌   | 1197/1813 [2:10:55<1:03:30,  6.19s/it]
+ 66%|██████▌   | 1198/1813 [2:11:01<1:03:05,  6.16s/it]
+ 66%|██████▌   | 1199/1813 [2:11:07<1:02:23,  6.10s/it]
+ 66%|██████▌   | 1200/1813 [2:11:14<1:06:04,  6.47s/it]
+                                                       
+{'loss': '0.859', 'grad_norm': '0.8802', 'learning_rate': '5.599e-05', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352, 'train_runtime': '7875', 'train_tokens_per_second': '1023'}
+
+ 66%|██████▌   | 1200/1813 [2:11:14<1:06:04,  6.47s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.82it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.25it/s]
+
+  1%|          | 7/1000 [00:00<01:58,  8.35it/s]
+
+  1%|          | 9/1000 [00:01<02:32,  6.49it/s]
+
+  1%|          | 11/1000 [00:01<02:01,  8.11it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.02it/s]
+
+  1%|▏         | 14/1000 [00:01<02:23,  6.85it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.98it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.74it/s]
+
+  2%|▏         | 18/1000 [00:02<02:41,  6.07it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.60it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.37it/s]
+
+  3%|▎         | 26/1000 [00:03<02:35,  6.26it/s]
+
+  3%|▎         | 28/1000 [00:04<02:42,  5.99it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.36it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 31/1000 [00:04<03:00,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  5.99it/s]
+
+  3%|▎         | 33/1000 [00:05<02:48,  5.75it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.11it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.58it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.76it/s]
+
+  4%|▍         | 42/1000 [00:06<01:44,  9.13it/s]
+
+  4%|▍         | 44/1000 [00:06<01:34, 10.17it/s]
+
+  5%|▍         | 46/1000 [00:06<01:46,  8.98it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.34it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.73it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.90it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.91it/s]
+
+  6%|▌         | 55/1000 [00:07<01:28, 10.73it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.31it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.20it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.75it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.37it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.55it/s]
+
+  7%|▋         | 69/1000 [00:09<01:53,  8.24it/s]
+
+  7%|▋         | 70/1000 [00:09<02:13,  6.99it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.71it/s]
+
+  7%|▋         | 74/1000 [00:09<01:54,  8.12it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.67it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.34it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.88it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.22it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.04it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.96it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.26it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.09it/s]
+
+  9%|▉         | 92/1000 [00:12<02:26,  6.18it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.52it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:13,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.73it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.81it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.31it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.44it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.22it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.06it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.54it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.79it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.19it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.09it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.81it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.11it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.72it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.16it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.67it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.90it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:35,  9.03it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.89it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.06it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.42it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.94it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.07it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.42it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.66it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.99it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.63it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.83it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.52it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:26,  5.69it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.19it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:23,  5.79it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.85it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.16it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.24it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.75it/s]
+
+ 18%|█▊        | 176/1000 [00:23<01:37,  8.46it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:37,  8.38it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:37,  8.37it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:02,  6.70it/s]
+
+ 18%|█▊        | 182/1000 [00:24<02:23,  5.69it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:21,  5.76it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.70it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.87it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.13it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:01,  6.64it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.39it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.77it/s]
+
+ 20%|█▉        | 198/1000 [00:26<01:26,  9.22it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.70it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.79it/s]
+
+ 20%|██        | 204/1000 [00:26<01:31,  8.66it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.96it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.53it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.68it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.76it/s]
+
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.17it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.51it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.79it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.55it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:07,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:30<01:53,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.38it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.05it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.23it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.96it/s]
+
+ 24%|██▎       | 237/1000 [00:31<01:20,  9.48it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:26,  8.76it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:32,  8.25it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:06,  5.97it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:19,  5.43it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.63it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.24it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.86it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.03it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:25,  8.73it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.68it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.33it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.59it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.91it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.04it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:13,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.71it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.71it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.08it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.83it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.63it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.04it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.72it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.72it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+
+ 29%|██▉       | 291/1000 [00:39<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.19it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.89it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:37,  7.14it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.78it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.09it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.15it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.96it/s]
+
+ 31%|███       | 307/1000 [00:41<01:41,  6.80it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.81it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.22it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.28it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.60it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:33,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.45it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.76it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.71it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.42it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.18it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.58it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:45<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.81it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.03it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.88it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:46<01:24,  7.73it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.23it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.43it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:33,  6.93it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:48,  5.97it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.65it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.57it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.37it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.98it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.62it/s]
+
+ 36%|███▌      | 361/1000 [00:47<01:07,  9.45it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.20it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.30it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:08,  4.93it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.92it/s]
+
+ 38%|███▊      | 380/1000 [00:51<01:10,  8.77it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.91it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.21it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.19it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.14it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.65it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.38it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.80it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.47it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.75it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.24it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.68it/s]
+
+ 40%|████      | 402/1000 [00:54<01:41,  5.92it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.50it/s]
+
+ 41%|████      | 408/1000 [00:54<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.44it/s]
+
+ 41%|████      | 411/1000 [00:55<01:41,  5.83it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.35it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.04it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:11,  8.08it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:09,  8.34it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.44it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.66it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.91it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.36it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.94it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.54it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.55it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:53, 10.52it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:49, 11.47it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:54, 10.37it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.04it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.26it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:09,  8.07it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.24it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.20it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:37,  5.60it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.51it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.80it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.26it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.84it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.64it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.02it/s]
+
+ 47%|████▋     | 467/1000 [01:03<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.32it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.86it/s]
+
+ 47%|████▋     | 472/1000 [01:03<01:04,  8.21it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:11,  7.38it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:24,  6.19it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:24,  6.19it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:38,  5.29it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.09it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.52it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.06it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.29it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.00it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.26it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.97it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:07<00:59,  8.44it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.56it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:12,  6.82it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.57it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:25,  5.81it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.43it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.93it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:49,  9.95it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.07it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.04it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.56it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.60it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.55it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:48,  9.72it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.71it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.66it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.77it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.06it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.51it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.70it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:15<01:01,  7.26it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.96it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.79it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:03,  7.03it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.85it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.04it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.39it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.99it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.46it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.46it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.31it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.84it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.26it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.56it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.93it/s]
+
+ 59%|█████▊    | 587/1000 [01:20<00:57,  7.16it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.55it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.76it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.19it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.81it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.41it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.90it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.69it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+
+ 60%|██████    | 601/1000 [01:22<01:02,  6.41it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.72it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:45,  8.48it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.75it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.82it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.75it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.10it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.04it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.22it/s]
+
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.20it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.23it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.87it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.43it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.73it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.08it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.96it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.19it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.97it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.61it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.57it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.33it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.71it/s]
+
+ 66%|██████▌   | 656/1000 [01:29<00:38,  8.92it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.57it/s]
+
+ 66%|██████▌   | 662/1000 [01:30<00:52,  6.49it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.34it/s]
+
+ 67%|██████▋   | 667/1000 [01:31<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.21it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.65it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.70it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.94it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.90it/s]
+
+ 68%|██████▊   | 685/1000 [01:33<00:31,  9.87it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:33<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.27it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.06it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.61it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:35<00:40,  7.23it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.43it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.50it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.72it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.88it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.38it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.07it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.15it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.30it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.13it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:23, 11.60it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.53it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:40<00:27,  9.51it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.76it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.31it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.37it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.62it/s]
+
+ 75%|███████▍  | 747/1000 [01:41<00:24, 10.47it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:28,  8.67it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.71it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:42<00:34,  7.25it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.24it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:37,  6.43it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.62it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.32it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.73it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.68it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.62it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.77it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.13it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.78it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.62it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.95it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:26,  8.38it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.17it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.54it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.38it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.01it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.74it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+
+ 80%|███████▉  | 797/1000 [01:47<00:29,  6.86it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.71it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.63it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:25,  7.82it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  8.98it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.85it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.12it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.16it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.03it/s]
+
+ 82%|████████▏ | 815/1000 [01:49<00:19,  9.29it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.14it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.20it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.06it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.81it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.03it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.76it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:27,  6.16it/s]
+
+ 83%|████████▎ | 834/1000 [01:53<00:27,  5.97it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.50it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.10it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.44it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.61it/s]
+
+ 84%|████████▍ | 842/1000 [01:54<00:19,  8.12it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.41it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.77it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.25it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.54it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.88it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.48it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.52it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.63it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.73it/s]
+
+ 87%|████████▋ | 867/1000 [01:57<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.28it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.31it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.79it/s]
+
+ 88%|████████▊ | 875/1000 [01:58<00:16,  7.64it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.89it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.88it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.95it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.85it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.71it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.87it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.08it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.06it/s]
+
+ 91%|█████████ | 908/1000 [02:02<00:11,  8.32it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.60it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.76it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.65it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.09it/s]
+
+ 93%|█████████▎| 928/1000 [02:05<00:10,  6.66it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.41it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.77it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.59it/s]
+
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.15it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.29it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.82it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.06it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.16it/s]
+
+ 94%|█████████▍| 942/1000 [02:07<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.29it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.63it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.80it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.40it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.14it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.37it/s]
+
+ 96%|█████████▌| 957/1000 [02:09<00:06,  6.39it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.76it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.75it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  5.99it/s]
+
+ 96%|█████████▋| 964/1000 [02:10<00:04,  7.35it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.37it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.28it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:04,  7.95it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.37it/s]
+
+ 97%|█████████▋| 972/1000 [02:11<00:03,  7.99it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.17it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.78it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.44it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.06it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.45it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.39it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.01it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.98it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.54it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.68it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.60it/s]
+                                                       
+
+                                                   
+{'eval_loss': '0.7542', 'eval_runtime': '135.3', 'eval_samples_per_second': '7.39', 'eval_steps_per_second': '7.39', 'epoch': '0.6621', 'num_input_tokens_seen': 8055352}
+
+ 66%|██████▌   | 1200/1813 [2:13:29<1:06:04,  6.47s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.60it/s]
+
+                                                   
+ 66%|██████▌   | 1201/1813 [2:13:35<7:56:09, 46.68s/it]
+ 66%|██████▋   | 1202/1813 [2:13:41<5:53:02, 34.67s/it]
+ 66%|██████▋   | 1203/1813 [2:13:48<4:26:06, 26.17s/it]
+ 66%|██████▋   | 1204/1813 [2:13:54<3:26:20, 20.33s/it]
+ 66%|██████▋   | 1205/1813 [2:14:01<2:44:29, 16.23s/it]
+ 67%|██████▋   | 1206/1813 [2:14:07<2:13:51, 13.23s/it]
+ 67%|██████▋   | 1207/1813 [2:14:13<1:52:11, 11.11s/it]
+ 67%|██████▋   | 1208/1813 [2:14:21<1:40:55, 10.01s/it]
+ 67%|██████▋   | 1209/1813 [2:14:28<1:30:50,  9.02s/it]
+ 67%|██████▋   | 1210/1813 [2:14:33<1:20:58,  8.06s/it]
+                                                       
+{'loss': '0.9206', 'grad_norm': '1.031', 'learning_rate': '5.438e-05', 'epoch': '0.6676', 'num_input_tokens_seen': 8125224, 'train_runtime': '8074', 'train_tokens_per_second': '1006'}
+
+ 67%|██████▋   | 1210/1813 [2:14:33<1:20:58,  8.06s/it]
+ 67%|██████▋   | 1211/1813 [2:14:40<1:17:15,  7.70s/it]
+ 67%|██████▋   | 1212/1813 [2:14:45<1:07:01,  6.69s/it]
+ 67%|██████▋   | 1213/1813 [2:14:51<1:06:14,  6.62s/it]
+ 67%|██████▋   | 1214/1813 [2:14:59<1:11:20,  7.15s/it]
+ 67%|██████▋   | 1215/1813 [2:15:06<1:08:42,  6.89s/it]
+ 67%|██████▋   | 1216/1813 [2:15:13<1:09:15,  6.96s/it]
+ 67%|██████▋   | 1217/1813 [2:15:19<1:06:04,  6.65s/it]
+ 67%|██████▋   | 1218/1813 [2:15:25<1:04:31,  6.51s/it]
+ 67%|██████▋   | 1219/1813 [2:15:30<1:01:11,  6.18s/it]
+ 67%|██████▋   | 1220/1813 [2:15:39<1:07:08,  6.79s/it]
+                                                       
+{'loss': '0.9264', 'grad_norm': '3.5', 'learning_rate': '5.279e-05', 'epoch': '0.6731', 'num_input_tokens_seen': 8197488, 'train_runtime': '8139', 'train_tokens_per_second': '1007'}
+
+ 67%|██████▋   | 1220/1813 [2:15:39<1:07:08,  6.79s/it]
+ 67%|██████▋   | 1221/1813 [2:15:44<1:03:35,  6.44s/it]
+ 67%|██████▋   | 1222/1813 [2:15:49<59:53,  6.08s/it]  
+ 67%|██████▋   | 1223/1813 [2:15:55<59:06,  6.01s/it]
+ 68%|██████▊   | 1224/1813 [2:16:00<56:34,  5.76s/it]
+ 68%|██████▊   | 1225/1813 [2:16:07<57:57,  5.91s/it]
+ 68%|██████▊   | 1226/1813 [2:16:13<58:21,  5.97s/it]
+ 68%|██████▊   | 1227/1813 [2:16:19<59:41,  6.11s/it]
+ 68%|██████▊   | 1228/1813 [2:16:24<56:29,  5.79s/it]
+ 68%|██████▊   | 1229/1813 [2:16:31<58:06,  5.97s/it]
+ 68%|██████▊   | 1230/1813 [2:16:37<57:46,  5.95s/it]
+                                                     
+{'loss': '0.7974', 'grad_norm': '0.8714', 'learning_rate': '5.121e-05', 'epoch': '0.6786', 'num_input_tokens_seen': 8257288, 'train_runtime': '8197', 'train_tokens_per_second': '1007'}
+
+ 68%|██████▊   | 1230/1813 [2:16:37<57:46,  5.95s/it]
+ 68%|██████▊   | 1231/1813 [2:16:44<1:02:01,  6.39s/it]
+ 68%|██████▊   | 1232/1813 [2:16:51<1:04:17,  6.64s/it]
+ 68%|██████▊   | 1233/1813 [2:16:57<1:02:40,  6.48s/it]
+ 68%|██████▊   | 1234/1813 [2:17:03<1:01:36,  6.38s/it]
+ 68%|██████▊   | 1235/1813 [2:17:10<1:00:52,  6.32s/it]
+ 68%|██████▊   | 1236/1813 [2:17:18<1:07:27,  7.02s/it]
+ 68%|██████▊   | 1237/1813 [2:17:25<1:05:49,  6.86s/it]
+ 68%|██████▊   | 1238/1813 [2:17:30<1:02:16,  6.50s/it]
+ 68%|██████▊   | 1239/1813 [2:17:36<58:53,  6.16s/it]  
+ 68%|██████▊   | 1240/1813 [2:17:41<56:12,  5.88s/it]
+                                                     
+{'loss': '0.7968', 'grad_norm': '0.9342', 'learning_rate': '4.964e-05', 'epoch': '0.6841', 'num_input_tokens_seen': 8327600, 'train_runtime': '8262', 'train_tokens_per_second': '1008'}
+
+ 68%|██████▊   | 1240/1813 [2:17:41<56:12,  5.88s/it]
+ 68%|██████▊   | 1241/1813 [2:17:47<55:12,  5.79s/it]
+ 69%|██████▊   | 1242/1813 [2:17:53<55:28,  5.83s/it]
+ 69%|██████▊   | 1243/1813 [2:17:59<56:19,  5.93s/it]
+ 69%|██████▊   | 1244/1813 [2:18:06<59:14,  6.25s/it]
+ 69%|██████▊   | 1245/1813 [2:18:13<1:01:51,  6.53s/it]
+ 69%|██████▊   | 1246/1813 [2:18:19<59:34,  6.30s/it]  
+ 69%|██████▉   | 1247/1813 [2:18:25<1:00:44,  6.44s/it]
+ 69%|██████▉   | 1248/1813 [2:18:32<1:01:31,  6.53s/it]
+ 69%|██████▉   | 1249/1813 [2:18:38<1:00:23,  6.42s/it]
+ 69%|██████▉   | 1250/1813 [2:18:43<55:00,  5.86s/it]  
+                                                     
+{'loss': '0.9414', 'grad_norm': '1.299', 'learning_rate': '4.81e-05', 'epoch': '0.6897', 'num_input_tokens_seen': 8394824, 'train_runtime': '8323', 'train_tokens_per_second': '1009'}
+
+ 69%|██████▉   | 1250/1813 [2:18:43<55:00,  5.86s/it]
+ 69%|██████▉   | 1251/1813 [2:18:49<55:05,  5.88s/it]
+ 69%|██████▉   | 1252/1813 [2:18:55<54:32,  5.83s/it]
+ 69%|██████▉   | 1253/1813 [2:19:00<54:26,  5.83s/it]
+ 69%|██████▉   | 1254/1813 [2:19:07<55:34,  5.96s/it]
+ 69%|██████▉   | 1255/1813 [2:19:13<55:29,  5.97s/it]
+ 69%|██████▉   | 1256/1813 [2:19:18<55:07,  5.94s/it]
+ 69%|██████▉   | 1257/1813 [2:19:24<54:22,  5.87s/it]
+ 69%|██████▉   | 1258/1813 [2:19:31<56:50,  6.15s/it]
+ 69%|██████▉   | 1259/1813 [2:19:38<58:19,  6.32s/it]
+ 69%|██████▉   | 1260/1813 [2:19:45<1:00:37,  6.58s/it]
+                                                       
+{'loss': '0.9102', 'grad_norm': '1.223', 'learning_rate': '4.656e-05', 'epoch': '0.6952', 'num_input_tokens_seen': 8461096, 'train_runtime': '8385', 'train_tokens_per_second': '1009'}
+
+ 69%|██████▉   | 1260/1813 [2:19:45<1:00:37,  6.58s/it]
+ 70%|██████▉   | 1261/1813 [2:19:53<1:04:53,  7.05s/it]
+ 70%|██████▉   | 1262/1813 [2:20:00<1:05:48,  7.17s/it]
+ 70%|██████▉   | 1263/1813 [2:20:08<1:05:49,  7.18s/it]
+ 70%|██████▉   | 1264/1813 [2:20:13<1:01:47,  6.75s/it]
+ 70%|██████▉   | 1265/1813 [2:20:19<57:24,  6.29s/it]  
+ 70%|██████▉   | 1266/1813 [2:20:26<1:01:17,  6.72s/it]
+ 70%|██████▉   | 1267/1813 [2:20:31<56:42,  6.23s/it]  
+ 70%|██████▉   | 1268/1813 [2:20:36<52:44,  5.81s/it]
+ 70%|██████▉   | 1269/1813 [2:20:42<51:33,  5.69s/it]
+ 70%|███████   | 1270/1813 [2:20:48<52:57,  5.85s/it]
+                                                     
+{'loss': '0.8934', 'grad_norm': '1.042', 'learning_rate': '4.505e-05', 'epoch': '0.7007', 'num_input_tokens_seen': 8529224, 'train_runtime': '8448', 'train_tokens_per_second': '1010'}
+
+ 70%|███████   | 1270/1813 [2:20:48<52:57,  5.85s/it]
+ 70%|███████   | 1271/1813 [2:20:53<50:45,  5.62s/it]
+ 70%|███████   | 1272/1813 [2:21:00<54:40,  6.06s/it]
+ 70%|███████   | 1273/1813 [2:21:07<56:28,  6.28s/it]
+ 70%|███████   | 1274/1813 [2:21:14<58:56,  6.56s/it]
+ 70%|███████   | 1275/1813 [2:21:21<59:09,  6.60s/it]
+ 70%|███████   | 1276/1813 [2:21:26<56:32,  6.32s/it]
+ 70%|███████   | 1277/1813 [2:21:34<58:31,  6.55s/it]
+ 70%|███████   | 1278/1813 [2:21:39<56:11,  6.30s/it]
+ 71%|███████   | 1279/1813 [2:21:44<52:18,  5.88s/it]
+ 71%|███████   | 1280/1813 [2:21:51<55:23,  6.23s/it]
+                                                     
+{'loss': '0.9203', 'grad_norm': '1.228', 'learning_rate': '4.355e-05', 'epoch': '0.7062', 'num_input_tokens_seen': 8596456, 'train_runtime': '8512', 'train_tokens_per_second': '1010'}
+
+ 71%|███████   | 1280/1813 [2:21:51<55:23,  6.23s/it]
+ 71%|███████   | 1281/1813 [2:21:56<52:40,  5.94s/it]
+ 71%|███████   | 1282/1813 [2:22:03<53:23,  6.03s/it]
+ 71%|███████   | 1283/1813 [2:22:09<55:03,  6.23s/it]
+ 71%|███████   | 1284/1813 [2:22:15<53:38,  6.08s/it]
+ 71%|███████   | 1285/1813 [2:22:21<51:41,  5.87s/it]
+ 71%|███████   | 1286/1813 [2:22:28<55:14,  6.29s/it]
+ 71%|███████   | 1287/1813 [2:22:34<54:19,  6.20s/it]
+ 71%|███████   | 1288/1813 [2:22:39<52:06,  5.96s/it]
+ 71%|███████   | 1289/1813 [2:22:46<54:30,  6.24s/it]
+ 71%|███████   | 1290/1813 [2:22:53<57:04,  6.55s/it]
+                                                     
+{'loss': '0.9681', 'grad_norm': '0.7653', 'learning_rate': '4.207e-05', 'epoch': '0.7117', 'num_input_tokens_seen': 8662472, 'train_runtime': '8574', 'train_tokens_per_second': '1010'}
+
+ 71%|███████   | 1290/1813 [2:22:53<57:04,  6.55s/it]
+ 71%|███████   | 1291/1813 [2:22:59<55:58,  6.43s/it]
+ 71%|███████▏  | 1292/1813 [2:23:06<56:49,  6.54s/it]
+ 71%|███████▏  | 1293/1813 [2:23:13<57:54,  6.68s/it]
+ 71%|███████▏  | 1294/1813 [2:23:20<56:36,  6.54s/it]
+ 71%|███████▏  | 1295/1813 [2:23:25<55:01,  6.37s/it]
+ 71%|███████▏  | 1296/1813 [2:23:31<52:37,  6.11s/it]
+ 72%|███████▏  | 1297/1813 [2:23:36<50:20,  5.85s/it]
+ 72%|███████▏  | 1298/1813 [2:23:43<52:00,  6.06s/it]
+ 72%|███████▏  | 1299/1813 [2:23:50<54:03,  6.31s/it]
+ 72%|███████▏  | 1300/1813 [2:23:56<53:17,  6.23s/it]
+                                                     
+{'loss': '0.8606', 'grad_norm': '1.126', 'learning_rate': '4.06e-05', 'epoch': '0.7172', 'num_input_tokens_seen': 8730600, 'train_runtime': '8636', 'train_tokens_per_second': '1011'}
+
+ 72%|███████▏  | 1300/1813 [2:23:56<53:17,  6.23s/it]
+ 72%|███████▏  | 1301/1813 [2:24:01<51:48,  6.07s/it]
+ 72%|███████▏  | 1302/1813 [2:24:08<53:31,  6.28s/it]
+ 72%|███████▏  | 1303/1813 [2:24:16<57:11,  6.73s/it]
+ 72%|███████▏  | 1304/1813 [2:24:21<53:36,  6.32s/it]
+ 72%|███████▏  | 1305/1813 [2:24:29<56:19,  6.65s/it]
+ 72%|███████▏  | 1306/1813 [2:24:35<54:00,  6.39s/it]
+ 72%|███████▏  | 1307/1813 [2:24:41<53:23,  6.33s/it]
+ 72%|███████▏  | 1308/1813 [2:24:47<53:28,  6.35s/it]
+ 72%|███████▏  | 1309/1813 [2:24:53<53:03,  6.32s/it]
+ 72%|███████▏  | 1310/1813 [2:25:00<54:29,  6.50s/it]
+                                                     
+{'loss': '0.9519', 'grad_norm': '1.012', 'learning_rate': '3.916e-05', 'epoch': '0.7228', 'num_input_tokens_seen': 8802000, 'train_runtime': '8701', 'train_tokens_per_second': '1012'}
+
+ 72%|███████▏  | 1310/1813 [2:25:00<54:29,  6.50s/it]
+ 72%|███████▏  | 1311/1813 [2:25:06<53:30,  6.39s/it]
+ 72%|███████▏  | 1312/1813 [2:25:12<51:15,  6.14s/it]
+ 72%|███████▏  | 1313/1813 [2:25:18<50:37,  6.08s/it]
+ 72%|███████▏  | 1314/1813 [2:25:24<50:54,  6.12s/it]
+ 73%|███████▎  | 1315/1813 [2:25:30<49:47,  6.00s/it]
+ 73%|███████▎  | 1316/1813 [2:25:36<49:23,  5.96s/it]
+ 73%|███████▎  | 1317/1813 [2:25:42<49:19,  5.97s/it]
+ 73%|███████▎  | 1318/1813 [2:25:47<48:07,  5.83s/it]
+ 73%|███████▎  | 1319/1813 [2:25:54<50:01,  6.08s/it]
+ 73%|███████▎  | 1320/1813 [2:26:01<51:46,  6.30s/it]
+                                                     
+{'loss': '0.9269', 'grad_norm': '0.8949', 'learning_rate': '3.773e-05', 'epoch': '0.7283', 'num_input_tokens_seen': 8866824, 'train_runtime': '8761', 'train_tokens_per_second': '1012'}
+
+ 73%|███████▎  | 1320/1813 [2:26:01<51:46,  6.30s/it]
+ 73%|███████▎  | 1321/1813 [2:26:07<52:48,  6.44s/it]
+ 73%|███████▎  | 1322/1813 [2:26:14<51:44,  6.32s/it]
+ 73%|███████▎  | 1323/1813 [2:26:20<52:24,  6.42s/it]
+ 73%|███████▎  | 1324/1813 [2:26:26<50:44,  6.23s/it]
+ 73%|███████▎  | 1325/1813 [2:26:32<49:23,  6.07s/it]
+ 73%|███████▎  | 1326/1813 [2:26:38<49:51,  6.14s/it]
+ 73%|███████▎  | 1327/1813 [2:26:46<53:37,  6.62s/it]
+ 73%|███████▎  | 1328/1813 [2:26:53<54:25,  6.73s/it]
+ 73%|███████▎  | 1329/1813 [2:26:59<53:01,  6.57s/it]
+ 73%|███████▎  | 1330/1813 [2:27:05<51:02,  6.34s/it]
+                                                     
+{'loss': '0.8767', 'grad_norm': '1.204', 'learning_rate': '3.633e-05', 'epoch': '0.7338', 'num_input_tokens_seen': 8935584, 'train_runtime': '8825', 'train_tokens_per_second': '1013'}
+
+ 73%|███████▎  | 1330/1813 [2:27:05<51:02,  6.34s/it]
+ 73%|███████▎  | 1331/1813 [2:27:10<49:38,  6.18s/it]
+ 73%|███████▎  | 1332/1813 [2:27:16<48:48,  6.09s/it]
+ 74%|███████▎  | 1333/1813 [2:27:22<48:22,  6.05s/it]
+ 74%|███████▎  | 1334/1813 [2:27:28<46:58,  5.88s/it]
+ 74%|███████▎  | 1335/1813 [2:27:33<45:44,  5.74s/it]
+ 74%|███████▎  | 1336/1813 [2:27:39<46:38,  5.87s/it]
+ 74%|███████▎  | 1337/1813 [2:27:47<50:30,  6.37s/it]
+ 74%|███████▍  | 1338/1813 [2:27:54<52:19,  6.61s/it]
+ 74%|███████▍  | 1339/1813 [2:27:59<48:33,  6.15s/it]
+ 74%|███████▍  | 1340/1813 [2:28:06<50:16,  6.38s/it]
+                                                     
+{'loss': '0.9722', 'grad_norm': '0.9797', 'learning_rate': '3.494e-05', 'epoch': '0.7393', 'num_input_tokens_seen': 9000416, 'train_runtime': '8887', 'train_tokens_per_second': '1013'}
+
+ 74%|███████▍  | 1340/1813 [2:28:06<50:16,  6.38s/it]
+ 74%|███████▍  | 1341/1813 [2:28:13<50:29,  6.42s/it]
+ 74%|███████▍  | 1342/1813 [2:28:18<47:32,  6.06s/it]
+ 74%|███████▍  | 1343/1813 [2:28:23<46:34,  5.95s/it]
+ 74%|███████▍  | 1344/1813 [2:28:30<46:40,  5.97s/it]
+ 74%|███████▍  | 1345/1813 [2:28:37<50:38,  6.49s/it]
+ 74%|███████▍  | 1346/1813 [2:28:44<50:49,  6.53s/it]
+ 74%|███████▍  | 1347/1813 [2:28:50<48:59,  6.31s/it]
+ 74%|███████▍  | 1348/1813 [2:28:56<49:13,  6.35s/it]
+ 74%|███████▍  | 1349/1813 [2:29:03<50:04,  6.48s/it]
+ 74%|███████▍  | 1350/1813 [2:29:09<48:05,  6.23s/it]
+                                                     
+{'loss': '0.9188', 'grad_norm': '1.067', 'learning_rate': '3.358e-05', 'epoch': '0.7448', 'num_input_tokens_seen': 9066864, 'train_runtime': '8949', 'train_tokens_per_second': '1013'}
+
+ 74%|███████▍  | 1350/1813 [2:29:09<48:05,  6.23s/it]
+ 75%|███████▍  | 1351/1813 [2:29:16<51:43,  6.72s/it]
+ 75%|███████▍  | 1352/1813 [2:29:23<50:32,  6.58s/it]
+ 75%|███████▍  | 1353/1813 [2:29:30<51:19,  6.69s/it]
+ 75%|███████▍  | 1354/1813 [2:29:36<51:37,  6.75s/it]
+ 75%|███████▍  | 1355/1813 [2:29:45<55:47,  7.31s/it]
+ 75%|███████▍  | 1356/1813 [2:29:51<52:32,  6.90s/it]
+ 75%|███████▍  | 1357/1813 [2:29:57<50:27,  6.64s/it]
+ 75%|███████▍  | 1358/1813 [2:30:03<49:53,  6.58s/it]
+ 75%|███████▍  | 1359/1813 [2:30:09<48:17,  6.38s/it]
+ 75%|███████▌  | 1360/1813 [2:30:16<47:41,  6.32s/it]
+                                                     
+{'loss': '0.8702', 'grad_norm': '1.209', 'learning_rate': '3.224e-05', 'epoch': '0.7503', 'num_input_tokens_seen': 9140520, 'train_runtime': '9016', 'train_tokens_per_second': '1014'}
+
+ 75%|███████▌  | 1360/1813 [2:30:16<47:41,  6.32s/it]
+ 75%|███████▌  | 1361/1813 [2:30:23<48:57,  6.50s/it]
+ 75%|███████▌  | 1362/1813 [2:30:29<48:30,  6.45s/it]
+ 75%|███████▌  | 1363/1813 [2:30:36<50:30,  6.74s/it]
+ 75%|███████▌  | 1364/1813 [2:30:42<48:38,  6.50s/it]
+ 75%|███████▌  | 1365/1813 [2:30:50<50:41,  6.79s/it]
+ 75%|███████▌  | 1366/1813 [2:30:56<48:38,  6.53s/it]
+ 75%|███████▌  | 1367/1813 [2:31:02<48:46,  6.56s/it]
+ 75%|███████▌  | 1368/1813 [2:31:07<45:35,  6.15s/it]
+ 76%|███████▌  | 1369/1813 [2:31:15<49:38,  6.71s/it]
+ 76%|███████▌  | 1370/1813 [2:31:21<47:51,  6.48s/it]
+                                                     
+{'loss': '0.8544', 'grad_norm': '0.9758', 'learning_rate': '3.092e-05', 'epoch': '0.7559', 'num_input_tokens_seen': 9213048, 'train_runtime': '9082', 'train_tokens_per_second': '1014'}
+
+ 76%|███████▌  | 1370/1813 [2:31:21<47:51,  6.48s/it]
+ 76%|███████▌  | 1371/1813 [2:31:29<51:13,  6.95s/it]
+ 76%|███████▌  | 1372/1813 [2:31:35<47:30,  6.46s/it]
+ 76%|███████▌  | 1373/1813 [2:31:41<46:45,  6.38s/it]
+ 76%|███████▌  | 1374/1813 [2:31:47<45:56,  6.28s/it]
+ 76%|███████▌  | 1375/1813 [2:31:53<45:09,  6.19s/it]
+ 76%|███████▌  | 1376/1813 [2:31:58<43:39,  5.99s/it]
+ 76%|███████▌  | 1377/1813 [2:32:05<44:09,  6.08s/it]
+ 76%|███████▌  | 1378/1813 [2:32:10<43:13,  5.96s/it]
+ 76%|███████▌  | 1379/1813 [2:32:16<43:02,  5.95s/it]
+ 76%|███████▌  | 1380/1813 [2:32:23<45:25,  6.30s/it]
+                                                     
+{'loss': '0.846', 'grad_norm': '1.103', 'learning_rate': '2.962e-05', 'epoch': '0.7614', 'num_input_tokens_seen': 9279088, 'train_runtime': '9144', 'train_tokens_per_second': '1015'}
+
+ 76%|███████▌  | 1380/1813 [2:32:23<45:25,  6.30s/it]
+ 76%|███████▌  | 1381/1813 [2:32:31<46:58,  6.53s/it]
+ 76%|███████▌  | 1382/1813 [2:32:36<44:43,  6.23s/it]
+ 76%|███████▋  | 1383/1813 [2:32:42<44:25,  6.20s/it]
+ 76%|███████▋  | 1384/1813 [2:32:48<43:30,  6.09s/it]
+ 76%|███████▋  | 1385/1813 [2:32:55<45:04,  6.32s/it]
+ 76%|███████▋  | 1386/1813 [2:33:01<45:08,  6.34s/it]
+ 77%|███████▋  | 1387/1813 [2:33:08<46:28,  6.55s/it]
+ 77%|███████▋  | 1388/1813 [2:33:14<44:20,  6.26s/it]
+ 77%|███████▋  | 1389/1813 [2:33:18<40:39,  5.75s/it]
+ 77%|███████▋  | 1390/1813 [2:33:25<43:07,  6.12s/it]
+                                                     
+{'loss': '0.9365', 'grad_norm': '1.031', 'learning_rate': '2.834e-05', 'epoch': '0.7669', 'num_input_tokens_seen': 9344880, 'train_runtime': '9206', 'train_tokens_per_second': '1015'}
+
+ 77%|███████▋  | 1390/1813 [2:33:25<43:07,  6.12s/it]
+ 77%|███████▋  | 1391/1813 [2:33:31<41:41,  5.93s/it]
+ 77%|███████▋  | 1392/1813 [2:33:38<43:21,  6.18s/it]
+ 77%|███████▋  | 1393/1813 [2:33:43<41:50,  5.98s/it]
+ 77%|███████▋  | 1394/1813 [2:33:50<43:20,  6.21s/it]
+ 77%|███████▋  | 1395/1813 [2:33:56<43:37,  6.26s/it]
+ 77%|███████▋  | 1396/1813 [2:34:03<43:25,  6.25s/it]
+ 77%|███████▋  | 1397/1813 [2:34:09<44:36,  6.43s/it]
+ 77%|███████▋  | 1398/1813 [2:34:14<40:52,  5.91s/it]
+ 77%|███████▋  | 1399/1813 [2:34:19<38:53,  5.64s/it]
+ 77%|███████▋  | 1400/1813 [2:34:26<41:01,  5.96s/it]
+                                                     
+{'loss': '0.8794', 'grad_norm': '0.8734', 'learning_rate': '2.709e-05', 'epoch': '0.7724', 'num_input_tokens_seen': 9408280, 'train_runtime': '9266', 'train_tokens_per_second': '1015'}
+
+ 77%|███████▋  | 1400/1813 [2:34:26<41:01,  5.96s/it]
+ 77%|███████▋  | 1401/1813 [2:34:31<40:00,  5.83s/it]
+ 77%|███████▋  | 1402/1813 [2:34:38<40:53,  5.97s/it]
+ 77%|███████▋  | 1403/1813 [2:34:44<42:20,  6.20s/it]
+ 77%|███████▋  | 1404/1813 [2:34:50<42:06,  6.18s/it]
+ 77%|███████▋  | 1405/1813 [2:34:55<38:57,  5.73s/it]
+ 78%|███████▊  | 1406/1813 [2:35:01<38:45,  5.71s/it]
+ 78%|███████▊  | 1407/1813 [2:35:08<40:41,  6.01s/it]
+ 78%|███████▊  | 1408/1813 [2:35:13<39:17,  5.82s/it]
+ 78%|███████▊  | 1409/1813 [2:35:18<37:20,  5.55s/it]
+ 78%|███████▊  | 1410/1813 [2:35:24<38:01,  5.66s/it]
+                                                     
+{'loss': '0.9228', 'grad_norm': '1.03', 'learning_rate': '2.586e-05', 'epoch': '0.7779', 'num_input_tokens_seen': 9467408, 'train_runtime': '9324', 'train_tokens_per_second': '1015'}
+
+ 78%|███████▊  | 1410/1813 [2:35:24<38:01,  5.66s/it]
+ 78%|███████▊  | 1411/1813 [2:35:29<37:08,  5.54s/it]
+ 78%|███████▊  | 1412/1813 [2:35:35<38:44,  5.80s/it]
+ 78%|███████▊  | 1413/1813 [2:35:41<38:13,  5.73s/it]
+ 78%|███████▊  | 1414/1813 [2:35:47<38:56,  5.86s/it]
+ 78%|███████▊  | 1415/1813 [2:35:54<41:19,  6.23s/it]
+ 78%|███████▊  | 1416/1813 [2:36:01<41:20,  6.25s/it]
+ 78%|███████▊  | 1417/1813 [2:36:06<39:24,  5.97s/it]
+ 78%|███████▊  | 1418/1813 [2:36:12<38:53,  5.91s/it]
+ 78%|███████▊  | 1419/1813 [2:36:20<44:03,  6.71s/it]
+ 78%|███████▊  | 1420/1813 [2:36:25<39:25,  6.02s/it]
+                                                     
+{'loss': '0.8772', 'grad_norm': '1.106', 'learning_rate': '2.465e-05', 'epoch': '0.7834', 'num_input_tokens_seen': 9532720, 'train_runtime': '9385', 'train_tokens_per_second': '1016'}
+
+ 78%|███████▊  | 1420/1813 [2:36:25<39:25,  6.02s/it]
+ 78%|███████▊  | 1421/1813 [2:36:31<40:48,  6.25s/it]
+ 78%|███████▊  | 1422/1813 [2:36:37<39:21,  6.04s/it]
+ 78%|███████▊  | 1423/1813 [2:36:43<38:30,  5.92s/it]
+ 79%|███████▊  | 1424/1813 [2:36:50<40:59,  6.32s/it]
+ 79%|███████▊  | 1425/1813 [2:36:54<37:22,  5.78s/it]
+ 79%|███████▊  | 1426/1813 [2:37:03<42:35,  6.60s/it]
+ 79%|███████▊  | 1427/1813 [2:37:08<40:05,  6.23s/it]
+ 79%|███████▉  | 1428/1813 [2:37:15<40:40,  6.34s/it]
+ 79%|███████▉  | 1429/1813 [2:37:21<40:27,  6.32s/it]
+ 79%|███████▉  | 1430/1813 [2:37:28<41:34,  6.51s/it]
+                                                     
+{'loss': '0.8426', 'grad_norm': '0.9835', 'learning_rate': '2.347e-05', 'epoch': '0.789', 'num_input_tokens_seen': 9602112, 'train_runtime': '9449', 'train_tokens_per_second': '1016'}
+
+ 79%|███████▉  | 1430/1813 [2:37:28<41:34,  6.51s/it]
+ 79%|███████▉  | 1431/1813 [2:37:34<40:58,  6.44s/it]
+ 79%|███████▉  | 1432/1813 [2:37:40<39:43,  6.26s/it]
+ 79%|███████▉  | 1433/1813 [2:37:46<39:31,  6.24s/it]
+ 79%|███████▉  | 1434/1813 [2:37:53<39:35,  6.27s/it]
+ 79%|███████▉  | 1435/1813 [2:37:58<37:58,  6.03s/it]
+ 79%|███████▉  | 1436/1813 [2:38:05<39:19,  6.26s/it]
+ 79%|███████▉  | 1437/1813 [2:38:11<38:06,  6.08s/it]
+ 79%|███████▉  | 1438/1813 [2:38:18<40:10,  6.43s/it]
+ 79%|███████▉  | 1439/1813 [2:38:23<38:32,  6.18s/it]
+ 79%|███████▉  | 1440/1813 [2:38:29<36:32,  5.88s/it]
+                                                     
+{'loss': '0.8589', 'grad_norm': '0.9937', 'learning_rate': '2.232e-05', 'epoch': '0.7945', 'num_input_tokens_seen': 9667112, 'train_runtime': '9509', 'train_tokens_per_second': '1017'}
+
+ 79%|███████▉  | 1440/1813 [2:38:29<36:32,  5.88s/it]
+ 79%|███████▉  | 1441/1813 [2:38:35<37:45,  6.09s/it]
+ 80%|███████▉  | 1442/1813 [2:38:41<36:19,  5.88s/it]
+ 80%|███████▉  | 1443/1813 [2:38:46<34:58,  5.67s/it]
+ 80%|███████▉  | 1444/1813 [2:38:53<37:09,  6.04s/it]
+ 80%|███████▉  | 1445/1813 [2:38:58<36:25,  5.94s/it]
+ 80%|███████▉  | 1446/1813 [2:39:04<35:47,  5.85s/it]
+ 80%|███████▉  | 1447/1813 [2:39:12<40:05,  6.57s/it]
+ 80%|███████▉  | 1448/1813 [2:39:20<41:05,  6.75s/it]
+ 80%|███████▉  | 1449/1813 [2:39:26<40:30,  6.68s/it]
+ 80%|███████▉  | 1450/1813 [2:39:32<39:56,  6.60s/it]
+                                                     
+{'loss': '0.9391', 'grad_norm': '1.023', 'learning_rate': '2.119e-05', 'epoch': '0.8', 'num_input_tokens_seen': 9735792, 'train_runtime': '9573', 'train_tokens_per_second': '1017'}
+
+ 80%|███████▉  | 1450/1813 [2:39:32<39:56,  6.60s/it]
+ 80%|████████  | 1451/1813 [2:39:39<40:27,  6.71s/it]
+ 80%|████████  | 1452/1813 [2:39:44<36:20,  6.04s/it]
+ 80%|████████  | 1453/1813 [2:39:52<40:08,  6.69s/it]
+ 80%|████████  | 1454/1813 [2:39:57<37:33,  6.28s/it]
+ 80%|████████  | 1455/1813 [2:40:04<37:14,  6.24s/it]
+ 80%|████████  | 1456/1813 [2:40:11<38:40,  6.50s/it]
+ 80%|████████  | 1457/1813 [2:40:15<33:56,  5.72s/it]
+ 80%|████████  | 1458/1813 [2:40:20<33:28,  5.66s/it]
+ 80%|████████  | 1459/1813 [2:40:27<35:43,  6.06s/it]
+ 81%|████████  | 1460/1813 [2:40:34<37:34,  6.39s/it]
+                                                     
+{'loss': '0.8958', 'grad_norm': '1.024', 'learning_rate': '2.008e-05', 'epoch': '0.8055', 'num_input_tokens_seen': 9802968, 'train_runtime': '9635', 'train_tokens_per_second': '1017'}
+
+ 81%|████████  | 1460/1813 [2:40:34<37:34,  6.39s/it]
+ 81%|████████  | 1461/1813 [2:40:40<36:30,  6.22s/it]
+ 81%|████████  | 1462/1813 [2:40:45<34:30,  5.90s/it]
+ 81%|████████  | 1463/1813 [2:40:53<37:02,  6.35s/it]
+ 81%|████████  | 1464/1813 [2:40:59<37:49,  6.50s/it]
+ 81%|████████  | 1465/1813 [2:41:08<40:51,  7.05s/it]
+ 81%|████████  | 1466/1813 [2:41:13<38:28,  6.65s/it]
+ 81%|████████  | 1467/1813 [2:41:21<39:46,  6.90s/it]
+ 81%|████████  | 1468/1813 [2:41:27<39:01,  6.79s/it]
+ 81%|████████  | 1469/1813 [2:41:36<42:11,  7.36s/it]
+ 81%|████████  | 1470/1813 [2:41:43<40:35,  7.10s/it]
+                                                     
+{'loss': '0.9034', 'grad_norm': '0.9483', 'learning_rate': '1.9e-05', 'epoch': '0.811', 'num_input_tokens_seen': 9879232, 'train_runtime': '9703', 'train_tokens_per_second': '1018'}
+
+ 81%|████████  | 1470/1813 [2:41:43<40:35,  7.10s/it]
+ 81%|████████  | 1471/1813 [2:41:49<38:45,  6.80s/it]
+ 81%|████████  | 1472/1813 [2:41:55<38:22,  6.75s/it]
+ 81%|████████  | 1473/1813 [2:42:01<36:26,  6.43s/it]
+ 81%|████████▏ | 1474/1813 [2:42:08<36:46,  6.51s/it]
+ 81%|████████▏ | 1475/1813 [2:42:14<36:24,  6.46s/it]
+ 81%|████████▏ | 1476/1813 [2:42:20<35:53,  6.39s/it]
+ 81%|████████▏ | 1477/1813 [2:42:27<35:33,  6.35s/it]
+ 82%|████████▏ | 1478/1813 [2:42:32<33:58,  6.09s/it]
+ 82%|████████▏ | 1479/1813 [2:42:38<33:50,  6.08s/it]
+ 82%|████████▏ | 1480/1813 [2:42:45<35:20,  6.37s/it]
+                                                     
+{'loss': '0.8723', 'grad_norm': '0.7927', 'learning_rate': '1.795e-05', 'epoch': '0.8166', 'num_input_tokens_seen': 9948032, 'train_runtime': '9766', 'train_tokens_per_second': '1019'}
+
+ 82%|████████▏ | 1480/1813 [2:42:45<35:20,  6.37s/it]
+ 82%|████████▏ | 1481/1813 [2:42:52<35:38,  6.44s/it]
+ 82%|████████▏ | 1482/1813 [2:42:57<34:14,  6.21s/it]
+ 82%|████████▏ | 1483/1813 [2:43:04<35:02,  6.37s/it]
+ 82%|████████▏ | 1484/1813 [2:43:11<34:58,  6.38s/it]
+ 82%|████████▏ | 1485/1813 [2:43:17<35:12,  6.44s/it]
+ 82%|████████▏ | 1486/1813 [2:43:22<33:03,  6.07s/it]
+ 82%|████████▏ | 1487/1813 [2:43:30<35:58,  6.62s/it]
+ 82%|████████▏ | 1488/1813 [2:43:37<35:16,  6.51s/it]
+ 82%|████████▏ | 1489/1813 [2:43:44<36:32,  6.77s/it]
+ 82%|████████▏ | 1490/1813 [2:43:51<36:24,  6.76s/it]
+                                                     
+{'loss': '0.8917', 'grad_norm': '1.072', 'learning_rate': '1.692e-05', 'epoch': '0.8221', 'num_input_tokens_seen': 10021400, 'train_runtime': '9831', 'train_tokens_per_second': '1019'}
+
+ 82%|████████▏ | 1490/1813 [2:43:51<36:24,  6.76s/it]
+ 82%|████████▏ | 1491/1813 [2:43:56<34:42,  6.47s/it]
+ 82%|████████▏ | 1492/1813 [2:44:05<37:23,  6.99s/it]
+ 82%|████████▏ | 1493/1813 [2:44:11<35:25,  6.64s/it]
+ 82%|████████▏ | 1494/1813 [2:44:16<34:11,  6.43s/it]
+ 82%|████████▏ | 1495/1813 [2:44:24<36:31,  6.89s/it]
+ 83%|████████▎ | 1496/1813 [2:44:30<33:34,  6.35s/it]
+ 83%|████████▎ | 1497/1813 [2:44:35<31:27,  5.97s/it]
+ 83%|████████▎ | 1498/1813 [2:44:41<31:57,  6.09s/it]
+ 83%|████████▎ | 1499/1813 [2:44:45<29:21,  5.61s/it]
+ 83%|████████▎ | 1500/1813 [2:44:53<32:11,  6.17s/it]
+                                                     
+{'loss': '0.8528', 'grad_norm': '0.8909', 'learning_rate': '1.593e-05', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904, 'train_runtime': '9893', 'train_tokens_per_second': '1020'}
+
+ 83%|████████▎ | 1500/1813 [2:44:53<32:11,  6.17s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:02, 15.86it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.23it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.34it/s]
+
+  1%|          | 9/1000 [00:01<02:32,  6.49it/s]
+
+  1%|          | 11/1000 [00:01<02:01,  8.12it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.02it/s]
+
+  1%|▏         | 14/1000 [00:01<02:23,  6.86it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.99it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.74it/s]
+
+  2%|▏         | 18/1000 [00:02<02:41,  6.07it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.16it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.61it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.26it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.38it/s]
+
+  3%|▎         | 26/1000 [00:03<02:35,  6.28it/s]
+
+  3%|▎         | 28/1000 [00:04<02:41,  6.01it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 31/1000 [00:04<03:00,  5.36it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  6.00it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.68it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.12it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.59it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.76it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.12it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.34it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.37it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.75it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.95it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.77it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.33it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.21it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.36it/s]
+
+  6%|▋         | 63/1000 [00:08<01:35,  9.78it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.39it/s]
+
+  7%|▋         | 67/1000 [00:08<02:02,  7.59it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.27it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.01it/s]
+
+  7%|▋         | 72/1000 [00:09<02:17,  6.73it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.14it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.88it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.23it/s]
+
+  8%|▊         | 84/1000 [00:11<01:53,  8.05it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.81it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.97it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.40it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.04it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.27it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.10it/s]
+
+  9%|▉         | 92/1000 [00:12<02:26,  6.20it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.80it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.55it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:12,  6.81it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.40it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.67it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.79it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.86it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.38it/s]
+
+ 11%|█         | 108/1000 [00:14<01:24, 10.50it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.25it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.07it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.54it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.81it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.22it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.93it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.10it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.45it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:21,  6.16it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.81it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.30it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.12it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.72it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.17it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.68it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:26,  9.92it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:25, 10.00it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.05it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.92it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.11it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:40,  8.46it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:39,  8.52it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.94it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.07it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:20, 10.44it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.67it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.98it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.63it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.82it/s]
+
+ 16%|█▋        | 163/1000 [00:20<01:51,  7.52it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.42it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:26,  5.68it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.19it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:23,  5.79it/s]
+
+ 17%|█▋        | 169/1000 [00:21<01:45,  7.86it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.17it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.17it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.24it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.75it/s]
+
+ 18%|█▊        | 176/1000 [00:22<01:37,  8.46it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.83it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.60it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.54it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:00,  6.78it/s]
+
+ 18%|█▊        | 182/1000 [00:23<02:22,  5.74it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.82it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:44,  7.77it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:42,  7.93it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:06,  6.41it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.14it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:01,  6.64it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.50it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.39it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.76it/s]
+
+ 20%|█▉        | 198/1000 [00:25<01:27,  9.21it/s]
+
+ 20%|██        | 200/1000 [00:26<01:14, 10.68it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.78it/s]
+
+ 20%|██        | 204/1000 [00:26<01:31,  8.65it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.94it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.52it/s]
+
+ 21%|██        | 210/1000 [00:27<01:32,  8.50it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.70it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.76it/s]
+
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.17it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.52it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.81it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.33it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.56it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:07,  6.07it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:06,  6.12it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.43it/s]
+
+ 23%|██▎       | 228/1000 [00:29<01:53,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.92it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.38it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.05it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.23it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.92it/s]
+
+ 24%|██▎       | 237/1000 [00:30<01:20,  9.45it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.97it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.38it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.03it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.48it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.68it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:30,  8.29it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.88it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:05,  5.98it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.88it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.05it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.76it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:40,  7.39it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.72it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.77it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:35,  7.71it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.53it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.44it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.92it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:36,  7.55it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.04it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:13,  5.45it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.71it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.71it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.08it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.82it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.61it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.02it/s]
+
+ 28%|██▊       | 283/1000 [00:37<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.75it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.79it/s]
+
+ 29%|██▉       | 291/1000 [00:38<01:41,  7.01it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.20it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.90it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.19it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.13it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.78it/s]
+
+ 30%|███       | 304/1000 [00:40<01:25,  8.10it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.16it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.96it/s]
+
+ 31%|███       | 307/1000 [00:41<01:41,  6.81it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.81it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.21it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.26it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.24it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.59it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:33,  7.25it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.91it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.44it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.75it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.71it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.43it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.19it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.59it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.98it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.40it/s]
+
+ 34%|███▎      | 336/1000 [00:44<01:22,  8.06it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.83it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:06,  9.95it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.83it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.61it/s]
+
+ 34%|███▍      | 345/1000 [00:45<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.22it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:28,  7.40it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:34,  6.91it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:49,  5.95it/s]
+
+ 35%|███▌      | 352/1000 [00:46<01:24,  7.63it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.55it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.36it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.96it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.59it/s]
+
+ 36%|███▌      | 361/1000 [00:47<01:08,  9.36it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.18it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.27it/s]
+
+ 36%|███▋      | 365/1000 [00:48<02:09,  4.92it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.38it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.04it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.54it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.71it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.93it/s]
+
+ 38%|███▊      | 380/1000 [00:50<01:10,  8.74it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.90it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:07,  9.12it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:07,  9.16it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.13it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.65it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.36it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:13,  8.31it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:23,  7.26it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:10,  8.55it/s]
+
+ 40%|███▉      | 398/1000 [00:53<01:24,  7.13it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:42,  5.89it/s]
+
+ 40%|████      | 400/1000 [00:53<01:51,  5.36it/s]
+
+ 40%|████      | 402/1000 [00:54<01:43,  5.75it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.58it/s]
+
+ 41%|████      | 407/1000 [00:54<01:23,  7.06it/s]
+
+ 41%|████      | 408/1000 [00:54<01:36,  6.16it/s]
+
+ 41%|████      | 410/1000 [00:55<01:33,  6.31it/s]
+
+ 41%|████      | 411/1000 [00:55<01:44,  5.65it/s]
+
+ 41%|████      | 412/1000 [00:55<01:53,  5.18it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.81it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:09,  8.33it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.50it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:16,  7.52it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:36,  5.94it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.96it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.56it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.85it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.56it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.05it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.83it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.54it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.10it/s]
+
+ 44%|████▍     | 441/1000 [00:59<00:59,  9.33it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.12it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.27it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.03it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.20it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:00<01:15,  7.20it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.53it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.83it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.29it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.86it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.65it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.76it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.03it/s]
+
+ 47%|████▋     | 467/1000 [01:02<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.32it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:09,  7.60it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:10,  7.44it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:21,  6.44it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:22,  6.36it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.47it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:13,  7.14it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:19,  6.58it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.05it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.30it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.02it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:16,  6.65it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.23it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.29it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50, 10.00it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.43it/s]
+
+ 50%|████▉     | 498/1000 [01:06<00:59,  8.47it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:14,  6.69it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.94it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.56it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:12,  6.81it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.57it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.82it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.44it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.97it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.30it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.55it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:07,  7.12it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.08it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.69it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.57it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.60it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.55it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.42it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.14it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:47,  9.75it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.71it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.38it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.68it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.95it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.77it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.95it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.08it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.51it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.71it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:15,  5.94it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.53it/s]
+
+ 55%|█████▌    | 552/1000 [01:14<01:01,  7.28it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.96it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.04it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.88it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.09it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.77it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.57it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.46it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.33it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:06,  6.37it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.85it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.27it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  7.00it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.58it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.60it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.93it/s]
+
+ 59%|█████▊    | 587/1000 [01:19<00:57,  7.17it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.55it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.77it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.19it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.82it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.42it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.90it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.69it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.49it/s]
+
+ 60%|██████    | 601/1000 [01:21<01:02,  6.41it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.73it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.70it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.13it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.89it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:45,  8.51it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.78it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.85it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.64it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.77it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:42,  9.02it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.01it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.20it/s]
+
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.43it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.62it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.20it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.23it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.87it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.46it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.73it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.08it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.97it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.21it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.97it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.61it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.13it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.60it/s]
+
+ 65%|██████▍   | 648/1000 [01:27<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.58it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.34it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.73it/s]
+
+ 66%|██████▌   | 656/1000 [01:28<00:38,  8.94it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.12it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.00it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.56it/s]
+
+ 66%|██████▌   | 662/1000 [01:29<00:52,  6.49it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.34it/s]
+
+ 67%|██████▋   | 667/1000 [01:30<00:59,  5.63it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.21it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.90it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.65it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.29it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.68it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.72it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:28, 11.02it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.95it/s]
+
+ 68%|██████▊   | 685/1000 [01:32<00:31,  9.92it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.89it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.18it/s]
+
+ 69%|██████▉   | 690/1000 [01:33<00:45,  6.82it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.85it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:41,  7.29it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:48,  6.32it/s]
+
+ 70%|██████▉   | 698/1000 [01:34<00:43,  6.93it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.04it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:35<00:40,  7.23it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.64it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.51it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.72it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.87it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.24it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.01it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.79it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.46it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.15it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.23it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:32,  8.34it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.17it/s]
+
+ 73%|███████▎  | 729/1000 [01:38<00:23, 11.66it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.34it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:27,  9.54it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.91it/s]
+
+ 74%|███████▎  | 737/1000 [01:39<00:27,  9.55it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.80it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.32it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.38it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.63it/s]
+
+ 75%|███████▍  | 747/1000 [01:40<00:24, 10.48it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:28,  8.67it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.70it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.18it/s]
+
+ 75%|███████▌  | 753/1000 [01:41<00:34,  7.26it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.25it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:37,  6.42it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.63it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.34it/s]
+
+ 76%|███████▌  | 761/1000 [01:42<00:27,  8.73it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.68it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.63it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.49it/s]
+
+ 77%|███████▋  | 768/1000 [01:43<00:29,  7.81it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.65it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.57it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.14it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.40it/s]
+
+ 77%|███████▋  | 774/1000 [01:44<00:33,  6.80it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.90it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.56it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.94it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:26,  8.37it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.15it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.53it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.37it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.01it/s]
+
+ 79%|███████▉  | 791/1000 [01:46<00:25,  8.24it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.74it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.99it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.21it/s]
+
+ 80%|███████▉  | 797/1000 [01:47<00:29,  6.88it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.03it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.74it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.66it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:24,  7.86it/s]
+
+ 81%|████████  | 806/1000 [01:48<00:21,  9.20it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  9.00it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.25it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.26it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.06it/s]
+
+ 82%|████████▏ | 815/1000 [01:49<00:20,  9.23it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.11it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.17it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.01it/s]
+
+ 82%|████████▏ | 823/1000 [01:50<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:51<00:32,  5.24it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.75it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.04it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.77it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:52<00:27,  6.02it/s]
+
+ 84%|████████▎ | 835/1000 [01:52<00:25,  6.55it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.14it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.48it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.64it/s]
+
+ 84%|████████▍ | 842/1000 [01:53<00:19,  8.16it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.43it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.79it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:54<00:19,  7.57it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.50it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.88it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.48it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.06it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:14, 10.20it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.51it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.76it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.64it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.73it/s]
+
+ 87%|████████▋ | 867/1000 [01:56<00:14,  9.34it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.29it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.32it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.81it/s]
+
+ 88%|████████▊ | 875/1000 [01:57<00:16,  7.66it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.90it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.89it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.62it/s]
+
+ 88%|████████▊ | 882/1000 [01:58<00:17,  6.56it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.98it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.19it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.38it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.68it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.71it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.60it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.84it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:00<00:12,  7.70it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.87it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.07it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.04it/s]
+
+ 91%|█████████ | 908/1000 [02:01<00:11,  8.31it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.60it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.80it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.40it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.10it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.39it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.62it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.73it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.96it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.20it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.68it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:08,  8.12it/s]
+
+ 93%|█████████▎| 928/1000 [02:04<00:10,  6.65it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.49it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.85it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.63it/s]
+
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.18it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.32it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.84it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.07it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.16it/s]
+
+ 94%|█████████▍| 942/1000 [02:06<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.41it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.75it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.86it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.43it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.18it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.42it/s]
+
+ 96%|█████████▌| 957/1000 [02:08<00:06,  6.42it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.79it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.84it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.06it/s]
+
+ 96%|█████████▋| 964/1000 [02:09<00:04,  7.51it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.36it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.04it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.44it/s]
+
+ 97%|█████████▋| 972/1000 [02:10<00:03,  8.04it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.22it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.83it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.85it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.45it/s]
+
+ 98%|█████████▊| 981/1000 [02:11<00:02,  7.07it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.45it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.95it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.39it/s]
+
+ 99%|█████████▉| 988/1000 [02:12<00:01,  6.75it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.01it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.17it/s]
+
+ 99%|█████████▉| 992/1000 [02:13<00:01,  4.99it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.56it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.55it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.07it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.78it/s]
+
+100%|██████████| 1000/1000 [02:14<00:00,  6.68it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7431', 'eval_runtime': '135.1', 'eval_samples_per_second': '7.401', 'eval_steps_per_second': '7.401', 'epoch': '0.8276', 'num_input_tokens_seen': 10089904}
+
+ 83%|████████▎ | 1500/1813 [2:47:08<32:11,  6.17s/it]
+
+100%|██████████| 1000/1000 [02:14<00:00,  6.68it/s]
+
+                                                   
+ 83%|████████▎ | 1501/1813 [2:47:14<4:03:14, 46.78s/it]
+ 83%|████████▎ | 1502/1813 [2:47:19<2:57:00, 34.15s/it]
+ 83%|████████▎ | 1503/1813 [2:47:25<2:13:07, 25.76s/it]
+ 83%|████████▎ | 1504/1813 [2:47:30<1:40:35, 19.53s/it]
+ 83%|████████▎ | 1505/1813 [2:47:36<1:18:37, 15.32s/it]
+ 83%|████████▎ | 1506/1813 [2:47:43<1:05:45, 12.85s/it]
+ 83%|████████▎ | 1507/1813 [2:47:48<53:41, 10.53s/it]  
+ 83%|████████▎ | 1508/1813 [2:47:55<48:03,  9.46s/it]
+ 83%|████████▎ | 1509/1813 [2:48:01<42:27,  8.38s/it]
+ 83%|████████▎ | 1510/1813 [2:48:07<38:39,  7.66s/it]
+                                                     
+{'loss': '0.8215', 'grad_norm': '1.2', 'learning_rate': '1.496e-05', 'epoch': '0.8331', 'num_input_tokens_seen': 10151928, 'train_runtime': '1.009e+04', 'train_tokens_per_second': '1006'}
+
+ 83%|████████▎ | 1510/1813 [2:48:07<38:39,  7.66s/it]
+ 83%|████████▎ | 1511/1813 [2:48:13<36:33,  7.26s/it]
+ 83%|████████▎ | 1512/1813 [2:48:19<34:06,  6.80s/it]
+ 83%|████████▎ | 1513/1813 [2:48:25<32:19,  6.46s/it]
+ 84%|████████▎ | 1514/1813 [2:48:31<32:37,  6.55s/it]
+ 84%|████████▎ | 1515/1813 [2:48:39<33:52,  6.82s/it]
+ 84%|████████▎ | 1516/1813 [2:48:46<33:49,  6.83s/it]
+ 84%|████████▎ | 1517/1813 [2:48:52<33:14,  6.74s/it]
+ 84%|████████▎ | 1518/1813 [2:49:00<34:19,  6.98s/it]
+ 84%|████████▍ | 1519/1813 [2:49:07<34:22,  7.02s/it]
+ 84%|████████▍ | 1520/1813 [2:49:13<33:01,  6.76s/it]
+                                                     
+{'loss': '0.8583', 'grad_norm': '1.102', 'learning_rate': '1.401e-05', 'epoch': '0.8386', 'num_input_tokens_seen': 10225024, 'train_runtime': '1.015e+04', 'train_tokens_per_second': '1007'}
+
+ 84%|████████▍ | 1520/1813 [2:49:13<33:01,  6.76s/it]
+ 84%|████████▍ | 1521/1813 [2:49:20<33:02,  6.79s/it]
+ 84%|████████▍ | 1522/1813 [2:49:26<32:04,  6.61s/it]
+ 84%|████████▍ | 1523/1813 [2:49:33<32:53,  6.81s/it]
+ 84%|████████▍ | 1524/1813 [2:49:40<32:02,  6.65s/it]
+ 84%|████████▍ | 1525/1813 [2:49:44<29:11,  6.08s/it]
+ 84%|████████▍ | 1526/1813 [2:49:51<29:27,  6.16s/it]
+ 84%|████████▍ | 1527/1813 [2:49:57<29:55,  6.28s/it]
+ 84%|████████▍ | 1528/1813 [2:50:03<29:50,  6.28s/it]
+ 84%|████████▍ | 1529/1813 [2:50:10<30:39,  6.48s/it]
+ 84%|████████▍ | 1530/1813 [2:50:16<29:15,  6.20s/it]
+                                                     
+{'loss': '0.9042', 'grad_norm': '0.9425', 'learning_rate': '1.31e-05', 'epoch': '0.8441', 'num_input_tokens_seen': 10294368, 'train_runtime': '1.022e+04', 'train_tokens_per_second': '1008'}
+
+ 84%|████████▍ | 1530/1813 [2:50:16<29:15,  6.20s/it]
+ 84%|████████▍ | 1531/1813 [2:50:21<27:37,  5.88s/it]
+ 85%|████████▍ | 1532/1813 [2:50:28<28:58,  6.19s/it]
+ 85%|████████▍ | 1533/1813 [2:50:33<27:41,  5.93s/it]
+ 85%|████████▍ | 1534/1813 [2:50:41<30:36,  6.58s/it]
+ 85%|████████▍ | 1535/1813 [2:50:47<29:07,  6.28s/it]
+ 85%|████████▍ | 1536/1813 [2:50:55<30:45,  6.66s/it]
+ 85%|████████▍ | 1537/1813 [2:51:01<30:29,  6.63s/it]
+ 85%|████████▍ | 1538/1813 [2:51:06<27:29,  6.00s/it]
+ 85%|████████▍ | 1539/1813 [2:51:11<26:56,  5.90s/it]
+ 85%|████████▍ | 1540/1813 [2:51:17<26:23,  5.80s/it]
+                                                     
+{'loss': '0.9007', 'grad_norm': '1.356', 'learning_rate': '1.221e-05', 'epoch': '0.8497', 'num_input_tokens_seen': 10359416, 'train_runtime': '1.028e+04', 'train_tokens_per_second': '1008'}
+
+ 85%|████████▍ | 1540/1813 [2:51:17<26:23,  5.80s/it]
+ 85%|████████▍ | 1541/1813 [2:51:23<27:19,  6.03s/it]
+ 85%|████████▌ | 1542/1813 [2:51:30<27:41,  6.13s/it]
+ 85%|████████▌ | 1543/1813 [2:51:35<25:51,  5.75s/it]
+ 85%|████████▌ | 1544/1813 [2:51:40<24:35,  5.48s/it]
+ 85%|████████▌ | 1545/1813 [2:51:47<27:32,  6.17s/it]
+ 85%|████████▌ | 1546/1813 [2:51:53<27:07,  6.09s/it]
+ 85%|████████▌ | 1547/1813 [2:51:58<25:46,  5.81s/it]
+ 85%|████████▌ | 1548/1813 [2:52:07<28:43,  6.50s/it]
+ 85%|████████▌ | 1549/1813 [2:52:12<26:57,  6.13s/it]
+ 85%|████████▌ | 1550/1813 [2:52:19<28:32,  6.51s/it]
+                                                     
+{'loss': '0.9711', 'grad_norm': '0.9562', 'learning_rate': '1.136e-05', 'epoch': '0.8552', 'num_input_tokens_seen': 10427368, 'train_runtime': '1.034e+04', 'train_tokens_per_second': '1008'}
+
+ 85%|████████▌ | 1550/1813 [2:52:19<28:32,  6.51s/it]
+ 86%|████████▌ | 1551/1813 [2:52:27<30:03,  6.88s/it]
+ 86%|████████▌ | 1552/1813 [2:52:33<28:47,  6.62s/it]
+ 86%|████████▌ | 1553/1813 [2:52:39<27:39,  6.38s/it]
+ 86%|████████▌ | 1554/1813 [2:52:46<28:05,  6.51s/it]
+ 86%|████████▌ | 1555/1813 [2:52:52<28:11,  6.56s/it]
+ 86%|████████▌ | 1556/1813 [2:52:58<27:27,  6.41s/it]
+ 86%|████████▌ | 1557/1813 [2:53:06<29:04,  6.81s/it]
+ 86%|████████▌ | 1558/1813 [2:53:12<27:46,  6.53s/it]
+ 86%|████████▌ | 1559/1813 [2:53:18<27:14,  6.44s/it]
+ 86%|████████▌ | 1560/1813 [2:53:23<25:28,  6.04s/it]
+                                                     
+{'loss': '0.9794', 'grad_norm': '0.8591', 'learning_rate': '1.053e-05', 'epoch': '0.8607', 'num_input_tokens_seen': 10497472, 'train_runtime': '1.04e+04', 'train_tokens_per_second': '1009'}
+
+ 86%|████████▌ | 1560/1813 [2:53:23<25:28,  6.04s/it]
+ 86%|████████▌ | 1561/1813 [2:53:30<26:27,  6.30s/it]
+ 86%|████████▌ | 1562/1813 [2:53:36<26:18,  6.29s/it]
+ 86%|████████▌ | 1563/1813 [2:53:43<25:59,  6.24s/it]
+ 86%|████████▋ | 1564/1813 [2:53:49<25:43,  6.20s/it]
+ 86%|████████▋ | 1565/1813 [2:53:53<23:16,  5.63s/it]
+ 86%|████████▋ | 1566/1813 [2:53:59<23:30,  5.71s/it]
+ 86%|████████▋ | 1567/1813 [2:54:05<23:42,  5.78s/it]
+ 86%|████████▋ | 1568/1813 [2:54:10<22:17,  5.46s/it]
+ 87%|████████▋ | 1569/1813 [2:54:17<24:36,  6.05s/it]
+ 87%|████████▋ | 1570/1813 [2:54:24<25:58,  6.41s/it]
+                                                     
+{'loss': '0.7987', 'grad_norm': '0.9577', 'learning_rate': '9.73e-06', 'epoch': '0.8662', 'num_input_tokens_seen': 10564336, 'train_runtime': '1.046e+04', 'train_tokens_per_second': '1010'}
+
+ 87%|████████▋ | 1570/1813 [2:54:24<25:58,  6.41s/it]
+ 87%|████████▋ | 1571/1813 [2:54:29<24:26,  6.06s/it]
+ 87%|████████▋ | 1572/1813 [2:54:35<23:36,  5.88s/it]
+ 87%|████████▋ | 1573/1813 [2:54:40<23:03,  5.77s/it]
+ 87%|████████▋ | 1574/1813 [2:54:47<23:39,  5.94s/it]
+ 87%|████████▋ | 1575/1813 [2:54:52<23:12,  5.85s/it]
+ 87%|████████▋ | 1576/1813 [2:55:00<24:55,  6.31s/it]
+ 87%|████████▋ | 1577/1813 [2:55:07<25:35,  6.51s/it]
+ 87%|████████▋ | 1578/1813 [2:55:13<25:24,  6.49s/it]
+ 87%|████████▋ | 1579/1813 [2:55:19<24:57,  6.40s/it]
+ 87%|████████▋ | 1580/1813 [2:55:25<23:24,  6.03s/it]
+                                                     
+{'loss': '0.9294', 'grad_norm': '1.193', 'learning_rate': '8.961e-06', 'epoch': '0.8717', 'num_input_tokens_seen': 10628400, 'train_runtime': '1.053e+04', 'train_tokens_per_second': '1010'}
+
+ 87%|████████▋ | 1580/1813 [2:55:25<23:24,  6.03s/it]
+ 87%|████████▋ | 1581/1813 [2:55:31<23:15,  6.02s/it]
+ 87%|████████▋ | 1582/1813 [2:55:35<21:30,  5.59s/it]
+ 87%|████████▋ | 1583/1813 [2:55:42<22:25,  5.85s/it]
+ 87%|████████▋ | 1584/1813 [2:55:48<23:05,  6.05s/it]
+ 87%|████████▋ | 1585/1813 [2:55:55<24:29,  6.44s/it]
+ 87%|████████▋ | 1586/1813 [2:56:03<25:29,  6.74s/it]
+ 88%|████████▊ | 1587/1813 [2:56:10<26:17,  6.98s/it]
+ 88%|████████▊ | 1588/1813 [2:56:16<24:56,  6.65s/it]
+ 88%|████████▊ | 1589/1813 [2:56:21<23:07,  6.20s/it]
+ 88%|████████▊ | 1590/1813 [2:56:28<23:34,  6.34s/it]
+                                                     
+{'loss': '0.871', 'grad_norm': '1.112', 'learning_rate': '8.223e-06', 'epoch': '0.8772', 'num_input_tokens_seen': 10695288, 'train_runtime': '1.059e+04', 'train_tokens_per_second': '1010'}
+
+ 88%|████████▊ | 1590/1813 [2:56:28<23:34,  6.34s/it]
+ 88%|████████▊ | 1591/1813 [2:56:34<23:20,  6.31s/it]
+ 88%|████████▊ | 1592/1813 [2:56:41<23:25,  6.36s/it]
+ 88%|████████▊ | 1593/1813 [2:56:48<23:45,  6.48s/it]
+ 88%|████████▊ | 1594/1813 [2:56:54<23:56,  6.56s/it]
+ 88%|████████▊ | 1595/1813 [2:57:02<24:39,  6.79s/it]
+ 88%|████████▊ | 1596/1813 [2:57:10<26:41,  7.38s/it]
+ 88%|████████▊ | 1597/1813 [2:57:17<25:57,  7.21s/it]
+ 88%|████████▊ | 1598/1813 [2:57:24<25:09,  7.02s/it]
+ 88%|████████▊ | 1599/1813 [2:57:30<23:43,  6.65s/it]
+ 88%|████████▊ | 1600/1813 [2:57:35<21:45,  6.13s/it]
+                                                     
+{'loss': '0.874', 'grad_norm': '0.9599', 'learning_rate': '7.515e-06', 'epoch': '0.8828', 'num_input_tokens_seen': 10767512, 'train_runtime': '1.066e+04', 'train_tokens_per_second': '1011'}
+
+ 88%|████████▊ | 1600/1813 [2:57:35<21:45,  6.13s/it]
+ 88%|████████▊ | 1601/1813 [2:57:43<23:41,  6.71s/it]
+ 88%|████████▊ | 1602/1813 [2:57:49<22:58,  6.53s/it]
+ 88%|████████▊ | 1603/1813 [2:57:55<23:05,  6.60s/it]
+ 88%|████████▊ | 1604/1813 [2:58:02<23:05,  6.63s/it]
+ 89%|████████▊ | 1605/1813 [2:58:10<24:40,  7.12s/it]
+ 89%|████████▊ | 1606/1813 [2:58:16<22:33,  6.54s/it]
+ 89%|████████▊ | 1607/1813 [2:58:22<22:42,  6.61s/it]
+ 89%|████████▊ | 1608/1813 [2:58:30<23:20,  6.83s/it]
+ 89%|████████▊ | 1609/1813 [2:58:35<21:53,  6.44s/it]
+ 89%|████████▉ | 1610/1813 [2:58:41<21:29,  6.35s/it]
+                                                     
+{'loss': '0.8963', 'grad_norm': '1.343', 'learning_rate': '6.838e-06', 'epoch': '0.8883', 'num_input_tokens_seen': 10840800, 'train_runtime': '1.072e+04', 'train_tokens_per_second': '1011'}
+
+ 89%|████████▉ | 1610/1813 [2:58:41<21:29,  6.35s/it]
+ 89%|████████▉ | 1611/1813 [2:58:47<20:15,  6.02s/it]
+ 89%|████████▉ | 1612/1813 [2:58:53<20:14,  6.04s/it]
+ 89%|████████▉ | 1613/1813 [2:58:57<18:42,  5.61s/it]
+ 89%|████████▉ | 1614/1813 [2:59:03<18:54,  5.70s/it]
+ 89%|████████▉ | 1615/1813 [2:59:10<19:39,  5.96s/it]
+ 89%|████████▉ | 1616/1813 [2:59:16<20:09,  6.14s/it]
+ 89%|████████▉ | 1617/1813 [2:59:23<20:41,  6.34s/it]
+ 89%|████████▉ | 1618/1813 [2:59:28<19:34,  6.02s/it]
+ 89%|████████▉ | 1619/1813 [2:59:34<19:15,  5.96s/it]
+ 89%|████████▉ | 1620/1813 [2:59:39<18:19,  5.70s/it]
+                                                     
+{'loss': '0.9016', 'grad_norm': '1.042', 'learning_rate': '6.191e-06', 'epoch': '0.8938', 'num_input_tokens_seen': 10899640, 'train_runtime': '1.078e+04', 'train_tokens_per_second': '1011'}
+
+ 89%|████████▉ | 1620/1813 [2:59:39<18:19,  5.70s/it]
+ 89%|████████▉ | 1621/1813 [2:59:45<18:08,  5.67s/it]
+ 89%|████████▉ | 1622/1813 [2:59:53<19:55,  6.26s/it]
+ 90%|████████▉ | 1623/1813 [3:00:00<20:30,  6.48s/it]
+ 90%|████████▉ | 1624/1813 [3:00:06<19:59,  6.35s/it]
+ 90%|████████▉ | 1625/1813 [3:00:13<20:50,  6.65s/it]
+ 90%|████████▉ | 1626/1813 [3:00:19<20:19,  6.52s/it]
+ 90%|████████▉ | 1627/1813 [3:00:25<19:33,  6.31s/it]
+ 90%|████████▉ | 1628/1813 [3:00:31<18:57,  6.15s/it]
+ 90%|████████▉ | 1629/1813 [3:00:38<19:46,  6.45s/it]
+ 90%|████████▉ | 1630/1813 [3:00:43<18:41,  6.13s/it]
+                                                     
+{'loss': '0.8953', 'grad_norm': '1.339', 'learning_rate': '5.575e-06', 'epoch': '0.8993', 'num_input_tokens_seen': 10968528, 'train_runtime': '1.084e+04', 'train_tokens_per_second': '1012'}
+
+ 90%|████████▉ | 1630/1813 [3:00:43<18:41,  6.13s/it]
+ 90%|████████▉ | 1631/1813 [3:00:49<18:01,  5.94s/it]
+ 90%|█████████ | 1632/1813 [3:00:54<16:50,  5.59s/it]
+ 90%|█████████ | 1633/1813 [3:01:01<18:20,  6.12s/it]
+ 90%|█████████ | 1634/1813 [3:01:06<17:07,  5.74s/it]
+ 90%|█████████ | 1635/1813 [3:01:12<17:53,  6.03s/it]
+ 90%|█████████ | 1636/1813 [3:01:18<17:15,  5.85s/it]
+ 90%|█████████ | 1637/1813 [3:01:24<17:33,  5.99s/it]
+ 90%|█████████ | 1638/1813 [3:01:29<16:44,  5.74s/it]
+ 90%|█████████ | 1639/1813 [3:01:36<17:31,  6.05s/it]
+ 90%|█████████ | 1640/1813 [3:01:41<16:39,  5.77s/it]
+                                                     
+{'loss': '0.9005', 'grad_norm': '0.9101', 'learning_rate': '4.991e-06', 'epoch': '0.9048', 'num_input_tokens_seen': 11030048, 'train_runtime': '1.09e+04', 'train_tokens_per_second': '1012'}
+
+ 90%|█████████ | 1640/1813 [3:01:41<16:39,  5.77s/it]
+ 91%|█████████ | 1641/1813 [3:01:47<16:41,  5.82s/it]
+ 91%|█████████ | 1642/1813 [3:01:54<17:18,  6.08s/it]
+ 91%|█████████ | 1643/1813 [3:02:00<17:10,  6.06s/it]
+ 91%|█████████ | 1644/1813 [3:02:07<17:35,  6.25s/it]
+ 91%|█████████ | 1645/1813 [3:02:16<20:09,  7.20s/it]
+ 91%|█████████ | 1646/1813 [3:02:23<19:28,  7.00s/it]
+ 91%|█████████ | 1647/1813 [3:02:29<18:36,  6.73s/it]
+ 91%|█████████ | 1648/1813 [3:02:35<18:23,  6.69s/it]
+ 91%|█████████ | 1649/1813 [3:02:40<17:03,  6.24s/it]
+ 91%|█████████ | 1650/1813 [3:02:46<16:05,  5.92s/it]
+                                                     
+{'loss': '0.8626', 'grad_norm': '0.9827', 'learning_rate': '4.438e-06', 'epoch': '0.9103', 'num_input_tokens_seen': 11098768, 'train_runtime': '1.097e+04', 'train_tokens_per_second': '1012'}
+
+ 91%|█████████ | 1650/1813 [3:02:46<16:05,  5.92s/it]
+ 91%|█████████ | 1651/1813 [3:02:52<16:21,  6.06s/it]
+ 91%|█████████ | 1652/1813 [3:02:58<16:33,  6.17s/it]
+ 91%|█████████ | 1653/1813 [3:03:05<16:48,  6.30s/it]
+ 91%|█████████ | 1654/1813 [3:03:11<16:31,  6.23s/it]
+ 91%|█████████▏| 1655/1813 [3:03:18<16:37,  6.31s/it]
+ 91%|█████████▏| 1656/1813 [3:03:23<16:07,  6.16s/it]
+ 91%|█████████▏| 1657/1813 [3:03:31<17:08,  6.59s/it]
+ 91%|█████████▏| 1658/1813 [3:03:38<16:59,  6.58s/it]
+ 92%|█████████▏| 1659/1813 [3:03:44<17:09,  6.68s/it]
+ 92%|█████████▏| 1660/1813 [3:03:51<16:48,  6.59s/it]
+                                                     
+{'loss': '0.924', 'grad_norm': '0.9582', 'learning_rate': '3.917e-06', 'epoch': '0.9159', 'num_input_tokens_seen': 11168848, 'train_runtime': '1.103e+04', 'train_tokens_per_second': '1012'}
+
+ 92%|█████████▏| 1660/1813 [3:03:51<16:48,  6.59s/it]
+ 92%|█████████▏| 1661/1813 [3:03:58<17:02,  6.73s/it]
+ 92%|█████████▏| 1662/1813 [3:04:05<16:58,  6.74s/it]
+ 92%|█████████▏| 1663/1813 [3:04:12<17:04,  6.83s/it]
+ 92%|█████████▏| 1664/1813 [3:04:19<16:58,  6.84s/it]
+ 92%|█████████▏| 1665/1813 [3:04:26<17:25,  7.06s/it]
+ 92%|█████████▏| 1666/1813 [3:04:31<15:27,  6.31s/it]
+ 92%|█████████▏| 1667/1813 [3:04:38<15:43,  6.46s/it]
+ 92%|█████████▏| 1668/1813 [3:04:44<15:17,  6.33s/it]
+ 92%|█████████▏| 1669/1813 [3:04:54<17:48,  7.42s/it]
+ 92%|█████████▏| 1670/1813 [3:04:59<16:10,  6.79s/it]
+                                                     
+{'loss': '0.8752', 'grad_norm': '1.092', 'learning_rate': '3.428e-06', 'epoch': '0.9214', 'num_input_tokens_seen': 11244288, 'train_runtime': '1.11e+04', 'train_tokens_per_second': '1013'}
+
+ 92%|█████████▏| 1670/1813 [3:04:59<16:10,  6.79s/it]
+ 92%|█████████▏| 1671/1813 [3:05:04<14:49,  6.27s/it]
+ 92%|█████████▏| 1672/1813 [3:05:10<14:18,  6.09s/it]
+ 92%|█████████▏| 1673/1813 [3:05:15<13:58,  5.99s/it]
+ 92%|█████████▏| 1674/1813 [3:05:22<14:29,  6.26s/it]
+ 92%|█████████▏| 1675/1813 [3:05:29<14:59,  6.51s/it]
+ 92%|█████████▏| 1676/1813 [3:05:37<15:24,  6.75s/it]
+ 92%|█████████▏| 1677/1813 [3:05:43<15:01,  6.63s/it]
+ 93%|█████████▎| 1678/1813 [3:05:49<14:40,  6.52s/it]
+ 93%|█████████▎| 1679/1813 [3:05:55<14:04,  6.30s/it]
+ 93%|█████████▎| 1680/1813 [3:06:04<15:32,  7.01s/it]
+                                                     
+{'loss': '0.9472', 'grad_norm': '0.8809', 'learning_rate': '2.971e-06', 'epoch': '0.9269', 'num_input_tokens_seen': 11317056, 'train_runtime': '1.116e+04', 'train_tokens_per_second': '1014'}
+
+ 93%|█████████▎| 1680/1813 [3:06:04<15:32,  7.01s/it]
+ 93%|█████████▎| 1681/1813 [3:06:08<13:55,  6.33s/it]
+ 93%|█████████▎| 1682/1813 [3:06:14<13:00,  5.96s/it]
+ 93%|█████████▎| 1683/1813 [3:06:18<12:01,  5.55s/it]
+ 93%|█████████▎| 1684/1813 [3:06:25<12:43,  5.92s/it]
+ 93%|█████████▎| 1685/1813 [3:06:31<13:03,  6.12s/it]
+ 93%|█████████▎| 1686/1813 [3:06:38<13:24,  6.34s/it]
+ 93%|█████████▎| 1687/1813 [3:06:45<13:18,  6.34s/it]
+ 93%|█████████▎| 1688/1813 [3:06:51<13:29,  6.47s/it]
+ 93%|█████████▎| 1689/1813 [3:06:56<12:16,  5.94s/it]
+ 93%|█████████▎| 1690/1813 [3:07:03<12:30,  6.10s/it]
+                                                     
+{'loss': '0.8724', 'grad_norm': '0.8967', 'learning_rate': '2.546e-06', 'epoch': '0.9324', 'num_input_tokens_seen': 11380560, 'train_runtime': '1.122e+04', 'train_tokens_per_second': '1014'}
+
+ 93%|█████████▎| 1690/1813 [3:07:03<12:30,  6.10s/it]
+ 93%|█████████▎| 1691/1813 [3:07:09<12:24,  6.10s/it]
+ 93%|█████████▎| 1692/1813 [3:07:16<13:14,  6.56s/it]
+ 93%|█████████▎| 1693/1813 [3:07:23<12:53,  6.45s/it]
+ 93%|█████████▎| 1694/1813 [3:07:31<13:51,  6.98s/it]
+ 93%|█████████▎| 1695/1813 [3:07:37<13:08,  6.68s/it]
+ 94%|█████████▎| 1696/1813 [3:07:43<12:35,  6.46s/it]
+ 94%|█████████▎| 1697/1813 [3:07:49<12:33,  6.49s/it]
+ 94%|█████████▎| 1698/1813 [3:07:55<11:51,  6.19s/it]
+ 94%|█████████▎| 1699/1813 [3:08:01<11:53,  6.26s/it]
+ 94%|█████████▍| 1700/1813 [3:08:08<11:51,  6.30s/it]
+                                                     
+{'loss': '0.8517', 'grad_norm': '1.081', 'learning_rate': '2.154e-06', 'epoch': '0.9379', 'num_input_tokens_seen': 11450712, 'train_runtime': '1.129e+04', 'train_tokens_per_second': '1014'}
+
+ 94%|█████████▍| 1700/1813 [3:08:08<11:51,  6.30s/it]
+ 94%|█████████▍| 1701/1813 [3:08:12<10:48,  5.79s/it]
+ 94%|█████████▍| 1702/1813 [3:08:17<10:26,  5.64s/it]
+ 94%|█████████▍| 1703/1813 [3:08:23<10:28,  5.71s/it]
+ 94%|█████████▍| 1704/1813 [3:08:31<11:29,  6.32s/it]
+ 94%|█████████▍| 1705/1813 [3:08:38<11:30,  6.39s/it]
+ 94%|█████████▍| 1706/1813 [3:08:44<11:19,  6.35s/it]
+ 94%|█████████▍| 1707/1813 [3:08:49<10:46,  6.10s/it]
+ 94%|█████████▍| 1708/1813 [3:08:55<10:10,  5.82s/it]
+ 94%|█████████▍| 1709/1813 [3:09:01<10:22,  5.98s/it]
+ 94%|█████████▍| 1710/1813 [3:09:07<10:12,  5.95s/it]
+                                                     
+{'loss': '0.9361', 'grad_norm': '1.249', 'learning_rate': '1.793e-06', 'epoch': '0.9434', 'num_input_tokens_seen': 11511984, 'train_runtime': '1.135e+04', 'train_tokens_per_second': '1015'}
+
+ 94%|█████████▍| 1710/1813 [3:09:07<10:12,  5.95s/it]
+ 94%|█████████▍| 1711/1813 [3:09:12<09:54,  5.83s/it]
+ 94%|█████████▍| 1712/1813 [3:09:18<09:44,  5.79s/it]
+ 94%|█████████▍| 1713/1813 [3:09:24<09:40,  5.80s/it]
+ 95%|█████████▍| 1714/1813 [3:09:30<09:29,  5.75s/it]
+ 95%|█████████▍| 1715/1813 [3:09:35<09:13,  5.64s/it]
+ 95%|█████████▍| 1716/1813 [3:09:40<08:46,  5.43s/it]
+ 95%|█████████▍| 1717/1813 [3:09:45<08:33,  5.35s/it]
+ 95%|█████████▍| 1718/1813 [3:09:51<08:50,  5.58s/it]
+ 95%|█████████▍| 1719/1813 [3:09:57<09:03,  5.78s/it]
+ 95%|█████████▍| 1720/1813 [3:10:03<09:02,  5.83s/it]
+                                                     
+{'loss': '0.8587', 'grad_norm': '0.949', 'learning_rate': '1.466e-06', 'epoch': '0.949', 'num_input_tokens_seen': 11571368, 'train_runtime': '1.14e+04', 'train_tokens_per_second': '1015'}
+
+ 95%|█████████▍| 1720/1813 [3:10:03<09:02,  5.83s/it]
+ 95%|█████████▍| 1721/1813 [3:10:11<09:36,  6.26s/it]
+ 95%|█████████▍| 1722/1813 [3:10:17<09:23,  6.19s/it]
+ 95%|█████████▌| 1723/1813 [3:10:23<09:12,  6.14s/it]
+ 95%|█████████▌| 1724/1813 [3:10:28<08:58,  6.06s/it]
+ 95%|█████████▌| 1725/1813 [3:10:36<09:25,  6.43s/it]
+ 95%|█████████▌| 1726/1813 [3:10:43<09:43,  6.71s/it]
+ 95%|█████████▌| 1727/1813 [3:10:49<09:15,  6.46s/it]
+ 95%|█████████▌| 1728/1813 [3:10:54<08:29,  6.00s/it]
+ 95%|█████████▌| 1729/1813 [3:10:59<08:02,  5.74s/it]
+ 95%|█████████▌| 1730/1813 [3:11:06<08:21,  6.04s/it]
+                                                     
+{'loss': '0.9201', 'grad_norm': '0.818', 'learning_rate': '1.171e-06', 'epoch': '0.9545', 'num_input_tokens_seen': 11640392, 'train_runtime': '1.147e+04', 'train_tokens_per_second': '1015'}
+
+ 95%|█████████▌| 1730/1813 [3:11:06<08:21,  6.04s/it]
+ 95%|█████████▌| 1731/1813 [3:11:12<08:16,  6.06s/it]
+ 96%|█████████▌| 1732/1813 [3:11:19<08:26,  6.26s/it]
+ 96%|█████████▌| 1733/1813 [3:11:24<07:51,  5.90s/it]
+ 96%|█████████▌| 1734/1813 [3:11:30<07:43,  5.87s/it]
+ 96%|█████████▌| 1735/1813 [3:11:36<07:49,  6.02s/it]
+ 96%|█████████▌| 1736/1813 [3:11:43<08:01,  6.26s/it]
+ 96%|█████████▌| 1737/1813 [3:11:49<07:52,  6.22s/it]
+ 96%|█████████▌| 1738/1813 [3:11:54<07:15,  5.81s/it]
+ 96%|█████████▌| 1739/1813 [3:12:02<07:56,  6.44s/it]
+ 96%|█████████▌| 1740/1813 [3:12:08<07:51,  6.46s/it]
+                                                     
+{'loss': '0.9088', 'grad_norm': '1.028', 'learning_rate': '9.094e-07', 'epoch': '0.96', 'num_input_tokens_seen': 11709120, 'train_runtime': '1.153e+04', 'train_tokens_per_second': '1016'}
+
+ 96%|█████████▌| 1740/1813 [3:12:08<07:51,  6.46s/it]
+ 96%|█████████▌| 1741/1813 [3:12:14<07:28,  6.23s/it]
+ 96%|█████████▌| 1742/1813 [3:12:21<07:33,  6.38s/it]
+ 96%|█████████▌| 1743/1813 [3:12:28<07:39,  6.56s/it]
+ 96%|█████████▌| 1744/1813 [3:12:35<07:42,  6.71s/it]
+ 96%|█████████▌| 1745/1813 [3:12:40<07:19,  6.46s/it]
+ 96%|█████████▋| 1746/1813 [3:12:48<07:30,  6.72s/it]
+ 96%|█████████▋| 1747/1813 [3:12:53<06:47,  6.17s/it]
+ 96%|█████████▋| 1748/1813 [3:13:00<07:03,  6.52s/it]
+ 96%|█████████▋| 1749/1813 [3:13:05<06:34,  6.17s/it]
+ 97%|█████████▋| 1750/1813 [3:13:11<06:12,  5.91s/it]
+                                                     
+{'loss': '0.8587', 'grad_norm': '0.8952', 'learning_rate': '6.805e-07', 'epoch': '0.9655', 'num_input_tokens_seen': 11777768, 'train_runtime': '1.159e+04', 'train_tokens_per_second': '1016'}
+
+ 97%|█████████▋| 1750/1813 [3:13:11<06:12,  5.91s/it]
+ 97%|█████████▋| 1751/1813 [3:13:17<06:05,  5.90s/it]
+ 97%|█████████▋| 1752/1813 [3:13:22<05:57,  5.87s/it]
+ 97%|█████████▋| 1753/1813 [3:13:30<06:22,  6.37s/it]
+ 97%|█████████▋| 1754/1813 [3:13:37<06:30,  6.63s/it]
+ 97%|█████████▋| 1755/1813 [3:13:43<06:20,  6.55s/it]
+ 97%|█████████▋| 1756/1813 [3:13:49<05:52,  6.18s/it]
+ 97%|█████████▋| 1757/1813 [3:13:54<05:33,  5.96s/it]
+ 97%|█████████▋| 1758/1813 [3:14:00<05:21,  5.85s/it]
+ 97%|█████████▋| 1759/1813 [3:14:06<05:16,  5.87s/it]
+ 97%|█████████▋| 1760/1813 [3:14:11<05:00,  5.67s/it]
+                                                     
+{'loss': '0.869', 'grad_norm': '1.069', 'learning_rate': '4.847e-07', 'epoch': '0.971', 'num_input_tokens_seen': 11843192, 'train_runtime': '1.165e+04', 'train_tokens_per_second': '1016'}
+
+ 97%|█████████▋| 1760/1813 [3:14:11<05:00,  5.67s/it]
+ 97%|█████████▋| 1761/1813 [3:14:16<04:49,  5.56s/it]
+ 97%|█████████▋| 1762/1813 [3:14:21<04:28,  5.27s/it]
+ 97%|█████████▋| 1763/1813 [3:14:28<04:58,  5.96s/it]
+ 97%|█████████▋| 1764/1813 [3:14:33<04:39,  5.70s/it]
+ 97%|█████████▋| 1765/1813 [3:14:39<04:29,  5.61s/it]
+ 97%|█████████▋| 1766/1813 [3:14:45<04:23,  5.61s/it]
+ 97%|█████████▋| 1767/1813 [3:14:50<04:18,  5.61s/it]
+ 98%|█████████▊| 1768/1813 [3:14:56<04:17,  5.72s/it]
+ 98%|█████████▊| 1769/1813 [3:15:02<04:14,  5.78s/it]
+ 98%|█████████▊| 1770/1813 [3:15:09<04:22,  6.10s/it]
+                                                     
+{'loss': '0.8808', 'grad_norm': '0.9307', 'learning_rate': '3.219e-07', 'epoch': '0.9766', 'num_input_tokens_seen': 11903728, 'train_runtime': '1.171e+04', 'train_tokens_per_second': '1017'}
+
+ 98%|█████████▊| 1770/1813 [3:15:09<04:22,  6.10s/it]
+ 98%|█████████▊| 1771/1813 [3:15:15<04:20,  6.20s/it]
+ 98%|█████████▊| 1772/1813 [3:15:22<04:17,  6.28s/it]
+ 98%|█████████▊| 1773/1813 [3:15:29<04:19,  6.49s/it]
+ 98%|█████████▊| 1774/1813 [3:15:33<03:50,  5.92s/it]
+ 98%|█████████▊| 1775/1813 [3:15:41<04:01,  6.37s/it]
+ 98%|█████████▊| 1776/1813 [3:15:48<04:06,  6.65s/it]
+ 98%|█████████▊| 1777/1813 [3:15:55<04:07,  6.89s/it]
+ 98%|█████████▊| 1778/1813 [3:16:01<03:46,  6.47s/it]
+ 98%|█████████▊| 1779/1813 [3:16:06<03:29,  6.15s/it]
+ 98%|█████████▊| 1780/1813 [3:16:12<03:19,  6.04s/it]
+                                                     
+{'loss': '0.8571', 'grad_norm': '0.8858', 'learning_rate': '1.922e-07', 'epoch': '0.9821', 'num_input_tokens_seen': 11972864, 'train_runtime': '1.177e+04', 'train_tokens_per_second': '1017'}
+
+ 98%|█████████▊| 1780/1813 [3:16:12<03:19,  6.04s/it]
+ 98%|█████████▊| 1781/1813 [3:16:18<03:14,  6.07s/it]
+ 98%|█████████▊| 1782/1813 [3:16:24<03:05,  6.00s/it]
+ 98%|█████████▊| 1783/1813 [3:16:30<02:58,  5.96s/it]
+ 98%|█████████▊| 1784/1813 [3:16:37<03:00,  6.24s/it]
+ 98%|█████████▊| 1785/1813 [3:16:43<02:53,  6.21s/it]
+ 99%|█████████▊| 1786/1813 [3:16:49<02:48,  6.22s/it]
+ 99%|█████████▊| 1787/1813 [3:16:55<02:36,  6.01s/it]
+ 99%|█████████▊| 1788/1813 [3:17:02<02:36,  6.26s/it]
+ 99%|█████████▊| 1789/1813 [3:17:10<02:46,  6.95s/it]
+ 99%|█████████▊| 1790/1813 [3:17:18<02:42,  7.05s/it]
+                                                     
+{'loss': '0.8864', 'grad_norm': '0.8395', 'learning_rate': '9.58e-08', 'epoch': '0.9876', 'num_input_tokens_seen': 12043496, 'train_runtime': '1.184e+04', 'train_tokens_per_second': '1017'}
+
+ 99%|█████████▊| 1790/1813 [3:17:18<02:42,  7.05s/it]
+ 99%|█████████▉| 1791/1813 [3:17:23<02:25,  6.61s/it]
+ 99%|█████████▉| 1792/1813 [3:17:29<02:16,  6.52s/it]
+ 99%|█████████▉| 1793/1813 [3:17:36<02:13,  6.66s/it]
+ 99%|█████████▉| 1794/1813 [3:17:44<02:09,  6.81s/it]
+ 99%|█████████▉| 1795/1813 [3:17:50<02:02,  6.81s/it]
+ 99%|█████████▉| 1796/1813 [3:17:57<01:55,  6.80s/it]
+ 99%|█████████▉| 1797/1813 [3:18:04<01:47,  6.70s/it]
+ 99%|█████████▉| 1798/1813 [3:18:13<01:52,  7.50s/it]
+ 99%|█████████▉| 1799/1813 [3:18:19<01:40,  7.17s/it]
+ 99%|█████████▉| 1800/1813 [3:18:24<01:22,  6.35s/it]
+                                                     
+{'loss': '0.8567', 'grad_norm': '0.9589', 'learning_rate': '3.26e-08', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448, 'train_runtime': '1.19e+04', 'train_tokens_per_second': '1018'}
+
+ 99%|█████████▉| 1800/1813 [3:18:24<01:22,  6.35s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.75it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.24it/s]
+
+  1%|          | 7/1000 [00:00<01:58,  8.35it/s]
+
+  1%|          | 9/1000 [00:01<02:32,  6.48it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.10it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.02it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.85it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.98it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.74it/s]
+
+  2%|▏         | 18/1000 [00:02<02:42,  6.06it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.61it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.25it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.36it/s]
+
+  3%|▎         | 26/1000 [00:04<02:35,  6.27it/s]
+
+  3%|▎         | 28/1000 [00:04<02:41,  6.00it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.97it/s]
+
+  3%|▎         | 31/1000 [00:04<03:01,  5.35it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  6.00it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.67it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.10it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.57it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.75it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.10it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.33it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.37it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.76it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.91it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.95it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.77it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.32it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.19it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.33it/s]
+
+  6%|▋         | 63/1000 [00:08<01:36,  9.76it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.40it/s]
+
+  7%|▋         | 67/1000 [00:09<02:03,  7.57it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.26it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.00it/s]
+
+  7%|▋         | 72/1000 [00:09<02:17,  6.74it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.14it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.87it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.34it/s]
+
+  8%|▊         | 80/1000 [00:10<01:56,  7.88it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.23it/s]
+
+  8%|▊         | 84/1000 [00:11<01:53,  8.04it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.80it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.96it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.40it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.03it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.26it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.09it/s]
+
+  9%|▉         | 92/1000 [00:12<02:26,  6.18it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.54it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:12,  6.79it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.37it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.63it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.76it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.83it/s]
+
+ 11%|█         | 106/1000 [00:14<01:36,  9.31it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.45it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.22it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.05it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.54it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.83it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.22it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.92it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.09it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.43it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.72it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.81it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.11it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.72it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.16it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.65it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:26,  9.92it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.98it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.88it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:25, 10.05it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:41,  8.43it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.05it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.40it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.65it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.98it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.63it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.82it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.42it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:26,  5.69it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.19it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:23,  5.79it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.85it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.17it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.17it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.24it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.75it/s]
+
+ 18%|█▊        | 176/1000 [00:22<01:37,  8.45it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.82it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.59it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.52it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+
+ 18%|█▊        | 182/1000 [00:23<02:22,  5.73it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.38it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.12it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:01,  6.65it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.49it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.76it/s]
+
+ 20%|█▉        | 198/1000 [00:25<01:27,  9.21it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.66it/s]
+
+ 20%|██        | 202/1000 [00:26<01:30,  8.78it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.65it/s]
+
+ 21%|██        | 206/1000 [00:26<01:19,  9.96it/s]
+
+ 21%|██        | 208/1000 [00:27<01:32,  8.53it/s]
+
+ 21%|██        | 210/1000 [00:27<01:32,  8.51it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.70it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.76it/s]
+
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.17it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.51it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.81it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.32it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.56it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.05it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:06,  6.10it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.43it/s]
+
+ 23%|██▎       | 228/1000 [00:29<01:53,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.91it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.38it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.04it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.23it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.99it/s]
+
+ 24%|██▎       | 237/1000 [00:30<01:20,  9.48it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  9.00it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.39it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.03it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.48it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.68it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:30,  8.29it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.88it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:05,  5.97it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.88it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.04it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.76it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.70it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.75it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.70it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.53it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.44it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.60it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.92it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:37,  7.54it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.05it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:14,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.71it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:47,  6.71it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.08it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.82it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.63it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.05it/s]
+
+ 28%|██▊       | 283/1000 [00:37<02:02,  5.87it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.71it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:36,  7.44it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.76it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.79it/s]
+
+ 29%|██▉       | 291/1000 [00:38<01:41,  7.00it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.20it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:42,  6.90it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.18it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.47it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:38,  7.13it/s]
+
+ 30%|███       | 303/1000 [00:40<01:20,  8.68it/s]
+
+ 30%|███       | 304/1000 [00:40<01:26,  8.05it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.12it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.94it/s]
+
+ 31%|███       | 307/1000 [00:41<01:42,  6.79it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.81it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.21it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.26it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.25it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.59it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:34,  7.24it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.91it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.45it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.76it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.71it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.41it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.17it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.80it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.59it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.96it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.38it/s]
+
+ 34%|███▎      | 336/1000 [00:44<01:22,  8.04it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:15,  8.82it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.02it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.85it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.60it/s]
+
+ 34%|███▍      | 345/1000 [00:45<01:24,  7.72it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.23it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.42it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:33,  6.93it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:48,  5.97it/s]
+
+ 35%|███▌      | 352/1000 [00:47<01:24,  7.65it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.57it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.38it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.98it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.60it/s]
+
+ 36%|███▌      | 361/1000 [00:47<01:07,  9.44it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:30,  7.00it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:42,  6.18it/s]
+
+ 36%|███▋      | 365/1000 [00:49<02:10,  4.87it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:58,  5.33it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:06,  5.01it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.60it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.55it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.72it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.08it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.22it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.93it/s]
+
+ 38%|███▊      | 380/1000 [00:50<01:10,  8.80it/s]
+
+ 38%|███▊      | 382/1000 [00:51<01:18,  7.92it/s]
+
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.22it/s]
+
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.21it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.16it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.67it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.40it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:08,  8.82it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.49it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.77it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.24it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.50it/s]
+
+ 41%|████      | 408/1000 [00:54<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.37it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:23,  7.06it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.26it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.46it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:17,  7.51it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.69it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:37,  5.93it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.38it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.96it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.55it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.84it/s]
+
+ 43%|████▎     | 431/1000 [00:58<00:59,  9.56it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.03it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.79it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.54it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.10it/s]
+
+ 44%|████▍     | 441/1000 [00:59<00:59,  9.32it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.26it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.74it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.20it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:37,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:01<01:15,  7.19it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.51it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.81it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.27it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.85it/s]
+
+ 46%|████▋     | 463/1000 [01:02<01:10,  7.64it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.75it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.02it/s]
+
+ 47%|████▋     | 467/1000 [01:02<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.32it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.87it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.58it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.51it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.40it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.07it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.33it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.33it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.01it/s]
+
+ 49%|████▉     | 489/1000 [01:06<01:17,  6.63it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.22it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.28it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50,  9.97it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.40it/s]
+
+ 50%|████▉     | 498/1000 [01:06<00:59,  8.44it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.66it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.56it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:12,  6.82it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.57it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.82it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.44it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.98it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48,  9.98it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.28it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.53it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.12it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.09it/s]
+
+ 52%|█████▏    | 519/1000 [01:10<01:15,  6.39it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.04it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:21,  5.82it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.58it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.60it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.55it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.13it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:47,  9.74it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.71it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.67it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.77it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.95it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.07it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.51it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.71it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.52it/s]
+
+ 55%|█████▌    | 552/1000 [01:14<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.95it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.04it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.86it/s]
+
+ 56%|█████▌    | 561/1000 [01:16<00:43, 10.05it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.98it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.75it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.56it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.32it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.35it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.84it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.75it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.26it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.57it/s]
+
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.61it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.93it/s]
+
+ 59%|█████▊    | 587/1000 [01:19<00:57,  7.17it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.55it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.77it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.20it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.82it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.41it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.49it/s]
+
+ 60%|██████    | 601/1000 [01:21<01:02,  6.41it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.72it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.85it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:46,  8.47it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.73it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.82it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.11it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.04it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.22it/s]
+
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.46it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.23it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.87it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.45it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.72it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.07it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.96it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.19it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.97it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.61it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.57it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.33it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.70it/s]
+
+ 66%|██████▌   | 656/1000 [01:28<00:38,  8.91it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.58it/s]
+
+ 66%|██████▌   | 662/1000 [01:29<00:52,  6.49it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:30<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.20it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.65it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.67it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.71it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 11.00it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:32<00:31,  9.89it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+
+ 69%|██████▉   | 690/1000 [01:33<00:45,  6.81it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.84it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.42it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.95it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.60it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:53,  5.61it/s]
+
+ 70%|███████   | 704/1000 [01:35<00:40,  7.22it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.03it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.63it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.51it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:37,  7.73it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.88it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.39it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.08it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.81it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.95it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.17it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.24it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.32it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:27, 10.11it/s]
+
+ 73%|███████▎  | 729/1000 [01:38<00:23, 11.60it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.33it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.53it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.89it/s]
+
+ 74%|███████▎  | 737/1000 [01:39<00:27,  9.53it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.78it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.33it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.38it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.63it/s]
+
+ 75%|███████▍  | 747/1000 [01:40<00:24, 10.46it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:28,  8.66it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.71it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.18it/s]
+
+ 75%|███████▌  | 753/1000 [01:41<00:34,  7.25it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.24it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:37,  6.43it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.63it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.34it/s]
+
+ 76%|███████▌  | 761/1000 [01:42<00:27,  8.73it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.66it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.47it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.79it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.63it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.12it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:44<00:33,  6.79it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.79it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.50it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.91it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:26,  8.33it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.14it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.52it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.36it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  8.99it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.23it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.75it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  8.00it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.21it/s]
+
+ 80%|███████▉  | 797/1000 [01:47<00:29,  6.88it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.03it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.74it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.67it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:24,  7.84it/s]
+
+ 81%|████████  | 806/1000 [01:48<00:21,  9.17it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.23it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.24it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.07it/s]
+
+ 82%|████████▏ | 815/1000 [01:49<00:19,  9.33it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.15it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.21it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.07it/s]
+
+ 82%|████████▏ | 823/1000 [01:50<00:22,  7.84it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.77it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.81it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.25it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.76it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.25it/s]
+
+ 83%|████████▎ | 834/1000 [01:52<00:27,  6.04it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.54it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.12it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.46it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.64it/s]
+
+ 84%|████████▍ | 842/1000 [01:53<00:20,  7.87it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.32it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:27,  5.71it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.20it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:20,  7.50it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.47it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.86it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.46it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.04it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:14, 10.18it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.51it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.78it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.66it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.74it/s]
+
+ 87%|████████▋ | 867/1000 [01:56<00:14,  9.32it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.28it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.29it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.80it/s]
+
+ 88%|████████▊ | 875/1000 [01:57<00:16,  7.64it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.89it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.89it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.62it/s]
+
+ 88%|████████▊ | 882/1000 [01:58<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.97it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.17it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.35it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.62it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.86it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.55it/s]
+
+ 90%|█████████ | 900/1000 [02:00<00:12,  7.72it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.88it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.09it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.06it/s]
+
+ 91%|█████████ | 908/1000 [02:01<00:11,  8.33it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.61it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.80it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.40it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.10it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.38it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.61it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.73it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.03it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.52it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.00it/s]
+
+ 93%|█████████▎| 928/1000 [02:04<00:10,  6.60it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.43it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.79it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.60it/s]
+
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.17it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.30it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.83it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:06<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.40it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.73it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.56it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.21it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.08it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:06,  7.31it/s]
+
+ 96%|█████████▌| 957/1000 [02:08<00:06,  6.34it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.75it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.79it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.03it/s]
+
+ 96%|█████████▋| 964/1000 [02:09<00:04,  7.47it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.43it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.34it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.81it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.83it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.32it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.00it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.41it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.91it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.35it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.99it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.53it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.05it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.77it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7406', 'eval_runtime': '135.2', 'eval_samples_per_second': '7.395', 'eval_steps_per_second': '7.395', 'epoch': '0.9931', 'num_input_tokens_seen': 12115448}
+
+ 99%|█████████▉| 1800/1813 [3:20:39<01:22,  6.35s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_008_lr_1.865e-04/checkpoint-1800
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+ 99%|█████████▉| 1801/1813 [3:20:48<09:32, 47.67s/it]
+ 99%|█████████▉| 1802/1813 [3:20:53<06:24, 34.94s/it]
+ 99%|█████████▉| 1803/1813 [3:20:59<04:21, 26.12s/it]
+100%|█████████▉| 1804/1813 [3:21:06<03:04, 20.55s/it]
+100%|█████████▉| 1805/1813 [3:21:12<02:09, 16.17s/it]
+100%|█████████▉| 1806/1813 [3:21:19<01:32, 13.23s/it]
+100%|█████████▉| 1807/1813 [3:21:26<01:08, 11.43s/it]
+100%|█████████▉| 1808/1813 [3:21:33<00:51, 10.24s/it]
+100%|█████████▉| 1809/1813 [3:21:39<00:36,  9.04s/it]
+100%|█████████▉| 1810/1813 [3:21:45<00:23,  7.95s/it]
+                                                     
+{'loss': '0.8564', 'grad_norm': '0.8714', 'learning_rate': '2.662e-09', 'epoch': '0.9986', 'num_input_tokens_seen': 12187432, 'train_runtime': '1.211e+04', 'train_tokens_per_second': '1007'}
+
+100%|█████████▉| 1810/1813 [3:21:45<00:23,  7.95s/it]
+100%|█████████▉| 1811/1813 [3:21:50<00:14,  7.19s/it]
+100%|█████████▉| 1812/1813 [3:21:55<00:06,  6.59s/it]
+100%|██████████| 1813/1813 [3:21:58<00:00,  5.50s/it]
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+
+  0%|          | 3/1000 [00:00<01:03, 15.80it/s]
+
+  0%|          | 5/1000 [00:00<02:17,  7.22it/s]
+
+  1%|          | 7/1000 [00:00<01:59,  8.32it/s]
+
+  1%|          | 9/1000 [00:01<02:32,  6.48it/s]
+
+  1%|          | 11/1000 [00:01<02:02,  8.08it/s]
+
+  1%|▏         | 13/1000 [00:01<02:20,  7.01it/s]
+
+  1%|▏         | 14/1000 [00:01<02:24,  6.84it/s]
+
+  2%|▏         | 15/1000 [00:02<02:44,  5.98it/s]
+
+  2%|▏         | 16/1000 [00:02<02:51,  5.73it/s]
+
+  2%|▏         | 18/1000 [00:02<02:41,  6.07it/s]
+
+  2%|▏         | 19/1000 [00:02<03:10,  5.16it/s]
+
+  2%|▏         | 20/1000 [00:03<03:32,  4.61it/s]
+
+  2%|▏         | 22/1000 [00:03<02:36,  6.26it/s]
+
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+
+  2%|▎         | 25/1000 [00:03<02:12,  7.38it/s]
+
+  3%|▎         | 26/1000 [00:03<02:35,  6.27it/s]
+
+  3%|▎         | 28/1000 [00:04<02:41,  6.00it/s]
+
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+
+  3%|▎         | 30/1000 [00:04<02:42,  5.98it/s]
+
+  3%|▎         | 31/1000 [00:04<03:00,  5.36it/s]
+
+  3%|▎         | 32/1000 [00:05<02:41,  6.00it/s]
+
+  3%|▎         | 33/1000 [00:05<02:47,  5.76it/s]
+
+  4%|▎         | 35/1000 [00:05<02:05,  7.68it/s]
+
+  4%|▎         | 37/1000 [00:05<01:45,  9.12it/s]
+
+  4%|▍         | 39/1000 [00:05<02:06,  7.59it/s]
+
+  4%|▍         | 40/1000 [00:05<02:03,  7.76it/s]
+
+  4%|▍         | 42/1000 [00:06<01:45,  9.12it/s]
+
+  4%|▍         | 44/1000 [00:06<01:32, 10.35it/s]
+
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+
+  5%|▍         | 47/1000 [00:06<02:09,  7.38it/s]
+
+  5%|▍         | 49/1000 [00:06<01:48,  8.78it/s]
+
+  5%|▌         | 51/1000 [00:07<01:35,  9.93it/s]
+
+  5%|▌         | 53/1000 [00:07<01:26, 10.96it/s]
+
+  6%|▌         | 55/1000 [00:07<01:27, 10.77it/s]
+
+  6%|▌         | 57/1000 [00:07<01:41,  9.32it/s]
+
+  6%|▌         | 59/1000 [00:08<01:54,  8.22it/s]
+
+  6%|▌         | 61/1000 [00:08<01:40,  9.36it/s]
+
+  6%|▋         | 63/1000 [00:08<01:35,  9.79it/s]
+
+  6%|▋         | 65/1000 [00:08<01:51,  8.38it/s]
+
+  7%|▋         | 67/1000 [00:08<02:02,  7.60it/s]
+
+  7%|▋         | 69/1000 [00:09<01:52,  8.28it/s]
+
+  7%|▋         | 70/1000 [00:09<02:12,  7.01it/s]
+
+  7%|▋         | 72/1000 [00:09<02:18,  6.72it/s]
+
+  7%|▋         | 74/1000 [00:09<01:53,  8.13it/s]
+
+  8%|▊         | 75/1000 [00:10<02:14,  6.88it/s]
+
+  8%|▊         | 76/1000 [00:10<02:18,  6.69it/s]
+
+  8%|▊         | 78/1000 [00:10<02:25,  6.33it/s]
+
+  8%|▊         | 80/1000 [00:10<01:57,  7.86it/s]
+
+  8%|▊         | 82/1000 [00:10<01:39,  9.18it/s]
+
+  8%|▊         | 84/1000 [00:11<01:54,  8.02it/s]
+
+  8%|▊         | 85/1000 [00:11<02:14,  6.79it/s]
+
+  9%|▊         | 86/1000 [00:11<02:33,  5.95it/s]
+
+  9%|▊         | 87/1000 [00:11<02:49,  5.39it/s]
+
+  9%|▉         | 89/1000 [00:12<02:09,  7.01it/s]
+
+  9%|▉         | 90/1000 [00:12<02:05,  7.26it/s]
+
+  9%|▉         | 91/1000 [00:12<02:29,  6.08it/s]
+
+  9%|▉         | 92/1000 [00:12<02:26,  6.18it/s]
+
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+
+ 10%|▉         | 95/1000 [00:13<02:43,  5.53it/s]
+
+ 10%|▉         | 97/1000 [00:13<02:12,  6.80it/s]
+
+ 10%|▉         | 99/1000 [00:13<01:47,  8.38it/s]
+
+ 10%|█         | 101/1000 [00:13<01:33,  9.65it/s]
+
+ 10%|█         | 103/1000 [00:14<01:55,  7.76it/s]
+
+ 10%|█         | 104/1000 [00:14<01:54,  7.83it/s]
+
+ 11%|█         | 106/1000 [00:14<01:35,  9.32it/s]
+
+ 11%|█         | 108/1000 [00:14<01:25, 10.45it/s]
+
+ 11%|█         | 110/1000 [00:14<01:19, 11.22it/s]
+
+ 11%|█         | 112/1000 [00:14<01:13, 12.05it/s]
+
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.54it/s]
+
+ 12%|█▏        | 116/1000 [00:15<01:08, 12.84it/s]
+
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.26it/s]
+
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.93it/s]
+
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.10it/s]
+
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.44it/s]
+
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+
+ 13%|█▎        | 126/1000 [00:16<02:22,  6.15it/s]
+
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.81it/s]
+
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.30it/s]
+
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.11it/s]
+
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.17it/s]
+
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.62it/s]
+
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.88it/s]
+
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.90it/s]
+
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.08it/s]
+
+ 15%|█▍        | 147/1000 [00:19<01:40,  8.45it/s]
+
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.05it/s]
+
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.41it/s]
+
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.66it/s]
+
+ 16%|█▌        | 158/1000 [00:20<01:45,  7.99it/s]
+
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.63it/s]
+
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.82it/s]
+
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.51it/s]
+
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.42it/s]
+
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.68it/s]
+
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.19it/s]
+
+ 17%|█▋        | 167/1000 [00:21<02:23,  5.79it/s]
+
+ 17%|█▋        | 169/1000 [00:22<01:45,  7.85it/s]
+
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.17it/s]
+
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.16it/s]
+
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.23it/s]
+
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.76it/s]
+
+ 18%|█▊        | 176/1000 [00:22<01:37,  8.47it/s]
+
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.82it/s]
+
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.61it/s]
+
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.53it/s]
+
+ 18%|█▊        | 181/1000 [00:23<02:00,  6.77it/s]
+
+ 18%|█▊        | 182/1000 [00:23<02:22,  5.73it/s]
+
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.81it/s]
+
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.74it/s]
+
+ 19%|█▊        | 186/1000 [00:24<01:43,  7.90it/s]
+
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.40it/s]
+
+ 19%|█▉        | 189/1000 [00:24<01:41,  8.02it/s]
+
+ 19%|█▉        | 190/1000 [00:25<02:02,  6.60it/s]
+
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.48it/s]
+
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.37it/s]
+
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.75it/s]
+
+ 20%|█▉        | 198/1000 [00:25<01:28,  9.09it/s]
+
+ 20%|██        | 200/1000 [00:26<01:15, 10.60it/s]
+
+ 20%|██        | 202/1000 [00:26<01:31,  8.75it/s]
+
+ 20%|██        | 204/1000 [00:26<01:32,  8.63it/s]
+
+ 21%|██        | 206/1000 [00:26<01:20,  9.89it/s]
+
+ 21%|██        | 208/1000 [00:27<01:33,  8.50it/s]
+
+ 21%|██        | 210/1000 [00:27<01:33,  8.49it/s]
+
+ 21%|██        | 212/1000 [00:27<01:21,  9.69it/s]
+
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.74it/s]
+
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.16it/s]
+
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.52it/s]
+
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.81it/s]
+
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.32it/s]
+
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.31it/s]
+
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.57it/s]
+
+ 22%|██▏       | 224/1000 [00:29<02:07,  6.07it/s]
+
+ 22%|██▎       | 225/1000 [00:29<02:06,  6.11it/s]
+
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.42it/s]
+
+ 23%|██▎       | 228/1000 [00:29<01:52,  6.83it/s]
+
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.92it/s]
+
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.38it/s]
+
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.05it/s]
+
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.23it/s]
+
+ 24%|██▎       | 235/1000 [00:30<01:17,  9.93it/s]
+
+ 24%|██▎       | 237/1000 [00:30<01:20,  9.46it/s]
+
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.99it/s]
+
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.37it/s]
+
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.02it/s]
+
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.47it/s]
+
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.66it/s]
+
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.27it/s]
+
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.86it/s]
+
+ 25%|██▍       | 249/1000 [00:32<02:05,  5.96it/s]
+
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+
+ 25%|██▌       | 252/1000 [00:33<02:03,  6.04it/s]
+
+ 25%|██▌       | 253/1000 [00:33<02:09,  5.75it/s]
+
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.37it/s]
+
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.71it/s]
+
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.78it/s]
+
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.70it/s]
+
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.53it/s]
+
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.44it/s]
+
+ 26%|██▋       | 264/1000 [00:35<01:56,  6.34it/s]
+
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.61it/s]
+
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.92it/s]
+
+ 27%|██▋       | 268/1000 [00:35<01:36,  7.55it/s]
+
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.04it/s]
+
+ 27%|██▋       | 271/1000 [00:36<02:13,  5.44it/s]
+
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.71it/s]
+
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.71it/s]
+
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.08it/s]
+
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.82it/s]
+
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.63it/s]
+
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.04it/s]
+
+ 28%|██▊       | 283/1000 [00:38<02:02,  5.86it/s]
+
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.72it/s]
+
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.45it/s]
+
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.77it/s]
+
+ 29%|██▉       | 290/1000 [00:38<01:30,  7.80it/s]
+
+ 29%|██▉       | 291/1000 [00:38<01:40,  7.02it/s]
+
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+
+ 29%|██▉       | 294/1000 [00:39<01:37,  7.21it/s]
+
+ 30%|██▉       | 296/1000 [00:39<01:41,  6.91it/s]
+
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.20it/s]
+
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.48it/s]
+
+ 30%|███       | 300/1000 [00:40<01:42,  6.86it/s]
+
+ 30%|███       | 301/1000 [00:40<01:37,  7.14it/s]
+
+ 30%|███       | 303/1000 [00:40<01:19,  8.78it/s]
+
+ 30%|███       | 304/1000 [00:40<01:25,  8.10it/s]
+
+ 30%|███       | 305/1000 [00:40<01:25,  8.15it/s]
+
+ 31%|███       | 306/1000 [00:41<01:39,  6.96it/s]
+
+ 31%|███       | 307/1000 [00:41<01:41,  6.81it/s]
+
+ 31%|███       | 309/1000 [00:41<01:18,  8.82it/s]
+
+ 31%|███       | 311/1000 [00:41<01:07, 10.21it/s]
+
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.26it/s]
+
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.26it/s]
+
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.60it/s]
+
+ 32%|███▏      | 319/1000 [00:42<01:33,  7.25it/s]
+
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.45it/s]
+
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.76it/s]
+
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.71it/s]
+
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.42it/s]
+
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.18it/s]
+
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.80it/s]
+
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.59it/s]
+
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.97it/s]
+
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.39it/s]
+
+ 34%|███▎      | 336/1000 [00:44<01:22,  8.06it/s]
+
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.84it/s]
+
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.05it/s]
+
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.91it/s]
+
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.62it/s]
+
+ 34%|███▍      | 345/1000 [00:45<01:24,  7.75it/s]
+
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.24it/s]
+
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.43it/s]
+
+ 35%|███▍      | 349/1000 [00:46<01:33,  6.93it/s]
+
+ 35%|███▌      | 350/1000 [00:46<01:48,  5.97it/s]
+
+ 35%|███▌      | 352/1000 [00:46<01:24,  7.66it/s]
+
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.57it/s]
+
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.38it/s]
+
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.99it/s]
+
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.62it/s]
+
+ 36%|███▌      | 361/1000 [00:47<01:07,  9.45it/s]
+
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.19it/s]
+
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.29it/s]
+
+ 36%|███▋      | 365/1000 [00:48<02:08,  4.93it/s]
+
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.38it/s]
+
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.04it/s]
+
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.73it/s]
+
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.09it/s]
+
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.23it/s]
+
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.50it/s]
+
+ 38%|███▊      | 377/1000 [00:50<01:40,  6.20it/s]
+
+ 38%|███▊      | 379/1000 [00:50<01:16,  8.17it/s]
+
+ 38%|███▊      | 381/1000 [00:51<01:07,  9.19it/s]
+
+ 38%|███▊      | 383/1000 [00:51<01:11,  8.57it/s]
+
+ 38%|███▊      | 385/1000 [00:51<01:07,  9.10it/s]
+
+ 39%|███▊      | 387/1000 [00:51<01:20,  7.57it/s]
+
+ 39%|███▉      | 388/1000 [00:52<01:23,  7.31it/s]
+
+ 39%|███▉      | 390/1000 [00:52<01:17,  7.86it/s]
+
+ 39%|███▉      | 391/1000 [00:52<01:21,  7.51it/s]
+
+ 39%|███▉      | 393/1000 [00:52<01:07,  9.03it/s]
+
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.53it/s]
+
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.86it/s]
+
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.24it/s]
+
+ 40%|████      | 400/1000 [00:53<01:45,  5.69it/s]
+
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+
+ 40%|████      | 404/1000 [00:54<01:22,  7.25it/s]
+
+ 41%|████      | 406/1000 [00:54<01:09,  8.52it/s]
+
+ 41%|████      | 408/1000 [00:54<01:32,  6.40it/s]
+
+ 41%|████      | 410/1000 [00:55<01:31,  6.46it/s]
+
+ 41%|████      | 411/1000 [00:55<01:40,  5.85it/s]
+
+ 41%|████      | 412/1000 [00:55<01:49,  5.37it/s]
+
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.07it/s]
+
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.27it/s]
+
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.46it/s]
+
+ 42%|████▏     | 421/1000 [00:56<01:16,  7.52it/s]
+
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.70it/s]
+
+ 42%|████▏     | 424/1000 [00:57<01:36,  5.94it/s]
+
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.38it/s]
+
+ 43%|████▎     | 427/1000 [00:57<01:11,  7.97it/s]
+
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.56it/s]
+
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.86it/s]
+
+ 43%|████▎     | 431/1000 [00:57<00:59,  9.57it/s]
+
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.05it/s]
+
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.81it/s]
+
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.52it/s]
+
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.10it/s]
+
+ 44%|████▍     | 441/1000 [00:59<01:00,  9.31it/s]
+
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.10it/s]
+
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.25it/s]
+
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.72it/s]
+
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.49it/s]
+
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.02it/s]
+
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.19it/s]
+
+ 45%|████▌     | 452/1000 [01:00<01:38,  5.59it/s]
+
+ 45%|████▌     | 454/1000 [01:00<01:15,  7.20it/s]
+
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.52it/s]
+
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.82it/s]
+
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.27it/s]
+
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.85it/s]
+
+ 46%|████▋     | 463/1000 [01:01<01:10,  7.64it/s]
+
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.76it/s]
+
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.02it/s]
+
+ 47%|████▋     | 467/1000 [01:02<01:50,  4.83it/s]
+
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.32it/s]
+
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.87it/s]
+
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.58it/s]
+
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.52it/s]
+
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.40it/s]
+
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.07it/s]
+
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.32it/s]
+
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.02it/s]
+
+ 49%|████▉     | 489/1000 [01:05<01:16,  6.65it/s]
+
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.23it/s]
+
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.29it/s]
+
+ 49%|████▉     | 494/1000 [01:06<00:50, 10.01it/s]
+
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.43it/s]
+
+ 50%|████▉     | 498/1000 [01:06<00:59,  8.46it/s]
+
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.68it/s]
+
+ 50%|█████     | 500/1000 [01:07<01:24,  5.94it/s]
+
+ 50%|█████     | 502/1000 [01:07<01:05,  7.56it/s]
+
+ 50%|█████     | 503/1000 [01:07<01:12,  6.82it/s]
+
+ 50%|█████     | 505/1000 [01:08<01:15,  6.58it/s]
+
+ 51%|█████     | 506/1000 [01:08<01:24,  5.82it/s]
+
+ 51%|█████     | 508/1000 [01:08<01:06,  7.44it/s]
+
+ 51%|█████     | 510/1000 [01:08<00:54,  8.98it/s]
+
+ 51%|█████     | 512/1000 [01:08<00:48, 10.00it/s]
+
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.30it/s]
+
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.54it/s]
+
+ 52%|█████▏    | 516/1000 [01:09<01:08,  7.11it/s]
+
+ 52%|█████▏    | 518/1000 [01:09<01:00,  7.93it/s]
+
+ 52%|█████▏    | 519/1000 [01:09<01:16,  6.32it/s]
+
+ 52%|█████▏    | 521/1000 [01:10<01:00,  7.97it/s]
+
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.65it/s]
+
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.78it/s]
+
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.55it/s]
+
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.59it/s]
+
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.54it/s]
+
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.40it/s]
+
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.12it/s]
+
+ 53%|█████▎    | 533/1000 [01:11<00:47,  9.73it/s]
+
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.71it/s]
+
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.67it/s]
+
+ 54%|█████▍    | 539/1000 [01:13<01:17,  5.94it/s]
+
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.77it/s]
+
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.95it/s]
+
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.08it/s]
+
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.52it/s]
+
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.71it/s]
+
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.51it/s]
+
+ 55%|█████▌    | 552/1000 [01:14<01:01,  7.27it/s]
+
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.96it/s]
+
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.79it/s]
+
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.04it/s]
+
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.87it/s]
+
+ 56%|█████▌    | 561/1000 [01:15<00:43, 10.07it/s]
+
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.40it/s]
+
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.99it/s]
+
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.76it/s]
+
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.57it/s]
+
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.47it/s]
+
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.32it/s]
+
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.36it/s]
+
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.85it/s]
+
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.26it/s]
+
+ 58%|█████▊    | 580/1000 [01:19<01:00,  6.98it/s]
+
+ 58%|█████▊    | 582/1000 [01:19<00:50,  8.20it/s]
+
+ 58%|█████▊    | 583/1000 [01:19<01:00,  6.85it/s]
+
+ 58%|█████▊    | 585/1000 [01:19<00:59,  7.00it/s]
+
+ 59%|█████▊    | 587/1000 [01:19<00:57,  7.20it/s]
+
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.51it/s]
+
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.78it/s]
+
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.19it/s]
+
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.82it/s]
+
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.39it/s]
+
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.87it/s]
+
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.48it/s]
+
+ 60%|██████    | 601/1000 [01:21<01:02,  6.41it/s]
+
+ 60%|██████    | 602/1000 [01:22<00:59,  6.71it/s]
+
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+
+ 61%|██████    | 606/1000 [01:22<00:55,  7.11it/s]
+
+ 61%|██████    | 608/1000 [01:22<00:49,  7.86it/s]
+
+ 61%|██████    | 610/1000 [01:23<00:45,  8.48it/s]
+
+ 61%|██████    | 612/1000 [01:23<00:39,  9.74it/s]
+
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.81it/s]
+
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.76it/s]
+
+ 62%|██████▏   | 619/1000 [01:24<00:41,  9.11it/s]
+
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.04it/s]
+
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.22it/s]
+
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.47it/s]
+
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.64it/s]
+
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.19it/s]
+
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.22it/s]
+
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.87it/s]
+
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.46it/s]
+
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.73it/s]
+
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.08it/s]
+
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.97it/s]
+
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.21it/s]
+
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.98it/s]
+
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.62it/s]
+
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.12it/s]
+
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.60it/s]
+
+ 65%|██████▍   | 648/1000 [01:28<00:53,  6.53it/s]
+
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.59it/s]
+
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.34it/s]
+
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.72it/s]
+
+ 66%|██████▌   | 656/1000 [01:28<00:38,  8.93it/s]
+
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.01it/s]
+
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.57it/s]
+
+ 66%|██████▌   | 662/1000 [01:29<00:52,  6.49it/s]
+
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.33it/s]
+
+ 67%|██████▋   | 667/1000 [01:30<00:59,  5.62it/s]
+
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.21it/s]
+
+ 67%|██████▋   | 669/1000 [01:31<00:57,  5.71it/s]
+
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.65it/s]
+
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.29it/s]
+
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.67it/s]
+
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.70it/s]
+
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.96it/s]
+
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.93it/s]
+
+ 68%|██████▊   | 685/1000 [01:32<00:31,  9.91it/s]
+
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.86it/s]
+
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.16it/s]
+
+ 69%|██████▉   | 690/1000 [01:33<00:45,  6.82it/s]
+
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.85it/s]
+
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+
+ 70%|██████▉   | 698/1000 [01:35<00:43,  6.96it/s]
+
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.06it/s]
+
+ 70%|███████   | 701/1000 [01:35<00:39,  7.62it/s]
+
+ 70%|███████   | 702/1000 [01:35<00:52,  5.62it/s]
+
+ 70%|███████   | 704/1000 [01:35<00:40,  7.23it/s]
+
+ 70%|███████   | 705/1000 [01:36<00:54,  5.42it/s]
+
+ 71%|███████   | 707/1000 [01:36<00:41,  7.04it/s]
+
+ 71%|███████   | 709/1000 [01:36<00:38,  7.64it/s]
+
+ 71%|███████   | 710/1000 [01:36<00:44,  6.52it/s]
+
+ 71%|███████   | 712/1000 [01:37<00:38,  7.56it/s]
+
+ 71%|███████▏  | 713/1000 [01:37<00:37,  7.74it/s]
+
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.26it/s]
+
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.02it/s]
+
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.77it/s]
+
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.93it/s]
+
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.16it/s]
+
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.25it/s]
+
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.26it/s]
+
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.30it/s]
+
+ 73%|███████▎  | 727/1000 [01:38<00:27,  9.90it/s]
+
+ 73%|███████▎  | 729/1000 [01:39<00:24, 10.94it/s]
+
+ 73%|███████▎  | 731/1000 [01:39<00:33,  8.14it/s]
+
+ 73%|███████▎  | 733/1000 [01:39<00:28,  9.37it/s]
+
+ 74%|███████▎  | 735/1000 [01:39<00:27,  9.78it/s]
+
+ 74%|███████▎  | 737/1000 [01:39<00:27,  9.44it/s]
+
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.71it/s]
+
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.29it/s]
+
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.36it/s]
+
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.61it/s]
+
+ 75%|███████▍  | 747/1000 [01:40<00:24, 10.43it/s]
+
+ 75%|███████▍  | 749/1000 [01:41<00:29,  8.65it/s]
+
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.68it/s]
+
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.17it/s]
+
+ 75%|███████▌  | 753/1000 [01:41<00:34,  7.25it/s]
+
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.24it/s]
+
+ 76%|███████▌  | 756/1000 [01:42<00:39,  6.20it/s]
+
+ 76%|███████▌  | 758/1000 [01:42<00:32,  7.43it/s]
+
+ 76%|███████▌  | 759/1000 [01:42<00:33,  7.19it/s]
+
+ 76%|███████▌  | 761/1000 [01:43<00:27,  8.61it/s]
+
+ 76%|███████▋  | 763/1000 [01:43<00:31,  7.56it/s]
+
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.56it/s]
+
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.45it/s]
+
+ 77%|███████▋  | 768/1000 [01:44<00:29,  7.77it/s]
+
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.62it/s]
+
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.55it/s]
+
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.11it/s]
+
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.39it/s]
+
+ 77%|███████▋  | 774/1000 [01:45<00:33,  6.80it/s]
+
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.91it/s]
+
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.33it/s]
+
+ 78%|███████▊  | 780/1000 [01:45<00:32,  6.86it/s]
+
+ 78%|███████▊  | 782/1000 [01:45<00:26,  8.29it/s]
+
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.12it/s]
+
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.51it/s]
+
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.35it/s]
+
+ 79%|███████▉  | 790/1000 [01:46<00:23,  8.98it/s]
+
+ 79%|███████▉  | 791/1000 [01:47<00:25,  8.22it/s]
+
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.72it/s]
+
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.97it/s]
+
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.20it/s]
+
+ 80%|███████▉  | 797/1000 [01:47<00:29,  6.87it/s]
+
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.03it/s]
+
+ 80%|████████  | 800/1000 [01:48<00:29,  6.73it/s]
+
+ 80%|████████  | 802/1000 [01:48<00:25,  7.66it/s]
+
+ 80%|████████  | 804/1000 [01:48<00:24,  7.85it/s]
+
+ 81%|████████  | 806/1000 [01:49<00:21,  9.18it/s]
+
+ 81%|████████  | 807/1000 [01:49<00:21,  8.99it/s]
+
+ 81%|████████  | 809/1000 [01:49<00:18, 10.23it/s]
+
+ 81%|████████  | 811/1000 [01:49<00:16, 11.24it/s]
+
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.06it/s]
+
+ 82%|████████▏ | 815/1000 [01:49<00:19,  9.31it/s]
+
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.15it/s]
+
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.21it/s]
+
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+
+ 82%|████████▏ | 823/1000 [01:51<00:22,  7.82it/s]
+
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.76it/s]
+
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.80it/s]
+
+ 83%|████████▎ | 828/1000 [01:52<00:32,  5.23it/s]
+
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.74it/s]
+
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.76it/s]
+
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.24it/s]
+
+ 83%|████████▎ | 834/1000 [01:52<00:27,  6.01it/s]
+
+ 84%|████████▎ | 835/1000 [01:53<00:25,  6.53it/s]
+
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.12it/s]
+
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.47it/s]
+
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+
+ 84%|████████▍ | 842/1000 [01:53<00:19,  8.17it/s]
+
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.42it/s]
+
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.78it/s]
+
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.26it/s]
+
+ 85%|████████▍ | 849/1000 [01:55<00:19,  7.56it/s]
+
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.50it/s]
+
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.48it/s]
+
+ 86%|████████▌ | 855/1000 [01:55<00:14, 10.07it/s]
+
+ 86%|████████▌ | 857/1000 [01:55<00:13, 10.23it/s]
+
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.53it/s]
+
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.77it/s]
+
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.65it/s]
+
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.73it/s]
+
+ 87%|████████▋ | 867/1000 [01:56<00:14,  9.34it/s]
+
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.27it/s]
+
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.28it/s]
+
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.79it/s]
+
+ 88%|████████▊ | 875/1000 [01:57<00:16,  7.64it/s]
+
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.90it/s]
+
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.88it/s]
+
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.61it/s]
+
+ 88%|████████▊ | 882/1000 [01:59<00:18,  6.55it/s]
+
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.36it/s]
+
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.77it/s]
+
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.76it/s]
+
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.61it/s]
+
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.86it/s]
+
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.54it/s]
+
+ 90%|█████████ | 900/1000 [02:01<00:12,  7.71it/s]
+
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.88it/s]
+
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.08it/s]
+
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.06it/s]
+
+ 91%|█████████ | 908/1000 [02:01<00:11,  8.33it/s]
+
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.59it/s]
+
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.77it/s]
+
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.39it/s]
+
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.09it/s]
+
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.37it/s]
+
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.60it/s]
+
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.72it/s]
+
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.59it/s]
+
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.94it/s]
+
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.11it/s]
+
+ 93%|█████████▎| 928/1000 [02:04<00:10,  6.66it/s]
+
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.48it/s]
+
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.83it/s]
+
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.62it/s]
+
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.19it/s]
+
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.32it/s]
+
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.85it/s]
+
+ 94%|█████████▍| 939/1000 [02:06<00:07,  8.05it/s]
+
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.14it/s]
+
+ 94%|█████████▍| 942/1000 [02:06<00:09,  6.34it/s]
+
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.09it/s]
+
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.41it/s]
+
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.72it/s]
+
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.83it/s]
+
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.42it/s]
+
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.41it/s]
+
+ 96%|█████████▌| 957/1000 [02:08<00:06,  6.41it/s]
+
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.82it/s]
+
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.05it/s]
+
+ 96%|█████████▋| 964/1000 [02:09<00:04,  7.51it/s]
+
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.45it/s]
+
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.03it/s]
+
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.42it/s]
+
+ 97%|█████████▋| 972/1000 [02:10<00:03,  8.02it/s]
+
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.21it/s]
+
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.80it/s]
+
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.82it/s]
+
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.44it/s]
+
+ 98%|█████████▊| 981/1000 [02:12<00:02,  7.08it/s]
+
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.45it/s]
+
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.94it/s]
+
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.39it/s]
+
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.00it/s]
+
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+
+ 99%|█████████▉| 992/1000 [02:14<00:01,  4.99it/s]
+
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.55it/s]
+
+100%|█████████▉| 995/1000 [02:14<00:00,  5.54it/s]
+
+100%|█████████▉| 997/1000 [02:14<00:00,  7.06it/s]
+
+100%|█████████▉| 999/1000 [02:14<00:00,  6.78it/s]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+                                                     
+
+                                                   
+{'eval_loss': '0.7406', 'eval_runtime': '135.2', 'eval_samples_per_second': '7.394', 'eval_steps_per_second': '7.394', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:24:14<00:00,  5.50s/it]
+
+100%|██████████| 1000/1000 [02:15<00:00,  6.68it/s]
+
+                                                   Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_008_lr_1.865e-04/checkpoint-1813
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+
+
+Training completed. Do not forget to share your model on huggingface.co/models =)
+
+
+
+                                                     
+{'train_runtime': '1.226e+04', 'train_samples_per_second': '2.366', 'train_steps_per_second': '0.148', 'train_loss': '0.9515', 'epoch': '1', 'num_input_tokens_seen': 12201816}
+
+100%|██████████| 1813/1813 [3:24:15<00:00,  5.50s/it]
+100%|██████████| 1813/1813 [3:24:15<00:00,  6.76s/it]
+Saving model checkpoint to /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_008_lr_1.865e-04
+loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
+Model config MistralConfig {
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}
+
+***** train metrics *****
+  epoch                    =         1.0
+  num_input_tokens_seen    =    12201816
+  total_flos               = 487685654GF
+  train_loss               =      0.9515
+  train_runtime            =  3:24:15.17
+  train_samples_per_second =       2.366
+  train_steps_per_second   =       0.148
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_008_lr_1.865e-04/training_loss.png
+Figure saved at: /home/schwarc/diplomovka/optuna_lr_hpo_1gpu/trials/trial_008_lr_1.865e-04/training_eval_loss.png
+[WARNING|2026-10-16 04:59:54] llamafactory.extras.ploting:149 >> No metric eval_accuracy to plot.
+
+***** Running Evaluation *****
+  Num examples = 1000
+  Batch size = 1
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+/home/schwarc/miniconda3/envs/llama_factory_unsloth/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
+  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
+
+  0%|          | 0/1000 [00:00<?, ?it/s]
+  0%|          | 3/1000 [00:00<01:03, 15.82it/s]
+  0%|          | 5/1000 [00:00<02:17,  7.25it/s]
+  1%|          | 7/1000 [00:00<01:58,  8.38it/s]
+  1%|          | 9/1000 [00:01<02:32,  6.50it/s]
+  1%|          | 11/1000 [00:01<02:02,  8.10it/s]
+  1%|▏         | 13/1000 [00:01<02:20,  7.02it/s]
+  1%|▏         | 14/1000 [00:01<02:23,  6.85it/s]
+  2%|▏         | 15/1000 [00:02<02:44,  5.98it/s]
+  2%|▏         | 16/1000 [00:02<02:51,  5.74it/s]
+  2%|▏         | 18/1000 [00:02<02:41,  6.07it/s]
+  2%|▏         | 19/1000 [00:02<03:10,  5.15it/s]
+  2%|▏         | 20/1000 [00:03<03:32,  4.60it/s]
+  2%|▏         | 22/1000 [00:03<02:36,  6.26it/s]
+  2%|▏         | 23/1000 [00:03<02:54,  5.60it/s]
+  2%|▎         | 25/1000 [00:03<02:12,  7.37it/s]
+  3%|▎         | 26/1000 [00:03<02:35,  6.27it/s]
+  3%|▎         | 28/1000 [00:04<02:42,  6.00it/s]
+  3%|▎         | 29/1000 [00:04<02:32,  6.37it/s]
+  3%|▎         | 30/1000 [00:04<02:42,  5.98it/s]
+  3%|▎         | 31/1000 [00:04<03:00,  5.36it/s]
+  3%|▎         | 32/1000 [00:05<02:41,  6.01it/s]
+  3%|▎         | 33/1000 [00:05<02:47,  5.77it/s]
+  4%|▎         | 35/1000 [00:05<02:05,  7.69it/s]
+  4%|▎         | 37/1000 [00:05<01:45,  9.13it/s]
+  4%|▍         | 39/1000 [00:05<02:06,  7.59it/s]
+  4%|▍         | 40/1000 [00:05<02:03,  7.77it/s]
+  4%|▍         | 42/1000 [00:06<01:44,  9.13it/s]
+  4%|▍         | 44/1000 [00:06<01:32, 10.35it/s]
+  5%|▍         | 46/1000 [00:06<01:45,  9.06it/s]
+  5%|▍         | 47/1000 [00:06<02:09,  7.37it/s]
+  5%|▍         | 49/1000 [00:06<01:48,  8.76it/s]
+  5%|▌         | 51/1000 [00:07<01:35,  9.92it/s]
+  5%|▌         | 53/1000 [00:07<01:26, 10.93it/s]
+  6%|▌         | 55/1000 [00:07<01:27, 10.75it/s]
+  6%|▌         | 57/1000 [00:07<01:41,  9.31it/s]
+  6%|▌         | 59/1000 [00:08<01:54,  8.21it/s]
+  6%|▌         | 61/1000 [00:08<01:40,  9.36it/s]
+  6%|▋         | 63/1000 [00:08<01:35,  9.77it/s]
+  6%|▋         | 65/1000 [00:08<01:51,  8.39it/s]
+  7%|▋         | 67/1000 [00:08<02:02,  7.60it/s]
+  7%|▋         | 69/1000 [00:09<01:52,  8.28it/s]
+  7%|▋         | 70/1000 [00:09<02:12,  7.01it/s]
+  7%|▋         | 72/1000 [00:09<02:17,  6.73it/s]
+  7%|▋         | 74/1000 [00:09<01:53,  8.15it/s]
+  8%|▊         | 75/1000 [00:10<02:14,  6.88it/s]
+  8%|▊         | 76/1000 [00:10<02:18,  6.68it/s]
+  8%|▊         | 78/1000 [00:10<02:25,  6.35it/s]
+  8%|▊         | 80/1000 [00:10<01:56,  7.89it/s]
+  8%|▊         | 82/1000 [00:10<01:39,  9.23it/s]
+  8%|▊         | 84/1000 [00:11<01:53,  8.04it/s]
+  8%|▊         | 85/1000 [00:11<02:14,  6.81it/s]
+  9%|▊         | 86/1000 [00:11<02:33,  5.97it/s]
+  9%|▊         | 87/1000 [00:11<02:49,  5.39it/s]
+  9%|▉         | 89/1000 [00:12<02:09,  7.02it/s]
+  9%|▉         | 90/1000 [00:12<02:05,  7.26it/s]
+  9%|▉         | 91/1000 [00:12<02:29,  6.09it/s]
+  9%|▉         | 92/1000 [00:12<02:26,  6.19it/s]
+  9%|▉         | 93/1000 [00:12<02:36,  5.79it/s]
+ 10%|▉         | 95/1000 [00:13<02:43,  5.53it/s]
+ 10%|▉         | 97/1000 [00:13<02:12,  6.80it/s]
+ 10%|▉         | 99/1000 [00:13<01:47,  8.39it/s]
+ 10%|█         | 101/1000 [00:13<01:33,  9.65it/s]
+ 10%|█         | 103/1000 [00:14<01:55,  7.76it/s]
+ 10%|█         | 104/1000 [00:14<01:54,  7.84it/s]
+ 11%|█         | 106/1000 [00:14<01:35,  9.33it/s]
+ 11%|█         | 108/1000 [00:14<01:25, 10.47it/s]
+ 11%|█         | 110/1000 [00:14<01:19, 11.23it/s]
+ 11%|█         | 112/1000 [00:14<01:13, 12.06it/s]
+ 11%|█▏        | 114/1000 [00:14<01:10, 12.54it/s]
+ 12%|█▏        | 116/1000 [00:15<01:09, 12.81it/s]
+ 12%|█▏        | 118/1000 [00:15<01:06, 13.25it/s]
+ 12%|█▏        | 120/1000 [00:15<02:07,  6.93it/s]
+ 12%|█▏        | 122/1000 [00:15<01:48,  8.10it/s]
+ 12%|█▏        | 124/1000 [00:16<01:57,  7.44it/s]
+ 12%|█▎        | 125/1000 [00:16<02:32,  5.73it/s]
+ 13%|█▎        | 126/1000 [00:16<02:21,  6.16it/s]
+ 13%|█▎        | 127/1000 [00:16<02:30,  5.81it/s]
+ 13%|█▎        | 128/1000 [00:17<02:44,  5.29it/s]
+ 13%|█▎        | 130/1000 [00:17<02:02,  7.10it/s]
+ 13%|█▎        | 132/1000 [00:17<01:39,  8.71it/s]
+ 13%|█▎        | 134/1000 [00:17<01:25, 10.16it/s]
+ 14%|█▎        | 136/1000 [00:17<01:29,  9.67it/s]
+ 14%|█▍        | 138/1000 [00:18<01:27,  9.89it/s]
+ 14%|█▍        | 140/1000 [00:18<01:26,  9.97it/s]
+ 14%|█▍        | 142/1000 [00:18<01:34,  9.04it/s]
+ 14%|█▍        | 143/1000 [00:18<01:36,  8.91it/s]
+ 14%|█▍        | 145/1000 [00:18<01:24, 10.10it/s]
+ 15%|█▍        | 147/1000 [00:19<01:40,  8.45it/s]
+ 15%|█▍        | 148/1000 [00:19<01:40,  8.50it/s]
+ 15%|█▌        | 150/1000 [00:19<01:25,  9.93it/s]
+ 15%|█▌        | 152/1000 [00:19<01:16, 11.07it/s]
+ 15%|█▌        | 154/1000 [00:19<01:21, 10.44it/s]
+ 16%|█▌        | 156/1000 [00:20<02:06,  6.66it/s]
+ 16%|█▌        | 158/1000 [00:20<01:45,  8.00it/s]
+ 16%|█▌        | 160/1000 [00:20<01:37,  8.63it/s]
+ 16%|█▌        | 162/1000 [00:20<01:47,  7.83it/s]
+ 16%|█▋        | 163/1000 [00:21<01:51,  7.52it/s]
+ 16%|█▋        | 164/1000 [00:21<02:10,  6.41it/s]
+ 16%|█▋        | 165/1000 [00:21<02:27,  5.67it/s]
+ 17%|█▋        | 166/1000 [00:21<02:40,  5.19it/s]
+ 17%|█▋        | 167/1000 [00:21<02:23,  5.80it/s]
+ 17%|█▋        | 169/1000 [00:21<01:45,  7.86it/s]
+ 17%|█▋        | 171/1000 [00:22<01:55,  7.16it/s]
+ 17%|█▋        | 172/1000 [00:22<02:14,  6.15it/s]
+ 17%|█▋        | 173/1000 [00:22<02:12,  6.22it/s]
+ 17%|█▋        | 174/1000 [00:22<02:02,  6.75it/s]
+ 18%|█▊        | 176/1000 [00:22<01:37,  8.45it/s]
+ 18%|█▊        | 177/1000 [00:23<02:00,  6.81it/s]
+ 18%|█▊        | 179/1000 [00:23<01:35,  8.58it/s]
+ 18%|█▊        | 180/1000 [00:23<01:36,  8.52it/s]
+ 18%|█▊        | 181/1000 [00:23<02:01,  6.76it/s]
+ 18%|█▊        | 182/1000 [00:23<02:22,  5.73it/s]
+ 18%|█▊        | 183/1000 [00:24<02:20,  5.80it/s]
+ 18%|█▊        | 185/1000 [00:24<01:45,  7.76it/s]
+ 19%|█▊        | 186/1000 [00:24<01:42,  7.91it/s]
+ 19%|█▊        | 187/1000 [00:24<02:07,  6.39it/s]
+ 19%|█▉        | 189/1000 [00:24<01:39,  8.13it/s]
+ 19%|█▉        | 190/1000 [00:25<02:01,  6.64it/s]
+ 19%|█▉        | 192/1000 [00:25<02:04,  6.50it/s]
+ 19%|█▉        | 194/1000 [00:25<02:06,  6.38it/s]
+ 20%|█▉        | 196/1000 [00:25<01:43,  7.76it/s]
+ 20%|█▉        | 198/1000 [00:25<01:27,  9.21it/s]
+ 20%|██        | 200/1000 [00:26<01:14, 10.70it/s]
+ 20%|██        | 202/1000 [00:26<01:30,  8.79it/s]
+ 20%|██        | 204/1000 [00:26<01:32,  8.65it/s]
+ 21%|██        | 206/1000 [00:26<01:19,  9.94it/s]
+ 21%|██        | 208/1000 [00:27<01:32,  8.53it/s]
+ 21%|██        | 210/1000 [00:27<01:32,  8.50it/s]
+ 21%|██        | 212/1000 [00:27<01:21,  9.69it/s]
+ 21%|██▏       | 214/1000 [00:27<01:29,  8.75it/s]
+ 22%|██▏       | 216/1000 [00:27<01:25,  9.16it/s]
+ 22%|██▏       | 217/1000 [00:28<01:44,  7.51it/s]
+ 22%|██▏       | 219/1000 [00:28<01:28,  8.80it/s]
+ 22%|██▏       | 220/1000 [00:28<02:03,  6.32it/s]
+ 22%|██▏       | 221/1000 [00:28<02:03,  6.31it/s]
+ 22%|██▏       | 223/1000 [00:29<02:19,  5.56it/s]
+ 22%|██▏       | 224/1000 [00:29<02:08,  6.06it/s]
+ 22%|██▎       | 225/1000 [00:29<02:06,  6.11it/s]
+ 23%|██▎       | 226/1000 [00:29<02:22,  5.43it/s]
+ 23%|██▎       | 228/1000 [00:29<01:52,  6.83it/s]
+ 23%|██▎       | 229/1000 [00:30<02:10,  5.92it/s]
+ 23%|██▎       | 230/1000 [00:30<02:00,  6.39it/s]
+ 23%|██▎       | 232/1000 [00:30<01:35,  8.05it/s]
+ 23%|██▎       | 233/1000 [00:30<01:33,  8.24it/s]
+ 24%|██▎       | 235/1000 [00:30<01:16,  9.99it/s]
+ 24%|██▎       | 237/1000 [00:30<01:20,  9.49it/s]
+ 24%|██▍       | 239/1000 [00:31<01:24,  8.99it/s]
+ 24%|██▍       | 241/1000 [00:31<01:30,  8.38it/s]
+ 24%|██▍       | 242/1000 [00:31<02:05,  6.03it/s]
+ 24%|██▍       | 243/1000 [00:32<02:18,  5.48it/s]
+ 24%|██▍       | 245/1000 [00:32<01:53,  6.67it/s]
+ 25%|██▍       | 247/1000 [00:32<01:31,  8.27it/s]
+ 25%|██▍       | 248/1000 [00:32<01:49,  6.87it/s]
+ 25%|██▍       | 249/1000 [00:32<02:06,  5.96it/s]
+ 25%|██▌       | 250/1000 [00:33<02:07,  5.87it/s]
+ 25%|██▌       | 252/1000 [00:33<02:04,  6.02it/s]
+ 25%|██▌       | 253/1000 [00:33<02:10,  5.74it/s]
+ 26%|██▌       | 255/1000 [00:33<01:41,  7.36it/s]
+ 26%|██▌       | 256/1000 [00:33<01:36,  7.69it/s]
+ 26%|██▌       | 258/1000 [00:34<01:24,  8.76it/s]
+ 26%|██▌       | 260/1000 [00:34<01:36,  7.69it/s]
+ 26%|██▌       | 261/1000 [00:34<01:53,  6.52it/s]
+ 26%|██▌       | 262/1000 [00:34<01:54,  6.43it/s]
+ 26%|██▋       | 264/1000 [00:35<01:55,  6.35it/s]
+ 26%|██▋       | 265/1000 [00:35<01:51,  6.61it/s]
+ 27%|██▋       | 266/1000 [00:35<01:46,  6.92it/s]
+ 27%|██▋       | 268/1000 [00:35<01:36,  7.55it/s]
+ 27%|██▋       | 270/1000 [00:35<01:43,  7.04it/s]
+ 27%|██▋       | 271/1000 [00:36<02:13,  5.45it/s]
+ 27%|██▋       | 273/1000 [00:36<02:07,  5.71it/s]
+ 28%|██▊       | 275/1000 [00:36<01:48,  6.71it/s]
+ 28%|██▊       | 277/1000 [00:36<01:29,  8.07it/s]
+ 28%|██▊       | 278/1000 [00:37<01:45,  6.82it/s]
+ 28%|██▊       | 280/1000 [00:37<01:48,  6.63it/s]
+ 28%|██▊       | 282/1000 [00:37<01:29,  8.04it/s]
+ 28%|██▊       | 283/1000 [00:37<02:02,  5.86it/s]
+ 28%|██▊       | 284/1000 [00:38<02:05,  5.72it/s]
+ 29%|██▊       | 286/1000 [00:38<01:35,  7.44it/s]
+ 29%|██▉       | 288/1000 [00:38<01:21,  8.71it/s]
+ 29%|██▉       | 290/1000 [00:38<01:31,  7.76it/s]
+ 29%|██▉       | 291/1000 [00:38<01:41,  6.99it/s]
+ 29%|██▉       | 292/1000 [00:39<01:56,  6.08it/s]
+ 29%|██▉       | 294/1000 [00:39<01:38,  7.20it/s]
+ 30%|██▉       | 296/1000 [00:39<01:41,  6.90it/s]
+ 30%|██▉       | 298/1000 [00:39<01:25,  8.20it/s]
+ 30%|██▉       | 299/1000 [00:40<01:48,  6.48it/s]
+ 30%|███       | 300/1000 [00:40<01:41,  6.87it/s]
+ 30%|███       | 301/1000 [00:40<01:37,  7.15it/s]
+ 30%|███       | 303/1000 [00:40<01:19,  8.79it/s]
+ 30%|███       | 304/1000 [00:40<01:25,  8.10it/s]
+ 30%|███       | 305/1000 [00:40<01:25,  8.16it/s]
+ 31%|███       | 306/1000 [00:41<01:39,  6.97it/s]
+ 31%|███       | 307/1000 [00:41<01:41,  6.81it/s]
+ 31%|███       | 309/1000 [00:41<01:18,  8.81it/s]
+ 31%|███       | 311/1000 [00:41<01:07, 10.23it/s]
+ 31%|███▏      | 313/1000 [00:41<01:00, 11.30it/s]
+ 32%|███▏      | 315/1000 [00:41<01:06, 10.26it/s]
+ 32%|███▏      | 317/1000 [00:42<01:11,  9.58it/s]
+ 32%|███▏      | 319/1000 [00:42<01:33,  7.25it/s]
+ 32%|███▏      | 321/1000 [00:42<01:38,  6.90it/s]
+ 32%|███▏      | 322/1000 [00:43<01:45,  6.45it/s]
+ 32%|███▏      | 324/1000 [00:43<01:27,  7.76it/s]
+ 32%|███▎      | 325/1000 [00:43<01:58,  5.71it/s]
+ 33%|███▎      | 327/1000 [00:43<01:30,  7.42it/s]
+ 33%|███▎      | 328/1000 [00:43<01:33,  7.18it/s]
+ 33%|███▎      | 330/1000 [00:44<01:38,  6.79it/s]
+ 33%|███▎      | 332/1000 [00:44<01:41,  6.60it/s]
+ 33%|███▎      | 333/1000 [00:44<01:35,  6.98it/s]
+ 33%|███▎      | 334/1000 [00:44<01:44,  6.39it/s]
+ 34%|███▎      | 336/1000 [00:44<01:22,  8.06it/s]
+ 34%|███▍      | 338/1000 [00:45<01:14,  8.84it/s]
+ 34%|███▍      | 340/1000 [00:45<01:05, 10.06it/s]
+ 34%|███▍      | 342/1000 [00:45<01:00, 10.90it/s]
+ 34%|███▍      | 344/1000 [00:45<01:26,  7.62it/s]
+ 34%|███▍      | 345/1000 [00:45<01:24,  7.74it/s]
+ 35%|███▍      | 346/1000 [00:46<01:30,  7.24it/s]
+ 35%|███▍      | 347/1000 [00:46<01:27,  7.43it/s]
+ 35%|███▍      | 349/1000 [00:46<01:33,  6.93it/s]
+ 35%|███▌      | 350/1000 [00:46<01:48,  5.97it/s]
+ 35%|███▌      | 352/1000 [00:46<01:24,  7.65it/s]
+ 35%|███▌      | 354/1000 [00:47<01:25,  7.57it/s]
+ 36%|███▌      | 355/1000 [00:47<01:27,  7.37it/s]
+ 36%|███▌      | 357/1000 [00:47<01:11,  8.97it/s]
+ 36%|███▌      | 359/1000 [00:47<01:00, 10.60it/s]
+ 36%|███▌      | 361/1000 [00:47<01:07,  9.44it/s]
+ 36%|███▋      | 363/1000 [00:48<01:28,  7.20it/s]
+ 36%|███▋      | 364/1000 [00:48<01:41,  6.29it/s]
+ 36%|███▋      | 365/1000 [00:48<02:08,  4.93it/s]
+ 37%|███▋      | 367/1000 [00:49<01:57,  5.37it/s]
+ 37%|███▋      | 368/1000 [00:49<02:05,  5.03it/s]
+ 37%|███▋      | 369/1000 [00:49<01:52,  5.61it/s]
+ 37%|███▋      | 371/1000 [00:49<01:23,  7.56it/s]
+ 37%|███▋      | 372/1000 [00:49<01:21,  7.74it/s]
+ 37%|███▋      | 374/1000 [00:50<01:17,  8.11it/s]
+ 38%|███▊      | 375/1000 [00:50<01:40,  6.23it/s]
+ 38%|███▊      | 376/1000 [00:50<01:53,  5.49it/s]
+ 38%|███▊      | 378/1000 [00:50<01:29,  6.93it/s]
+ 38%|███▊      | 380/1000 [00:50<01:10,  8.78it/s]
+ 38%|███▊      | 382/1000 [00:51<01:17,  7.92it/s]
+ 38%|███▊      | 384/1000 [00:51<01:06,  9.23it/s]
+ 39%|███▊      | 386/1000 [00:51<01:06,  9.22it/s]
+ 39%|███▉      | 388/1000 [00:52<01:25,  7.15it/s]
+ 39%|███▉      | 390/1000 [00:52<01:19,  7.66it/s]
+ 39%|███▉      | 391/1000 [00:52<01:22,  7.38it/s]
+ 39%|███▉      | 393/1000 [00:52<01:09,  8.79it/s]
+ 40%|███▉      | 395/1000 [00:52<01:20,  7.48it/s]
+ 40%|███▉      | 397/1000 [00:53<01:08,  8.76it/s]
+ 40%|███▉      | 399/1000 [00:53<01:36,  6.24it/s]
+ 40%|████      | 400/1000 [00:53<01:45,  5.70it/s]
+ 40%|████      | 402/1000 [00:54<01:40,  5.93it/s]
+ 40%|████      | 404/1000 [00:54<01:22,  7.24it/s]
+ 41%|████      | 406/1000 [00:54<01:09,  8.49it/s]
+ 41%|████      | 408/1000 [00:54<01:32,  6.40it/s]
+ 41%|████      | 410/1000 [00:55<01:31,  6.45it/s]
+ 41%|████      | 411/1000 [00:55<01:40,  5.84it/s]
+ 41%|████      | 412/1000 [00:55<01:49,  5.37it/s]
+ 41%|████▏     | 414/1000 [00:55<01:22,  7.07it/s]
+ 42%|████▏     | 416/1000 [00:56<01:25,  6.82it/s]
+ 42%|████▏     | 418/1000 [00:56<01:10,  8.28it/s]
+ 42%|████▏     | 420/1000 [00:56<01:08,  8.46it/s]
+ 42%|████▏     | 421/1000 [00:56<01:16,  7.52it/s]
+ 42%|████▏     | 423/1000 [00:57<01:26,  6.70it/s]
+ 42%|████▏     | 424/1000 [00:57<01:36,  5.94it/s]
+ 42%|████▎     | 425/1000 [00:57<01:30,  6.37it/s]
+ 43%|████▎     | 427/1000 [00:57<01:12,  7.95it/s]
+ 43%|████▎     | 428/1000 [00:57<01:15,  7.55it/s]
+ 43%|████▎     | 429/1000 [00:57<01:12,  7.86it/s]
+ 43%|████▎     | 431/1000 [00:57<00:59,  9.57it/s]
+ 43%|████▎     | 433/1000 [00:58<00:51, 11.05it/s]
+ 44%|████▎     | 435/1000 [00:58<00:47, 11.82it/s]
+ 44%|████▎     | 437/1000 [00:58<00:53, 10.53it/s]
+ 44%|████▍     | 439/1000 [00:58<01:09,  8.10it/s]
+ 44%|████▍     | 441/1000 [00:58<00:59,  9.33it/s]
+ 44%|████▍     | 443/1000 [00:59<01:08,  8.13it/s]
+ 44%|████▍     | 444/1000 [00:59<01:16,  7.27it/s]
+ 45%|████▍     | 446/1000 [00:59<01:03,  8.73it/s]
+ 45%|████▍     | 448/1000 [01:00<01:13,  7.48it/s]
+ 45%|████▌     | 450/1000 [01:00<01:18,  7.03it/s]
+ 45%|████▌     | 451/1000 [01:00<01:28,  6.20it/s]
+ 45%|████▌     | 452/1000 [01:00<01:37,  5.59it/s]
+ 45%|████▌     | 454/1000 [01:00<01:15,  7.20it/s]
+ 46%|████▌     | 456/1000 [01:01<01:03,  8.52it/s]
+ 46%|████▌     | 457/1000 [01:01<01:09,  7.83it/s]
+ 46%|████▌     | 459/1000 [01:01<00:58,  9.29it/s]
+ 46%|████▌     | 461/1000 [01:01<00:49, 10.86it/s]
+ 46%|████▋     | 463/1000 [01:01<01:10,  7.64it/s]
+ 46%|████▋     | 465/1000 [01:02<01:19,  6.76it/s]
+ 47%|████▋     | 466/1000 [01:02<01:28,  6.03it/s]
+ 47%|████▋     | 467/1000 [01:02<01:50,  4.83it/s]
+ 47%|████▋     | 469/1000 [01:03<01:24,  6.32it/s]
+ 47%|████▋     | 471/1000 [01:03<01:07,  7.87it/s]
+ 47%|████▋     | 473/1000 [01:03<01:09,  7.59it/s]
+ 47%|████▋     | 474/1000 [01:03<01:20,  6.52it/s]
+ 48%|████▊     | 476/1000 [01:04<01:21,  6.40it/s]
+ 48%|████▊     | 477/1000 [01:04<01:35,  5.50it/s]
+ 48%|████▊     | 479/1000 [01:04<01:12,  7.16it/s]
+ 48%|████▊     | 480/1000 [01:04<01:18,  6.59it/s]
+ 48%|████▊     | 482/1000 [01:04<01:04,  8.05it/s]
+ 48%|████▊     | 484/1000 [01:05<01:10,  7.31it/s]
+ 48%|████▊     | 485/1000 [01:05<01:21,  6.32it/s]
+ 49%|████▊     | 486/1000 [01:05<01:21,  6.31it/s]
+ 49%|████▉     | 488/1000 [01:05<01:03,  8.01it/s]
+ 49%|████▉     | 489/1000 [01:05<01:17,  6.64it/s]
+ 49%|████▉     | 491/1000 [01:06<01:01,  8.23it/s]
+ 49%|████▉     | 492/1000 [01:06<01:01,  8.30it/s]
+ 49%|████▉     | 494/1000 [01:06<00:50, 10.01it/s]
+ 50%|████▉     | 496/1000 [01:06<00:59,  8.43it/s]
+ 50%|████▉     | 498/1000 [01:06<00:59,  8.43it/s]
+ 50%|████▉     | 499/1000 [01:07<01:15,  6.67it/s]
+ 50%|█████     | 500/1000 [01:07<01:24,  5.93it/s]
+ 50%|█████     | 502/1000 [01:07<01:05,  7.55it/s]
+ 50%|█████     | 503/1000 [01:07<01:12,  6.81it/s]
+ 50%|█████     | 505/1000 [01:08<01:15,  6.57it/s]
+ 51%|█████     | 506/1000 [01:08<01:24,  5.82it/s]
+ 51%|█████     | 508/1000 [01:08<01:06,  7.44it/s]
+ 51%|█████     | 510/1000 [01:08<00:54,  8.97it/s]
+ 51%|█████     | 512/1000 [01:08<00:48,  9.99it/s]
+ 51%|█████▏    | 514/1000 [01:09<01:06,  7.30it/s]
+ 52%|█████▏    | 515/1000 [01:09<01:04,  7.55it/s]
+ 52%|█████▏    | 516/1000 [01:09<01:07,  7.12it/s]
+ 52%|█████▏    | 518/1000 [01:09<00:59,  8.09it/s]
+ 52%|█████▏    | 519/1000 [01:09<01:15,  6.39it/s]
+ 52%|█████▏    | 521/1000 [01:10<00:59,  8.03it/s]
+ 52%|█████▏    | 522/1000 [01:10<01:11,  6.68it/s]
+ 52%|█████▏    | 523/1000 [01:10<01:22,  5.81it/s]
+ 52%|█████▏    | 524/1000 [01:10<01:25,  5.57it/s]
+ 53%|█████▎    | 526/1000 [01:11<01:11,  6.60it/s]
+ 53%|█████▎    | 528/1000 [01:11<01:12,  6.55it/s]
+ 53%|█████▎    | 529/1000 [01:11<01:13,  6.41it/s]
+ 53%|█████▎    | 531/1000 [01:11<00:57,  8.14it/s]
+ 53%|█████▎    | 533/1000 [01:11<00:47,  9.74it/s]
+ 54%|█████▎    | 535/1000 [01:12<01:09,  6.70it/s]
+ 54%|█████▎    | 536/1000 [01:12<01:12,  6.37it/s]
+ 54%|█████▎    | 537/1000 [01:12<01:21,  5.67it/s]
+ 54%|█████▍    | 539/1000 [01:12<01:17,  5.94it/s]
+ 54%|█████▍    | 540/1000 [01:13<01:19,  5.76it/s]
+ 54%|█████▍    | 541/1000 [01:13<01:27,  5.27it/s]
+ 54%|█████▍    | 543/1000 [01:13<01:05,  6.94it/s]
+ 55%|█████▍    | 545/1000 [01:13<01:04,  7.06it/s]
+ 55%|█████▍    | 546/1000 [01:14<01:09,  6.50it/s]
+ 55%|█████▍    | 547/1000 [01:14<01:19,  5.71it/s]
+ 55%|█████▍    | 549/1000 [01:14<01:16,  5.92it/s]
+ 55%|█████▌    | 551/1000 [01:14<00:59,  7.52it/s]
+ 55%|█████▌    | 552/1000 [01:14<01:01,  7.27it/s]
+ 55%|█████▌    | 553/1000 [01:15<01:12,  6.19it/s]
+ 55%|█████▌    | 554/1000 [01:15<01:14,  5.96it/s]
+ 56%|█████▌    | 556/1000 [01:15<00:57,  7.78it/s]
+ 56%|█████▌    | 557/1000 [01:15<01:02,  7.03it/s]
+ 56%|█████▌    | 559/1000 [01:15<00:49,  8.86it/s]
+ 56%|█████▌    | 561/1000 [01:15<00:43, 10.06it/s]
+ 56%|█████▋    | 563/1000 [01:16<00:52,  8.38it/s]
+ 56%|█████▋    | 565/1000 [01:16<00:54,  7.99it/s]
+ 57%|█████▋    | 566/1000 [01:16<01:04,  6.76it/s]
+ 57%|█████▋    | 568/1000 [01:17<01:05,  6.57it/s]
+ 57%|█████▋    | 569/1000 [01:17<01:06,  6.46it/s]
+ 57%|█████▋    | 571/1000 [01:17<01:06,  6.45it/s]
+ 57%|█████▋    | 572/1000 [01:17<01:07,  6.32it/s]
+ 57%|█████▋    | 574/1000 [01:18<01:07,  6.36it/s]
+ 58%|█████▊    | 576/1000 [01:18<00:54,  7.85it/s]
+ 58%|█████▊    | 577/1000 [01:18<01:13,  5.74it/s]
+ 58%|█████▊    | 578/1000 [01:18<01:20,  5.26it/s]
+ 58%|█████▊    | 580/1000 [01:18<01:00,  6.98it/s]
+ 58%|█████▊    | 582/1000 [01:19<00:48,  8.57it/s]
+ 58%|█████▊    | 584/1000 [01:19<00:54,  7.61it/s]
+ 58%|█████▊    | 585/1000 [01:19<00:59,  6.92it/s]
+ 59%|█████▊    | 587/1000 [01:19<00:57,  7.16it/s]
+ 59%|█████▉    | 589/1000 [01:20<00:48,  8.55it/s]
+ 59%|█████▉    | 590/1000 [01:20<01:00,  6.76it/s]
+ 59%|█████▉    | 592/1000 [01:20<00:49,  8.19it/s]
+ 59%|█████▉    | 593/1000 [01:20<00:59,  6.81it/s]
+ 60%|█████▉    | 595/1000 [01:20<00:48,  8.39it/s]
+ 60%|█████▉    | 596/1000 [01:21<00:58,  6.88it/s]
+ 60%|█████▉    | 597/1000 [01:21<01:00,  6.68it/s]
+ 60%|█████▉    | 599/1000 [01:21<01:01,  6.47it/s]
+ 60%|██████    | 601/1000 [01:21<01:02,  6.41it/s]
+ 60%|██████    | 602/1000 [01:22<00:59,  6.72it/s]
+ 60%|██████    | 604/1000 [01:22<01:09,  5.69it/s]
+ 61%|██████    | 606/1000 [01:22<00:55,  7.10it/s]
+ 61%|██████    | 608/1000 [01:22<00:49,  7.86it/s]
+ 61%|██████    | 610/1000 [01:23<00:45,  8.48it/s]
+ 61%|██████    | 612/1000 [01:23<00:39,  9.75it/s]
+ 61%|██████▏   | 614/1000 [01:23<00:35, 10.81it/s]
+ 62%|██████▏   | 616/1000 [01:23<00:50,  7.63it/s]
+ 62%|██████▏   | 617/1000 [01:23<00:49,  7.77it/s]
+ 62%|██████▏   | 619/1000 [01:23<00:41,  9.08it/s]
+ 62%|██████▏   | 621/1000 [01:24<00:47,  8.03it/s]
+ 62%|██████▏   | 623/1000 [01:24<00:45,  8.21it/s]
+ 62%|██████▎   | 625/1000 [01:24<00:50,  7.46it/s]
+ 63%|██████▎   | 627/1000 [01:25<00:43,  8.65it/s]
+ 63%|██████▎   | 628/1000 [01:25<00:51,  7.20it/s]
+ 63%|██████▎   | 629/1000 [01:25<00:59,  6.23it/s]
+ 63%|██████▎   | 631/1000 [01:25<00:46,  7.88it/s]
+ 63%|██████▎   | 633/1000 [01:25<00:38,  9.46it/s]
+ 64%|██████▎   | 635/1000 [01:26<00:54,  6.73it/s]
+ 64%|██████▎   | 636/1000 [01:26<00:51,  7.08it/s]
+ 64%|██████▍   | 638/1000 [01:26<00:45,  7.97it/s]
+ 64%|██████▍   | 640/1000 [01:26<00:39,  9.20it/s]
+ 64%|██████▍   | 642/1000 [01:27<00:44,  7.98it/s]
+ 64%|██████▍   | 643/1000 [01:27<00:46,  7.60it/s]
+ 64%|██████▍   | 645/1000 [01:27<00:49,  7.11it/s]
+ 65%|██████▍   | 647/1000 [01:27<00:46,  7.59it/s]
+ 65%|██████▍   | 648/1000 [01:27<00:53,  6.53it/s]
+ 65%|██████▌   | 650/1000 [01:28<00:53,  6.56it/s]
+ 65%|██████▌   | 652/1000 [01:28<00:54,  6.33it/s]
+ 65%|██████▌   | 654/1000 [01:28<00:44,  7.71it/s]
+ 66%|██████▌   | 656/1000 [01:28<00:38,  8.93it/s]
+ 66%|██████▌   | 658/1000 [01:29<00:42,  8.13it/s]
+ 66%|██████▌   | 659/1000 [01:29<00:56,  6.02it/s]
+ 66%|██████▌   | 661/1000 [01:29<00:44,  7.58it/s]
+ 66%|██████▌   | 662/1000 [01:29<00:52,  6.49it/s]
+ 66%|██████▋   | 664/1000 [01:30<00:58,  5.78it/s]
+ 66%|██████▋   | 665/1000 [01:30<01:02,  5.34it/s]
+ 67%|██████▋   | 667/1000 [01:30<00:59,  5.62it/s]
+ 67%|██████▋   | 668/1000 [01:31<01:03,  5.21it/s]
+ 67%|██████▋   | 669/1000 [01:31<00:58,  5.70it/s]
+ 67%|██████▋   | 671/1000 [01:31<00:55,  5.89it/s]
+ 67%|██████▋   | 673/1000 [01:31<00:49,  6.64it/s]
+ 68%|██████▊   | 675/1000 [01:32<00:44,  7.28it/s]
+ 68%|██████▊   | 677/1000 [01:32<00:37,  8.66it/s]
+ 68%|██████▊   | 679/1000 [01:32<00:33,  9.69it/s]
+ 68%|██████▊   | 681/1000 [01:32<00:29, 10.96it/s]
+ 68%|██████▊   | 683/1000 [01:32<00:35,  8.91it/s]
+ 68%|██████▊   | 685/1000 [01:32<00:31,  9.88it/s]
+ 69%|██████▊   | 687/1000 [01:33<00:45,  6.87it/s]
+ 69%|██████▉   | 688/1000 [01:33<00:43,  7.17it/s]
+ 69%|██████▉   | 690/1000 [01:33<00:45,  6.81it/s]
+ 69%|██████▉   | 692/1000 [01:34<00:39,  7.85it/s]
+ 69%|██████▉   | 694/1000 [01:34<00:42,  7.26it/s]
+ 70%|██████▉   | 695/1000 [01:34<00:41,  7.43it/s]
+ 70%|██████▉   | 696/1000 [01:34<00:47,  6.34it/s]
+ 70%|██████▉   | 698/1000 [01:34<00:43,  6.95it/s]
+ 70%|██████▉   | 699/1000 [01:35<00:49,  6.05it/s]
+ 70%|███████   | 701/1000 [01:35<00:39,  7.62it/s]
+ 70%|███████   | 702/1000 [01:35<00:52,  5.62it/s]
+ 70%|███████   | 704/1000 [01:35<00:40,  7.23it/s]
+ 70%|███████   | 705/1000 [01:36<00:54,  5.43it/s]
+ 71%|███████   | 707/1000 [01:36<00:41,  7.04it/s]
+ 71%|███████   | 709/1000 [01:36<00:38,  7.64it/s]
+ 71%|███████   | 710/1000 [01:36<00:44,  6.51it/s]
+ 71%|███████   | 712/1000 [01:36<00:37,  7.71it/s]
+ 71%|███████▏  | 713/1000 [01:37<00:36,  7.86it/s]
+ 72%|███████▏  | 715/1000 [01:37<00:30,  9.36it/s]
+ 72%|███████▏  | 717/1000 [01:37<00:35,  8.07it/s]
+ 72%|███████▏  | 718/1000 [01:37<00:41,  6.81it/s]
+ 72%|███████▏  | 719/1000 [01:38<00:47,  5.94it/s]
+ 72%|███████▏  | 720/1000 [01:38<00:43,  6.47it/s]
+ 72%|███████▏  | 722/1000 [01:38<00:34,  8.17it/s]
+ 72%|███████▏  | 723/1000 [01:38<00:33,  8.25it/s]
+ 72%|███████▏  | 724/1000 [01:38<00:33,  8.26it/s]
+ 72%|███████▎  | 725/1000 [01:38<00:33,  8.33it/s]
+ 73%|███████▎  | 727/1000 [01:38<00:26, 10.14it/s]
+ 73%|███████▎  | 729/1000 [01:38<00:23, 11.63it/s]
+ 73%|███████▎  | 731/1000 [01:39<00:32,  8.34it/s]
+ 73%|███████▎  | 733/1000 [01:39<00:27,  9.55it/s]
+ 74%|███████▎  | 735/1000 [01:39<00:26,  9.91it/s]
+ 74%|███████▎  | 737/1000 [01:39<00:27,  9.54it/s]
+ 74%|███████▍  | 739/1000 [01:40<00:26,  9.79it/s]
+ 74%|███████▍  | 741/1000 [01:40<00:31,  8.32it/s]
+ 74%|███████▍  | 743/1000 [01:40<00:30,  8.38it/s]
+ 74%|███████▍  | 745/1000 [01:40<00:26,  9.64it/s]
+ 75%|███████▍  | 747/1000 [01:40<00:24, 10.47it/s]
+ 75%|███████▍  | 749/1000 [01:41<00:28,  8.66it/s]
+ 75%|███████▌  | 750/1000 [01:41<00:32,  7.70it/s]
+ 75%|███████▌  | 751/1000 [01:41<00:40,  6.18it/s]
+ 75%|███████▌  | 753/1000 [01:41<00:34,  7.25it/s]
+ 75%|███████▌  | 754/1000 [01:42<00:39,  6.23it/s]
+ 76%|███████▌  | 756/1000 [01:42<00:38,  6.39it/s]
+ 76%|███████▌  | 758/1000 [01:42<00:31,  7.60it/s]
+ 76%|███████▌  | 759/1000 [01:42<00:32,  7.32it/s]
+ 76%|███████▌  | 761/1000 [01:42<00:27,  8.71it/s]
+ 76%|███████▋  | 763/1000 [01:43<00:30,  7.65it/s]
+ 76%|███████▋  | 764/1000 [01:43<00:35,  6.61it/s]
+ 77%|███████▋  | 766/1000 [01:43<00:36,  6.47it/s]
+ 77%|███████▋  | 768/1000 [01:43<00:29,  7.79it/s]
+ 77%|███████▋  | 769/1000 [01:44<00:34,  6.64it/s]
+ 77%|███████▋  | 770/1000 [01:44<00:35,  6.56it/s]
+ 77%|███████▋  | 772/1000 [01:44<00:28,  8.13it/s]
+ 77%|███████▋  | 773/1000 [01:44<00:35,  6.39it/s]
+ 77%|███████▋  | 774/1000 [01:44<00:33,  6.79it/s]
+ 78%|███████▊  | 776/1000 [01:45<00:28,  7.91it/s]
+ 78%|███████▊  | 778/1000 [01:45<00:23,  9.63it/s]
+ 78%|███████▊  | 780/1000 [01:45<00:31,  6.96it/s]
+ 78%|███████▊  | 782/1000 [01:45<00:25,  8.39it/s]
+ 78%|███████▊  | 784/1000 [01:46<00:30,  7.17it/s]
+ 79%|███████▊  | 786/1000 [01:46<00:28,  7.55it/s]
+ 79%|███████▉  | 788/1000 [01:46<00:25,  8.38it/s]
+ 79%|███████▉  | 790/1000 [01:46<00:23,  9.01it/s]
+ 79%|███████▉  | 791/1000 [01:46<00:25,  8.24it/s]
+ 79%|███████▉  | 792/1000 [01:47<00:26,  7.74it/s]
+ 79%|███████▉  | 793/1000 [01:47<00:25,  7.98it/s]
+ 80%|███████▉  | 795/1000 [01:47<00:28,  7.19it/s]
+ 80%|███████▉  | 797/1000 [01:47<00:29,  6.86it/s]
+ 80%|███████▉  | 798/1000 [01:48<00:33,  6.02it/s]
+ 80%|████████  | 800/1000 [01:48<00:29,  6.72it/s]
+ 80%|████████  | 802/1000 [01:48<00:25,  7.62it/s]
+ 80%|████████  | 804/1000 [01:48<00:25,  7.82it/s]
+ 81%|████████  | 806/1000 [01:48<00:21,  9.16it/s]
+ 81%|████████  | 807/1000 [01:49<00:21,  8.97it/s]
+ 81%|████████  | 809/1000 [01:49<00:18, 10.21it/s]
+ 81%|████████  | 811/1000 [01:49<00:16, 11.24it/s]
+ 81%|████████▏ | 813/1000 [01:49<00:20,  9.05it/s]
+ 82%|████████▏ | 815/1000 [01:49<00:19,  9.33it/s]
+ 82%|████████▏ | 817/1000 [01:50<00:22,  8.15it/s]
+ 82%|████████▏ | 819/1000 [01:50<00:19,  9.21it/s]
+ 82%|████████▏ | 821/1000 [01:50<00:22,  8.05it/s]
+ 82%|████████▏ | 823/1000 [01:50<00:22,  7.84it/s]
+ 82%|████████▏ | 824/1000 [01:51<00:26,  6.77it/s]
+ 82%|████████▎ | 825/1000 [01:51<00:24,  7.01it/s]
+ 83%|████████▎ | 826/1000 [01:51<00:29,  5.80it/s]
+ 83%|████████▎ | 828/1000 [01:51<00:32,  5.25it/s]
+ 83%|████████▎ | 829/1000 [01:52<00:29,  5.75it/s]
+ 83%|████████▎ | 831/1000 [01:52<00:24,  7.02it/s]
+ 83%|████████▎ | 832/1000 [01:52<00:29,  5.75it/s]
+ 83%|████████▎ | 833/1000 [01:52<00:26,  6.23it/s]
+ 83%|████████▎ | 834/1000 [01:52<00:27,  6.02it/s]
+ 84%|████████▎ | 835/1000 [01:52<00:25,  6.54it/s]
+ 84%|████████▎ | 836/1000 [01:53<00:26,  6.12it/s]
+ 84%|████████▍ | 838/1000 [01:53<00:21,  7.45it/s]
+ 84%|████████▍ | 840/1000 [01:53<00:24,  6.63it/s]
+ 84%|████████▍ | 842/1000 [01:53<00:19,  8.03it/s]
+ 84%|████████▍ | 844/1000 [01:54<00:24,  6.38it/s]
+ 84%|████████▍ | 845/1000 [01:54<00:26,  5.75it/s]
+ 85%|████████▍ | 847/1000 [01:54<00:24,  6.23it/s]
+ 85%|████████▍ | 849/1000 [01:54<00:20,  7.54it/s]
+ 85%|████████▌ | 850/1000 [01:55<00:23,  6.49it/s]
+ 85%|████████▌ | 851/1000 [01:55<00:21,  6.87it/s]
+ 85%|████████▌ | 853/1000 [01:55<00:17,  8.48it/s]
+ 86%|████████▌ | 855/1000 [01:55<00:14,  9.82it/s]
+ 86%|████████▌ | 857/1000 [01:55<00:14, 10.07it/s]
+ 86%|████████▌ | 859/1000 [01:56<00:16,  8.46it/s]
+ 86%|████████▌ | 861/1000 [01:56<00:14,  9.72it/s]
+ 86%|████████▋ | 863/1000 [01:56<00:12, 10.60it/s]
+ 86%|████████▋ | 865/1000 [01:56<00:11, 11.44it/s]
+ 87%|████████▋ | 867/1000 [01:56<00:14,  9.22it/s]
+ 87%|████████▋ | 869/1000 [01:57<00:12, 10.20it/s]
+ 87%|████████▋ | 871/1000 [01:57<00:11, 11.23it/s]
+ 87%|████████▋ | 873/1000 [01:57<00:16,  7.76it/s]
+ 88%|████████▊ | 875/1000 [01:57<00:16,  7.61it/s]
+ 88%|████████▊ | 877/1000 [01:58<00:13,  8.87it/s]
+ 88%|████████▊ | 879/1000 [01:58<00:17,  6.87it/s]
+ 88%|████████▊ | 881/1000 [01:58<00:15,  7.60it/s]
+ 88%|████████▊ | 882/1000 [01:58<00:18,  6.54it/s]
+ 88%|████████▊ | 884/1000 [01:59<00:14,  7.96it/s]
+ 89%|████████▊ | 886/1000 [01:59<00:12,  9.18it/s]
+ 89%|████████▉ | 888/1000 [01:59<00:10, 10.37it/s]
+ 89%|████████▉ | 890/1000 [01:59<00:11,  9.79it/s]
+ 89%|████████▉ | 892/1000 [01:59<00:10, 10.78it/s]
+ 89%|████████▉ | 894/1000 [02:00<00:13,  7.62it/s]
+ 90%|████████▉ | 896/1000 [02:00<00:11,  8.85it/s]
+ 90%|████████▉ | 898/1000 [02:00<00:15,  6.55it/s]
+ 90%|█████████ | 900/1000 [02:00<00:12,  7.71it/s]
+ 90%|█████████ | 902/1000 [02:01<00:12,  7.87it/s]
+ 90%|█████████ | 904/1000 [02:01<00:10,  9.09it/s]
+ 91%|█████████ | 906/1000 [02:01<00:13,  7.07it/s]
+ 91%|█████████ | 908/1000 [02:01<00:11,  8.33it/s]
+ 91%|█████████ | 910/1000 [02:02<00:11,  7.61it/s]
+ 91%|█████████ | 912/1000 [02:02<00:10,  8.79it/s]
+ 91%|█████████▏| 914/1000 [02:02<00:11,  7.40it/s]
+ 92%|█████████▏| 916/1000 [02:03<00:11,  7.05it/s]
+ 92%|█████████▏| 917/1000 [02:03<00:11,  7.33it/s]
+ 92%|█████████▏| 918/1000 [02:03<00:14,  5.59it/s]
+ 92%|█████████▏| 919/1000 [02:03<00:14,  5.70it/s]
+ 92%|█████████▏| 921/1000 [02:04<00:14,  5.57it/s]
+ 92%|█████████▏| 922/1000 [02:04<00:13,  5.93it/s]
+ 92%|█████████▏| 924/1000 [02:04<00:10,  7.17it/s]
+ 93%|█████████▎| 926/1000 [02:04<00:08,  8.66it/s]
+ 93%|█████████▎| 927/1000 [02:04<00:09,  8.10it/s]
+ 93%|█████████▎| 928/1000 [02:04<00:10,  6.65it/s]
+ 93%|█████████▎| 930/1000 [02:05<00:10,  6.49it/s]
+ 93%|█████████▎| 931/1000 [02:05<00:10,  6.84it/s]
+ 93%|█████████▎| 933/1000 [02:05<00:10,  6.63it/s]
+ 93%|█████████▎| 934/1000 [02:05<00:10,  6.19it/s]
+ 94%|█████████▎| 936/1000 [02:06<00:08,  7.32it/s]
+ 94%|█████████▍| 938/1000 [02:06<00:07,  8.71it/s]
+ 94%|█████████▍| 939/1000 [02:06<00:07,  7.98it/s]
+ 94%|█████████▍| 940/1000 [02:06<00:07,  8.10it/s]
+ 94%|█████████▍| 942/1000 [02:06<00:09,  6.33it/s]
+ 94%|█████████▍| 944/1000 [02:07<00:07,  7.06it/s]
+ 95%|█████████▍| 946/1000 [02:07<00:06,  8.37it/s]
+ 95%|█████████▍| 948/1000 [02:07<00:05,  9.71it/s]
+ 95%|█████████▌| 950/1000 [02:07<00:05,  8.83it/s]
+ 95%|█████████▌| 952/1000 [02:08<00:06,  7.41it/s]
+ 95%|█████████▌| 954/1000 [02:08<00:07,  6.17it/s]
+ 96%|█████████▌| 956/1000 [02:08<00:05,  7.41it/s]
+ 96%|█████████▌| 957/1000 [02:08<00:06,  6.41it/s]
+ 96%|█████████▌| 959/1000 [02:09<00:07,  5.78it/s]
+ 96%|█████████▌| 961/1000 [02:09<00:05,  6.83it/s]
+ 96%|█████████▌| 962/1000 [02:09<00:06,  6.06it/s]
+ 96%|█████████▋| 964/1000 [02:09<00:04,  7.51it/s]
+ 96%|█████████▋| 965/1000 [02:10<00:05,  6.46it/s]
+ 97%|█████████▋| 966/1000 [02:10<00:05,  6.35it/s]
+ 97%|█████████▋| 968/1000 [02:10<00:03,  8.03it/s]
+ 97%|█████████▋| 970/1000 [02:10<00:03,  9.43it/s]
+ 97%|█████████▋| 972/1000 [02:10<00:03,  8.03it/s]
+ 97%|█████████▋| 974/1000 [02:11<00:02,  9.20it/s]
+ 98%|█████████▊| 976/1000 [02:11<00:02,  9.82it/s]
+ 98%|█████████▊| 978/1000 [02:11<00:02, 10.84it/s]
+ 98%|█████████▊| 980/1000 [02:11<00:02,  8.45it/s]
+ 98%|█████████▊| 981/1000 [02:11<00:02,  7.07it/s]
+ 98%|█████████▊| 982/1000 [02:12<00:03,  5.45it/s]
+ 98%|█████████▊| 984/1000 [02:12<00:02,  6.95it/s]
+ 99%|█████████▊| 986/1000 [02:12<00:01,  8.38it/s]
+ 99%|█████████▉| 988/1000 [02:13<00:01,  6.74it/s]
+ 99%|█████████▉| 989/1000 [02:13<00:01,  6.01it/s]
+ 99%|█████████▉| 991/1000 [02:13<00:01,  6.16it/s]
+ 99%|█████████▉| 992/1000 [02:13<00:01,  4.98it/s]
+ 99%|█████████▉| 994/1000 [02:14<00:00,  6.54it/s]
+100%|█████████▉| 995/1000 [02:14<00:00,  5.54it/s]
+100%|█████████▉| 997/1000 [02:14<00:00,  6.89it/s]
+100%|█████████▉| 999/1000 [02:14<00:00,  6.69it/s]
+100%|██████████| 1000/1000 [02:14<00:00,  6.62it/s]
+100%|██████████| 1000/1000 [02:14<00:00,  7.41it/s]
+***** eval metrics *****
+  epoch                   =        1.0
+  eval_loss               =     0.7406
+  eval_runtime            = 0:02:15.16
+  eval_samples_per_second =      7.399
+  eval_steps_per_second   =      7.399
+  num_input_tokens_seen   =   12201816
+Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
diff --git a/optuna_30k_experiments/results/lr_vs_eval_loss.png b/optuna_30k_experiments/results/lr_vs_eval_loss.png
new file mode 100644
index 0000000..bc7ee1c
Binary files /dev/null and b/optuna_30k_experiments/results/lr_vs_eval_loss.png differ
diff --git a/optuna_30k_experiments/results/lr_vs_max_grad_norm.png b/optuna_30k_experiments/results/lr_vs_max_grad_norm.png
new file mode 100644
index 0000000..7f9cc7d
Binary files /dev/null and b/optuna_30k_experiments/results/lr_vs_max_grad_norm.png differ
diff --git a/optuna_30k_experiments/results/optuna_lr_results.csv b/optuna_30k_experiments/results/optuna_lr_results.csv
new file mode 100644
index 0000000..c3595b2
--- /dev/null
+++ b/optuna_30k_experiments/results/optuna_lr_results.csv
@@ -0,0 +1,9 @@
+rank,trial,state,learning_rate,final_eval_loss,final_eval_step,min_eval_loss,min_eval_step,max_grad_norm,grad_status,train_loss,train_runtime_seconds,global_step,config,results_dir
+1,3,COMPLETE,0.0002,0.7404094338417053,1813,0.7404094338417053,1813,14.712687492370605,OK,0.9520373807319014,12634.3569,1813,configs/trial_003_lr_2.000e-04.yaml,trials/trial_003_lr_2.000e-04
+2,8,COMPLETE,0.00018645202585249604,0.7405665516853333,1813,0.7405665516853333,1813,3.499601125717163,OK,0.9515243444532172,12255.1774,1813,configs/trial_008_lr_1.865e-04.yaml,trials/trial_008_lr_1.865e-04
+3,4,COMPLETE,0.00018690446893209942,0.741250216960907,1813,0.741250216960907,1813,29.025802612304688,OK,0.9521475063130399,12306.5607,1813,configs/trial_004_lr_1.869e-04.yaml,trials/trial_004_lr_1.869e-04
+4,2,COMPLETE,0.0001,0.7528900504112244,1813,0.7528843879699707,1800,7.803459644317627,OK,0.9646503870091383,12301.2835,1813,configs/trial_002_lr_1.000e-04.yaml,trials/trial_002_lr_1.000e-04
+5,6,COMPLETE,6.257495330311199e-05,0.7667444944381714,1813,0.7667444944381714,1813,8.260318756103516,OK,0.9794918145517979,12296.8874,1813,configs/trial_006_lr_6.257e-05.yaml,trials/trial_006_lr_6.257e-05
+6,1,COMPLETE,5e-05,0.7741077542304993,1813,0.7741077542304993,1813,9.544248580932617,OK,0.9867739501212489,12328.0877,1813,configs/trial_001_lr_5.000e-05.yaml,trials/trial_001_lr_5.000e-05
+7,7,COMPLETE,2.0278096180761755e-05,0.8000637888908386,1813,0.8000125288963318,1800,16.131792068481445,OK,1.0186243988306432,12284.0985,1813,configs/trial_007_lr_2.028e-05.yaml,trials/trial_007_lr_2.028e-05
+8,0,COMPLETE,2e-05,0.8005238175392151,1813,0.8004471659660339,1800,17.875633239746094,OK,1.0190646677338084,12292.1318,1813,configs/trial_000_lr_2.000e-05.yaml,trials/trial_000_lr_2.000e-05
diff --git a/optuna_30k_experiments/results/optuna_lr_results.json b/optuna_30k_experiments/results/optuna_lr_results.json
new file mode 100644
index 0000000..83d7868
--- /dev/null
+++ b/optuna_30k_experiments/results/optuna_lr_results.json
@@ -0,0 +1,130 @@
+[
+  {
+    "trial": 3,
+    "state": "COMPLETE",
+    "learning_rate": 0.0002,
+    "final_eval_loss": 0.7404094338417053,
+    "final_eval_step": 1813,
+    "min_eval_loss": 0.7404094338417053,
+    "min_eval_step": 1813,
+    "max_grad_norm": 14.712687492370605,
+    "grad_status": "OK",
+    "train_loss": 0.9520373807319014,
+    "train_runtime_seconds": 12634.3569,
+    "global_step": 1813,
+    "config": "configs/trial_003_lr_2.000e-04.yaml",
+    "results_dir": "trials/trial_003_lr_2.000e-04"
+  },
+  {
+    "trial": 8,
+    "state": "COMPLETE",
+    "learning_rate": 0.00018645202585249604,
+    "final_eval_loss": 0.7405665516853333,
+    "final_eval_step": 1813,
+    "min_eval_loss": 0.7405665516853333,
+    "min_eval_step": 1813,
+    "max_grad_norm": 3.499601125717163,
+    "grad_status": "OK",
+    "train_loss": 0.9515243444532172,
+    "train_runtime_seconds": 12255.1774,
+    "global_step": 1813,
+    "config": "configs/trial_008_lr_1.865e-04.yaml",
+    "results_dir": "trials/trial_008_lr_1.865e-04"
+  },
+  {
+    "trial": 4,
+    "state": "COMPLETE",
+    "learning_rate": 0.00018690446893209942,
+    "final_eval_loss": 0.741250216960907,
+    "final_eval_step": 1813,
+    "min_eval_loss": 0.741250216960907,
+    "min_eval_step": 1813,
+    "max_grad_norm": 29.025802612304688,
+    "grad_status": "OK",
+    "train_loss": 0.9521475063130399,
+    "train_runtime_seconds": 12306.5607,
+    "global_step": 1813,
+    "config": "configs/trial_004_lr_1.869e-04.yaml",
+    "results_dir": "trials/trial_004_lr_1.869e-04"
+  },
+  {
+    "trial": 2,
+    "state": "COMPLETE",
+    "learning_rate": 0.0001,
+    "final_eval_loss": 0.7528900504112244,
+    "final_eval_step": 1813,
+    "min_eval_loss": 0.7528843879699707,
+    "min_eval_step": 1800,
+    "max_grad_norm": 7.803459644317627,
+    "grad_status": "OK",
+    "train_loss": 0.9646503870091383,
+    "train_runtime_seconds": 12301.2835,
+    "global_step": 1813,
+    "config": "configs/trial_002_lr_1.000e-04.yaml",
+    "results_dir": "trials/trial_002_lr_1.000e-04"
+  },
+  {
+    "trial": 6,
+    "state": "COMPLETE",
+    "learning_rate": 6.257495330311199e-05,
+    "final_eval_loss": 0.7667444944381714,
+    "final_eval_step": 1813,
+    "min_eval_loss": 0.7667444944381714,
+    "min_eval_step": 1813,
+    "max_grad_norm": 8.260318756103516,
+    "grad_status": "OK",
+    "train_loss": 0.9794918145517979,
+    "train_runtime_seconds": 12296.8874,
+    "global_step": 1813,
+    "config": "configs/trial_006_lr_6.257e-05.yaml",
+    "results_dir": "trials/trial_006_lr_6.257e-05"
+  },
+  {
+    "trial": 1,
+    "state": "COMPLETE",
+    "learning_rate": 5e-05,
+    "final_eval_loss": 0.7741077542304993,
+    "final_eval_step": 1813,
+    "min_eval_loss": 0.7741077542304993,
+    "min_eval_step": 1813,
+    "max_grad_norm": 9.544248580932617,
+    "grad_status": "OK",
+    "train_loss": 0.9867739501212489,
+    "train_runtime_seconds": 12328.0877,
+    "global_step": 1813,
+    "config": "configs/trial_001_lr_5.000e-05.yaml",
+    "results_dir": "trials/trial_001_lr_5.000e-05"
+  },
+  {
+    "trial": 7,
+    "state": "COMPLETE",
+    "learning_rate": 2.0278096180761755e-05,
+    "final_eval_loss": 0.8000637888908386,
+    "final_eval_step": 1813,
+    "min_eval_loss": 0.8000125288963318,
+    "min_eval_step": 1800,
+    "max_grad_norm": 16.131792068481445,
+    "grad_status": "OK",
+    "train_loss": 1.0186243988306432,
+    "train_runtime_seconds": 12284.0985,
+    "global_step": 1813,
+    "config": "configs/trial_007_lr_2.028e-05.yaml",
+    "results_dir": "trials/trial_007_lr_2.028e-05"
+  },
+  {
+    "trial": 0,
+    "state": "COMPLETE",
+    "learning_rate": 2e-05,
+    "final_eval_loss": 0.8005238175392151,
+    "final_eval_step": 1813,
+    "min_eval_loss": 0.8004471659660339,
+    "min_eval_step": 1800,
+    "max_grad_norm": 17.875633239746094,
+    "grad_status": "OK",
+    "train_loss": 1.0190646677338084,
+    "train_runtime_seconds": 12292.1318,
+    "global_step": 1813,
+    "config": "configs/trial_000_lr_2.000e-05.yaml",
+    "results_dir": "trials/trial_000_lr_2.000e-05"
+  }
+]
diff --git a/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/eval_results.json b/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/eval_results.json
new file mode 100644
index 0000000..6f5da1e
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/eval_results.json
@@ -0,0 +1,8 @@
+{
+    "epoch": 1.0,
+    "eval_loss": 0.8005238175392151,
+    "eval_runtime": 135.4368,
+    "eval_samples_per_second": 7.384,
+    "eval_steps_per_second": 7.384,
+    "num_input_tokens_seen": 12201816
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/optuna_summary.json b/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/optuna_summary.json
new file mode 100644
index 0000000..6ab3dbe
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/optuna_summary.json
@@ -0,0 +1,11 @@
+{
+  "trial": 0,
+  "learning_rate": 2e-05,
+  "final_eval_step": 1813,
+  "final_eval_loss": 0.8005238175392151,
+  "min_eval_step": 1800,
+  "min_eval_loss": 0.8004471659660339,
+  "max_grad_norm": 17.875633239746094,
+  "grad_nonfinite": false,
+  "grad_status": "OK"
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/train_results.json b/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/train_results.json
new file mode 100644
index 0000000..9271f42
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/train_results.json
@@ -0,0 +1,9 @@
+{
+    "epoch": 1.0,
+    "num_input_tokens_seen": 12201816,
+    "total_flos": 5.2364848431896986e+17,
+    "train_loss": 1.0190646677338084,
+    "train_runtime": 12292.1318,
+    "train_samples_per_second": 2.359,
+    "train_steps_per_second": 0.147
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/training_eval_loss.png b/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/training_eval_loss.png
new file mode 100644
index 0000000..a2b23fa
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/training_eval_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/training_loss.png b/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/training_loss.png
new file mode 100644
index 0000000..6ff6b42
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_000_lr_2.000e-05/training_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/eval_results.json b/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/eval_results.json
new file mode 100644
index 0000000..2c84bd2
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/eval_results.json
@@ -0,0 +1,8 @@
+{
+    "epoch": 1.0,
+    "eval_loss": 0.7741077542304993,
+    "eval_runtime": 135.3685,
+    "eval_samples_per_second": 7.387,
+    "eval_steps_per_second": 7.387,
+    "num_input_tokens_seen": 12201816
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/optuna_summary.json b/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/optuna_summary.json
new file mode 100644
index 0000000..083ee42
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/optuna_summary.json
@@ -0,0 +1,11 @@
+{
+  "trial": 1,
+  "learning_rate": 5e-05,
+  "final_eval_step": 1813,
+  "final_eval_loss": 0.7741077542304993,
+  "min_eval_step": 1813,
+  "min_eval_loss": 0.7741077542304993,
+  "max_grad_norm": 9.544248580932617,
+  "grad_nonfinite": false,
+  "grad_status": "OK"
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/train_results.json b/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/train_results.json
new file mode 100644
index 0000000..428e7c3
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/train_results.json
@@ -0,0 +1,9 @@
+{
+    "epoch": 1.0,
+    "num_input_tokens_seen": 12201816,
+    "total_flos": 5.2364848431896986e+17,
+    "train_loss": 0.9867739501212489,
+    "train_runtime": 12328.0877,
+    "train_samples_per_second": 2.352,
+    "train_steps_per_second": 0.147
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/training_eval_loss.png b/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/training_eval_loss.png
new file mode 100644
index 0000000..bf0acec
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/training_eval_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/training_loss.png b/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/training_loss.png
new file mode 100644
index 0000000..68b4db9
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_001_lr_5.000e-05/training_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/eval_results.json b/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/eval_results.json
new file mode 100644
index 0000000..b95a02c
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/eval_results.json
@@ -0,0 +1,8 @@
+{
+    "epoch": 1.0,
+    "eval_loss": 0.7528900504112244,
+    "eval_runtime": 135.5345,
+    "eval_samples_per_second": 7.378,
+    "eval_steps_per_second": 7.378,
+    "num_input_tokens_seen": 12201816
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/optuna_summary.json b/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/optuna_summary.json
new file mode 100644
index 0000000..50eb7fa
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/optuna_summary.json
@@ -0,0 +1,11 @@
+{
+  "trial": 2,
+  "learning_rate": 0.0001,
+  "final_eval_step": 1813,
+  "final_eval_loss": 0.7528900504112244,
+  "min_eval_step": 1800,
+  "min_eval_loss": 0.7528843879699707,
+  "max_grad_norm": 7.803459644317627,
+  "grad_nonfinite": false,
+  "grad_status": "OK"
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/train_results.json b/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/train_results.json
new file mode 100644
index 0000000..5061aaa
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/train_results.json
@@ -0,0 +1,9 @@
+{
+    "epoch": 1.0,
+    "num_input_tokens_seen": 12201816,
+    "total_flos": 5.2364848431896986e+17,
+    "train_loss": 0.9646503870091383,
+    "train_runtime": 12301.2835,
+    "train_samples_per_second": 2.357,
+    "train_steps_per_second": 0.147
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/training_eval_loss.png b/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/training_eval_loss.png
new file mode 100644
index 0000000..bf8a88a
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/training_eval_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/training_loss.png b/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/training_loss.png
new file mode 100644
index 0000000..5a03bcf
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_002_lr_1.000e-04/training_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/eval_results.json b/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/eval_results.json
new file mode 100644
index 0000000..792faa8
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/eval_results.json
@@ -0,0 +1,8 @@
+{
+    "epoch": 1.0,
+    "eval_loss": 0.7404094338417053,
+    "eval_runtime": 158.3367,
+    "eval_samples_per_second": 6.316,
+    "eval_steps_per_second": 6.316,
+    "num_input_tokens_seen": 12201816
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/optuna_summary.json b/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/optuna_summary.json
new file mode 100644
index 0000000..2b787db
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/optuna_summary.json
@@ -0,0 +1,11 @@
+{
+  "trial": 3,
+  "learning_rate": 0.0002,
+  "final_eval_step": 1813,
+  "final_eval_loss": 0.7404094338417053,
+  "min_eval_step": 1813,
+  "min_eval_loss": 0.7404094338417053,
+  "max_grad_norm": 14.712687492370605,
+  "grad_nonfinite": false,
+  "grad_status": "OK"
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/train_results.json b/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/train_results.json
new file mode 100644
index 0000000..473f060
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/train_results.json
@@ -0,0 +1,9 @@
+{
+    "epoch": 1.0,
+    "num_input_tokens_seen": 12201816,
+    "total_flos": 5.2364848431896986e+17,
+    "train_loss": 0.9520373807319014,
+    "train_runtime": 12634.3569,
+    "train_samples_per_second": 2.295,
+    "train_steps_per_second": 0.143
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/training_eval_loss.png b/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/training_eval_loss.png
new file mode 100644
index 0000000..b29c8b6
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/training_eval_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/training_loss.png b/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/training_loss.png
new file mode 100644
index 0000000..ae929db
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_003_lr_2.000e-04/training_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/eval_results.json b/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/eval_results.json
new file mode 100644
index 0000000..079fa5f
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/eval_results.json
@@ -0,0 +1,8 @@
+{
+    "epoch": 1.0,
+    "eval_loss": 0.741250216960907,
+    "eval_runtime": 135.4566,
+    "eval_samples_per_second": 7.382,
+    "eval_steps_per_second": 7.382,
+    "num_input_tokens_seen": 12201816
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/optuna_summary.json b/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/optuna_summary.json
new file mode 100644
index 0000000..e334c50
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/optuna_summary.json
@@ -0,0 +1,11 @@
+{
+  "trial": 4,
+  "learning_rate": 0.00018690446893209942,
+  "final_eval_step": 1813,
+  "final_eval_loss": 0.741250216960907,
+  "min_eval_step": 1813,
+  "min_eval_loss": 0.741250216960907,
+  "max_grad_norm": 29.025802612304688,
+  "grad_nonfinite": false,
+  "grad_status": "OK"
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/train_results.json b/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/train_results.json
new file mode 100644
index 0000000..556f7e8
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/train_results.json
@@ -0,0 +1,9 @@
+{
+    "epoch": 1.0,
+    "num_input_tokens_seen": 12201816,
+    "total_flos": 5.2364848431896986e+17,
+    "train_loss": 0.9521475063130399,
+    "train_runtime": 12306.5607,
+    "train_samples_per_second": 2.356,
+    "train_steps_per_second": 0.147
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/training_eval_loss.png b/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/training_eval_loss.png
new file mode 100644
index 0000000..65e418b
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/training_eval_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/training_loss.png b/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/training_loss.png
new file mode 100644
index 0000000..2725672
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_004_lr_1.869e-04/training_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/eval_results.json b/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/eval_results.json
new file mode 100644
index 0000000..03b76c5
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/eval_results.json
@@ -0,0 +1,8 @@
+{
+    "epoch": 1.0,
+    "eval_loss": 0.7667444944381714,
+    "eval_runtime": 135.2654,
+    "eval_samples_per_second": 7.393,
+    "eval_steps_per_second": 7.393,
+    "num_input_tokens_seen": 12201816
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/optuna_summary.json b/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/optuna_summary.json
new file mode 100644
index 0000000..04ebd7b
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/optuna_summary.json
@@ -0,0 +1,11 @@
+{
+  "trial": 6,
+  "learning_rate": 6.257495330311199e-05,
+  "final_eval_step": 1813,
+  "final_eval_loss": 0.7667444944381714,
+  "min_eval_step": 1813,
+  "min_eval_loss": 0.7667444944381714,
+  "max_grad_norm": 8.260318756103516,
+  "grad_nonfinite": false,
+  "grad_status": "OK"
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/train_results.json b/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/train_results.json
new file mode 100644
index 0000000..6938636
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/train_results.json
@@ -0,0 +1,9 @@
+{
+    "epoch": 1.0,
+    "num_input_tokens_seen": 12201816,
+    "total_flos": 5.2364848431896986e+17,
+    "train_loss": 0.9794918145517979,
+    "train_runtime": 12296.8874,
+    "train_samples_per_second": 2.358,
+    "train_steps_per_second": 0.147
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/training_eval_loss.png b/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/training_eval_loss.png
new file mode 100644
index 0000000..d231697
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/training_eval_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/training_loss.png b/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/training_loss.png
new file mode 100644
index 0000000..4a9b8c8
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_006_lr_6.257e-05/training_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/eval_results.json b/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/eval_results.json
new file mode 100644
index 0000000..e19d8ed
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/eval_results.json
@@ -0,0 +1,8 @@
+{
+    "epoch": 1.0,
+    "eval_loss": 0.8000637888908386,
+    "eval_runtime": 135.1761,
+    "eval_samples_per_second": 7.398,
+    "eval_steps_per_second": 7.398,
+    "num_input_tokens_seen": 12201816
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/optuna_summary.json b/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/optuna_summary.json
new file mode 100644
index 0000000..4ddb700
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/optuna_summary.json
@@ -0,0 +1,11 @@
+{
+  "trial": 7,
+  "learning_rate": 2.0278096180761755e-05,
+  "final_eval_step": 1813,
+  "final_eval_loss": 0.8000637888908386,
+  "min_eval_step": 1800,
+  "min_eval_loss": 0.8000125288963318,
+  "max_grad_norm": 16.131792068481445,
+  "grad_nonfinite": false,
+  "grad_status": "OK"
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/train_results.json b/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/train_results.json
new file mode 100644
index 0000000..608d8a6
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/train_results.json
@@ -0,0 +1,9 @@
+{
+    "epoch": 1.0,
+    "num_input_tokens_seen": 12201816,
+    "total_flos": 5.2364848431896986e+17,
+    "train_loss": 1.0186243988306432,
+    "train_runtime": 12284.0985,
+    "train_samples_per_second": 2.361,
+    "train_steps_per_second": 0.148
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/training_eval_loss.png b/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/training_eval_loss.png
new file mode 100644
index 0000000..921afed
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/training_eval_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/training_loss.png b/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/training_loss.png
new file mode 100644
index 0000000..b792a30
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_007_lr_2.028e-05/training_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/eval_results.json b/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/eval_results.json
new file mode 100644
index 0000000..c2dad7b
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/eval_results.json
@@ -0,0 +1,8 @@
+{
+    "epoch": 1.0,
+    "eval_loss": 0.7405665516853333,
+    "eval_runtime": 135.1617,
+    "eval_samples_per_second": 7.399,
+    "eval_steps_per_second": 7.399,
+    "num_input_tokens_seen": 12201816
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/optuna_summary.json b/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/optuna_summary.json
new file mode 100644
index 0000000..adaf6a5
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/optuna_summary.json
@@ -0,0 +1,11 @@
+{
+  "trial": 8,
+  "learning_rate": 0.00018645202585249604,
+  "final_eval_step": 1813,
+  "final_eval_loss": 0.7405665516853333,
+  "min_eval_step": 1813,
+  "min_eval_loss": 0.7405665516853333,
+  "max_grad_norm": 3.499601125717163,
+  "grad_nonfinite": false,
+  "grad_status": "OK"
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/train_results.json b/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/train_results.json
new file mode 100644
index 0000000..dc25e04
--- /dev/null
+++ b/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/train_results.json
@@ -0,0 +1,9 @@
+{
+    "epoch": 1.0,
+    "num_input_tokens_seen": 12201816,
+    "total_flos": 5.2364848431896986e+17,
+    "train_loss": 0.9515243444532172,
+    "train_runtime": 12255.1774,
+    "train_samples_per_second": 2.366,
+    "train_steps_per_second": 0.148
+}
\ No newline at end of file
diff --git a/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/training_eval_loss.png b/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/training_eval_loss.png
new file mode 100644
index 0000000..40603ed
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/training_eval_loss.png differ
diff --git a/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/training_loss.png b/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/training_loss.png
new file mode 100644
index 0000000..866483d
Binary files /dev/null and b/optuna_30k_experiments/trials/trial_008_lr_1.865e-04/training_loss.png differ