Diplomovka/aya_300k_lr_experiments/benchmarks/skbench/results/family_summary.csv
Jakub 67061cf373 Add Aya 300k learning-rate study and benchmark results
Document six completed LR runs as preparation for full Aya training on Perun. Record selection of LR 0.00010 by lowest final validation loss, selected-model lm-eval and partial SK-bench results, reproducible provenance, and dataset-access/context-truncation limitations. Preserve existing 30k and Titan material.
2026-10-25 00:33:24 +02:00

4.0 KiB

1familymetrictask_variantsprompt_countevaluated_instancestask_macro_meansample_weighted_meanmin_task_valuemax_task_value
2arcem3335070.0236669518106643850.0236669518106643850.0136869118905047040.03507271171941831
3arcem_conservative3335070.190191046478471630.190191046478471630.17108639863130880.223267750213858
4arcem_relaxed3335070.190761334473909330.190761334473909330.17108639863130880.22412318220701455
5belebeleem3327000.0222222222222222230.0222222222222222230.0166666666666666660.03
6belebeleem_conservative3327000.212592592592592580.212592592592592580.21111111111111110.21555555555555556
7belebeleem_relaxed3327000.213333333333333350.213333333333333350.21111111111111110.21555555555555556
8financial_examsem3340020.0297351324337831080.0297351324337831080.00374812593703148420.07571214392803598
9financial_examsem_conservative3340020.218390804597701160.218390804597701160.200899550224887560.24887556221889057
10financial_examsem_relaxed3340020.229385307346326840.229385307346326840.200899550224887560.24887556221889057
11globalmmlu-liteem3312000.0283333333333333350.0283333333333333320.0250.035
12globalmmlu-liteem_conservative3312000.198333333333333330.198333333333333330.18250.21
13globalmmlu-liteem_relaxed3312000.198333333333333330.198333333333333330.18250.21
14mmluem1713391860.031486753401983350.0295768897055070680.00.13
15mmluem_conservative1713391860.156655506503479790.154647067830347570.00267022696929238980.3553299492385787
16mmluem_relaxed1713391860.160094125302573720.157224518960853380.00267022696929238980.3553299492385787
17movie_reviewem_conservative_norm3360240.401394422310756950.401394422310756950.094123505976095620.6324701195219123
18movie_reviewem_norm3360240.148406374501992040.148406374501992040.010458167330677290.38745019920318724
19movie_reviewem_norm_relaxed3360240.409030544488711840.409030544488711840.111055776892430280.6329681274900398
20multi_wiki_qaexact_match_slk_prefix33149910.354679474351277450.354679474351277450.340004002401440850.36401841104662797
21multi_wiki_qaf1_slk33149910.50990470743193630.50990470743193630.50500717864701990.5156574509584114
22senti_skem_conservative_norm63252150.327348539698882560.326551655760460.19903433476394850.49302575107296137
23senti_skem_norm63252150.097193679507332470.099424945468966880.00375536480686695280.22642719692110327
24senti_skem_norm_relaxed63252150.36210092008101010.363077533214356540.251609442060085840.4965128755364807
25skj1em333000.00.00.00.0
26skj1em_conservative333000.033333333333333330.033333333333333330.020.05
27skj1em_relaxed333000.0433333333333333350.0433333333333333350.020.06
28sklepem_conservative_norm123270750.347470540259125270.212779316712834730.0013988808952837730.8800383877159309
29sklepem_norm123270750.0390421250709196840.018171745152354570.00.44817658349328215
30sklepem_norm_relaxed123270750.35079945102686430.214404432132963980.0013988808952837730.8809980806142035
31sklepexact_match_slk_prefix33287490.163727433997704260.163727433997704260.161431701972242520.16518835437754356
32sklepf1_slk33287490.37504150889284840.37504150889284840.37301512210960530.3769392643579436
33slovakcopaem3315000.00.00.00.0
34slovakcopaem_conservative3315000.3060.3060.0840.508
35slovakcopaem_relaxed3315000.40466666666666670.40466666666666670.0860.618
36toxic_skem_conservative_norm3326430.47597427166099130.47597427166099130.456299659477866050.4960272417707151
37toxic_skem_norm3326430.00264850548618993580.00264850548618993580.00.007945516458569807
38toxic_skem_norm_relaxed3326430.47597427166099130.47597427166099130.456299659477866050.4960272417707151
39truthfulqaem3323340.00.00.00.0
40truthfulqaem_conservative3323340.0119965724078834620.0119965724078834620.0077120822622107970.016709511568123392
41truthfulqaem_relaxed3323340.0119965724078834620.0119965724078834620.0077120822622107970.016709511568123392