Kontroly odpovedí agenta

README.md

@@ -15,7 +15,7 @@ FTS5/BM25 + embeddingy + RRF. Nad týmto baseline sa vo vetve GraphRAG
 dopĺňa knowledge graph uložený v Neo4j. Aktuálne je implementovaná
 grafová infraštruktúra, deterministický builder, vizualizácia grafu a
 samostatné grafové vyhľadávanie. Grafová vrstva má zdrojovo overený benchmark a zmrazenú opravenú
-verziu v5.1 s regresiou 320 historických otázok, kontrolou všetkých vrátených
+verziu v6 s regresiou 320 historických otázok, kontrolou všetkých vrátených
 typovaných faktov a samostatným overením generovania odpovedí cez OpenWebUI API. Ďalším krokom je natívne zapojenie grafového
 nástroja a hlavného agenta do OpenWebUI.
 
@@ -870,13 +870,17 @@ grafové vyhľadávanie je implementované v `tools/graphrag/graph_search.py`.
 GraphRAG zatiaľ nezasahuje do stabilného klasického `/rag` toku; ďalšia
 etapa spojí grafové výsledky s klasickým RAG a OpenWebUI nástrojom.
 
-Aktuálna evaluácia GraphRAG **v5.1** má DEV **722/722** a historickú regresiu
+Aktuálna evaluácia GraphRAG **v6** má DEV **722/722** a historickú regresiu
 **320/320**, vrátane kontroly všetkých vrátených typovaných faktov oproti raw
-Markdownom. Generovanie sa meria samostatne: 39 DEV a finálna regresia 320
+Markdownom. Generovanie sa meria samostatne: 42 DEV a finálna regresia 320
-historických otázok vrátane týchto 39; nejde o nový nezávislý TEST.
+historických otázok vrátane týchto 42; nejde o nový nezávislý TEST.
-LLM regresia dosiahla **317/320 (99,06 %)** bez neplatných JSON odpovedí.
+Strict výsledok generovania je **316/320 (98,75 %)**. Celá sada vrátane
-Finálne metriky, opravy JSON, historické porovnanie a obmedzenia sú v
+živých testov má **565 úspešných testov**, bez preskočených prípadov.
-[finálnej správe v5.1](tools/graphrag/evaluation/results/final_v5_1_2026-10-25/REPORT.md).
+V6 prenáša úplnosť a verziu dát do kontextu, kontroluje všetky citované URL
+oproti podkladom konkrétneho dotazu a spája názov/rok s tagom tej istej osoby.
+Priame čítanie zdrojov opravilo 15 očakávaní; v1–v5.1 ostávajú zachované.
+Finálne metriky, prvotné odpovede aj opravné pokusy a obmedzenia sú v
+[finálnej správe v6](tools/graphrag/evaluation/results/final_v6_2026-10-25/REPORT.md).
 
 Historická v4 mala grafový strict TEST 320/320, doplnkovú kontrolu kontextových
 faktov 319/320 a LLM TEST 281/300 (93,67 %). Pôvodné otázky aj výsledky
@@ -1104,9 +1108,9 @@ spoľahlivého zdroja.
 
 ### 2. GraphRAG evaluácia
 
-V5.1 má zmrazený graf a úplnú historickú regresiu **320/320**.
+V6 má zmrazený graf a úplnú historickú regresiu **320/320**.
-[Finálna správa](tools/graphrag/evaluation/results/final_v5_1_2026-10-25/REPORT.md)
+[Finálna správa](tools/graphrag/evaluation/results/final_v6_2026-10-25/REPORT.md)
-oddeľuje retrieval a generovanie odpovedí; pôvodné v1–v4 sa nemenia.
+oddeľuje retrieval a generovanie odpovedí; pôvodné v1–v5.1 sa nemenia.
 Pred formálnym porovnaním s klasickým RAG potrebujeme spoločnú metodiku
 a ideálne otázky anotované nezávislým človekom. Nový nezávislý TEST vyžaduje
 nové otázky; opakované meranie odhalených otázok je regresia.

docs/graphrag_v6_2026-10-25_worklog.md

@@ -0,0 +1,165 @@
+# GraphRAG v6 — kontext, citácie a kontrola zdrojov
+
+Práce z 25. 10. 2026 nadväzujú na historickú v5.1. Výsledky a otázky v1–v5.1
+sa neprepisujú. Záloha vstupného projektu je v
+`/tmp/zp-v6-backup-20261025T170922Z/before.tar.gz`, dodatočný snapshot grafu
+v `/tmp/zp-v6-graph-before.json`. Sú to lokálne zálohy, nie verzované artefakty.
+
+## Osem úloh
+
+1. RAG vkladá `complete`, `truncated`, `truncation_reasons`, `index_version`
+   aj do textového kontextu. Grafový adaptér tieto hodnoty prenáša bez
+   domýšľania; starý výsledok bez verzie ostáva s `index_version=null`.
+2. Oba prompty rozlišujú úplné, neúplné a neznáme pokrytie. Kontrola generovania
+   vyžaduje upozornenie na neúplnosť. Presný agregovaný počet môže byť úplný,
+   aj keď sú orezané zobrazené dôkazy. Z neúplnej agregácie sa celkový počet
+   nesmie odvodzovať.
+3. Spoločný modul `context_policy.py` kontroluje `source_urls` aj odkazy priamo
+   v texte vrátane Markdownu, relatívnych cieľov a adries s `www.`. Povolené
+   sú presné URL zdrojov konkrétneho dotazu, nie ľubovoľné známe wiki URL.
+   Úvodzovky a bežná interpunkcia nie sú súčasťou citovanej URL.
+4. Nové testy obsahujú správnu povinnú citáciu s ďalším chybným odkazom,
+   správnu URL iného dotazu, relatívny a ne-HTTP odkaz aj korektné citácie.
+   Archivované odpovede sa znovu skontrolovali bez volania modelu a bez
+   prepisovania ich pôvodných skóre. Nový prepočet je v
+   `results/v6_preparation/citation_regrade.json`.
+   Kontrola zahŕňa 56 archivovaných súborov a 6 044 odpovedí (opakované
+   experimenty sa prekrývajú), s piatimi porušeniami prísneho kontraktu URL.
+5. Typ práce sa nesmie vydávať za `keyword`. `answer_candidates` spája
+   doložený názov/rok a doslovný tag cez tú istú osobu aj cez jej dcérske
+   dokumenty. Nespája ľudí navzájom; názov s rokom musí patriť jednému
+   akademickému záznamu a kategória s autorom rovnakému dokumentu.
+6. Vzorka 34 otázok zahŕňa všetkých desať kategórií a známe rizikové prípady.
+   Priame čítanie pôvodných Markdownov je oddelené od grafového retrievera;
+   vykonala ho AI. Výber, výňatky, SHA-256, závery a hárok pre človeka sú
+   v `results/v6_preparation/`. **Ľudské posúdenie neprebehlo.**
+7. Celý grafový DEV, regresia a generované odpovede sa merajú samostatne.
+   Zmrazenie zahŕňa spoločné moduly, schému odpovede, kontroly kontextu,
+   publikačnú politiku a kód RAG evaluátora; generovanie navyše fixuje prompt,
+   model, jeho parametre a vybrané ID. Finálne opakovanie známych otázok je
+   historická regresia, nie nový nezávislý TEST.
+8. Nasadený Docker sa overuje voči zmrazeniu, `/health`, reálnemu `/rag`,
+   grafovým dotazom v kontajneri a spoločnej generácii JSON/SQLite/Neo4j.
+   Synchronizácia sa vykonala cez `/sync` s `pull_git=false`, bez zmeny wiki.
+
+## Kontroly a opravný pokus
+
+`generate_checked_answer()` nedostáva benchmark, očakávanú odpoveď ani oracle.
+Používa iba otázku, skutočný kontext, prompt a klienta modelu. Najviac jeden
+spoločný opravný požiadavok rieši neplatný JSON alebo porušenie pravidiel
+kontextu. Gold sa používa až po ukončení generovania. Chybná odpoveď, prípadný
+opravný pokus aj konečné odmietnutie sú uložené; odmietnutá odpoveď sa počíta
+ako zlyhanie, nevynecháva sa z menovateľa.
+
+`accepted_for_delivery=false` znamená, že konzument odpoveď nesmie zobraziť
+ako overenú. Nejde o záruku pravdivosti každého voľného súvetia. Kontroly
+overujú citácie, štruktúrované fakty a vybrané rozpory; formálna evaluácia
+navyše kontroluje požadované väzby, hodnoty v texte a zdrojové dôkazy.
+Budúci OpenWebUI agent musí tieto kontroly volať a rešpektovať ich výsledok;
+natívne rozhodovanie a smerovanie nástrojov ostáva bodom 4.
+
+Klasický RAG má samostatnú prezentačnú funkciu `finalize_rag_answer()`.
+Ak model uviedol správne podložené údaje, ale vynechal povinné upozornenie,
+pridá iba vetu „Výpis je neúplný.“ a uchová presný pôvodný `model_answer`,
+pôvodné kontrolné dôvody aj príznak `display_notice_added`. Nepridáva fakty
+ani ďalší požiadavok na model. Neznáme citácie a nepodložené tvrdenie o úplnom
+zozname týmto spôsobom neobchádza. RAG evaluátor používa túto funkciu pred
+hodnotením a budúci nástroj ju musí použiť pred doručením odpovede.
+
+Päť vopred vybraných živých RAG prípadov malo správne fakty a zdroje 5/5,
+ale chýbajúce upozornenie 5/5. Pôvodné výstupy ostávajú v `rag_live_smoke.json`.
+Prehratie tých istých odpovedí cez prezentačnú funkciu má 5/5 prijateľných
+výstupov v `rag_delivery_replay.json`. Nie je to nové meranie modelu ani
+odhad presnosti klasického RAG z piatich prípadov. Zmena tejto čisto RAG
+vrstvy nezasahuje do zmrazeného grafového generovania; celkový release
+navyše fixuje všetky runtime moduly v `frozen_release_v6.json`.
+
+## Zistenia zo zdrojovej kontroly
+
+Generický odkaz `Písomná práca: [rešerš](...)` nie je doložený názov projektu.
+V Dáriusovom profile je v sekcii TP 2019, zatiaľ čo cieľový dokument má vlastný
+názov a kategóriu `tp2020`. Názov ani rok dcérskeho dokumentu sa neprenášajú
+do nadradeného akademického záznamu. To isté pravidlo platí pre všetkých ľudí,
+nie iba pre dve známe zlyhania modelu. Konkrétny názov typu „Rešerš na tému
+prekladu“ ostáva platným kandidátom.
+
+Nový benchmark v6 zachováva všetkých 1 042 ID, texty, kategórie a rozdelenie
+722 DEV / 320 regresia. Opravených je 15 očakávaní vrátane štyroch
+`wide-116`, `wide-142`, `wide-145`, `wide-165`; tieto štyri otázky ostávajú
+v pôvodných kategóriách a očakávajú `no_answer`. Regresia má preto 284
+kladných a 36 záporných očakávaní. Úplné pred/po je v
+`source_adjudications.json`. Vyššie skóre oproti v5.1 nemožno pripísať iba
+zlepšeniu modelu, keďže sa opravil aj gold a zmenili kontroly.
+
+Zo vzorky: 31 očakávaní podporených, dve opravené a jedno podmienené zavedenou
+konvenciou. Pri `wide-112` Tomáš Kuchárik nemá výslovnú vetu o začiatku
+štúdia; rok 2016 sa odvodzuje z ročníka adresára `students/2016`. Tento
+fallback používa aj existujúci systém. Nie je nezávislým dôkazom skutočného
+dátumu nástupu a vyžaduje potvrdenie človekom pred interpretáciou ako
+administratívny fakt.
+
+Pri prísnej politike „všetky URL len zo sources“ historický RAG prípad
+`q0892` neprejde, lebo v texte navyše uvádza `http://skner.tukekemt.xyz`.
+Táto adresa v súčasnom Markdowne reálne existuje; nie je to dôkaz halucinácie.
+Nie je však v archivovanom zozname nástrojových `source_url`. Nové skóre je
+skóre dodržania prísneho kontraktu, nie nový úsudok o pravdivosti tej adresy.
+
+## Protokol generovania
+
+DEV má vopred vybraných 42 známych otázok: predchádzajúcich 39, historickú
+chybu `wide-129` a opravené title otázky. Prvý DEV kandidát mal 41/42;
+`wide-283` nesprávne odmietol spojenie tagu a roku cez profil a dcérsky dokument.
+Nasledovala oprava všeobecného spojenia osoby a nový DEV pred finálnym behom.
+Pôvodný kandidátsky DEV je zachovaný v `answers_v6_dev_2026-10-25`.
+`frozen_candidate.json` a overený `candidate_to_final.patch` zachovávajú
+prvú kandidátsku implementáciu; spätné aplikovanie patchu ju obnoví.
+Kandidátske retrieval výpisy sú iba v lokálnej zálohe
+`/tmp/zp-v6-candidate-before-year-join/retrieval`; pre finálne grafové čísla
+sa používajú výhradne plné uložené v6 výsledky. Kandidát nie je release.
+Finálna regresia vždy obsahuje všetkých 320 otázok vrátane DEV prekryvu.
+
+## Príkazy a reprodukcia
+
+Prostredie načítavajte bez vypisovania `.env`. Existujúce výstupné adresáre
+ani zmrazené súbory neprepisujte; pri reprodukcii zvoľte nové cesty.
+
+```bash
+python -m tools.graphrag.evaluation.benchmark_v6
+RUN_NEO4J_TESTS=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 python -m pytest -q
+python -m tools.graphrag.evaluation.evaluate_graphrag \
+  --questions tools/graphrag/evaluation/graphrag_questions_v6.json --split dev \
+  --audit tools/graphrag/evaluation/results/v6_preparation/source_audit.json \
+  --output-dir NOVY_RETRIEVAL_ADRESAR
+python -m tools.graphrag.evaluation.freeze_graphrag \
+  --dev-results NOVY_RETRIEVAL_ADRESAR/graphrag_dev.json \
+  --questions tools/graphrag/evaluation/graphrag_questions_v6.json \
+  --audit tools/graphrag/evaluation/results/v6_preparation/source_audit.json \
+  --purpose historical_regression --output NOVY_MANIFEST.json
+python -m tools.graphrag.evaluation.evaluate_graphrag \
+  --questions tools/graphrag/evaluation/graphrag_questions_v6.json --split test \
+  --audit tools/graphrag/evaluation/results/v6_preparation/source_audit.json \
+  --freeze-manifest NOVY_MANIFEST.json --output-dir NOVY_RETRIEVAL_ADRESAR
+python -m tools.answer_evaluation.evaluate_graph_answers \
+  --questions tools/graphrag/evaluation/graphrag_questions_v6.json \
+  --audit tools/graphrag/evaluation/results/v6_preparation/source_audit.json \
+  --freeze-manifest NOVY_MANIFEST.json \
+  --dev-ids tools/graphrag/evaluation/results/v6_preparation/generation_dev_ids.json \
+  --split dev --output-dir NOVY_LLM_DEV
+python -m tools.answer_evaluation.evaluate_graph_answers \
+  --questions tools/graphrag/evaluation/graphrag_questions_v6.json \
+  --audit tools/graphrag/evaluation/results/v6_preparation/source_audit.json \
+  --freeze-manifest NOVY_MANIFEST.json \
+  --dev-ids tools/graphrag/evaluation/results/v6_preparation/generation_dev_ids.json \
+  --dev-results NOVY_LLM_DEV/answers_dev.json \
+  --split regression --output-dir NOVY_LLM_REGRESSION
+docker compose build zp-agent-api
+docker compose up -d zp-agent-api
+```
+
+`benchmark_v6` slúži na jednorazovú migráciu a odmietne prepísať hotový dataset.
+Pri obyčajnej reprodukcii používajte uložené v6 otázky. Živé syntetické
+funkčné scenáre sa spúšťajú cez `tools.answer_evaluation.check_context_scenarios`;
+nepripočítavajú sa k 320 otázkam ani k meraniu presnosti na wiki.
+
+Finálne čísla, zlyhania a stav nasadenia:
+[správa v6](../tools/graphrag/evaluation/results/final_v6_2026-10-25/REPORT.md).

docs/vystupny_kontrakt.md

@@ -1,5 +1,9 @@
 # Spoločný výstupný kontrakt RAG a GraphRAG
 
+V6 prenáša polia úplnosti a verzie aj do kontextu LLM a kontroluje ich pri
+generovaní. Kontrola citácií sa vzťahuje na zoznam aj odkazy v texte odpovede.
+[Implementácia, meranie a obmedzenia](graphrag_v6_2026-10-25_worklog.md).
+
 Kontrakt v1 zjednocuje polia pre hlavného agenta a budúce OpenWebUI nástroje.
 Vyhľadávanie naďalej vracia svoje pôvodné dáta. Klasický RAG zachováva
 `context`, `instructions`, `answer_format` a rozšírené zdroje; GraphRAG

test/test_context_policy.py

@@ -0,0 +1,191 @@
+import copy
+import json
+
+import pytest
+
+from tools.answer_evaluation.context_policy import citation_check, answer_context_issues, joined_candidates
+from tools.answer_evaluation.checked_generation import generate_checked_answer
+from tools.answer_evaluation.evaluate_graph_answers import graph_context,grade_answer
+
+
+@pytest.fixture
+def context():
+    return {'status':'ok','reason':None,'complete':True,'truncated':False,'truncation_reasons':[],
+            'index_version':{'version':'generation-1'},'aggregation':None,
+            'sources':[{'source_id':'S1','document_path':'a.md','source_url':'https://example.test/a'}],
+            'facts':[{'person':'Eva Nová','title':'Vyhľadávanie informácií','year':2026,
+                      'document_path':'a.md','source_url':'https://example.test/a'},
+                     {'person':'Eva Nová','keyword':'RAG','document_path':'a.md','source_url':'https://example.test/a'}],
+            'answer_candidates':[]}
+
+
+@pytest.fixture
+def answer():
+    return {'status':'ok','answer':'Eva Nová má prácu Vyhľadávanie informácií z roku 2026. https://example.test/a',
+            'claims':[{'person':'Eva Nová','title':'Vyhľadávanie informácií','year':2026}],
+            'source_urls':['https://example.test/a'],'count':None}
+
+
+def test_graph_context_forwards_exact_contract_without_inventing_version(context):
+    raw={**context,'constraints':{'mentions_title':True,'mentions_academic_year':False,'years':[],
+                                'mentions_start_year':False},'documents':[]}
+    out=graph_context(raw)
+    for key in ['complete','truncated','truncation_reasons','index_version','sources']:
+        assert out[key]==context[key]
+    legacy=copy.deepcopy(raw);legacy.pop('index_version')
+    assert graph_context(legacy)['index_version'] is None
+
+
+@pytest.mark.parametrize('location',['source_urls','answer'])
+def test_unknown_citation_is_rejected_in_list_and_prose(answer,location):
+    if location=='source_urls':answer['source_urls'].append('https://invented.test/x')
+    else:answer['answer']+=' https://invented.test/x'
+    assert not citation_check(answer,{'https://example.test/a'})['citations_ok']
+
+
+def test_legitimate_markdown_quoted_and_punctuated_urls_are_accepted(answer):
+    answer['answer']='[Zdroj](https://example.test/a), `https://example.test/a` a „https://example.test/a“. '
+    assert citation_check(answer,{'https://example.test/a'})['citations_ok']
+
+
+@pytest.mark.parametrize('link', ['[zdroj](/invented)', 'www.invented.test/a', '[zdroj](mailto:fake@example.test)'])
+def test_non_http_and_relative_citations_are_checked(answer, link):
+    answer['answer'] += ' ' + link
+    assert not citation_check(answer, {'https://example.test/a'})['citations_ok']
+
+
+def test_rag_grader_rejects_extra_citation_despite_required_source_present():
+    from tools.rag.evaluation.rag_metrics import evaluate_answer
+    question = {'should_answer': True, 'expected_answer_contains': ['SQLite'],
+                'expected_source_urls': ['https://example.test/a']}
+    text = 'SQLite. https://example.test/a'
+    assert evaluate_answer(question, text, tool_called=True,
+                           allowed_source_urls=['https://example.test/a'])['strict_pass']
+    result = evaluate_answer(question, text + ' https://invented.test', tool_called=True,
+                             allowed_source_urls=['https://example.test/a'])
+    assert not result['strict_pass'] and result['unsupported_urls'] == ['https://invented.test']
+
+
+@pytest.mark.parametrize('complete,reasons', [(None, []), (False, []), (False, ['source_limit'])])
+def test_rag_prompt_metadata_matches_top_level_contract(monkeypatch, tmp_path, complete, reasons):
+    from tools.rag import rag_utils
+    version = {'version': 'test-generation'}
+    monkeypatch.setattr(rag_utils, 'search_database', lambda *a, **k: {
+        'results': [], 'engine': 'test', 'strategies': [], 'complete': complete,
+        'truncated': bool(reasons), 'truncation_reasons': reasons, 'index_version': version})
+    result = rag_utils.build_rag_context(tmp_path / 'test.sqlite', 'Neznáma osoba')
+    metadata = json.loads(result['context'].splitlines()[1])
+    assert metadata == {key: result[key] for key in
+                        ['complete', 'truncated', 'truncation_reasons', 'index_version']}
+    assert metadata['index_version'] == version
+
+
+def test_known_wiki_url_outside_this_context_is_rejected(context,answer):
+    answer['source_urls'].append('https://example.test/other-person')
+    assert 'unsupported_citation' in answer_context_issues(answer,context)
+
+
+def test_type_of_work_cannot_become_a_keyword(context,answer):
+    answer['claims'][0]['keyword']='diplomový projekt'
+    assert 'unsupported_keyword' in answer_context_issues(answer,context)
+
+
+def test_title_and_year_must_share_one_actual_record(context,answer):
+    context['facts'].append({'person':'Eva Nová','title':'Iný názov','year':2025})
+    answer['claims'][0]['year']=2025
+    assert 'unsupported_academic_binding' in answer_context_issues(answer,context)
+
+
+def test_person_join_supports_child_document_but_never_another_person(context,answer):
+    context['facts'][1]['document_path']='child.md'
+    intent={'mentions_title':True,'mentions_keyword':True}
+    candidates=joined_candidates(context['facts'],intent)
+    assert candidates[0]['evidence_documents']==['a.md','child.md']
+    context['answer_candidates']=candidates
+    answer['claims'][0]['keyword']='RAG'
+    assert answer_context_issues(answer,context)==[]
+    context['facts'][1]['person']='Boris Iný'
+    assert joined_candidates(context['facts'],intent)==[]
+
+
+def test_year_and_keyword_join_does_not_require_a_title(context, answer):
+    context['facts'][0].pop('title')
+    context['facts'][1]['document_path'] = 'child.md'
+    candidates = joined_candidates(context['facts'], {'years': [2026], 'mentions_keyword': True})
+    assert candidates[0]['year'] == 2026 and 'title' not in candidates[0]
+    assert candidates[0]['evidence_documents'] == ['a.md', 'child.md']
+    context['answer_candidates'] = candidates
+    answer.update(status='no_answer', claims=[])
+    assert 'rejected_supported_person_join' in answer_context_issues(answer, context)
+    answer.update(status='ok', claims=[{'person': 'Eva Nová', 'year': 2026, 'keyword': 'RAG'}])
+    assert answer_context_issues(answer, context) == []
+
+
+def test_known_join_cannot_be_rejected_or_split(context,answer):
+    context['answer_candidates']=joined_candidates(context['facts'],{'mentions_title':True,'mentions_keyword':True})
+    answer['status']='no_answer';answer['claims']=[]
+    assert 'rejected_supported_person_join' in answer_context_issues(answer,context)
+    answer['status']='ok';answer['claims']=[{'person':'Eva Nová','title':'Vyhľadávanie informácií','year':2026},
+                                           {'person':'Eva Nová','keyword':'RAG'}]
+    assert 'split_or_missing_person_join' in answer_context_issues(answer,context)
+
+
+@pytest.mark.parametrize('complete,truncated',[(False,False),(True,True),(None,False)])
+def test_incomplete_or_unknown_context_cannot_claim_full_list(context,answer,complete,truncated):
+    context.update(complete=complete,truncated=truncated)
+    answer['answer']='Toto je úplný zoznam: Eva Nová.'
+    assert 'unsupported_exhaustive_statement' in answer_context_issues(answer,context)
+    answer['answer']='Eva Nová. Výpis je neúplný.'
+    assert answer_context_issues(answer,context)==[]
+
+
+def test_exact_count_survives_truncated_evidence_with_notice(context,answer):
+    context.update(complete=False,truncated=True,aggregation={'count':2,'complete':True})
+    answer.update(count=2,answer='Počet je úplný, dôkazy sú orezané.')
+    assert answer_context_issues(answer,context)==[]
+    answer['count']=3
+    assert 'incorrect_context_count' in answer_context_issues(answer,context)
+    context['aggregation']['complete']=False
+    assert 'unsupported_total_count' in answer_context_issues(answer,context)
+
+
+def response(answer):
+    return {'choices':[{'message':{'content':answer if isinstance(answer,str) else json.dumps(answer)}}]}
+
+
+def test_one_correction_for_bad_citation_preserves_both_attempts(context,answer):
+    bad=copy.deepcopy(answer);bad['source_urls'].append('https://invented.test/x')
+    replies=iter([response(bad),response(answer)]);messages=[]
+    def complete(rows):
+        messages.append(copy.deepcopy(rows));return next(replies)
+    out=generate_checked_answer('Otázka',context,'Pokyny',complete)
+    assert out['accepted_for_delivery'] and len(out['attempts'])==2
+    assert out['attempts'][0]['context_issues']==['unsupported_citation']
+    assert len(messages[0])==2 and len(messages[1])==4
+    assert 'expected_bindings' not in json.dumps(messages)
+
+
+def test_one_total_correction_covers_format_then_context_failure(context,answer):
+    bad=copy.deepcopy(answer);bad['claims'][0]['year']=1901
+    replies=iter([response('{bad json'),response(bad)])
+    out=generate_checked_answer('Otázka',context,'Pokyny',lambda _:next(replies))
+    assert not out['accepted_for_delivery'] and len(out['attempts'])==2
+    assert out['attempts'][0]['format_error'] and out['context_rejected']
+
+
+def test_valid_context_answer_never_retries_based_on_gold(context,answer):
+    calls=[]
+    def complete(rows):calls.append(rows);return response(answer)
+    out=generate_checked_answer('Otázka',context,'Pokyny',complete)
+    assert out['accepted_for_delivery'] and len(calls)==1
+
+
+def test_grader_rejects_extra_url_without_rejecting_correct_answer(context,answer):
+    oracle={'people':{'Eva Nová':{'documents':['a.md'],'academics':[{'year':2026,'title':'Vyhľadávanie informácií'}]}},
+            'documents':{'a.md':{'source_url':'https://example.test/a','tags':[],'categories':[],'author':None}}}
+    case={'expected_answer_contains':['Eva Nová','2026'],'graph':{
+        'expected_bindings':[{'person':'Eva Nová','year':2026}],
+        'expected_evidence_documents':[{'any_of':[{'source_url':'https://example.test/a'}]}]}}
+    assert grade_answer(case,answer,oracle,context=context)['strict']
+    answer['answer']+=' https://invented.test/x'
+    assert not grade_answer(case,answer,oracle,context=context)['strict']

test/test_rag_delivery.py

@@ -0,0 +1,36 @@
+import copy
+import pytest
+
+from tools.answer_evaluation.rag_delivery import finalize_rag_answer
+
+
+def test_notice_is_rendered_without_changing_facts_or_raw_answer():
+    original = {'answer': 'Eva má prácu z roku 2026.', 'context_issues': ['missing_incompleteness_notice'],
+                'citations_ok': True, 'accepted_for_delivery': False}
+    before = copy.deepcopy(original)
+    out = finalize_rag_answer(original)
+    assert original == before
+    assert out['model_answer'] == before['answer']
+    assert out['answer'] == 'Výpis je neúplný.\n\n' + before['answer']
+    assert out['accepted_for_delivery'] and out['display_notice_added']
+    assert not out['first_pass_accepted_for_delivery']
+    assert finalize_rag_answer(out) == out
+
+
+@pytest.mark.parametrize('issues,citations_ok', [
+    (['unsupported_exhaustive_statement', 'missing_incompleteness_notice'], True),
+    (['missing_incompleteness_notice'], False),
+    ([], False)])
+def test_notice_never_bypasses_other_rejections(issues, citations_ok):
+    original = {'answer': 'Úplný zoznam.', 'context_issues': issues,
+                'citations_ok': citations_ok, 'accepted_for_delivery': False}
+    out = finalize_rag_answer(original)
+    assert not out['accepted_for_delivery'] and not out['display_notice_added']
+    assert out['answer'] == original['answer']
+
+
+def test_complete_valid_answer_does_not_get_a_notice():
+    original = {'answer': 'Eva.', 'context_issues': [], 'citations_ok': True, 'accepted_for_delivery': True}
+    out = finalize_rag_answer(original)
+    assert out['answer'] == original['answer'] and not out['display_notice_added']
+    assert out['accepted_for_delivery']

tools/answer_evaluation/README.md

@@ -1,6 +1,33 @@
 # Generovanie odpovedí nad zmrazeným grafom
 
-Aktuálne meranie **v5.1** je historická regresia. Výsledky sú v
+Aktuálne meranie **v6** je historická regresia 320 známych otázok vrátane
+42 DEV otázok. [Výsledná správa](../graphrag/evaluation/results/final_v6_2026-10-25/REPORT.md)
+uvádza samostatne prvý pokus, výsledok po jednom povolenom opravnom požiadavku,
+odmietnuté odpovede a JSON chyby. Žiadna chyba sa nevynecháva z menovateľa.
+Finálny strict výsledok je **316/320 (98,75 %)**: tri neplatné JSON/schema
+výstupy a jedno prísne odmietnutie skloňovaného mena pri správnych claims.
+
+Spoločný adaptér prenáša úplnosť, orezanie, dôvody aj verziu dát. Guard používa
+iba skutočný kontext; kontroluje všetky citované URL, typy hodnôt, ich väzby
+a upozornenie pri neúplnom výpise. Môže vyžiadať najviac jednu opravu spolu
+pre formát alebo porušenie kontextu. Gold sa používa až pri hodnotení.
+Oba evaluačné grafové vstupy používajú tento istý adaptér a guard.
+
+`accepted_for_delivery=false` zakazuje považovať výstup za overenú odpoveď.
+Integrácia tejto kontroly do budúceho natívneho OpenWebUI agenta je súčasťou
+bodu 4; dnešné meranie grafu dodáva kontext cez lokálny klient.
+Klasický RAG pred doručením používa `finalize_rag_answer()`: môže pridať iba
+chýbajúce upozornenie na neúplnosť, bez zmeny faktov. Zachová pôvodný text
+modelu a neobíde nepovolené URL ani tvrdenie o úplnom zozname.
+
+[Podrobný postup a príkazy](../../docs/graphrag_v6_2026-10-25_worklog.md).
+Skripty `check_context_scenarios`, `check_rag_runtime`, `check_deployment`
+a `regrade_citations` oddeľujú syntetické funkčné kontroly, živý RAG smoke,
+overenie nasadenia a prepočet historických citácií od hlavnej presnosti.
+
+## Historická v5.1
+
+Meranie **v5.1** je historická regresia. Výsledky sú v
 [spoločnej správe](../graphrag/evaluation/results/final_v5_1_2026-10-25/REPORT.md).
 Používa 39 vopred vybraných DEV otázok (20 pôvodných a 19 historických zlyhaní)
 a 320 finálnych regresných otázok vrátane týchto 39. Nejde o nezávislý holdout.

tools/answer_evaluation/check_context_scenarios.py

@@ -0,0 +1,99 @@
+"""Live synthetic functional checks; separate from corpus accuracy metrics."""
+import argparse
+import concurrent.futures
+import copy
+import json
+import os
+from pathlib import Path
+
+from tools.rag.evaluation.rag_runner import request_json, OPENWEBUI_URL, DEFAULT_MODEL
+from tools.graphrag.evaluation.reproducibility import sha256_file
+from .checked_generation import generate_checked_answer
+from .context_policy import joined_candidates
+
+HERE = Path(__file__).parent
+
+
+def scenarios():
+    base = {'status': 'ok', 'reason': None, 'complete': True, 'truncated': False,
+            'truncation_reasons': [], 'index_version': {'version': 'synthetic-functional-fixture'},
+            'sources': [{'source_id': 'S1', 'document_path': 'a.md', 'source_url': 'https://example.test/a'}],
+            'facts': [{'person': 'Eva Vzorová', 'title': 'Vyhľadávanie informácií', 'year': 2026,
+                       'document_path': 'a.md', 'source_url': 'https://example.test/a'}],
+            'aggregation': None, 'answer_candidates': []}
+    query = 'Uveď názov práce osoby Eva Vzorová a rok práce.'
+    result = []
+    for name, fields in [
+        ('complete', {}), ('incomplete_without_truncated', {'complete': False}),
+        ('truncated', {'complete': False, 'truncated': True, 'truncation_reasons': ['limit']}),
+        ('unknown_completeness', {'complete': None})]:
+        context = copy.deepcopy(base); context.update(fields)
+        result.append({'id': name, 'question': query, 'context': context,
+                       'expected': {'status': 'ok', 'claims': [{'person': 'Eva Vzorová',
+                                      'title': 'Vyhľadávanie informácií', 'year': 2026}]}})
+    context = copy.deepcopy(base)
+    context.update(complete=False, truncated=True, truncation_reasons=['evidence_limit'],
+                   aggregation={'count': 2, 'complete': True})
+    result.append({'id': 'exact_count_partial_evidence', 'question': 'Aký je počet vyhovujúcich osôb?',
+                   'context': context, 'expected': {'status': 'ok', 'count': 2}})
+    context = copy.deepcopy(base)
+    context['sources'].append({'source_id': 'S2', 'document_path': 'child.md', 'source_url': 'https://example.test/child'})
+    context['facts'].append({'person': 'Eva Vzorová', 'keyword': 'python',
+                             'document_path': 'child.md', 'source_url': 'https://example.test/child'})
+    context['answer_candidates'] = joined_candidates(context['facts'], {'mentions_title': True, 'mentions_keyword': True})
+    result.append({'id': 'same_person_child_tag',
+                   'question': 'Uveď názov práce osoby Eva Vzorová v roku 2026 s kľúčovým slovom python.',
+                   'context': context, 'expected': {'status': 'ok', 'claims': [
+                       {'person': 'Eva Vzorová', 'title': 'Vyhľadávanie informácií', 'year': 2026, 'keyword': 'python'}]}})
+    context = copy.deepcopy(context)
+    context['facts'][0].pop('title')
+    context['answer_candidates'] = joined_candidates(context['facts'], {'years': [2026], 'mentions_keyword': True})
+    result.append({'id': 'same_person_year_child_tag',
+                   'question': 'Ktorá osoba spĺňa kľúčové slovo python a rok práce 2026?',
+                   'context': context, 'expected': {'status': 'ok', 'claims': [
+                       {'person': 'Eva Vzorová', 'year': 2026, 'keyword': 'python'}]}})
+    result.append({'id': 'type_is_not_keyword', 'question': 'Uveď názov diplomového projektu osoby Eva Vzorová a rok.',
+                   'context': copy.deepcopy(base), 'expected': {'status': 'ok', 'no_keyword': True}})
+    context = copy.deepcopy(base)
+    context.update(status='no_answer', reason='unknown_person', facts=[], sources=[])
+    result.append({'id': 'no_answer', 'question': 'Akú prácu má Neexistujúca Osoba?',
+                   'context': context, 'expected': {'status': 'no_answer'}})
+    return result
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--output', type=Path, required=True)
+    args = parser.parse_args()
+    if args.output.exists():
+        parser.error('Refusing to overwrite results.')
+    files = [Path(__file__), HERE/'graph_answer_prompt.txt', HERE/'context_policy.py', HERE/'checked_generation.py', HERE/'answer_format.py']
+    hashes = {str(p): sha256_file(p) for p in files}
+    declared = scenarios()
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.with_suffix('.declaration.json').write_text(json.dumps(
+        {'method': __doc__, 'model': DEFAULT_MODEL, 'hashes': hashes, 'scenarios': declared}, ensure_ascii=False, indent=2)+'\n')
+    prompt = (HERE/'graph_answer_prompt.txt').read_text()
+    def run(case):
+        def complete(messages):
+            return request_json(OPENWEBUI_URL, method='POST', timeout=90, max_attempts=2,
+                headers={'Authorization': 'Bearer '+os.environ['OPENWEBUI_API_KEY'], 'Content-Type': 'application/json'},
+                payload={'model': DEFAULT_MODEL, 'temperature': 0, 'stream': False,
+                         'response_format': {'type': 'json_object'}, 'messages': messages})
+        out = generate_checked_answer(case['question'], case['context'], prompt, complete)
+        answer = out.get('answer', {}); expected = case['expected']
+        ok = answer.get('status') == expected['status']
+        for binding in expected.get('claims', []):
+            ok &= any(all(claim.get(k) == v for k, v in binding.items()) for claim in answer.get('claims', []))
+        if 'count' in expected: ok &= answer.get('count') == expected['count']
+        if expected.get('no_keyword'): ok &= not any(c.get('keyword') for c in answer.get('claims', []))
+        return {'id': case['id'], **out, 'passed': bool(ok and out['accepted_for_delivery'])}
+    with concurrent.futures.ThreadPoolExecutor(max_workers=3) as executor:
+        rows = list(executor.map(run, declared))
+    assert hashes == {str(p): sha256_file(p) for p in files}
+    output = {'method': __doc__, 'questions': rows, 'summary': {'total': len(rows), 'passed': sum(r['passed'] for r in rows)}}
+    args.output.write_text(json.dumps(output, ensure_ascii=False, indent=2)+'\n')
+    print(json.dumps(output['summary']))
+
+
+if __name__ == '__main__': main()

tools/answer_evaluation/check_deployment.py

@@ -0,0 +1,82 @@
+"""Verify the running API and graph library against a frozen release."""
+import argparse
+import json
+import os
+import platform
+from datetime import datetime, timezone
+from pathlib import Path
+from importlib.metadata import version as package_version
+
+from tools.rag.evaluation.rag_runner import request_json
+from tools.graphrag.graph_db import create_neo4j_driver
+from tools.graphrag.graph_search import search_graph
+from tools.graphrag.build_graphrag import graph_version
+from tools.graphrag.evaluation.reproducibility import graph_fingerprint, run_metadata, verify_freeze, sha256_file
+from tools.index_state import read_version, sqlite_version
+from tools.rag.common import DB_FILE
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--output', type=Path, required=True)
+    args = parser.parse_args()
+    if args.output.exists(): parser.error('Refusing to overwrite deployment verification.')
+    base = Path('tools/graphrag/evaluation')
+    release_path = base / 'frozen_release_v6.json'
+    release_match = None
+    if release_path.exists():
+        release = json.loads(release_path.read_text())
+        for path, digest in release['runtime_files'].items():
+            assert sha256_file(Path(path)) == digest, f'Release code mismatch: {path}'
+        assert sha256_file(base/'frozen_v6.json') == release['graph_freeze_sha256']
+        release_match = True
+    freeze = json.loads((base/'frozen_v6.json').read_text())
+    health = request_json('http://localhost:8000/health')
+    assert health['ready'] and all(v['ready'] for v in health['tools'].values())
+    with create_neo4j_driver() as driver, driver.session() as session:
+        graph = graph_fingerprint(session)
+        verify_freeze(freeze, run_metadata(base/'graphrag_questions_v6.json'), graph,
+                      audit=base/'results/v6_preparation/source_audit.json', **freeze['parameters'])
+        current = read_version(Path('data'))
+        assert current == sqlite_version(DB_FILE) == graph_version(session) == health['index_version']
+        documents = session.run('MATCH (d:Document {graph_scope:"zpwiki"}) RETURN d.path AS path, d.published AS published').data()
+    headers = {'X-API-Key': os.environ['SEARCH_API_KEY'], 'Content-Type': 'application/json'}
+    rag = request_json('http://localhost:8000/rag', method='POST', headers=headers,
+                       payload={'query': 'Aký je názov práce osoby Ján Holp a rok práce?', 'limit': 5}, timeout=120)
+    assert rag['status'] == 'ok' and rag['sources'] and rag['index_version'] == current
+    coverage = json.loads(rag['context'].splitlines()[1])
+    assert coverage == {k: rag[k] for k in ['complete', 'truncated', 'truncation_reasons', 'index_version']}
+    cases = {}
+    for id, question, expected in [
+        ('known_title', 'Aký je názov práce osoby Ján Holp a rok práce?', 'ok'),
+        ('generic_label_not_title', 'Uveď názov práce osoby Dárius Lindvai v roku 2019 s kľúčovým slovom python.', 'no_answer'),
+        ('unknown_person', 'Akú prácu má osoba Jozef Neznámy Overovací?', 'no_answer')]:
+        result = search_graph(question)
+        assert result['status'] == expected and result['index_version'] == current
+        cases[id] = {'status': result['status'], 'reason': result['reason'], 'complete': result['complete']}
+    hidden = {row['path'] for row in documents if row['published'] is False}
+    raw_documents = json.loads(Path('data/documents.json').read_text())
+    missing = sum(row['published'] is None for row in raw_documents)
+    assert len(hidden) == 1 and missing == 16
+    assert {row['path']: row['published'] for row in documents} == {
+        row['path']: row['published'] is not False for row in raw_documents}
+    hidden_rag = request_json('http://localhost:8000/rag', method='POST', headers=headers,
+        payload={'query': 'Tvorba nezodpovedateľných otázok', 'limit': 5}, timeout=120)
+    assert not hidden & {s['document_path'] for s in hidden_rag['sources']}
+    result = {'checked_at_utc': datetime.now(timezone.utc).isoformat(), 'health': health,
+        'index_version': current, 'frozen_implementation_inputs_and_graph_match': True,
+        'graph': graph, 'graph_cases': cases, 'rag_live_query': 'ok',
+        'release_code_match': release_match,
+        'rag_context_metadata_matches_contract': True,
+        'publication': {'documents': len(documents), 'explicit_false': len(hidden),
+                        'missing_included': missing, 'visible_by_policy': len(documents)-len(hidden),
+                        'hidden_document_absent_from_rag': True},
+        'runtime': {'python': platform.python_version(), **{p: package_version(p) for p in
+                     ['fastapi', 'pydantic', 'neo4j', 'sentence-transformers', 'torch']}},
+        'scope': 'RAG HTTP API plus GraphRAG library/CLI; native OpenWebUI graph-tool routing is point 4.'}
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(result, ensure_ascii=False, indent=2)+'\n')
+    print(json.dumps({'ready': True, 'version': current['version'], 'graph_cases': cases}))
+
+
+if __name__ == '__main__': main()

tools/answer_evaluation/check_rag_runtime.py

@@ -0,0 +1,49 @@
+"""Five preselected historical RAG smoke cases; not a new accuracy estimate."""
+import argparse
+import concurrent.futures
+import json
+import os
+from pathlib import Path
+
+from tools.rag.evaluation.rag_runner import build_rag_tool, request_json, run_question, LOCAL_OPENAPI_URL, DEFAULT_MODEL
+from tools.rag.evaluation.rag_metrics import evaluate_answer
+from tools.graphrag.evaluation.reproducibility import sha256_file
+from .rag_delivery import finalize_rag_answer
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--output', type=Path, required=True)
+    args = parser.parse_args()
+    if args.output.exists(): parser.error('Refusing to overwrite smoke results.')
+    source = Path('tools/rag/evaluation/results/test_classic_rag_frozen/rag_answers_test.json')
+    historic = json.loads(source.read_text())['results']
+    selected = ['q0053', 'q0060', 'q0608', 'q1147', 'q0888']
+    cases = [next(row for row in historic if row['id'] == id) for id in selected]
+    operation, tool = build_rag_tool(request_json(LOCAL_OPENAPI_URL))
+    metadata = {'method': __doc__, 'selected_ids': selected, 'model': DEFAULT_MODEL,
+                'source_file': str(source), 'source_sha256': sha256_file(source),
+                'generation_files': {str(p): sha256_file(p) for p in [Path(__file__),
+                    Path('tools/rag/evaluation/rag_runner.py'), Path('tools/rag/evaluation/rag_metrics.py'),
+                    Path('tools/answer_evaluation/context_policy.py'), Path('tools/answer_evaluation/rag_delivery.py')]}}
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.with_suffix('.declaration.json').write_text(json.dumps(metadata, ensure_ascii=False, indent=2)+'\n')
+    def run(case):
+        out = run_question(case['question'], model=DEFAULT_MODEL, operation_id=operation,
+            rag_tool=tool, openwebui_api_key=os.environ['OPENWEBUI_API_KEY'],
+            search_api_key=os.environ['SEARCH_API_KEY'], timeout=120, max_attempts=2)
+        out = finalize_rag_answer(out)
+        score = evaluate_answer(case, out['answer'], tool_called=out['tool_called'],
+                                allowed_source_urls=out['rag_source_urls'])
+        return {'id': case['id'], 'question': case['question'], **out, **score,
+                'strict_pass': bool(score['strict_pass'] and out['accepted_for_delivery'])}
+    with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor:
+        rows = list(executor.map(run, cases))
+    assert metadata['generation_files'] == {p: sha256_file(Path(p)) for p in metadata['generation_files']}
+    result = {'configuration': metadata, 'results': rows,
+              'summary': {'questions': len(rows), 'strict_pass': sum(row['strict_pass'] for row in rows)}}
+    args.output.write_text(json.dumps(result, ensure_ascii=False, indent=2)+'\n')
+    print(json.dumps(result['summary']))
+
+
+if __name__ == '__main__': main()

tools/answer_evaluation/checked_generation.py

@@ -0,0 +1,46 @@
+"""One bounded corrective request based solely on retrieval evidence."""
+import json
+from .answer_format import parse_answer, Answer
+from .context_policy import answer_context_issues
+
+
+def generate_checked_answer(question: str, context: dict, prompt: str, complete) -> dict:
+    messages=[{'role':'system','content':prompt},{'role':'user','content':json.dumps(
+        {'question':question,'graph_context':context},ensure_ascii=False)}]
+    result={'attempts':[],'accepted_for_delivery':False}
+    for number in range(2):
+        record={};text=''
+        try:
+            response=complete(messages)
+            result['response']=response;record['response']=response
+            text=response['choices'][0]['message']['content'].strip()
+        except Exception as error:
+            result['error']=f'{type(error).__name__}: {str(error)[:500]}'
+            result['attempts'].append(record)
+            return result
+        result['attempts'].append(record)
+        try:
+            answer,normalized=parse_answer(text)
+            record['answer']=answer;record['format_normalized']=normalized
+        except (ValueError,TypeError) as error:
+            record['format_error']=type(error).__name__
+            if number==1:
+                result['error']='InvalidAnswerFormat: two invalid JSON/schema responses'
+                return result
+            feedback='Vráť jediný platný JSON objekt podľa schémy, bez zmeny faktov z pôvodného kontextu. Schéma: '+json.dumps(Answer.model_json_schema(),ensure_ascii=False)
+        else:
+            issues=answer_context_issues(answer,context)
+            record['context_issues']=issues
+            result['answer']=answer
+            if not issues:
+                result['accepted_for_delivery']=True
+                return result
+            if number==1:
+                result['context_rejected']=True
+                return result
+            feedback=('Odpoveď porušila pravidlá dodaného kontextu: '+', '.join(issues)+'. '
+                'Oprav jediný JSON objekt výhradne z pôvodných podkladov. Použi iba source_urls zo sources. '
+                'Typ práce nie je keyword. answer_candidates dokladajú spojenie požadovaného názvu alebo roku a tagu cez tú istú osobu. '
+                'Pri neúplnom výstupe uveď: Výpis je neúplný.')
+        messages += [{'role':'assistant','content':text},{'role':'user','content':feedback}]
+    return result

tools/answer_evaluation/context_policy.py

@@ -0,0 +1,109 @@
+"""Answer guards using only actual retrieval context; no benchmark or oracle."""
+from __future__ import annotations
+
+import re
+import unicodedata
+
+
+def normalized(value) -> str:
+    text=unicodedata.normalize('NFKD',str(value or ''))
+    return ' '.join(re.findall(r'[a-z0-9]+',''.join(c for c in text if not unicodedata.combining(c)).casefold()))
+
+
+def text_urls(text: str) -> set[str]:
+    urls = re.findall(r'(?:[a-zA-Z][a-zA-Z0-9+.-]*://|www\.)[^\s<>"\x27`“”„«»‘’\]\)]+', text)
+    # Relative Markdown targets are citations too, and must be supplied by
+    # the tool just like absolute links. Never resolve/guess another URL.
+    urls += re.findall(r'\[[^\]]*\]\(\s*<?([^\s)>]+)>?', text)
+    return {url.rstrip('.,;:!?') for url in urls}
+
+
+def citation_check(answer: dict, allowed: set[str]) -> dict:
+    declared=answer.get('source_urls',[])
+    if not isinstance(declared,list) or any(not isinstance(url,str) for url in declared):
+        return {'citations_ok':False,'unsupported_urls':[],'reason':'invalid_source_urls'}
+    used=set(declared)|text_urls(str(answer.get('answer','')))
+    unsupported=sorted(used-allowed)
+    return {'citations_ok':not unsupported,'unsupported_urls':unsupported}
+
+
+def allowed_urls(context: dict) -> set[str]:
+    return {row['source_url'] for row in context.get('sources',[]) if row.get('source_url')}
+
+
+def joined_candidates(facts: list[dict], intent: dict) -> list[dict]:
+    """Join requested academic fields and tag by person, retaining real sources."""
+    needs_title = intent.get('mentions_title')
+    needs_year = intent.get('mentions_academic_year') or intent.get('years')
+    if not (needs_title or needs_year) or not (intent.get('mentions_keyword') or intent.get('keyword_phrase')):
+        return []
+    result=[]
+    for record in facts:
+        if (needs_title and not record.get('title')) or (not needs_title and record.get('year') is None):
+            continue
+        for tag in facts:
+            if normalized(tag.get('person'))!=normalized(record.get('person')) or not tag.get('keyword'):
+                continue
+            fields = ['person', 'title', 'year'] if needs_title else ['person', 'year']
+            candidate={k:record[k] for k in fields if record.get(k) is not None}
+            candidate['keyword']=tag['keyword']
+            candidate['evidence_documents']=sorted({p for p in [record.get('document_path'),tag.get('document_path')] if p})
+            candidate['source_urls']=sorted({u for u in [record.get('source_url'),tag.get('source_url')] if u})
+            if candidate not in result:result.append(candidate)
+    return result
+
+
+def coverage_issues(answer: dict, context: dict) -> list[str]:
+    incomplete=context.get('truncated') is True or context.get('complete') is False
+    unknown=context.get('complete') is None
+    text=normalized(answer.get('answer',''))
+    caveat=any(phrase in text for phrase in ['vysledok je neuplny','vysledky su neuplne','vypis je neuplny',
+        'zoznam je neuplny','dokazy su orezane','vysledok je orezany','vysledky su orezane',
+        'nemusi byt uplny','nemusi byt uplna','nie je uplny','nie su uplne','nemozno zarucit uplnost'])
+    issues=[]
+    if incomplete and answer.get('status')=='ok' and not caveat:
+        issues.append('missing_incompleteness_notice')
+    exhaustive=any(phrase in text for phrase in ['uplny zoznam','kompletny zoznam','vsetci studenti','vsetky osoby','vsetky prace'])
+    if (incomplete or unknown) and exhaustive and not caveat:
+        issues.append('unsupported_exhaustive_statement')
+    return issues
+
+
+def answer_context_issues(answer: dict, context: dict) -> list[str]:
+    issues=[]
+    if not citation_check(answer,allowed_urls(context))['citations_ok']:
+        issues.append('unsupported_citation')
+    facts=context.get('facts',[])
+    aggregation=context.get('aggregation') or {}
+    if answer.get('count') is not None and aggregation:
+        if aggregation.get('complete') is not True:
+            issues.append('unsupported_total_count')
+        elif answer['count']!=aggregation.get('count'):
+            issues.append('incorrect_context_count')
+    for claim in answer.get('claims',[]):
+        person=normalized(claim.get('person'))
+        rows=[f for f in facts if normalized(f.get('person'))==person]
+        if not rows:
+            issues.append('unsupported_person');continue
+        for key in ['keyword','category','document_author','start_year']:
+            if claim.get(key) is not None and not any(normalized(row.get(key))==normalized(claim[key]) for row in rows):
+                issues.append('unsupported_'+key)
+        academic={k:claim[k] for k in ['title','year'] if claim.get(k) is not None}
+        if academic and not any(all(normalized(row.get(k))==normalized(v) for k,v in academic.items()) for row in rows):
+            issues.append('unsupported_academic_binding')
+        if claim.get('category') is not None and claim.get('document_author') is not None:
+            if not any(a.get('document_path') and a.get('document_path')==b.get('document_path')
+                for a in rows if normalized(a.get('category'))==normalized(claim['category'])
+                for b in rows if normalized(b.get('document_author'))==normalized(claim['document_author'])):
+                issues.append('disconnected_author_category')
+    candidates=context.get('answer_candidates',[])
+    if context.get('status')=='ok' and candidates:
+        if answer.get('status')=='no_answer':
+            issues.append('rejected_supported_person_join')
+        elif not all(any(all(normalized(claim.get(k))==normalized(candidate[k]) for k in ['person','title','year','keyword'] if k in candidate)
+                         for claim in answer.get('claims',[])) for candidate in candidates):
+            issues.append('split_or_missing_person_join')
+    if context.get('status') in {'no_answer','ambiguous','error'} and answer.get('status')=='ok':
+        issues.append('unsupported_positive_status')
+    issues.extend(coverage_issues(answer,context))
+    return list(dict.fromkeys(issues))

tools/answer_evaluation/evaluate_graph_answers.py

@@ -1,7 +1,7 @@
-"""Evaluate 300 fresh LLM outputs after 20 exposed development outputs.
+"""Measure generated answers on a declared, frozen graph benchmark.
 
-The 320-query graph benchmark stays frozen. This separate experiment supplies
+Regression mode repeats known questions; it is never a new independent TEST.
-the retriever's context to OpenWebUI; it does not test native tool routing.
+The harness supplies retrieval context and does not test native tool routing.
 """
 from __future__ import annotations
 
@@ -17,12 +17,14 @@ from pathlib import Path
 
 from tools.graphrag.graph_db import create_neo4j_driver, get_neo4j_settings
 from tools.graphrag.graph_search import search_graph_with_session
+from tools.graphrag.build_graphrag import graph_version
 from tools.graphrag.evaluation.benchmark_v2 import BASE
 from tools.graphrag.evaluation.check_openwebui import grade_claims
 from tools.graphrag.evaluation.reproducibility import graph_fingerprint, run_metadata, sha256_file, verify_freeze
 from tools.graphrag.evaluation.source_oracle import load_source_oracle, norm
 from tools.rag.evaluation.rag_runner import DEFAULT_MODEL, OPENWEBUI_URL, request_json
-from .answer_format import parse_answer, Answer
+from .checked_generation import generate_checked_answer
+from .context_policy import joined_candidates, citation_check, allowed_urls, answer_context_issues
 
 HERE = Path(__file__).resolve().parent
 
@@ -50,19 +52,26 @@ def graph_context(result: dict) -> dict:
         allowed = {'person', 'document_path', 'source_url', 'keyword', 'category', 'document_author'}
         if intent['mentions_title']:
             allowed.add('title')
-        if intent['mentions_academic_year'] or intent['years']:
+        if intent['mentions_title'] or intent['mentions_academic_year'] or intent['years']:
             allowed.add('year')
         if intent['mentions_start_year']:
             allowed.add('start_year')
         fact = {k: v for k, v in row.items() if k in allowed and v is not None}
         if fact not in facts:
             facts.append(fact)
-    return {'status': result['status'], 'reason': result['reason'], 'facts': facts,
+    sources=result.get('sources')
-            'sources': [{k: d[k] for k in ['document_path', 'source_url'] if d.get(k)}
+    if sources is None:
-                        for d in result['documents']], 'aggregation': result['aggregation']}
+        sources=[{k:d[k] for k in ['document_path','source_url'] if d.get(k)} for d in result.get('documents',[])]
+    context={'status':result['status'],'reason':result['reason'],'facts':facts,'sources':sources,
+             'aggregation':result['aggregation'],'complete':result.get('complete'),
+             'truncated':result.get('truncated',False),
+             'truncation_reasons':result.get('truncation_reasons',[]),
+             'index_version':result.get('index_version')}
+    context['answer_candidates']=joined_candidates(facts,intent)
+    return context
 
 
-def grade_answer(case: dict, answer: dict, oracle: dict) -> dict:
+def grade_answer(case: dict, answer: dict, oracle: dict, *, context: dict | None = None) -> dict:
     # JSON null is absence, not a positive factual assertion. The same rule is
     # applicable to all rounds, including the archived initial sample.
     normalized = dict(answer)
@@ -70,6 +79,12 @@ def grade_answer(case: dict, answer: dict, oracle: dict) -> dict:
         normalized['claims'] = [{k: v for k, v in claim.items() if v is not None}
                                 if isinstance(claim, dict) else claim for claim in answer['claims']]
     grade = grade_claims(case, normalized, oracle)
+    urls=(allowed_urls(context) if context is not None else
+          {d['source_url'] for d in oracle.get('documents',{}).values() if d.get('source_url')})
+    citations=citation_check(answer,urls)
+    grade.update(citations)
+    policy_issues=answer_context_issues(normalized,context) if context is not None else []
+    grade['context_issues']=policy_issues
     text = norm(answer.get('answer', ''))
     missing = [v for v in case.get('expected_answer_contains', []) if norm(v) not in text]
     # The initial sample exposed a real contradiction in otherwise correct
@@ -79,7 +94,7 @@ def grade_answer(case: dict, answer: dict, oracle: dict) -> dict:
                                str(b.get('category', '')).startswith('phd') for b in case['graph']['expected_bindings'])
     grade.update(text_values_ok=not missing, missing_text_values=missing,
                  contradictory_thesis_label=wrong_thesis_label)
-    grade['strict'] = bool(grade['strict'] and not missing and not wrong_thesis_label)
+    grade['strict'] = bool(grade['strict'] and not missing and not wrong_thesis_label and citations['citations_ok'] and not policy_issues)
     return grade
 
 
@@ -120,18 +135,26 @@ def main():
     metadata = run_metadata(dataset); settings = get_neo4j_settings(); contexts = {}
     with create_neo4j_driver(settings) as driver, driver.session(database=settings.database) as session:
         verify_freeze(manifest, metadata, graph_fingerprint(session), audit=audit, **manifest['parameters'])
+        metadata['index_version']=graph_version(session)
         # Generate every context before making any external request.
         for case in selected:
-            contexts[case['id']] = graph_context(search_graph_with_session(session, case['question'], **manifest['parameters']))
+            result=search_graph_with_session(session,case['question'],**manifest['parameters'])
-    code_files = [Path(__file__), prompt_path, HERE/'answer_format.py', Path('tools/rag/evaluation/rag_runner.py')]
+            result['index_version']=metadata['index_version']
+            contexts[case['id']]=graph_context(result)
+    code_files = [Path(__file__), prompt_path, HERE/'answer_format.py',HERE/'context_policy.py',HERE/'checked_generation.py', Path('tools/rag/evaluation/rag_runner.py')]
     hashes = {str(p): sha256_file(p) for p in code_files}
     if args.split=='regression':
         dev=json.loads(args.dev_results.read_text())
         if (dev['configuration']['selected_split']!='dev'
             or set(dev['configuration']['selected_ids'])!=development
             or dev['summary']['questions']!=len(development)
+            or {row['id'] for row in dev['questions']}!=development
+            or len(dev['questions'])!=len(development)
             or dev['configuration']['generation_files']!=hashes
-            or dev['run_metadata']['fingerprints']!=metadata['fingerprints']):
+            or dev['run_metadata']['fingerprints']!=metadata['fingerprints']
+            or dev['configuration']['graph_freeze_sha256']!=sha256_file(frozen)
+            or dev['configuration']['model']!=args.model
+            or dev['run_metadata'].get('index_version')!=metadata['index_version']):
             raise RuntimeError('Generation code, prompt, data or complete DEV differs; rerun DEV before final freeze.')
     declaration = {'method': ('Previously exposed LLM questions; regression, not independent TEST.'
                     if manifest.get('evaluation_kind')=='historical_regression' else
@@ -142,13 +165,16 @@ def main():
         'dev_count': len(development), 'test_count': len(test), 'target_accuracy': 0.92,
         'selected_split': args.split, 'selected_ids': [q['id'] for q in selected], 'selected_count':len(selected),
         'selected_dev_overlap': len(development & {q['id'] for q in selected}),
-        'format_repair_limit':1, 'repair_policy':'Only JSON/schema failures; same question/context, no gold; retain all attempts.',
+        'corrective_request_limit':1,'format_repair_limit':1,
+        'repair_policy':'At most one total corrective request for format, citations or retrieval-context violations. Same question/context, no gold; retain all attempts.',
         'dev_results_sha256':sha256_file(args.dev_results) if args.dev_results else None,
         'policy': ('Evaluate every declared regression case, including DEV overlap; retain all failures and stop tuning after this freeze.'
                    if manifest.get('evaluation_kind')=='historical_regression' else
                    'Evaluate all 300; retain every failure; do not tune on this TEST.'),
         'selected_categories': dict(Counter(q['category'] for q in selected)),
-        'limitations': ['Known corpus/profiles and templates; model outputs for these 300 questions were not previously observed.',
+        'limitations': [('All questions and previous model outputs are known; this is historical regression.'
+                         if manifest.get('evaluation_kind')=='historical_regression' else
+                         'Known corpus/profiles and templates; model outputs for these 300 questions were not previously observed.'),
             'Harness supplies graph context; native OpenWebUI tool choice and browser UI are outside this experiment.',
             'Checks structured factual claims, their source support/citations and required prose values; not a general semantic judge of arbitrary prose.']}
     (args.output_dir / 'frozen_answers.json').write_text(json.dumps(declaration, ensure_ascii=False, indent=2) + '\n')
@@ -157,36 +183,21 @@ def main():
         started = time.perf_counter(); context = contexts[case['id']]
         row = {'id': case['id'], 'category': case['category'], 'question': case['question'],
                'graph_context': context, 'strict': False}
-        messages=[{'role':'system','content':prompt},{'role':'user','content':json.dumps(
+        def complete(messages):
-                    {'question':case['question'],'graph_context':context},ensure_ascii=False)}]
+            return request_json(OPENWEBUI_URL, method='POST', timeout=90, max_attempts=2,
-        row['attempts']=[]
+                headers={'Authorization':f'Bearer {os.environ["OPENWEBUI_API_KEY"]}','Content-Type':'application/json'},
-        try:
+                payload={'model':args.model,'temperature':0,'stream':False,
-          for attempt in range(2):
+                         'response_format':{'type':'json_object'},'messages':messages})
-            response = request_json(OPENWEBUI_URL, method='POST', timeout=90, max_attempts=2,
+        generated=generate_checked_answer(case['question'],context,prompt,complete)
-                headers={'Authorization': f'Bearer {os.environ["OPENWEBUI_API_KEY"]}', 'Content-Type': 'application/json'},
+        row.update(generated)
-                payload={'model': args.model, 'temperature': 0, 'stream': False,
+        # Grading occurs after all generation/correction decisions, never inside
-                    'response_format': {'type': 'json_object'}, 'messages': messages})
+        # the generator. Gold cannot influence whether another request is sent.
-            row['response'] = response
+        for attempt in row['attempts']:
-            text = response['choices'][0]['message']['content'].strip()
+            attempt['strict']=(grade_answer(case,attempt['answer'],oracle,context=context)['strict']
-            record={'response':response}
+                               if attempt.get('answer') else False)
-            row['attempts'].append(record)
+        if row.get('answer'):
-            try:
+            row.update(grade_answer(case,row['answer'],oracle,context=context))
-                answer,normalized=parse_answer(text)
+        row['strict']=bool(row['strict'] and row['accepted_for_delivery'])
-            except (ValueError, TypeError) as exc:
-                record['format_error']=type(exc).__name__
-                if attempt==1:
-                    raise
-                messages += [{'role':'assistant','content':text}, {'role':'user','content':
-                    'Predchádzajúci výstup nie je platný JSON podľa schémy. Vráť len opravený jediný JSON objekt '
-                    'bez zmeny vecných údajov; používaj iba pôvodný kontext. Schéma: '+json.dumps(Answer.model_json_schema(),ensure_ascii=False)}]
-                continue
-            record['format_normalized']=normalized
-            grade=grade_answer(case,answer,oracle)
-            record['strict']=grade['strict']
-            row['answer']=answer; row.update(grade)
-            break
-        except Exception as error:
-            row['error'] = f'{type(error).__name__}: {str(error)[:500]}'
         row['latency_seconds'] = time.perf_counter() - started
         return row
     output = {'configuration': declaration, 'run_metadata': metadata, 'questions': []}
@@ -199,12 +210,15 @@ def main():
                 'accuracy': sum(r['strict'] for r in results.values()) / len(results),
                 'errors': sum(bool(r.get('error')) for r in results.values()),
                 'first_pass_correct':sum(bool(r.get('attempts') and r['attempts'][0].get('strict')) for r in results.values()),
-                'format_repair_attempted':sum(len(r.get('attempts',[]))>1 for r in results.values()),
+                'format_repair_attempted':sum(bool(r.get('attempts') and r['attempts'][0].get('format_error') and len(r['attempts'])>1) for r in results.values()),
+                'corrective_request_count':sum(len(r.get('attempts',[]))>1 for r in results.values()),
+                'context_repair_attempted':sum(bool(r.get('attempts') and r['attempts'][0].get('context_issues') and len(r['attempts'])>1) for r in results.values()),
                 'normalized_first_response':sum(bool(r.get('attempts') and r['attempts'][0].get('format_normalized')) for r in results.values())}
             (args.output_dir / f'answers_{args.split}.json').write_text(json.dumps(output, ensure_ascii=False, indent=2) + '\n')
             print(f'{len(results)}/{len(selected)} {row["id"]}: {"OK" if row["strict"] else "FAIL"} {row["latency_seconds"]:.1f}s', flush=True)
     with create_neo4j_driver(settings) as driver, driver.session(database=settings.database) as session:
         verify_freeze(manifest, run_metadata(dataset), graph_fingerprint(session), audit=audit, **manifest['parameters'])
+        assert graph_version(session)==metadata['index_version'],'Index version changed during generation.'
     assert hashes == {str(p): sha256_file(p) for p in code_files}, 'Generation code changed during TEST.'
     output['by_category'] = {category: {'questions': len(rows), 'correct': sum(r['strict'] for r in rows),
         'accuracy': sum(r['strict'] for r in rows) / len(rows)} for category in sorted({q['category'] for q in selected})

tools/answer_evaluation/graph_answer_prompt.txt

@@ -11,3 +11,6 @@ Kľúčové slovo sa v tomto nástroji viaže na osobu cez jej vlastné aj dcér
 Pred odoslaním pri otázke na názov a tag over, že claim s názvom obsahuje súčasne person, title, year aj keyword. Nerozdeľ túto požadovanú väzbu na samostatnú claim s názvom a samostatnú claim s tagom.
 Pri rokoch porovnaj len akademické záznamy dodané v kontexte, nie poznámky o príprave. Pri počte nemiešaj nevyhovujúcich kandidátov do claims.
 Vráť iba jediný platný JSON objekt bez úvodu, vysvetlení mimo JSON a bez Markdown ohraničenia. Reťazce ohraničuj ASCII dvojitými úvodzovkami. Vo vnútri answer názvy neobaľuj ďalšími ASCII dvojitými úvodzovkami; ak ich použiješ, musia byť escapované. Roky a count sú celé čísla. Nevypisuj čiarku za poslednou položkou.
+Rešpektuj complete, truncated, truncation_reasons a index_version. Ak truncated=true alebo complete=false a uvádzaš výsledky, do answer vlož vetu „Výpis je neúplný.“ Neoznačuj výpis za úplný zoznam ani za všetky existujúce práce. Pri complete=null úplnosť nemožno zaručiť. Ak aggregation.complete=true, počet môže byť presný aj pri orezaných dôkazoch; vtedy uveď „Počet je úplný, dôkazy sú orezané.“ Inak z počtu zobrazených položiek neodvodzuj celkový počet.
+Všetky URL v answer aj source_urls musia byť presne zo sources. Typ práce (napríklad diplomový projekt) nie je kľúčové slovo; keyword uveď iba ak ho dokladá fakt keyword pre tú istú person.
+answer_candidates sú doložené spojenia požadovaného názvu alebo roku a tagu cez tú istú osobu, nie nové odhady. Ak je taký kandidát poskytnutý a status kontextu je ok, odpovedz podľa neho a neodmietaj ho len preto, že tag pochádza z dcérskeho dokumentu. V claims ponechaj jeho person, year, keyword a prípadne title spolu v jednom objekte a cituj všetky jeho podklady. Aj pri samotnej kombinácii tag + rok práce sa tag viaže na osobu, nemusí byť v dokumente obsahujúcom rok práce.

tools/answer_evaluation/rag_delivery.py

@@ -0,0 +1,65 @@
+"""Render a truthful coverage notice without altering the model's facts.
+
+No gold or source oracle is accepted. Unsupported citations and unqualified
+exhaustive statements remain rejected. Keep the exact original model answer.
+"""
+import copy
+
+
+def finalize_rag_answer(result: dict) -> dict:
+    out = copy.deepcopy(result)
+    if out.get('display_notice_added'):
+        return out
+    issues = out.get('context_issues', [])
+    out['model_answer'] = out['answer']
+    out['first_pass_accepted_for_delivery'] = out.get('accepted_for_delivery')
+    out['first_pass_context_issues'] = list(issues)
+    out['display_notice_added'] = False
+    # Only a missing notice can be rendered automatically. Adding a caveat to
+    # a contradictory exhaustive statement would not repair that statement.
+    if issues == ['missing_incompleteness_notice'] and out.get('citations_ok') is True:
+        out['answer'] = 'Výpis je neúplný.\n\n' + out['model_answer']
+        out['context_issues'] = []
+        out['accepted_for_delivery'] = True
+        out['display_notice_added'] = True
+    return out
+
+
+def main():
+    import argparse
+    import json
+    from pathlib import Path
+    from tools.graphrag.evaluation.reproducibility import sha256_file
+    from tools.rag.evaluation.rag_metrics import evaluate_answer
+
+    parser = argparse.ArgumentParser(description='Replay saved RAG outputs through the delivery layer; no model calls.')
+    parser.add_argument('--input', type=Path, required=True)
+    parser.add_argument('--output', type=Path, required=True)
+    args = parser.parse_args()
+    if args.output.exists(): parser.error('Refusing to overwrite a replay.')
+    original = json.loads(args.input.read_text())
+    source = Path(original['configuration']['source_file'])
+    assert sha256_file(source) == original['configuration']['source_sha256']
+    gold = {case['id']: case for case in json.loads(source.read_text())['results']}
+    files = [Path(__file__), Path('tools/rag/evaluation/evaluate_rag_answers.py'),
+             Path('tools/rag/evaluation/rag_metrics.py'), Path('tools/rag/evaluation/rag_runner.py'),
+             Path('tools/answer_evaluation/context_policy.py')]
+    rows = []
+    for before in original['results']:
+        after = finalize_rag_answer(before)
+        score = evaluate_answer(gold[after['id']], after['answer'], tool_called=after['tool_called'],
+                                allowed_source_urls=after['rag_source_urls'])
+        after.update(score)
+        after['strict_pass'] = bool(score['strict_pass'] and after['accepted_for_delivery'])
+        rows.append(after)
+    result = {'configuration': {'method': 'Deterministic delivery replay on the same saved model responses, not new generation.',
+                  'input': str(args.input), 'input_sha256': sha256_file(args.input),
+                  'delivery_files': {str(p): sha256_file(p) for p in files}},
+              'results': rows, 'summary': {'questions': len(rows), 'strict_pass': sum(r['strict_pass'] for r in rows),
+                                         'display_notice_added': sum(r['display_notice_added'] for r in rows)}}
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(result, ensure_ascii=False, indent=2)+'\n')
+    print(json.dumps(result['summary']))
+
+
+if __name__ == '__main__': main()

tools/answer_evaluation/regrade_citations.py

@@ -0,0 +1,46 @@
+"""Recheck archived citations against archived tool sources, without generation.
+
+This changes only citation eligibility; it does not replace historical gold or
+claim a new full semantic evaluation. Missing archived context is unverifiable.
+"""
+import argparse
+import json
+from pathlib import Path
+
+from tools.graphrag.evaluation.reproducibility import sha256_file
+from .context_policy import citation_check, allowed_urls
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--input', type=Path, action='append', required=True)
+    parser.add_argument('--output', type=Path, required=True)
+    args = parser.parse_args()
+    if args.output.exists(): parser.error('Refusing to overwrite regrading.')
+    reports = []
+    for path in args.input:
+        data = json.loads(path.read_text()); rows = []
+        for old in data.get('questions', data.get('results', [])):
+            answer = old.get('answer') or ''
+            if isinstance(answer, str): answer = {'answer': answer, 'source_urls': []}
+            context = old.get('graph_context')
+            legacy_sources = ({row['source_url'] for row in context.get('documents', []) if row.get('source_url')}
+                              if isinstance(context, dict) and 'documents' in context else None)
+            urls = (allowed_urls(context) if isinstance(context, dict) and 'sources' in context else
+                    legacy_sources if legacy_sources is not None else
+                    set(old['rag_source_urls']) if isinstance(old.get('rag_source_urls'), list) else None)
+            check = citation_check(answer, urls) if urls is not None else {'citations_ok': None, 'unsupported_urls': []}
+            strict = bool(old.get('strict', old.get('strict_pass', False)))
+            rows.append({'id': old['id'], 'previous_strict': strict, **check,
+                         'citation_adjusted_strict': (strict and check['citations_ok']) if urls is not None else None})
+        reports.append({'file': str(path), 'sha256': sha256_file(path), 'rows': rows,
+            'summary': {'questions': len(rows), 'checked': sum(r['citations_ok'] is not None for r in rows),
+                        'invalid_citations': sum(r['citations_ok'] is False for r in rows),
+                        'previous_strict': sum(r['previous_strict'] for r in rows),
+                        'strict_after_citation_check': sum(r['citation_adjusted_strict'] is True for r in rows)}})
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps({'method': __doc__, 'files': reports}, ensure_ascii=False, indent=2)+'\n')
+    print(json.dumps([{'file': r['file'], **r['summary']} for r in reports], ensure_ascii=False))
+
+
+if __name__ == '__main__': main()

tools/graphrag/evaluation/README.md

@@ -1,6 +1,25 @@
 # GraphRAG evaluácia
 
-Aktuálna verzia je **v5.1**, tag `graphrag-v5.1-frozen`.
+Aktuálna verzia je **v6**, tag `graphrag-v6-frozen`.
+[Finálna správa v6](results/final_v6_2026-10-25/REPORT.md) oddeľuje grafový
+DEV 722/722, regresiu 320/320 a generované odpovede modelu.
+Generovanie má strict **316/320 (98,75 %)**; všetkých **565 testov** prešlo.
+Benchmark v6 ponecháva všetkých 1 042 otázok, ich ID a rozdelenie. Opravuje
+15 očakávaní po priamom čítaní zdrojov; generický odkaz „rešerš“ nie je názov
+akademického projektu. Všetky staršie dáta a výsledky ostávajú zachované.
+Modelový DEV má 42 známych otázok; finálna regresia obsahuje všetkých 320.
+
+Použite `--questions tools/graphrag/evaluation/graphrag_questions_v6.json`,
+`--audit tools/graphrag/evaluation/results/v6_preparation/source_audit.json`
+a `--freeze-manifest tools/graphrag/evaluation/frozen_v6.json`.
+[Postup, kontroly a zdrojová vzorka](../../../docs/graphrag_v6_2026-10-25_worklog.md).
+Kontrola vzorky je AI kontrola; ľudské posúdenie je pripravené v CSV, neprebehlo.
+Ročník adresára sa pri chýbajúcom explicitnom začiatku štúdia používa ako
+náhradná konvencia, nie ako nezávislý administratívny doklad.
+
+## Historická v5.1
+
+Historická verzia je **v5.1**, tag `graphrag-v5.1-frozen`.
 [Finálna správa](results/final_v5_1_2026-10-25/REPORT.md) a
 [strojový súhrn](results/final_v5_1_2026-10-25/summary.json) oddeľujú
 vyhľadávanie, zdrojovú kontrolu všetkých vrátených typovaných faktov a generovanie.

tools/graphrag/evaluation/benchmark_v6.py

@@ -0,0 +1,52 @@
+"""Version known regression questions after independent source adjudication.
+
+No retrieval output is used to choose, remove or relabel questions. Historical
+v4 remains immutable; all IDs, wording, categories and split membership remain.
+"""
+import copy
+import json
+from pathlib import Path
+
+from .benchmark_v2 import BASE, audit_dataset, binding_source_evidence, validate_case, write_json
+from .benchmark_v4 import refresh_titles, logical_group
+from .source_oracle import load_source_oracle
+
+
+def main():
+    output = BASE / 'graphrag_questions_v6.json'
+    artifacts = BASE / 'results/v6_preparation'
+    if output.exists():
+        raise RuntimeError('Refusing to overwrite versioned questions.')
+    original = json.loads((BASE / 'graphrag_questions_v4.json').read_text())
+    oracle = load_source_oracle()
+    cases = []; changes = []
+    for before in original:
+        after = refresh_titles(before, oracle)
+        after['graph']['partition_group'] = logical_group(after)
+        if after['should_answer']:
+            after['source_evidence'] = [proof for binding in after['graph']['expected_bindings']
+                                       for proof in binding_source_evidence(binding, oracle)]
+        if (before['should_answer'], before['graph']['expected_bindings']) != (
+                after['should_answer'], after['graph']['expected_bindings']):
+            changes.append({'id': before['id'], 'split': before['split'],
+                'before': copy.deepcopy(before), 'after': copy.deepcopy(after),
+                'reason': 'Generic written-deliverable link label is not a supported academic title.'})
+        assert before['question'] == after['question'] and before['category'] == after['category']
+        issues = validate_case(after, oracle)
+        if issues:
+            raise RuntimeError(f'{after["id"]}: {issues}')
+        cases.append(after)
+    write_json(output, cases)
+    write_json(artifacts / 'source_adjudications.json', {
+        'reviewer': 'AI direct Markdown review; not a human adjudication',
+        'policy': 'A bare artifact label (rešerš, tutoriál, projekt, dokument, repozitár) is not an academic title. '
+                  'Do not inherit a child-document title or category-year into its parent academic record.',
+        'measurement': 'All questions are known historical regression; corrected gold prevents direct accuracy comparison.',
+        'questions': changes})
+    write_json(artifacts / 'source_audit.json', audit_dataset(cases, oracle, output))
+    print(json.dumps({'questions': len(cases), 'source_issues': 0,
+                      'adjudicated_ids': [c['id'] for c in changes]}))
+
+
+if __name__ == '__main__':
+    main()

tools/graphrag/evaluation/check_openwebui.py

@@ -20,6 +20,7 @@ from .benchmark_v2 import BASE, academics, supporting_documents
 from .reproducibility import graph_fingerprint, run_metadata, verify_freeze
 from .source_oracle import load_source_oracle, norm
 from .source_validation import claim_issues
+from tools.answer_evaluation.context_policy import citation_check
 
 SYSTEM = '''Odpovedaj po slovensky výhradne podľa dodaného výsledku grafového vyhľadávania.
 Nedoplňuj vedomosti ani údaje z otázky bez potvrdenia v kontexte. Pri no_answer oznám,
@@ -65,6 +66,8 @@ def grade_claims(case: dict, answer: dict, oracle: dict) -> dict:
     urls = answer.get('source_urls', [])
     sources_ok = isinstance(urls, list) and all(any(a['source_url'] in urls for a in proof['any_of'])
                        for proof in graph.get('expected_evidence_documents', []))
+    sources_ok = sources_ok and citation_check(answer,{
+        doc['source_url'] for doc in oracle.get('documents',{}).values() if doc.get('source_url')})['citations_ok']
     negative = graph.get('expected_no_answer', False)
     status_ok = (answer.get('status') == 'no_answer' and not claims) if negative else answer.get('status') == 'ok'
     count_ok = 'expected_count' not in graph or answer.get('count') == graph['expected_count']
@@ -79,6 +82,11 @@ def grade_claims(case: dict, answer: dict, oracle: dict) -> dict:
 
 
 def main():
+    # Import the common adapter here: it imports grade_claims from this module.
+    from tools.answer_evaluation.evaluate_graph_answers import graph_context, grade_answer, HERE
+    from tools.answer_evaluation.checked_generation import generate_checked_answer
+    from tools.graphrag.build_graphrag import graph_version
+
     parser = argparse.ArgumentParser(description=__doc__)
     parser.add_argument('--questions', type=Path, default=BASE / 'graphrag_questions_v4.json')
     parser.add_argument('--freeze-manifest', type=Path, required=True)
@@ -97,8 +105,10 @@ def main():
             selected.append(case); counts[case['category']] += 1
     oracle = load_source_oracle(); manifest = json.loads(args.freeze_manifest.read_text())
     settings = get_neo4j_settings(); rows = []
-    payload = {'method': 'OpenWebUI API generation with locally supplied GraphRAG context; no autonomous tool routing.',
+    prompt = (HERE / 'graph_answer_prompt.txt').read_text()
-               'model': args.model, 'selected_ids': [q['id'] for q in selected], 'system_prompt': SYSTEM,
+    payload = {'method': 'Supplementary known-query check with common context guards; no autonomous tool routing.',
+               'model': args.model, 'selected_ids': [q['id'] for q in selected], 'system_prompt': prompt,
+               'corrective_request_limit': 1,
                'limitations': ['Small stratified supplementary sample; not the primary 320-case accuracy.',
                    'Structured claims, source citations and binding checks; free prose requires separate review.'],
                'questions': rows}
@@ -107,24 +117,24 @@ def main():
         metadata = run_metadata(args.questions)
         verify_freeze(manifest, metadata, graph_fingerprint(session), audit=args.audit, **manifest['parameters'])
         payload['run_metadata'] = metadata
+        version = graph_version(session)
         for case in selected:
             started = time.perf_counter()
             result = search_graph_with_session(session, case['question'], **manifest['parameters'])
-            context = {k: result[k] for k in ['status', 'reason', 'facts', 'documents', 'aggregation']}
+            result['index_version'] = version
+            context = graph_context(result)
             row = {'id': case['id'], 'category': case['category'], 'question': case['question'],
                    'graph_context': context, 'strict': False}
             try:
-                response = request_json(OPENWEBUI_URL, method='POST', timeout=90, max_attempts=1,
+                def complete(messages):
-                    headers={'Authorization': f'Bearer {os.environ["OPENWEBUI_API_KEY"]}', 'Content-Type': 'application/json'},
+                    return request_json(OPENWEBUI_URL, method='POST', timeout=90, max_attempts=1,
-                    payload={'model': args.model, 'stream': False, 'response_format': {'type': 'json_object'},
+                        headers={'Authorization': f'Bearer {os.environ["OPENWEBUI_API_KEY"]}', 'Content-Type': 'application/json'},
-                        'messages': [{'role': 'system', 'content': SYSTEM},
+                        payload={'model': args.model, 'stream': False, 'response_format': {'type': 'json_object'},
-                            {'role': 'user', 'content': json.dumps({'question': case['question'], 'graph_context': context}, ensure_ascii=False)}]})
+                                 'messages': messages})
-                row['response'] = response
+                row.update(generate_checked_answer(case['question'], context, prompt, complete))
-                text = response['choices'][0]['message']['content'].strip()
+                if row.get('answer'):
-                if text.startswith('```'):
+                    row.update(grade_answer(case, row['answer'], oracle, context=context))
-                    text = text.split('\n', 1)[1].rsplit('```', 1)[0]
+                row['strict'] = bool(row['strict'] and row['accepted_for_delivery'])
-                answer = json.loads(text)
-                row['answer'] = answer; row.update(grade_claims(case, answer, oracle))
             except Exception as error:
                 row['error'] = f'{type(error).__name__}: {str(error)[:500]}'
             row['latency_seconds'] = time.perf_counter() - started
@@ -134,6 +144,7 @@ def main():
             args.output.write_text(json.dumps(payload, ensure_ascii=False, indent=2) + '\n')
             print(f'{case["id"]}: {"OK" if row["strict"] else "FAIL"} ({row["latency_seconds"]:.1f}s)', flush=True)
         verify_freeze(manifest, run_metadata(args.questions), graph_fingerprint(session), audit=args.audit, **manifest['parameters'])
+        assert graph_version(session) == version, 'Index version changed during generation.'
     print(json.dumps(payload['summary']))
 
 

tools/graphrag/evaluation/frozen_release_v6.json

@@ -0,0 +1,134 @@
+{
+  "release": "graphrag-v6-frozen",
+  "evaluation_kind": "historical_regression",
+  "graph_freeze_sha256": "54946b476fd713f551414b00cf2b843627566f6a333e3b59ce3efbbdf2328629",
+  "runtime_files": {
+    "app/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+    "app/main.py": "ff1adeff8b770047139fe3f632ad837517c95ec04649849c4315d6f892f9eef3",
+    "app/routes.py": "dfb82171fea33d33b6e48cb238d2c19a29254b47a9a87308f4864cdbac53fa7b",
+    "app/security.py": "24f437aff6642477049f02fdccfb3429c15e610d044bbb66128903771510055f",
+    "requirements.txt": "26a6d097727eba03c1b6c71cd02b06373ef01ce03e567dd2bc4357ae330e5a1a",
+    "tools/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+    "tools/answer_evaluation/__init__.py": "795be75bf60e4aa9209e5b820207e52e8420c41a4dc2e37d0b6018cd2934a51a",
+    "tools/answer_evaluation/answer_format.py": "86a27b4aebc6a5077e13794b81957a1cf063a1f3e070c583120ba53c1788d2f8",
+    "tools/answer_evaluation/check_context_scenarios.py": "cbb2edf8fdcba61b9e5cc50e28cb1a1dcc25b0e0ab990177b41f40031223ca63",
+    "tools/answer_evaluation/check_deployment.py": "803c5a42ab61ff6529b830a4e1ab2f1401b59565b56086db65d449f0f9c1dfb5",
+    "tools/answer_evaluation/check_rag_runtime.py": "1e1b8bd48ac4d668ec057fee162ef30d34ae20ae026361bc8cc296bc455d4067",
+    "tools/answer_evaluation/checked_generation.py": "7e017471e6281db62b79f270210db3ab14c7e31a4623bbfa3d9a5fd8911b1c20",
+    "tools/answer_evaluation/context_policy.py": "e7b7c511e5e9cafdef04969eb4007a6a61ac4b99bcc60e9e722945a15bec1f26",
+    "tools/answer_evaluation/evaluate_graph_answers.py": "2553a0d17e164922b02046416fdf0d5df4496dc443466d8d9a1f4be86415b318",
+    "tools/answer_evaluation/graph_answer_prompt.txt": "8c8274a22bf5b44364d8fc8226a8f659b31c232f5a2aba5916d693b559e9f341",
+    "tools/answer_evaluation/rag_delivery.py": "b83f93d446f44fdd7d41c12eeeaae69c0b02908076624f4eb3f4f72b9b88dbc8",
+    "tools/answer_evaluation/regrade_citations.py": "330f437aec12927fd8c99b0aa601373f6ab26cb111971a6fb06098931d1e4aab",
+    "tools/graphrag/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+    "tools/graphrag/build_graphrag.py": "256292101d6d7d1946e926e2717ac45465b169f4091c51ecfed4f1a779dfd700",
+    "tools/graphrag/evaluation/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+    "tools/graphrag/evaluation/audit_benchmark.py": "6619c818a52fc284e87ed6c48a503a54a3c9dd95447463ed78f075b7c58bad80",
+    "tools/graphrag/evaluation/benchmark_v2.py": "129dd774faa414f06733e1ee6333e7b411b1c75b5601b895c405778df478a563",
+    "tools/graphrag/evaluation/benchmark_v3.py": "30bae6c605ed26f4b49973acec799087c4e2b97a883ccedcafb965dd909c4c4d",
+    "tools/graphrag/evaluation/benchmark_v4.py": "70a8b08f7c2b2cb3e2bdc77be9efedd23ea24ac564ec1c1eb224a9f6388b3a67",
+    "tools/graphrag/evaluation/benchmark_v6.py": "f955e600d17358fafcb33bab772aef09d787452f6b33352e299fdd42669aaf34",
+    "tools/graphrag/evaluation/check_openwebui.py": "e2a81b52f7431b27a5e2a7d2bd0801029a782ea3620c0b7a289c57bdebb994d7",
+    "tools/graphrag/evaluation/evaluate_graphrag.py": "89c364d15b70c9cb55df39d5970b4d0d79b681ba70eed7530a4e3fe64167812e",
+    "tools/graphrag/evaluation/evidence_contract.py": "78e24264c176e163e1782fef1634b6c45c7f31bd4692726d02a30ab45a06648e",
+    "tools/graphrag/evaluation/freeze_graphrag.py": "793c6d04a13c3217347283c609f647cd0d69b020e44bbde458bdc7ebbab8ec7e",
+    "tools/graphrag/evaluation/graphrag_evaluator.py": "7f99bf0f97974058650b619ea49a8ee32284cb4b324f1ffab9baf54f5c4495f4",
+    "tools/graphrag/evaluation/reproducibility.py": "8db9a0e0f86236b56239f41da11803538dfaa6745bbf74dbd7618e0a73bda0ae",
+    "tools/graphrag/evaluation/source_oracle.py": "f2265d0919a4019f10d2b4a986bd772684fe59c598d7ae2f5cd0daa41f81b87b",
+    "tools/graphrag/evaluation/source_validation.py": "9671b8978623888dbed8b8b2d1b6b01c1ce73cb0a512b03919e48873990f15ed",
+    "tools/graphrag/graph_db.py": "c4b1a2edb765856b807398e4c243a8d17753529a5d2dfdb51791e28bc8de17f7",
+    "tools/graphrag/graph_search.py": "8cc3668b4ad7beabf849e6978ea1d5defc818b142ee8b157b19c319998fb7541",
+    "tools/graphrag/graphrag.py": "a718efbd001301c4a809aa449a2f175ae7aca8f6f3248f539d6f0597faeb3187",
+    "tools/graphrag/neo4j_smoke.py": "fb5244c72c29ef2acf96d1e7f54335ff168eed4119db1692025eb1b99194b7bb",
+    "tools/graphrag/publication.py": "fba1fa0d0bed325fd87f8d3593baae9271de1766b772d7214c7f63fd3f1c37b6",
+    "tools/graphrag/tests/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+    "tools/graphrag/tests/test_evaluation_pipeline_v2.py": "63d1b3e80ae3c7b563374fd7ab4aa5ffad3c614eafbd15142c577abd7f756bf9",
+    "tools/graphrag/tests/test_evaluation_v5.py": "9387e9d1418ebab87530986b0e29dac31cf33d762777a29a31cff24cc72e552a",
+    "tools/graphrag/tests/test_evaluation_wide.py": "1b28bb74fd6b9457c869186b88606f1f7863d573aa3ab5226162e878f236d99e",
+    "tools/graphrag/tests/test_evaluator_connected_v2.py": "29d7c9838ed1ff37ea3276df6feb1b6142158c43a0525712f490b648181882de",
+    "tools/graphrag/tests/test_graph_publication.py": "96e53c1ae28e36c7a4290e1e60c8b398bf095ceaf883e819e1a477e2672b11c0",
+    "tools/graphrag/tests/test_graph_rebuild_atomic.py": "fe37d4fff0b1f0fb1d565f0a96152b9b4e336837bc78b5977cf3b4879d7fbaf3",
+    "tools/graphrag/tests/test_graph_search.py": "eb58661aef9dafe6bf7188fcdc2d79cef6652d57c64d1b81ae81a52e8e0204d7",
+    "tools/graphrag/tests/test_graph_search_entity_v2.py": "0ef1a8ebd513c5a4507eb38fcf5ae6137c69e9901b5a2975ba513be5902ac693",
+    "tools/graphrag/tests/test_graph_search_neo4j.py": "fec2bf72de8f752f17c9164fee5754bdb2e153828855e47efbf0000dbe82cf3b",
+    "tools/graphrag/tests/test_graph_search_regressions.py": "9db7779aeb8ed9e8c0f92c1657011a81080aff3c928649e811358e3009a18c48",
+    "tools/graphrag/tests/test_graphrag.py": "9ee8629d6b528dd78c2ddfc25cee95e17613f45b454fc0a38d4c591862637a9f",
+    "tools/graphrag/tests/test_graphrag_evaluator.py": "1c6d08aeec9ae56ac4123239a5aed12a0081d66cc3f34b2047d95764579d7298",
+    "tools/graphrag/tests/test_graphrag_freeze.py": "c4b760baf4c7ebf13b16bd4cd3c51874f5512eef3b4c61470dc4d428b0303d9f",
+    "tools/graphrag/tests/test_graphrag_research_v2.py": "711008a91ab8a58ae0854e7f90329bcd42e5189fde2e2001589e97b0c22c5198",
+    "tools/graphrag/tests/test_graphrag_root_title.py": "085651fe6e6081521a5dde9554abfee93456250108cf15978df25378d886a46c",
+    "tools/graphrag/tests/test_source_oracle_v2.py": "0afdbe235e185ffe9f38aa667f90f5a6c20504fe29e9ac572642b39a8eb7faca",
+    "tools/graphrag/tests/test_title_labels.py": "2983ead68ad6e2e346d24af0fc96dc6771c163e9c2a351af426a1e0383998e6f",
+    "tools/index_state.py": "c33920a63d04009f11ef23ca9e9eee9bf86453b7d0586d027bf3b39156afea0d",
+    "tools/publication.py": "f99d445073390a733afc8d849b9fcd4b4b0890bb54d136db67d4015b78436e1a",
+    "tools/rag/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+    "tools/rag/build_chunks.py": "fe060e25b138abcb689a6eb1b1e993d7017a4b3c5dd63e7cd275e1279da6053b",
+    "tools/rag/build_sqlite_index.py": "107c0987e9ec4086ff41f9e7ae30195a67be34616875635e705dc79236e79416",
+    "tools/rag/common.py": "8497c46d5c6c27a7c76df1568360ad5e9d00d5e5a7ef186e758960cc79d81d1c",
+    "tools/rag/embedding_utils.py": "a2ea0b5be1a1782efff1c85c0eabfedba043e0e8bf7eb8b6617eab60eb4559c3",
+    "tools/rag/evaluation/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+    "tools/rag/evaluation/evaluate_rag_answers.py": "945c2da655537fb37fe82a08e10dc3ed66a2d90980921f61f61efb171a4a26a2",
+    "tools/rag/evaluation/evaluate_retrieval.py": "aba865751ab74d9c2b962548b1e476508dbe6cf7bf811bda79e25f853061cbd7",
+    "tools/rag/evaluation/metrics.py": "40b7d16f7729599c1d2fa2370f7cb40dfa04a968cdadef3c4fae83a168202c28",
+    "tools/rag/evaluation/rag_answer_data.py": "9a369e9ced254cf723a5d3006877959c6f144f4c770e382835af776f7f569762",
+    "tools/rag/evaluation/rag_answer_state.py": "da4ab19f22378899d5c13e93e803a978518f8edcdd62e4990a06aae12d8eb469",
+    "tools/rag/evaluation/rag_metrics.py": "3e95922df90dd1b757a550040a2d9bd4f857e17d179f60170d7cc881a08f94b6",
+    "tools/rag/evaluation/rag_runner.py": "031abf7d5b1a026eb199aa2a579bfb8baaa01175279d0018a364bfe50b76c3c1",
+    "tools/rag/evaluation/retrieval_runner.py": "fe3dd05459f825d72252b7a0841f28004fc81537096a7de487807e4d5f0c4c45",
+    "tools/rag/rag_document_expansion.py": "d565b9794e30bc0f387c791a0557d5d03f2c1fa2fe5db53e49a9b19b1158f48e",
+    "tools/rag/rag_query_evidence.py": "90fced0fecd8ef976a1537506d2e7e8e9d3ba22a6371131f978fb91df5cc0b19",
+    "tools/rag/rag_supplemental_expansion.py": "a593579e543149633e3146c3886536a9196aff43f4df6f3d7ade94ca16dd243a",
+    "tools/rag/rag_utils.py": "4f898cf0facc2ae9f2ea28a36f48d4ed37bd7fc716e097b319bcf4d36a4a4f03",
+    "tools/rag/rebuild_index.py": "6a4d86a551d1141c403a0f6570c568896af37faeaa8c67cac170d2e6a393ca06",
+    "tools/rag/scan_zpwiki.py": "ef770ce4edb6352ff0690af5783766c95062a0e56f638cca6e3d929973728273",
+    "tools/rag/search_core.py": "624c8ddfe3edf78d06014579b747a59b2e33c0191a7cf90cdfd624b52ed5e96b",
+    "tools/rag/search_db.py": "cc63aeaf2854e87b7448c3978959569b097074cdb16a9b2c16b2c37a4677fdbb",
+    "tools/rag/search_utils.py": "69597bf5785f7852b1131a0648d9d2c65b6babba9ad279581f08b40d185ee646",
+    "tools/rag/tests/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+    "tools/rag/tests/conftest.py": "0c94402d5ca971233d9802ded81cdb39ae7dfddda5eb9126e2035f7599a8f1c7",
+    "tools/rag/tests/test_chunking.py": "94904a20cfed0512d508017eaadbb43ef45ca74a5e96631241f6ca481222c498",
+    "tools/rag/tests/test_common.py": "86f0aecb79b7a3764112c833ea3b732073c073a5d570ef8c99900fb2a3f04430",
+    "tools/rag/tests/test_database.py": "31fc3553d76127c2d7e2c113b9053e537a72bb92f57a9c24cad6a579a1ffc589",
+    "tools/rag/tests/test_e5_1_remaining.py": "a8a52d5204e7376f33bb1aef96aad20990126a0b92266596b7cf8dd0dcbea4bc",
+    "tools/rag/tests/test_evaluate_rag_answers.py": "2143f3c9efbacf7b9119eeb9ae69e6cd7ea3f428ff5076d8e9973d0943675296",
+    "tools/rag/tests/test_live_data.py": "3c5e04d4a29ec8360d89881be980dacacccf0e12594fe51a22ec23c32c98b3e3",
+    "tools/rag/tests/test_rag.py": "13e97c88bf4afe45a8d633b269d110890f0f646887cee861bbf55ada0be7ad24",
+    "tools/rag/tests/test_rag_document_expansion.py": "9adf39c7a9b683b1d40c36fce576272e71869b7c1236c371c1b077820874eb7e",
+    "tools/rag/tests/test_rag_metrics_morphology.py": "070ee314d326e4c7218e7c19959a94076a2b241c4f19b9ecd9ca1fb3cc2639d4",
+    "tools/rag/tests/test_rag_metrics_semantic.py": "3642394d2636edd7420589348fb75bf1d214b539983b7e620008d626deffabf0",
+    "tools/rag/tests/test_rag_query_evidence.py": "c3fb5115306171010759b0cfd0cd05d1439d6a57de049efea02330ae7c553bfc",
+    "tools/rag/tests/test_rag_query_evidence_topn.py": "ebf01ad57a52f030b908b7fb0312d8d32cf9f60ea2953c5a3fb8849c1f2dceaf",
+    "tools/rag/tests/test_rag_runner.py": "1813086999d1e9826b42c2ac7978ea77d29e1c4b12f5e606f1fba572829e24ea",
+    "tools/rag/tests/test_rag_runner_routing.py": "c9454e5dddfdb7afc013567811efec15545daf20733b7167f78f2dd9ff0380d6",
+    "tools/rag/tests/test_rag_runner_tool_prompt.py": "073403907717d8d388c28ea92d1d1f9426edab2e8bd320c8f6329bcbff082e6c",
+    "tools/rag/tests/test_rag_supplemental_expansion.py": "65449054e2d94396def0dc8d505a9573306adbf184820ddd6c9b5ea92e165f8b",
+    "tools/rag/tests/test_rebuild_index.py": "aa0688b81e4b699ce99e97a4b8c1a6d4a49c5c153caa02301ebd5a5d8be764a4",
+    "tools/rag/tests/test_scan_zpwiki.py": "324510ca504dff11ecf39b28aff26b44a358e14084b3b015c0c410690d8aa129",
+    "tools/rag/tests/test_search.py": "82c96d35719d2233e4636b6f25d89b22f004848d8bae46a9705ee384d4abe9a8",
+    "tools/retrieval_contract.py": "b8fa272c3babf924a3ce8b937fe7ae437cd959ba9f7667193ac67c18d2c5b58d",
+    "tools/sync_indexes.py": "5b909b4394c24e5ce22848594b16d9aea45fb06418f5cd92bb6aa6381a0ccf10",
+    "tools/tool_health.py": "8ad37345889ad58a737edac5960b5b6c5002af2397d60248f434e03fb4eddd89"
+  },
+  "deployment_configuration": {
+    "Dockerfile": "fc7aefe4a99222daedc321acf8359773dc24eff82be04e688507e135956a92a6",
+    "docker-compose.yml": "e33b65c58f155ac057b01be272bc323fab3ff7eb4f51c9e4ca1b53904cc17af4"
+  },
+  "measured_artifacts": {
+    "tools/graphrag/evaluation/results/final_v6_2026-10-25/graphrag_dev.json": "d4ac47c21aed4be149a3f1aec1f7658c862da2ee24e41cc680128d013a071707",
+    "tools/graphrag/evaluation/results/final_v6_2026-10-25/graphrag_test.json": "2d569267ada3eecb81cd793cc2c7e8f7715b3c9b931799d4ef0663ac3f11f1d5",
+    "tools/graphrag/evaluation/graphrag_questions_v6.json": "a7bb38be01e03603cf2fa2c7137817094dff0a0cbb3f801263609e55b28e9edc",
+    "tools/graphrag/evaluation/results/v6_preparation/source_audit.json": "d5922db25b6d40cb7e5dc7df7990e93a3caee263a6e747ae910ab340da340023",
+    "tools/graphrag/evaluation/results/v6_preparation/protocol.json": "3f33834c78d35586d8bd8eb2cd48d7db184df5a1d683806d40f0d743aec6c21e",
+    "tools/graphrag/evaluation/results/v6_preparation/source_review.csv": "ecf420330a6bb9a4a11fafa99ab617886d069d5c7e34f8045467794145f33cf9",
+    "tools/graphrag/evaluation/results/v6_preparation/rag_delivery_replay.json": "3ecb0ba20b515f968df5cd070e2b470d58d69f3fb7bfb2546b1bd01e8e308a83",
+    "tools/graphrag/evaluation/results/v6_preparation/citation_regrade.json": "238015f0f36010cb7f53570188a7e65e4a40e1ad778c9bab6bd37d2b4a697bc6",
+    "tools/graphrag/evaluation/results/answers_v6_final_dev_2026-10-25/answers_dev.json": "f70834bf4ee1fe106c0849ab56e0758dd50e07dfa9bbc25b05ae4d4e811a4edd",
+    "tools/graphrag/evaluation/results/answers_v6_regression_2026-10-25/answers_regression.json": "17a675405a79e3309602d0ac6f5b4ef447edde88a2c4c950aeb675d2de9aa4a4"
+  },
+  "policy": "Graph-generation files remained frozen during all 320 cases. RAG-only delivery notice verified separately and included in this full-runtime release freeze.",
+  "secrets_excluded": [
+    ".env",
+    "credentials",
+    "private keys"
+  ]
+}

tools/graphrag/evaluation/frozen_v6.json

@@ -0,0 +1,241 @@
+{
+  "schema_version": 2,
+  "evaluation_kind": "historical_regression",
+  "partition": {
+    "method": "Previously exposed questions; regression, not independent TEST",
+    "dev_profile_count": 72,
+    "test_profile_count": 72,
+    "source_profile_overlap": 72
+  },
+  "frozen_at_utc": "2026-10-25T21:18:46.238549+00:00",
+  "fingerprints": {
+    "implementation": {
+      "tools/graphrag/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "tools/graphrag/build_graphrag.py": "256292101d6d7d1946e926e2717ac45465b169f4091c51ecfed4f1a779dfd700",
+      "tools/graphrag/evaluation/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "tools/graphrag/evaluation/audit_benchmark.py": "6619c818a52fc284e87ed6c48a503a54a3c9dd95447463ed78f075b7c58bad80",
+      "tools/graphrag/evaluation/benchmark_v2.py": "129dd774faa414f06733e1ee6333e7b411b1c75b5601b895c405778df478a563",
+      "tools/graphrag/evaluation/benchmark_v3.py": "30bae6c605ed26f4b49973acec799087c4e2b97a883ccedcafb965dd909c4c4d",
+      "tools/graphrag/evaluation/benchmark_v4.py": "70a8b08f7c2b2cb3e2bdc77be9efedd23ea24ac564ec1c1eb224a9f6388b3a67",
+      "tools/graphrag/evaluation/benchmark_v6.py": "f955e600d17358fafcb33bab772aef09d787452f6b33352e299fdd42669aaf34",
+      "tools/graphrag/evaluation/check_openwebui.py": "e2a81b52f7431b27a5e2a7d2bd0801029a782ea3620c0b7a289c57bdebb994d7",
+      "tools/graphrag/evaluation/evaluate_graphrag.py": "89c364d15b70c9cb55df39d5970b4d0d79b681ba70eed7530a4e3fe64167812e",
+      "tools/graphrag/evaluation/evidence_contract.py": "78e24264c176e163e1782fef1634b6c45c7f31bd4692726d02a30ab45a06648e",
+      "tools/graphrag/evaluation/freeze_graphrag.py": "793c6d04a13c3217347283c609f647cd0d69b020e44bbde458bdc7ebbab8ec7e",
+      "tools/graphrag/evaluation/graphrag_evaluator.py": "7f99bf0f97974058650b619ea49a8ee32284cb4b324f1ffab9baf54f5c4495f4",
+      "tools/graphrag/evaluation/reproducibility.py": "8db9a0e0f86236b56239f41da11803538dfaa6745bbf74dbd7618e0a73bda0ae",
+      "tools/graphrag/evaluation/source_oracle.py": "f2265d0919a4019f10d2b4a986bd772684fe59c598d7ae2f5cd0daa41f81b87b",
+      "tools/graphrag/evaluation/source_validation.py": "9671b8978623888dbed8b8b2d1b6b01c1ce73cb0a512b03919e48873990f15ed",
+      "tools/graphrag/graph_db.py": "c4b1a2edb765856b807398e4c243a8d17753529a5d2dfdb51791e28bc8de17f7",
+      "tools/graphrag/graph_search.py": "8cc3668b4ad7beabf849e6978ea1d5defc818b142ee8b157b19c319998fb7541",
+      "tools/graphrag/graphrag.py": "a718efbd001301c4a809aa449a2f175ae7aca8f6f3248f539d6f0597faeb3187",
+      "tools/graphrag/neo4j_smoke.py": "fb5244c72c29ef2acf96d1e7f54335ff168eed4119db1692025eb1b99194b7bb",
+      "tools/graphrag/publication.py": "fba1fa0d0bed325fd87f8d3593baae9271de1766b772d7214c7f63fd3f1c37b6",
+      "tools/graphrag/tests/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "tools/graphrag/tests/test_evaluation_pipeline_v2.py": "63d1b3e80ae3c7b563374fd7ab4aa5ffad3c614eafbd15142c577abd7f756bf9",
+      "tools/graphrag/tests/test_evaluation_v5.py": "9387e9d1418ebab87530986b0e29dac31cf33d762777a29a31cff24cc72e552a",
+      "tools/graphrag/tests/test_evaluation_wide.py": "1b28bb74fd6b9457c869186b88606f1f7863d573aa3ab5226162e878f236d99e",
+      "tools/graphrag/tests/test_evaluator_connected_v2.py": "29d7c9838ed1ff37ea3276df6feb1b6142158c43a0525712f490b648181882de",
+      "tools/graphrag/tests/test_graph_publication.py": "96e53c1ae28e36c7a4290e1e60c8b398bf095ceaf883e819e1a477e2672b11c0",
+      "tools/graphrag/tests/test_graph_rebuild_atomic.py": "fe37d4fff0b1f0fb1d565f0a96152b9b4e336837bc78b5977cf3b4879d7fbaf3",
+      "tools/graphrag/tests/test_graph_search.py": "eb58661aef9dafe6bf7188fcdc2d79cef6652d57c64d1b81ae81a52e8e0204d7",
+      "tools/graphrag/tests/test_graph_search_entity_v2.py": "0ef1a8ebd513c5a4507eb38fcf5ae6137c69e9901b5a2975ba513be5902ac693",
+      "tools/graphrag/tests/test_graph_search_neo4j.py": "fec2bf72de8f752f17c9164fee5754bdb2e153828855e47efbf0000dbe82cf3b",
+      "tools/graphrag/tests/test_graph_search_regressions.py": "9db7779aeb8ed9e8c0f92c1657011a81080aff3c928649e811358e3009a18c48",
+      "tools/graphrag/tests/test_graphrag.py": "9ee8629d6b528dd78c2ddfc25cee95e17613f45b454fc0a38d4c591862637a9f",
+      "tools/graphrag/tests/test_graphrag_evaluator.py": "1c6d08aeec9ae56ac4123239a5aed12a0081d66cc3f34b2047d95764579d7298",
+      "tools/graphrag/tests/test_graphrag_freeze.py": "c4b760baf4c7ebf13b16bd4cd3c51874f5512eef3b4c61470dc4d428b0303d9f",
+      "tools/graphrag/tests/test_graphrag_research_v2.py": "711008a91ab8a58ae0854e7f90329bcd42e5189fde2e2001589e97b0c22c5198",
+      "tools/graphrag/tests/test_graphrag_root_title.py": "085651fe6e6081521a5dde9554abfee93456250108cf15978df25378d886a46c",
+      "tools/graphrag/tests/test_source_oracle_v2.py": "0afdbe235e185ffe9f38aa667f90f5a6c20504fe29e9ac572642b39a8eb7faca",
+      "tools/graphrag/tests/test_title_labels.py": "2983ead68ad6e2e346d24af0fc96dc6771c163e9c2a351af426a1e0383998e6f",
+      "requirements.txt": "26a6d097727eba03c1b6c71cd02b06373ef01ce03e567dd2bc4357ae330e5a1a",
+      "tools/publication.py": "f99d445073390a733afc8d849b9fcd4b4b0890bb54d136db67d4015b78436e1a",
+      "tools/retrieval_contract.py": "b8fa272c3babf924a3ce8b937fe7ae437cd959ba9f7667193ac67c18d2c5b58d",
+      "tools/index_state.py": "c33920a63d04009f11ef23ca9e9eee9bf86453b7d0586d027bf3b39156afea0d",
+      "tools/rag/common.py": "8497c46d5c6c27a7c76df1568360ad5e9d00d5e5a7ef186e758960cc79d81d1c",
+      "tools/sync_indexes.py": "5b909b4394c24e5ce22848594b16d9aea45fb06418f5cd92bb6aa6381a0ccf10",
+      "tools/rag/rag_utils.py": "4f898cf0facc2ae9f2ea28a36f48d4ed37bd7fc716e097b319bcf4d36a4a4f03",
+      "tools/answer_evaluation/context_policy.py": "e7b7c511e5e9cafdef04969eb4007a6a61ac4b99bcc60e9e722945a15bec1f26",
+      "tools/answer_evaluation/checked_generation.py": "7e017471e6281db62b79f270210db3ab14c7e31a4623bbfa3d9a5fd8911b1c20",
+      "tools/answer_evaluation/answer_format.py": "86a27b4aebc6a5077e13794b81957a1cf063a1f3e070c583120ba53c1788d2f8",
+      "tools/rag/evaluation/rag_metrics.py": "3e95922df90dd1b757a550040a2d9bd4f857e17d179f60170d7cc881a08f94b6",
+      "tools/rag/evaluation/rag_runner.py": "031abf7d5b1a026eb199aa2a579bfb8baaa01175279d0018a364bfe50b76c3c1"
+    },
+    "dataset_sha256": "a7bb38be01e03603cf2fa2c7137817094dff0a0cbb3f801263609e55b28e9edc",
+    "inputs": {
+      "data/documents.json": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+      "data/chunks.json": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9"
+    },
+    "source_corpus": {
+      "sha256": "440da12a7a991824e923f70cada7c616c3fb3b0f12767ec7d75039a6a3231b10",
+      "document_count": 114,
+      "documents": {
+        "pages/interns/bogdan_paul_chis/README.md": "3b4b946eb3c10e9f9a4092e6cffa01c5080e7579aded2f6ed3120c41ed62b647",
+        "pages/interns/cesar_gutierrez/README.md": "b0be60576eed28d4b1920e9acf198684519373dfc13171567247c853b3ed699e",
+        "pages/interns/oliver_pejic/README.md": "0470f3c8465be8f02204b08e4c75ef47f68c7e4ad1038faa5641cdf1811f33d5",
+        "pages/interns/sevval_bulburu/README.md": "ff2a9d66253802f1af0fdfd74832e40fae41b6df23c5e61dc266f6cd0cc16c4d",
+        "pages/interns/yussef_ressaissi/README.md": "f997a65699da0db3222ef7a3c37652d05dfc3419ad4ec74683bafc2c897bb987",
+        "pages/students/2016/darius_lindvai/README.md": "fcaa47bb9708a6439eec5c4b7f7b4ed6c7359e423e1d940e1aed9f44957e6d62",
+        "pages/students/2016/darius_lindvai/dp2021/README.md": "be3d0fea2a6d592b225eecabd9b19445c2fce62b8289c07ab6af93835db12970",
+        "pages/students/2016/darius_lindvai/timovy_projekt/resers/README.md": "352162de1b1e192228fa5d26506c028ac47e567e90df53b1bbaa6ad736ec4766",
+        "pages/students/2016/darius_lindvai/timovy_projekt/tutorial/README.md": "b4a81ff457624b089d618e4479afb1e3d748f6724f72104ef5518e535ae02fe5",
+        "pages/students/2016/dominik_nagy/README.md": "1c4037a158d84c45fa705a0c38e47142a9d46e226f0c138bc2d56d7c38f6d74d",
+        "pages/students/2016/dominik_nagy/timovy_projekt/README.md": "b81a4dab7316f7b846dcf36515f6aa9d44035c86f46f0e07b7a43308bcd31458",
+        "pages/students/2016/ediz_morochovic/README.md": "045e5eb9a3df2b84140808fbca0c3163266ad0efa144bc42d9d5c597881c6ac6",
+        "pages/students/2016/jakub_maruniak/README.md": "6734c41a95b9d0c1e3d226b393ad109ee4a7b82ebfffa73df2d7e0a743fefa14",
+        "pages/students/2016/jakub_maruniak/dp2021/README.md": "9cd884aec70705cdb2aaa0f514b513f16de263c37b749ced7f92916cd2f91c39",
+        "pages/students/2016/jakub_maruniak/dp2021/annotation/README.md": "debae0c50aea1accef0840912b9b1f37b2138f856002bb50e91d7ea2dbbe7963",
+        "pages/students/2016/jakub_maruniak/dp2021/train_experiments/README.md": "01dff38b37043414fa1db204394ddbe7aade00e14d02d495572b65c85d09b194",
+        "pages/students/2016/jakub_maruniak/timovy_projekt/README.md": "04114b1f3d84664aad5bdba5a0b614c5a6f23392be3c31fd9123d63db1be0a35",
+        "pages/students/2016/jan_holp/README.md": "80d6a8492766e5bfa61673a372e5f059d30c8e335de39842866c51485960e4d3",
+        "pages/students/2016/jan_holp/dp2021/README.md": "a62acb74009376766c7e2f9856ee01d21f84593eb9b3ce85cf2e6032aa412476",
+        "pages/students/2016/jan_holp/dp2021/zdrojove_subory/README.md": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+        "pages/students/2016/jan_holp/timovy_projekt/README.md": "79eebdfd3af7e8d5c279077e285a580731b1cf8cc9b4454f6596796e9915f70c",
+        "pages/students/2016/lukas_pokryvka/README.md": "f279456c5c277ff1ee0dd89a3464ed9f9f67b44bc185a5da9811ef5cfb675002",
+        "pages/students/2016/lukas_pokryvka/dp2021/README.md": "39b61376ce108e4a930f926ab2d6a997207457121bccfcf617129a3b210a15cf",
+        "pages/students/2016/lukas_pokryvka/timovy_projekt/README.md": "4de4b6c8cfed17b6c60090f82df1d55a66be70a79d3013850689b2baebdd801b",
+        "pages/students/2016/mark_feher/README.md": "bec37c0319fe0869a0377d771db23d090dbda93c27328f5697cdaeb3bbccfe9b",
+        "pages/students/2016/maros_harahus/README.md": "0da150e91918632c9e8cd1443f04da403595a372517e254928ce9f76c9d5033d",
+        "pages/students/2016/maros_harahus/timovy_projekt/README.md": "75af19f7c68f4b646fa1dbae75f3f3dfc52633a9f9da1400d21808bb20be0d83",
+        "pages/students/2016/patrik_pavlisin/README.md": "dc1c648d7cd9b0f0e021c57f9b98189e37c100871439e7e304390715913f2763",
+        "pages/students/2016/patrik_pavlisin/dp21/README.md": "ac1eb0fd0360ad34aa21b7c8da5bb010a6edf10be5721f28dedc3986cd1d8c54",
+        "pages/students/2016/patrik_pavlisin/dp22/README.md": "f2a8c619a812b99abab618ae388eb672407950e1f33fb8d7da7a528c4de81d50",
+        "pages/students/2016/patrik_pavlisin/tp20/README.md": "ae9c771f27e7e093bc29c6888c8d0b27ed61fb41519fd0d7e7f4f6cbb4626edc",
+        "pages/students/2016/tomas_kucharik/README.md": "6a1a215433eb1857660177b4cb2ecec03c6a86743d3a04cd6d5c9d9682c9e63b",
+        "pages/students/2017/dmytro_ushatenko/README.md": "65f4ac177d492162b08ca5da3d9eb048bd088a3c117a28a662113b7e4bc3a4ec",
+        "pages/students/2017/martin_jancura/README.md": "6a0a18d5095dcfb12abfdd72c9cfd8025d714fb5e29a7858cd2b285e4cb986b9",
+        "pages/students/2017/stanislav_matsunych/README.md": "90ffa11eff4a4360b0252dbc53a116c3a5777b3d5dd86ce5c89c02bdd2051c73",
+        "pages/students/2017/than_trung_thanh/README.md": "2150519b8580e7337520db4de64d2d1c82d2634380537d019f41c506dbf2979d",
+        "pages/students/2018/NaN1name_buday/README.md": "23c8c58090bdc954347e36f141bea91f43c8d5b1d777e346864a2ebe84c0bcce",
+        "pages/students/2018/adrian_remias/README.md": "e5bfe1d5d4e626beaa636a02d34ec58cf2cff9b0e57833905af35b80bf1465a2",
+        "pages/students/2018/david_ilas/README.md": "96c45ff9c5a821ddcb1a2fc86d487bd8f4729d1b76887914b0f5fbf99af45af0",
+        "pages/students/2018/david_omasta/README.md": "4164051870e1e1cf8949ed18a74196e9ea789dfbe4fee24bdeb736ca42848327",
+        "pages/students/2018/martin_wencel/README.md": "0e6a7a981689f22304f8baa5de97d4315c0695c94c9277de7851943eb012253b",
+        "pages/students/2018/oleh_bilykh/README.md": "4072c62dd8ee753a0c8fa43fdfbbdf566e394c2df13e6a79d08a302e332710b1",
+        "pages/students/2018/ondrej_megela/README.md": "969ff2e1a436b8f957e7168cf5eb678196b99aeda17701c502b016441f644065",
+        "pages/students/2018/samuel_sirotnik/README.md": "cc230ede67b6e8632511c92a5ed6d1e9deb42e70819280222bdfd369ee77fc70",
+        "pages/students/2019/alina_vitko/README.md": "9b071fe2aff470c275ae027be236fe836f741f774cabb9cbbb55304a8d5102cd",
+        "pages/students/2019/artem_yatsenko/README.md": "328b101e46c51c959e792cf7ca226942f81d2ab8fe9648ad6d0ecd07016429e2",
+        "pages/students/2019/david_stromp/README.md": "e46ca8691d5ec894128f6f6f2ed33e2c7bd141f80719bb0449b20f10f30cca09",
+        "pages/students/2019/dmytro_mural/README.md": "e881a47d847c3ef8d680844113fdbfbb87553b28982bf9eef4936f6d30923e93",
+        "pages/students/2019/filip_tomas/README.md": "75a1fe7b921e4bff0fdf9e2a957b4adad4b4a57aa7ebdffbb73dd82c02123f94",
+        "pages/students/2019/kristian_sopkovic/README.md": "3286960fb30f584df35455f9b6dfae81ca4827a1d711d2bcd6e48c3f150c9279",
+        "pages/students/2019/michal_stromko/README.md": "3ae1dd9b6638a6a249a94be3f3846286555666aefa225bb3a2c7288b35900ce7",
+        "pages/students/2019/michal_stromko/vp2021/README.md": "feb2e093fdd6461239e79b1e1f96e65a1c8cd96e8a2b70535ae556a9c198046a",
+        "pages/students/2019/michal_stromko/vp2021/tutorials/README.md": "40e3fa448c7329ef863e705435e23fe6b4788de440710f066fe0eed92d132bad",
+        "pages/students/2019/michal_stromko/vp2021/tutorials/img/README.md": "e8d3ac160b17dfdee2c3e996bf6a03af7a6f16f9140bf5cf7ee343a82dd1865f",
+        "pages/students/2019/patrik_pokrivcak/README.md": "3e3957ad0f7aca59d6eb64c38a3f5e21d8fd953200825d0d0e5cacabfbc35a7d",
+        "pages/students/2019/samuel_horani/README.md": "86e6959ccc612b069687b57996fa3f61660075d6a0d65bfec64f423d39a0a999",
+        "pages/students/2020/david_kostilnik/README.md": "2927524b75da59a64120fc364a0fc374986fa4d364d54e52c36800762c203264",
+        "pages/students/2020/jakub_kristian_lukas/README.md": "f139d9b1ebf5a3f45612f3df5703ecdd935b90701763c27c7c529044c0b0455a",
+        "pages/students/2020/kamil_tomcufcik/README.md": "9d865adf538bc6d7992ee4f4882713447da5c936fca17c43ca0f576072537048",
+        "pages/students/2020/matej_kobylan/README.md": "3d7b289e17b8f25678a07e9ee0e6741688485a0ccb1e50940a6a6acc6312a547",
+        "pages/students/2020/matus_suchanic/README.md": "770ca49538cca22df35f8e7ba736ee97483e4fe5788c221496553592b4e78dc6",
+        "pages/students/2020/pavol_hudak/README.md": "33d06fccc6451e5f19b27b0cad001a34e81635b4ea294a01dec7771c823abc40",
+        "pages/students/2020/vladyslav_krupko/README.md": "4c28e6d38651bb2cafcbf4a18e35210cd369aef2caff6c5b0aa52fac38f0fe7d",
+        "pages/students/2021/artur_hyrenko/README.md": "0568261f4ade4de5472edcfeb1ffed9abae18dc67e4050f4ce2ad088839a2bb3",
+        "pages/students/2021/eduard_matovka/README.md": "de2a7ae38ec87110e347956b37d0605a298ed2847f972961cc9b977da8221d43",
+        "pages/students/2021/manohar_gowdru_shridharu/README.md": "c00ceb620ed7087ead5d9448d476bbb98d273d1d9e7bf1bc4355c367b26c7a6b",
+        "pages/students/2021/martin_sarissky/README.md": "8e97b45a253745faa55f63c844e0b5dfe680aeaf6eb01fb8f9d06ac6e8ee8bcf",
+        "pages/students/2021/matej_novotny/README.md": "b083a7cadde532e3428dc9a57bc61456fcc5905bda91e011b96033d42bf5d9cb",
+        "pages/students/2021/matej_scislak/README.md": "b9a32cce8605e044a4f2828ff2d91a2995a62ef3a841e62f0a225c97221f0cc8",
+        "pages/students/2021/nikita_bodnar/README.md": "2016c0dd923c0f8e561d5cea42ac81da3610fdcc54a4527f56d078e34e53ce90",
+        "pages/students/2021/oleksandr_hryshchenko/README.md": "4b073992b715b6c75b79fd2c8f850a69f940966e3a61a82c625c32c945a93036",
+        "pages/students/2021/vladimir_ferko/README.md": "68e7d311896eda2171e5eec1ec3a8da90d326ed91acaace8058f68664a50d271",
+        "pages/students/2022/andrii_pervashov/README.md": "319dcaa1cc2e29dd7e3a710bab26cb3ac6944ab71b0b97e1de076b1199122334",
+        "pages/students/2022/daniil_huzenko/README.md": "390782fbff6220a9462060ce1a62959f2a2b88f42d06e2c51d1f406b75d73f2c",
+        "pages/students/2022/jakub_schwarc/README.md": "f541a8852c2b0f6f9d7182e01a34c4e3beee9bfa55427a06b331638b11eb8be6",
+        "pages/students/2022/jan_malinovsky/README.md": "8af021bfb131fa005a7aec3e2f2f12fc17fa5dc14e2675311adbbfc445a4bcab",
+        "pages/students/2022/jan_ptak/README.md": "131cf053764fd5423541ebe672c3b4f5da875cfb8e4b821f1335ddb967282282",
+        "pages/students/2022/oleh_poiasnik/README.md": "c2f7dc3f61f8b4a5b43ded1e3ae4c7d3e7f874439ee6b17dd9c8d45dabc25bc8",
+        "pages/students/2022/samuel_vasil/README.md": "67c807049147aacd04f3050f4ff0233a8b3f0a239ab173b430e13dfe113ad0a5",
+        "pages/students/2022/serhii_yemets/README.md": "ffbeac893947acafe855d576322a8826f9d4f87ac344cb31f8ceeefa244b9133",
+        "pages/students/2022/tetiana_mohorian/README.md": "c9efdc632bf1ca89a0bfde0dc00be7aaac8a85da3d92e2ce8506236c73e93cfb",
+        "pages/students/2022/valerii_kutsenko/README.md": "dc764c609ad3c1df59cfea997d4db82a591b1b1e0764453e319d841115507e4f",
+        "pages/students/2022/vladyslav_yanchenko/README.md": "4d1ce216e3b526ee437ddbc658e51d7bb83530087e43628c096a569a3854a47f",
+        "pages/students/2022/yevhenii_leonov/README.md": "b90e41db0cec2451027020a9102d32a24d777e8a38b604ccca87b36cd6b3a292",
+        "pages/students/2023/denis_le_thanh/README.md": "2cb83f942d86a02712b0c58778e46dc2211346d1a6eba606471dfa832245db61",
+        "pages/students/2023/maksym_zatirka/README.md": "7a7d2cbe276c352dacfbc6ba6de5f15303d30ecf34845ff8da8d085f3d67c3dd",
+        "pages/students/2023/oleksandr_dorybohov/README.md": "c24d6f941de0dafb4459cea8259cf147821cafc86ba65be5b528f419f734bc18",
+        "pages/students/2023/ronald_zalacko/README.md": "724b0f8e6e641236dba9bddbd77f4546e09514076c7dfb151807aaf4b882c96f",
+        "pages/students/2023/rostyslav_rodzhuk/README.md": "de356dd7427c8a0716c165a0f88ae47b691a86ee7a388f60e2d2e9306298415c",
+        "pages/students/2023/samuel_dzurina/README.md": "4818191d044b91219c796a35010c95da5cf2093e050d494f30b219b91e13fa0e",
+        "pages/students/2023/simona_bobrovcanova/README.md": "e473a1911dae10ab40e2cf6696a8be616a58c22aaf09af5953fbac475791a35e",
+        "pages/students/2023/yevhenii_medushivskyi/README.md": "5710ebf91c893a12fe6d19f5372311938d1e7c7bbdf203f2dc8ed2fe7c4f5a24",
+        "pages/topics/akopisat/README.md": "c7245e1fbfa4160618502c0baf8cc6bc47cd9a82b8a85bc1c645314829a7b76a",
+        "pages/topics/bert/README.md": "35e507f7ec2ad1d3526e2a15f94ea085ee25f680a02daf6df62fd429bd9c25c9",
+        "pages/topics/chatbot/README.md": "6f2f83ffb80109be47f9750cca9d3036ec3645abef9f2b590bd6825780c7c48a",
+        "pages/topics/faq/README.md": "15cfbd9c2a1649c80a7a58922e2b66dadab5c86a5ef33ef16c82bebd4d2dbf06",
+        "pages/topics/hatespeech/README.md": "364df94e70dd740c63544e2586ba7b957780f714fc0c941d1b04b02233e6cbc5",
+        "pages/topics/legal/README.md": "003bb9bfe1e0ca21652091c2d11496312eb04e0689fe459e14c2d5a738e8eeea",
+        "pages/topics/morphology/README.md": "e242cc5ff1a4cc98e227566c0ead999bbc95937b7c20be62cb2afb773d0d35c0",
+        "pages/topics/named-entity/README.md": "66b2e25f70211d432d596e377d552b3b8f0c525b14c31e3bc9c956c3aeadca84",
+        "pages/topics/named-entity/navod/README.md": "1ff87079a98411de0414754c60241e08d2214ddcd63ddf974e4340cce2ce5b02",
+        "pages/topics/nlp/README.md": "6f90abf9dc3f0ff33475283c13b4a08561abf631425ec2daacb076155a9e02c2",
+        "pages/topics/otvorene/README.md": "cd5544925553fe3840c0ea6f8ef60ab756e26ab6757bf810025c948d8d4f98fc",
+        "pages/topics/prodigy/README.md": "5e4f691f2754c2ec4afbaab2f41e6db77feb5b322dd9344465eb40c35f116499",
+        "pages/topics/python/README.md": "2ef073adba672142c39e9a59fa2e00a98ab52bda54233d46fbe6940ee23f9fcc",
+        "pages/topics/question/README.md": "408078a2a67cea2699e12359e85ab641136231e6104c2afdf874a705879525df",
+        "pages/topics/question/navod/README.md": "061e737c69184b2bd244316ca7ba841f7186fe01ee80ee993742a733dcad3ecd",
+        "pages/topics/question/nezodpovedatelne/README.md": "b6703d9690f4599b973efb27595ce157773ae0fc0d55cf2c0a09d9f50e8e4c92",
+        "pages/topics/question/validacie/README.md": "3216a7127abdfc6da50ed116f44ff2d6785e7da4af70d23aceda774e745ad562",
+        "pages/topics/resources/README.md": "09405397387f00cbcc5680f7d655f2d861e4d92e5df792a62d9dfd04f3534f48",
+        "pages/topics/seq2seq/README.md": "969d15c03a63a3f8d5edc9017fe5a0fb7311fdf4de4a3473a08266241f92c1e2",
+        "pages/topics/spacy/README.md": "f504e5db77cb8694f08a57fb8636e522d59a0e669216f4bc8273bc3c5f375f87",
+        "pages/topics/steel/README.md": "b4068cef2ed197711467ac47630d5b2064fdc8828f898f7af99664e94d833330",
+        "pages/topics/translation/README.md": "a7d4289e72199b307dc5ad8fc69cda39c1a33c27a6c7447af70aa9c3cc6b8635"
+      }
+    },
+    "retrieval_policy": {
+      "published_only": true,
+      "publication_policy": "explicit-false-hidden-v1",
+      "academic_preparation_is_work": false
+    }
+  },
+  "graph": {
+    "sha256": "6fd45dc58382641072d991a088fcdfcd1c0e81ce912befbae6fca700d630fe01",
+    "nodes": 446,
+    "relationships": 877
+  },
+  "parameters": {
+    "limit": 40,
+    "max_entities": 8
+  },
+  "dev_summary": {
+    "questions": 722,
+    "source_checked_count": 722,
+    "all_facts_supported_count": 722,
+    "strict_count": 722,
+    "strict_accuracy": 1.0,
+    "error_count": 0,
+    "value_score": 1.0,
+    "entity_score": 1.0,
+    "relation_score": 1.0,
+    "property_score": 1.0,
+    "relation_choice_score": 1.0,
+    "document_recall": 1.0,
+    "source_recall": 1.0,
+    "binding_score": 1.0,
+    "path_score": 1.0,
+    "answerability_accuracy": 1.0,
+    "latency_mean_seconds": 0.051461,
+    "latency_median_seconds": 0.04753,
+    "latency_p95_seconds": 0.079209
+  },
+  "source_audit_summary": {
+    "dev": {
+      "questions": 722,
+      "source_consistent": 722,
+      "issues": {}
+    },
+    "test": {
+      "questions": 320,
+      "source_consistent": 320,
+      "issues": {}
+    }
+  },
+  "policy": "Report all questions including negatives. No exclusions or holdout tuning.",
+  "dev_results_sha256": "d4ac47c21aed4be149a3f1aec1f7658c862da2ee24e41cc680128d013a071707",
+  "audit_sha256": "d5922db25b6d40cb7e5dc7df7990e93a3caee263a6e747ae910ab340da340023"
+}

tools/graphrag/evaluation/reproducibility.py

@@ -36,7 +36,10 @@ def content_fingerprints(questions: Path, root: Path = ROOT) -> dict[str, Any]:
     # requirements.txt and tools/ are present both in the checkout and image.
     files.extend(root / name for name in [
         'requirements.txt', 'tools/publication.py', 'tools/retrieval_contract.py',
-        'tools/index_state.py', 'tools/rag/common.py', 'tools/sync_indexes.py'])
+        'tools/index_state.py', 'tools/rag/common.py', 'tools/sync_indexes.py',
+        'tools/rag/rag_utils.py', 'tools/answer_evaluation/context_policy.py',
+        'tools/answer_evaluation/checked_generation.py', 'tools/answer_evaluation/answer_format.py',
+        'tools/rag/evaluation/rag_metrics.py', 'tools/rag/evaluation/rag_runner.py'])
     return {
         'implementation': {str(path.relative_to(root)): sha256_file(path) for path in files},
         'dataset_sha256': sha256_file(questions),

tools/graphrag/evaluation/results/answers_v6_dev_2026-10-25/answers_dev.csv

@@ -0,0 +1,43 @@
+id,category,strict,error,latency_seconds,attempts,first_pass_strict
+wide-001,shared_keyword_pair,True,,14.98031888899277,1,True
+wide-002,shared_keyword_pair,True,,8.397393244013074,1,True
+wide-041,same_academic_year_pair,True,,6.902643829002045,1,True
+wide-042,same_academic_year_pair,True,,6.882375492001302,1,True
+wide-080,same_academic_year_pair,True,,12.271757686001365,1,True
+wide-081,same_start_year_pair,True,,8.204727490010555,1,True
+wide-082,same_start_year_pair,True,,13.702611551998416,1,True
+wide-104,same_start_year_pair,True,,6.501820275996579,1,True
+wide-113,person_title_year,True,,10.964796315005515,1,True
+wide-114,person_title_year,True,,7.376625097997021,1,True
+wide-116,person_title_year,True,,7.847194762012805,1,True
+wide-119,person_title_year,True,,7.342980269997497,1,True
+wide-129,person_title_year,True,,8.418876146010007,1,True
+wide-137,person_keyword_title,True,,9.795194769991213,1,True
+wide-138,person_keyword_title,True,,8.844560971003375,1,True
+wide-139,person_keyword_title,True,,9.329334552996443,1,True
+wide-141,person_keyword_title,True,,9.592877750008483,1,True
+wide-142,person_keyword_title,True,,5.007802811000147,1,True
+wide-143,person_keyword_title,True,,11.664019524003379,1,True
+wide-145,person_keyword_title,True,,3.2914913540007547,1,True
+wide-155,person_keyword_title,True,,6.680825702002039,1,True
+wide-159,person_keyword_title,True,,13.560824561005575,1,True
+wide-161,person_keyword_title,True,,7.287550342996838,1,True
+wide-163,person_keyword_title,True,,12.33143888199993,1,True
+wide-165,person_keyword_title,True,,2.5120001080067595,1,True
+wide-169,document_author_category_multi_hop,True,,6.5130544929997995,1,True
+wide-170,document_author_category_multi_hop,True,,8.98161977599375,1,True
+wide-178,document_author_category_multi_hop,True,,6.077633890003199,1,True
+wide-181,document_author_category_multi_hop,True,,8.441153998006484,1,True
+wide-193,person_multiple_categories,True,,5.523557033011457,1,True
+wide-194,person_multiple_categories,True,,8.1642089090019,1,True
+wide-209,person_multiple_categories,True,,6.1462335319956765,1,True
+wide-217,candidate_set_aggregation,True,,10.333294509997359,1,True
+wide-218,candidate_set_aggregation,True,,8.658889918995555,1,True
+wide-232,candidate_set_aggregation,True,,14.336149934009882,1,True
+wide-236,candidate_set_aggregation,True,,7.896173950997763,1,True
+wide-239,candidate_set_aggregation,True,,13.960932583999238,1,True
+wide-249,conjunctive_disambiguation,True,,5.288597282997216,1,True
+wide-250,conjunctive_disambiguation,True,,10.146073770010844,1,True
+wide-283,conjunctive_disambiguation,False,,12.396372432995122,1,False
+wide-289,negative_lookup,True,,6.812868237000657,1,True
+wide-290,negative_lookup,True,,6.7916078380076215,1,True

tools/graphrag/evaluation/results/answers_v6_dev_2026-10-25/frozen_answers.json

@@ -0,0 +1,434 @@
+{
+  "method": "Previously exposed LLM questions; regression, not independent TEST.",
+  "graph_freeze_sha256": "e5c6c01277478b49e10b0ea3daee991b8a20d895b5fb435506a01d8a941535ef",
+  "audit_sha256": "d5922db25b6d40cb7e5dc7df7990e93a3caee263a6e747ae910ab340da340023",
+  "generation_files": {
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/evaluate_graph_answers.py": "2553a0d17e164922b02046416fdf0d5df4496dc443466d8d9a1f4be86415b318",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/graph_answer_prompt.txt": "c7104ffb08b5b384f38d48fbdad40ccd40de61beace984d8a118cf12bd229860",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/answer_format.py": "86a27b4aebc6a5077e13794b81957a1cf063a1f3e070c583120ba53c1788d2f8",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/context_policy.py": "ee3c423772800d3e7368fff9b48bcd464a98a741b87a0c79158398ed22a3dc00",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/checked_generation.py": "e82b95314b8c0ae9ba82353b98876da7b2a4dce464eb21129ad904bf58b8ef5e",
+    "tools/rag/evaluation/rag_runner.py": "031abf7d5b1a026eb199aa2a579bfb8baaa01175279d0018a364bfe50b76c3c1"
+  },
+  "model": "model120-fast",
+  "temperature": 0,
+  "workers": 3,
+  "dev_ids": [
+    "wide-001",
+    "wide-002",
+    "wide-041",
+    "wide-042",
+    "wide-080",
+    "wide-081",
+    "wide-082",
+    "wide-104",
+    "wide-113",
+    "wide-114",
+    "wide-116",
+    "wide-119",
+    "wide-129",
+    "wide-137",
+    "wide-138",
+    "wide-139",
+    "wide-141",
+    "wide-142",
+    "wide-143",
+    "wide-145",
+    "wide-155",
+    "wide-159",
+    "wide-161",
+    "wide-163",
+    "wide-165",
+    "wide-169",
+    "wide-170",
+    "wide-178",
+    "wide-181",
+    "wide-193",
+    "wide-194",
+    "wide-209",
+    "wide-217",
+    "wide-218",
+    "wide-232",
+    "wide-236",
+    "wide-239",
+    "wide-249",
+    "wide-250",
+    "wide-283",
+    "wide-289",
+    "wide-290"
+  ],
+  "test_ids": [
+    "wide-003",
+    "wide-004",
+    "wide-005",
+    "wide-006",
+    "wide-007",
+    "wide-008",
+    "wide-009",
+    "wide-010",
+    "wide-011",
+    "wide-012",
+    "wide-013",
+    "wide-014",
+    "wide-015",
+    "wide-016",
+    "wide-017",
+    "wide-018",
+    "wide-019",
+    "wide-020",
+    "wide-021",
+    "wide-022",
+    "wide-023",
+    "wide-024",
+    "wide-025",
+    "wide-026",
+    "wide-027",
+    "wide-028",
+    "wide-029",
+    "wide-030",
+    "wide-031",
+    "wide-032",
+    "wide-033",
+    "wide-034",
+    "wide-035",
+    "wide-036",
+    "wide-037",
+    "wide-038",
+    "wide-039",
+    "wide-040",
+    "wide-043",
+    "wide-044",
+    "wide-045",
+    "wide-046",
+    "wide-047",
+    "wide-048",
+    "wide-049",
+    "wide-050",
+    "wide-051",
+    "wide-052",
+    "wide-053",
+    "wide-054",
+    "wide-055",
+    "wide-056",
+    "wide-057",
+    "wide-058",
+    "wide-059",
+    "wide-060",
+    "wide-061",
+    "wide-062",
+    "wide-063",
+    "wide-064",
+    "wide-065",
+    "wide-066",
+    "wide-067",
+    "wide-068",
+    "wide-069",
+    "wide-070",
+    "wide-071",
+    "wide-072",
+    "wide-073",
+    "wide-074",
+    "wide-075",
+    "wide-076",
+    "wide-077",
+    "wide-078",
+    "wide-079",
+    "wide-080",
+    "wide-083",
+    "wide-084",
+    "wide-085",
+    "wide-086",
+    "wide-087",
+    "wide-088",
+    "wide-089",
+    "wide-090",
+    "wide-091",
+    "wide-092",
+    "wide-093",
+    "wide-094",
+    "wide-095",
+    "wide-096",
+    "wide-097",
+    "wide-098",
+    "wide-099",
+    "wide-100",
+    "wide-101",
+    "wide-102",
+    "wide-103",
+    "wide-104",
+    "wide-105",
+    "wide-106",
+    "wide-107",
+    "wide-108",
+    "wide-109",
+    "wide-110",
+    "wide-111",
+    "wide-112",
+    "wide-115",
+    "wide-116",
+    "wide-117",
+    "wide-118",
+    "wide-119",
+    "wide-120",
+    "wide-121",
+    "wide-122",
+    "wide-123",
+    "wide-124",
+    "wide-125",
+    "wide-126",
+    "wide-127",
+    "wide-128",
+    "wide-129",
+    "wide-130",
+    "wide-131",
+    "wide-132",
+    "wide-133",
+    "wide-134",
+    "wide-135",
+    "wide-136",
+    "wide-139",
+    "wide-140",
+    "wide-141",
+    "wide-142",
+    "wide-143",
+    "wide-144",
+    "wide-145",
+    "wide-146",
+    "wide-147",
+    "wide-148",
+    "wide-149",
+    "wide-150",
+    "wide-151",
+    "wide-152",
+    "wide-153",
+    "wide-154",
+    "wide-155",
+    "wide-156",
+    "wide-157",
+    "wide-158",
+    "wide-159",
+    "wide-160",
+    "wide-161",
+    "wide-162",
+    "wide-163",
+    "wide-164",
+    "wide-165",
+    "wide-166",
+    "wide-167",
+    "wide-168",
+    "wide-171",
+    "wide-172",
+    "wide-173",
+    "wide-174",
+    "wide-175",
+    "wide-176",
+    "wide-177",
+    "wide-178",
+    "wide-179",
+    "wide-180",
+    "wide-181",
+    "wide-182",
+    "wide-183",
+    "wide-184",
+    "wide-185",
+    "wide-186",
+    "wide-187",
+    "wide-188",
+    "wide-189",
+    "wide-190",
+    "wide-191",
+    "wide-192",
+    "wide-195",
+    "wide-196",
+    "wide-197",
+    "wide-198",
+    "wide-199",
+    "wide-200",
+    "wide-201",
+    "wide-202",
+    "wide-203",
+    "wide-204",
+    "wide-205",
+    "wide-206",
+    "wide-207",
+    "wide-208",
+    "wide-209",
+    "wide-210",
+    "wide-211",
+    "wide-212",
+    "wide-213",
+    "wide-214",
+    "wide-215",
+    "wide-216",
+    "wide-219",
+    "wide-220",
+    "wide-221",
+    "wide-222",
+    "wide-223",
+    "wide-224",
+    "wide-225",
+    "wide-226",
+    "wide-227",
+    "wide-228",
+    "wide-229",
+    "wide-230",
+    "wide-231",
+    "wide-232",
+    "wide-233",
+    "wide-234",
+    "wide-235",
+    "wide-236",
+    "wide-237",
+    "wide-238",
+    "wide-239",
+    "wide-240",
+    "wide-241",
+    "wide-242",
+    "wide-243",
+    "wide-244",
+    "wide-245",
+    "wide-246",
+    "wide-247",
+    "wide-248",
+    "wide-251",
+    "wide-252",
+    "wide-253",
+    "wide-254",
+    "wide-255",
+    "wide-256",
+    "wide-257",
+    "wide-258",
+    "wide-259",
+    "wide-260",
+    "wide-261",
+    "wide-262",
+    "wide-263",
+    "wide-264",
+    "wide-265",
+    "wide-266",
+    "wide-267",
+    "wide-268",
+    "wide-269",
+    "wide-270",
+    "wide-271",
+    "wide-272",
+    "wide-273",
+    "wide-274",
+    "wide-275",
+    "wide-276",
+    "wide-277",
+    "wide-278",
+    "wide-279",
+    "wide-280",
+    "wide-281",
+    "wide-282",
+    "wide-283",
+    "wide-284",
+    "wide-285",
+    "wide-286",
+    "wide-287",
+    "wide-288",
+    "wide-291",
+    "wide-292",
+    "wide-293",
+    "wide-294",
+    "wide-295",
+    "wide-296",
+    "wide-297",
+    "wide-298",
+    "wide-299",
+    "wide-300",
+    "wide-301",
+    "wide-302",
+    "wide-303",
+    "wide-304",
+    "wide-305",
+    "wide-306",
+    "wide-307",
+    "wide-308",
+    "wide-309",
+    "wide-310",
+    "wide-311",
+    "wide-312",
+    "wide-313",
+    "wide-314",
+    "wide-315",
+    "wide-316",
+    "wide-317",
+    "wide-318",
+    "wide-319",
+    "wide-320"
+  ],
+  "dev_count": 42,
+  "test_count": 300,
+  "target_accuracy": 0.92,
+  "selected_split": "dev",
+  "selected_ids": [
+    "wide-001",
+    "wide-002",
+    "wide-041",
+    "wide-042",
+    "wide-080",
+    "wide-081",
+    "wide-082",
+    "wide-104",
+    "wide-113",
+    "wide-114",
+    "wide-116",
+    "wide-119",
+    "wide-129",
+    "wide-137",
+    "wide-138",
+    "wide-139",
+    "wide-141",
+    "wide-142",
+    "wide-143",
+    "wide-145",
+    "wide-155",
+    "wide-159",
+    "wide-161",
+    "wide-163",
+    "wide-165",
+    "wide-169",
+    "wide-170",
+    "wide-178",
+    "wide-181",
+    "wide-193",
+    "wide-194",
+    "wide-209",
+    "wide-217",
+    "wide-218",
+    "wide-232",
+    "wide-236",
+    "wide-239",
+    "wide-249",
+    "wide-250",
+    "wide-283",
+    "wide-289",
+    "wide-290"
+  ],
+  "selected_count": 42,
+  "selected_dev_overlap": 42,
+  "corrective_request_limit": 1,
+  "format_repair_limit": 1,
+  "repair_policy": "At most one total corrective request for format, citations or retrieval-context violations. Same question/context, no gold; retain all attempts.",
+  "dev_results_sha256": null,
+  "policy": "Evaluate every declared regression case, including DEV overlap; retain all failures and stop tuning after this freeze.",
+  "selected_categories": {
+    "shared_keyword_pair": 2,
+    "same_academic_year_pair": 3,
+    "same_start_year_pair": 3,
+    "person_title_year": 5,
+    "person_keyword_title": 12,
+    "document_author_category_multi_hop": 4,
+    "person_multiple_categories": 3,
+    "candidate_set_aggregation": 5,
+    "conjunctive_disambiguation": 3,
+    "negative_lookup": 2
+  },
+  "limitations": [
+    "All questions and previous model outputs are known; this is historical regression.",
+    "Harness supplies graph context; native OpenWebUI tool choice and browser UI are outside this experiment.",
+    "Checks structured factual claims, their source support/citations and required prose values; not a general semantic judge of arbitrary prose."
+  ]
+}

tools/graphrag/evaluation/results/answers_v6_final_dev_2026-10-25/answers_dev.csv

@@ -0,0 +1,43 @@
+id,category,strict,error,latency_seconds,attempts,first_pass_strict
+wide-001,shared_keyword_pair,True,,9.046294871004648,1,True
+wide-002,shared_keyword_pair,True,,16.67808335200243,1,True
+wide-041,same_academic_year_pair,True,,12.044013594000717,1,True
+wide-042,same_academic_year_pair,True,,16.060333867004374,1,True
+wide-080,same_academic_year_pair,True,,14.174281418003375,1,True
+wide-081,same_start_year_pair,True,,16.653584496001713,1,True
+wide-082,same_start_year_pair,True,,15.326327786999173,1,True
+wide-104,same_start_year_pair,True,,17.865080415009288,1,True
+wide-113,person_title_year,True,,14.24573272500129,1,True
+wide-114,person_title_year,True,,9.858880985004362,1,True
+wide-116,person_title_year,True,,8.85577247099718,1,True
+wide-119,person_title_year,True,,8.421206099999836,1,True
+wide-129,person_title_year,True,,8.982687898998847,1,True
+wide-137,person_keyword_title,True,,8.925621253001736,1,True
+wide-138,person_keyword_title,False,InvalidAnswerFormat: two invalid JSON/schema responses,19.134980328992242,2,False
+wide-139,person_keyword_title,True,,8.484527300999616,1,True
+wide-141,person_keyword_title,True,,9.90471307298867,1,True
+wide-142,person_keyword_title,True,,5.409087655993062,1,True
+wide-143,person_keyword_title,False,InvalidAnswerFormat: two invalid JSON/schema responses,21.20253009199223,2,False
+wide-145,person_keyword_title,True,,4.431844910999644,1,True
+wide-155,person_keyword_title,True,,9.433868713997072,1,True
+wide-159,person_keyword_title,True,,12.901580277000903,1,True
+wide-161,person_keyword_title,True,,13.279490936998627,1,True
+wide-163,person_keyword_title,True,,10.156943395006238,1,True
+wide-165,person_keyword_title,True,,6.284939156001201,1,True
+wide-169,document_author_category_multi_hop,True,,6.316422595002223,1,True
+wide-170,document_author_category_multi_hop,True,,6.294689077010844,1,True
+wide-178,document_author_category_multi_hop,True,,8.572520501009421,1,True
+wide-181,document_author_category_multi_hop,True,,6.569023182004457,1,True
+wide-193,person_multiple_categories,True,,8.477885875996435,1,True
+wide-194,person_multiple_categories,True,,5.930385426996509,1,True
+wide-209,person_multiple_categories,True,,8.633006432995899,1,True
+wide-217,candidate_set_aggregation,True,,8.902099704995635,1,True
+wide-218,candidate_set_aggregation,True,,12.242024114006199,1,True
+wide-232,candidate_set_aggregation,True,,11.786965720995795,1,True
+wide-236,candidate_set_aggregation,True,,12.099168748012744,1,True
+wide-239,candidate_set_aggregation,True,,10.9651905639912,1,True
+wide-249,conjunctive_disambiguation,True,,9.097645240995917,1,True
+wide-250,conjunctive_disambiguation,True,,8.975287802997627,1,True
+wide-283,conjunctive_disambiguation,True,,8.098080616997322,1,True
+wide-289,negative_lookup,True,,8.07642724798643,1,True
+wide-290,negative_lookup,True,,5.626124244998209,1,True

tools/graphrag/evaluation/results/answers_v6_final_dev_2026-10-25/frozen_answers.json

@@ -0,0 +1,434 @@
+{
+  "method": "Previously exposed LLM questions; regression, not independent TEST.",
+  "graph_freeze_sha256": "54946b476fd713f551414b00cf2b843627566f6a333e3b59ce3efbbdf2328629",
+  "audit_sha256": "d5922db25b6d40cb7e5dc7df7990e93a3caee263a6e747ae910ab340da340023",
+  "generation_files": {
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/evaluate_graph_answers.py": "2553a0d17e164922b02046416fdf0d5df4496dc443466d8d9a1f4be86415b318",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/graph_answer_prompt.txt": "8c8274a22bf5b44364d8fc8226a8f659b31c232f5a2aba5916d693b559e9f341",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/answer_format.py": "86a27b4aebc6a5077e13794b81957a1cf063a1f3e070c583120ba53c1788d2f8",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/context_policy.py": "e7b7c511e5e9cafdef04969eb4007a6a61ac4b99bcc60e9e722945a15bec1f26",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/checked_generation.py": "7e017471e6281db62b79f270210db3ab14c7e31a4623bbfa3d9a5fd8911b1c20",
+    "tools/rag/evaluation/rag_runner.py": "031abf7d5b1a026eb199aa2a579bfb8baaa01175279d0018a364bfe50b76c3c1"
+  },
+  "model": "model120-fast",
+  "temperature": 0,
+  "workers": 3,
+  "dev_ids": [
+    "wide-001",
+    "wide-002",
+    "wide-041",
+    "wide-042",
+    "wide-080",
+    "wide-081",
+    "wide-082",
+    "wide-104",
+    "wide-113",
+    "wide-114",
+    "wide-116",
+    "wide-119",
+    "wide-129",
+    "wide-137",
+    "wide-138",
+    "wide-139",
+    "wide-141",
+    "wide-142",
+    "wide-143",
+    "wide-145",
+    "wide-155",
+    "wide-159",
+    "wide-161",
+    "wide-163",
+    "wide-165",
+    "wide-169",
+    "wide-170",
+    "wide-178",
+    "wide-181",
+    "wide-193",
+    "wide-194",
+    "wide-209",
+    "wide-217",
+    "wide-218",
+    "wide-232",
+    "wide-236",
+    "wide-239",
+    "wide-249",
+    "wide-250",
+    "wide-283",
+    "wide-289",
+    "wide-290"
+  ],
+  "test_ids": [
+    "wide-003",
+    "wide-004",
+    "wide-005",
+    "wide-006",
+    "wide-007",
+    "wide-008",
+    "wide-009",
+    "wide-010",
+    "wide-011",
+    "wide-012",
+    "wide-013",
+    "wide-014",
+    "wide-015",
+    "wide-016",
+    "wide-017",
+    "wide-018",
+    "wide-019",
+    "wide-020",
+    "wide-021",
+    "wide-022",
+    "wide-023",
+    "wide-024",
+    "wide-025",
+    "wide-026",
+    "wide-027",
+    "wide-028",
+    "wide-029",
+    "wide-030",
+    "wide-031",
+    "wide-032",
+    "wide-033",
+    "wide-034",
+    "wide-035",
+    "wide-036",
+    "wide-037",
+    "wide-038",
+    "wide-039",
+    "wide-040",
+    "wide-043",
+    "wide-044",
+    "wide-045",
+    "wide-046",
+    "wide-047",
+    "wide-048",
+    "wide-049",
+    "wide-050",
+    "wide-051",
+    "wide-052",
+    "wide-053",
+    "wide-054",
+    "wide-055",
+    "wide-056",
+    "wide-057",
+    "wide-058",
+    "wide-059",
+    "wide-060",
+    "wide-061",
+    "wide-062",
+    "wide-063",
+    "wide-064",
+    "wide-065",
+    "wide-066",
+    "wide-067",
+    "wide-068",
+    "wide-069",
+    "wide-070",
+    "wide-071",
+    "wide-072",
+    "wide-073",
+    "wide-074",
+    "wide-075",
+    "wide-076",
+    "wide-077",
+    "wide-078",
+    "wide-079",
+    "wide-080",
+    "wide-083",
+    "wide-084",
+    "wide-085",
+    "wide-086",
+    "wide-087",
+    "wide-088",
+    "wide-089",
+    "wide-090",
+    "wide-091",
+    "wide-092",
+    "wide-093",
+    "wide-094",
+    "wide-095",
+    "wide-096",
+    "wide-097",
+    "wide-098",
+    "wide-099",
+    "wide-100",
+    "wide-101",
+    "wide-102",
+    "wide-103",
+    "wide-104",
+    "wide-105",
+    "wide-106",
+    "wide-107",
+    "wide-108",
+    "wide-109",
+    "wide-110",
+    "wide-111",
+    "wide-112",
+    "wide-115",
+    "wide-116",
+    "wide-117",
+    "wide-118",
+    "wide-119",
+    "wide-120",
+    "wide-121",
+    "wide-122",
+    "wide-123",
+    "wide-124",
+    "wide-125",
+    "wide-126",
+    "wide-127",
+    "wide-128",
+    "wide-129",
+    "wide-130",
+    "wide-131",
+    "wide-132",
+    "wide-133",
+    "wide-134",
+    "wide-135",
+    "wide-136",
+    "wide-139",
+    "wide-140",
+    "wide-141",
+    "wide-142",
+    "wide-143",
+    "wide-144",
+    "wide-145",
+    "wide-146",
+    "wide-147",
+    "wide-148",
+    "wide-149",
+    "wide-150",
+    "wide-151",
+    "wide-152",
+    "wide-153",
+    "wide-154",
+    "wide-155",
+    "wide-156",
+    "wide-157",
+    "wide-158",
+    "wide-159",
+    "wide-160",
+    "wide-161",
+    "wide-162",
+    "wide-163",
+    "wide-164",
+    "wide-165",
+    "wide-166",
+    "wide-167",
+    "wide-168",
+    "wide-171",
+    "wide-172",
+    "wide-173",
+    "wide-174",
+    "wide-175",
+    "wide-176",
+    "wide-177",
+    "wide-178",
+    "wide-179",
+    "wide-180",
+    "wide-181",
+    "wide-182",
+    "wide-183",
+    "wide-184",
+    "wide-185",
+    "wide-186",
+    "wide-187",
+    "wide-188",
+    "wide-189",
+    "wide-190",
+    "wide-191",
+    "wide-192",
+    "wide-195",
+    "wide-196",
+    "wide-197",
+    "wide-198",
+    "wide-199",
+    "wide-200",
+    "wide-201",
+    "wide-202",
+    "wide-203",
+    "wide-204",
+    "wide-205",
+    "wide-206",
+    "wide-207",
+    "wide-208",
+    "wide-209",
+    "wide-210",
+    "wide-211",
+    "wide-212",
+    "wide-213",
+    "wide-214",
+    "wide-215",
+    "wide-216",
+    "wide-219",
+    "wide-220",
+    "wide-221",
+    "wide-222",
+    "wide-223",
+    "wide-224",
+    "wide-225",
+    "wide-226",
+    "wide-227",
+    "wide-228",
+    "wide-229",
+    "wide-230",
+    "wide-231",
+    "wide-232",
+    "wide-233",
+    "wide-234",
+    "wide-235",
+    "wide-236",
+    "wide-237",
+    "wide-238",
+    "wide-239",
+    "wide-240",
+    "wide-241",
+    "wide-242",
+    "wide-243",
+    "wide-244",
+    "wide-245",
+    "wide-246",
+    "wide-247",
+    "wide-248",
+    "wide-251",
+    "wide-252",
+    "wide-253",
+    "wide-254",
+    "wide-255",
+    "wide-256",
+    "wide-257",
+    "wide-258",
+    "wide-259",
+    "wide-260",
+    "wide-261",
+    "wide-262",
+    "wide-263",
+    "wide-264",
+    "wide-265",
+    "wide-266",
+    "wide-267",
+    "wide-268",
+    "wide-269",
+    "wide-270",
+    "wide-271",
+    "wide-272",
+    "wide-273",
+    "wide-274",
+    "wide-275",
+    "wide-276",
+    "wide-277",
+    "wide-278",
+    "wide-279",
+    "wide-280",
+    "wide-281",
+    "wide-282",
+    "wide-283",
+    "wide-284",
+    "wide-285",
+    "wide-286",
+    "wide-287",
+    "wide-288",
+    "wide-291",
+    "wide-292",
+    "wide-293",
+    "wide-294",
+    "wide-295",
+    "wide-296",
+    "wide-297",
+    "wide-298",
+    "wide-299",
+    "wide-300",
+    "wide-301",
+    "wide-302",
+    "wide-303",
+    "wide-304",
+    "wide-305",
+    "wide-306",
+    "wide-307",
+    "wide-308",
+    "wide-309",
+    "wide-310",
+    "wide-311",
+    "wide-312",
+    "wide-313",
+    "wide-314",
+    "wide-315",
+    "wide-316",
+    "wide-317",
+    "wide-318",
+    "wide-319",
+    "wide-320"
+  ],
+  "dev_count": 42,
+  "test_count": 300,
+  "target_accuracy": 0.92,
+  "selected_split": "dev",
+  "selected_ids": [
+    "wide-001",
+    "wide-002",
+    "wide-041",
+    "wide-042",
+    "wide-080",
+    "wide-081",
+    "wide-082",
+    "wide-104",
+    "wide-113",
+    "wide-114",
+    "wide-116",
+    "wide-119",
+    "wide-129",
+    "wide-137",
+    "wide-138",
+    "wide-139",
+    "wide-141",
+    "wide-142",
+    "wide-143",
+    "wide-145",
+    "wide-155",
+    "wide-159",
+    "wide-161",
+    "wide-163",
+    "wide-165",
+    "wide-169",
+    "wide-170",
+    "wide-178",
+    "wide-181",
+    "wide-193",
+    "wide-194",
+    "wide-209",
+    "wide-217",
+    "wide-218",
+    "wide-232",
+    "wide-236",
+    "wide-239",
+    "wide-249",
+    "wide-250",
+    "wide-283",
+    "wide-289",
+    "wide-290"
+  ],
+  "selected_count": 42,
+  "selected_dev_overlap": 42,
+  "corrective_request_limit": 1,
+  "format_repair_limit": 1,
+  "repair_policy": "At most one total corrective request for format, citations or retrieval-context violations. Same question/context, no gold; retain all attempts.",
+  "dev_results_sha256": null,
+  "policy": "Evaluate every declared regression case, including DEV overlap; retain all failures and stop tuning after this freeze.",
+  "selected_categories": {
+    "shared_keyword_pair": 2,
+    "same_academic_year_pair": 3,
+    "same_start_year_pair": 3,
+    "person_title_year": 5,
+    "person_keyword_title": 12,
+    "document_author_category_multi_hop": 4,
+    "person_multiple_categories": 3,
+    "candidate_set_aggregation": 5,
+    "conjunctive_disambiguation": 3,
+    "negative_lookup": 2
+  },
+  "limitations": [
+    "All questions and previous model outputs are known; this is historical regression.",
+    "Harness supplies graph context; native OpenWebUI tool choice and browser UI are outside this experiment.",
+    "Checks structured factual claims, their source support/citations and required prose values; not a general semantic judge of arbitrary prose."
+  ]
+}

tools/graphrag/evaluation/results/answers_v6_regression_2026-10-25/answers_regression.csv

@@ -0,0 +1,321 @@
+id,category,strict,error,latency_seconds,attempts,first_pass_strict
+wide-001,shared_keyword_pair,True,,46.673222578989225,1,True
+wide-002,shared_keyword_pair,True,,8.559933836993878,1,True
+wide-003,shared_keyword_pair,True,,7.599364289999357,1,True
+wide-004,shared_keyword_pair,True,,43.05431627100916,1,True
+wide-005,shared_keyword_pair,True,,46.60277532499458,1,True
+wide-006,shared_keyword_pair,True,,17.02198196199606,1,True
+wide-007,shared_keyword_pair,True,,37.58451674701064,1,True
+wide-008,shared_keyword_pair,True,,162.56305770600738,1,True
+wide-009,shared_keyword_pair,True,,96.59099388300092,1,True
+wide-010,shared_keyword_pair,True,,98.12694204699073,1,True
+wide-011,shared_keyword_pair,True,,5.023704371007625,1,True
+wide-012,shared_keyword_pair,True,,4.511661280004773,1,True
+wide-013,shared_keyword_pair,True,,4.7268635670043295,1,True
+wide-014,shared_keyword_pair,True,,4.411614174008719,1,True
+wide-015,shared_keyword_pair,True,,7.357604845994501,1,True
+wide-016,shared_keyword_pair,True,,8.766350423000404,1,True
+wide-017,shared_keyword_pair,True,,7.075148112009629,1,True
+wide-018,shared_keyword_pair,True,,7.273996427989914,1,True
+wide-019,shared_keyword_pair,True,,7.305565910995938,1,True
+wide-020,shared_keyword_pair,True,,6.627847782001481,1,True
+wide-021,shared_keyword_pair,True,,8.089955857009045,1,True
+wide-022,shared_keyword_pair,True,,7.479066304003936,1,True
+wide-023,shared_keyword_pair,True,,12.767835026999819,1,True
+wide-024,shared_keyword_pair,True,,12.18826178299787,1,True
+wide-025,shared_keyword_pair,True,,24.296105483997962,1,True
+wide-026,shared_keyword_pair,True,,26.256827491000877,1,True
+wide-027,shared_keyword_pair,True,,21.511969452010817,1,True
+wide-028,shared_keyword_pair,True,,13.690101789004984,1,True
+wide-029,shared_keyword_pair,True,,8.394414648006205,1,True
+wide-030,shared_keyword_pair,True,,13.538730853993911,1,True
+wide-031,shared_keyword_pair,True,,8.447307058013394,1,True
+wide-032,shared_keyword_pair,True,,13.313853904008283,1,True
+wide-033,shared_keyword_pair,True,,8.748317188990768,1,True
+wide-034,shared_keyword_pair,True,,15.075756493999506,1,True
+wide-035,shared_keyword_pair,True,,7.7471648280043155,1,True
+wide-036,shared_keyword_pair,True,,12.841984746992239,1,True
+wide-037,shared_keyword_pair,True,,8.577543838997371,1,True
+wide-038,shared_keyword_pair,True,,12.718846365998616,1,True
+wide-039,shared_keyword_pair,True,,9.08833715999208,1,True
+wide-040,shared_keyword_pair,True,,13.980110079006408,1,True
+wide-041,same_academic_year_pair,True,,7.145661453992943,1,True
+wide-042,same_academic_year_pair,True,,11.684578247994068,1,True
+wide-043,same_academic_year_pair,True,,10.049950553991948,1,True
+wide-044,same_academic_year_pair,True,,12.46243585999764,1,True
+wide-045,same_academic_year_pair,True,,9.949639684011345,1,True
+wide-046,same_academic_year_pair,True,,10.84967578999931,1,True
+wide-047,same_academic_year_pair,True,,11.929021324991481,1,True
+wide-048,same_academic_year_pair,True,,12.54277628500131,1,True
+wide-049,same_academic_year_pair,True,,12.512920577995828,1,True
+wide-050,same_academic_year_pair,True,,10.943723205011338,1,True
+wide-051,same_academic_year_pair,True,,11.361078971996903,1,True
+wide-052,same_academic_year_pair,True,,10.127026930989814,1,True
+wide-053,same_academic_year_pair,True,,9.671026055992115,1,True
+wide-054,same_academic_year_pair,True,,10.90942802200152,1,True
+wide-055,same_academic_year_pair,True,,9.650152500995318,1,True
+wide-056,same_academic_year_pair,True,,13.77639084700786,1,True
+wide-057,same_academic_year_pair,True,,7.519770080005401,1,True
+wide-058,same_academic_year_pair,True,,12.226794489004533,1,True
+wide-059,same_academic_year_pair,True,,9.18516255299619,1,True
+wide-060,same_academic_year_pair,True,,11.267676324001513,1,True
+wide-061,same_academic_year_pair,True,,8.971232945012162,1,True
+wide-062,same_academic_year_pair,True,,10.002240323010483,1,True
+wide-063,same_academic_year_pair,True,,9.58987016500032,1,True
+wide-064,same_academic_year_pair,True,,9.889049531993805,1,True
+wide-065,same_academic_year_pair,True,,10.5299181730079,1,True
+wide-066,same_academic_year_pair,True,,9.414292849003687,1,True
+wide-067,same_academic_year_pair,True,,12.795267630004673,1,True
+wide-068,same_academic_year_pair,True,,7.465757275000215,1,True
+wide-069,same_academic_year_pair,True,,13.18860708399734,1,True
+wide-070,same_academic_year_pair,True,,6.870617680004216,1,True
+wide-071,same_academic_year_pair,True,,13.928496929991525,1,True
+wide-072,same_academic_year_pair,True,,6.418771339012892,1,True
+wide-073,same_academic_year_pair,True,,11.917343320004875,1,True
+wide-074,same_academic_year_pair,True,,8.277425395994214,1,True
+wide-075,same_academic_year_pair,True,,10.830195090005873,1,True
+wide-076,same_academic_year_pair,True,,8.49058225400222,1,True
+wide-077,same_academic_year_pair,True,,10.718131281013484,1,True
+wide-078,same_academic_year_pair,True,,8.951186734004295,1,True
+wide-079,same_academic_year_pair,True,,10.618172678994597,1,True
+wide-080,same_academic_year_pair,True,,9.463216475996887,1,True
+wide-081,same_start_year_pair,True,,10.834732246003114,1,True
+wide-082,same_start_year_pair,True,,9.731701743992744,1,True
+wide-083,same_start_year_pair,True,,10.492485894006677,1,True
+wide-084,same_start_year_pair,True,,9.800166964996606,1,True
+wide-085,same_start_year_pair,True,,10.272105573007138,1,True
+wide-086,same_start_year_pair,True,,9.51915932400152,1,True
+wide-087,same_start_year_pair,True,,9.921699527010787,1,True
+wide-088,same_start_year_pair,True,,9.524695414002053,1,True
+wide-089,same_start_year_pair,True,,10.797489714997937,1,True
+wide-090,same_start_year_pair,True,,9.793365620993427,1,True
+wide-091,same_start_year_pair,True,,10.977525908994721,1,True
+wide-092,same_start_year_pair,True,,8.956933887006016,1,True
+wide-093,same_start_year_pair,True,,10.82372733700322,1,True
+wide-094,same_start_year_pair,True,,8.492465975010418,1,True
+wide-095,same_start_year_pair,True,,11.66025954700308,1,True
+wide-096,same_start_year_pair,True,,8.606951418012613,1,True
+wide-097,same_start_year_pair,True,,12.164914641994983,1,True
+wide-098,same_start_year_pair,True,,8.51734155100712,1,True
+wide-099,same_start_year_pair,True,,12.007975722997799,1,True
+wide-100,same_start_year_pair,True,,8.085574763012119,1,True
+wide-101,same_start_year_pair,True,,11.381393574003596,1,True
+wide-102,same_start_year_pair,True,,8.105480084996088,1,True
+wide-103,same_start_year_pair,True,,11.178495122003369,1,True
+wide-104,same_start_year_pair,False,,8.045926626000437,1,False
+wide-105,same_start_year_pair,True,,11.05217344200355,1,True
+wide-106,same_start_year_pair,True,,8.158618310000747,1,True
+wide-107,same_start_year_pair,True,,10.908150404997286,1,True
+wide-108,same_start_year_pair,True,,8.880417915992439,1,True
+wide-109,same_start_year_pair,True,,11.187253098003566,1,True
+wide-110,same_start_year_pair,True,,9.394831576995784,1,True
+wide-111,same_start_year_pair,True,,10.523962463004864,1,True
+wide-112,same_start_year_pair,True,,8.898946485001943,1,True
+wide-113,person_title_year,True,,9.418683277006494,1,True
+wide-114,person_title_year,True,,8.167781379001099,1,True
+wide-115,person_title_year,True,,8.713697127008345,1,True
+wide-116,person_title_year,True,,7.5155074930080445,1,True
+wide-117,person_title_year,True,,8.792661272003897,1,True
+wide-118,person_title_year,True,,7.729041151003912,1,True
+wide-119,person_title_year,False,InvalidAnswerFormat: two invalid JSON/schema responses,17.042694887000835,2,False
+wide-120,person_title_year,True,,7.293894077010918,1,True
+wide-121,person_title_year,True,,9.912295032001566,1,True
+wide-122,person_title_year,True,,10.336526605999097,1,True
+wide-123,person_title_year,True,,7.557256949992734,1,True
+wide-124,person_title_year,True,,9.972375639990787,1,True
+wide-125,person_title_year,True,,6.4579289499961305,1,True
+wide-126,person_title_year,True,,8.706965881996439,1,True
+wide-127,person_title_year,True,,6.528363082994474,1,True
+wide-128,person_title_year,True,,9.74250477401074,1,True
+wide-129,person_title_year,True,,7.816760465997504,1,True
+wide-130,person_title_year,True,,9.037185494991718,1,True
+wide-131,person_title_year,True,,6.980635786996572,1,True
+wide-132,person_title_year,True,,8.133284494004329,1,True
+wide-133,person_title_year,True,,7.697097384996596,1,True
+wide-134,person_title_year,True,,7.980329867990804,1,True
+wide-135,person_title_year,True,,7.890191981001408,1,True
+wide-136,person_title_year,True,,8.521550403995207,1,True
+wide-137,person_keyword_title,True,,8.654003389994614,1,True
+wide-138,person_keyword_title,True,,9.317925055991509,1,True
+wide-139,person_keyword_title,True,,8.443255108009907,1,True
+wide-140,person_keyword_title,True,,9.268355077001615,1,True
+wide-141,person_keyword_title,True,,8.619561607993091,1,True
+wide-142,person_keyword_title,True,,4.842823801009217,1,True
+wide-143,person_keyword_title,False,InvalidAnswerFormat: two invalid JSON/schema responses,25.41826143900107,2,False
+wide-144,person_keyword_title,True,,6.520040678005898,1,True
+wide-145,person_keyword_title,True,,6.482367881006212,1,True
+wide-146,person_keyword_title,True,,7.926933477006969,1,True
+wide-147,person_keyword_title,True,,9.258605299008195,1,True
+wide-148,person_keyword_title,True,,7.477349338005297,1,True
+wide-149,person_keyword_title,True,,11.021910049996222,1,True
+wide-150,person_keyword_title,True,,9.106626356995548,1,True
+wide-151,person_keyword_title,True,,9.87995294299617,1,True
+wide-152,person_keyword_title,True,,8.400023505993886,1,True
+wide-153,person_keyword_title,True,,8.574925572000211,1,True
+wide-154,person_keyword_title,True,,8.580053354002303,1,True
+wide-155,person_keyword_title,True,,9.083858401005273,1,True
+wide-156,person_keyword_title,True,,8.619125420998898,1,True
+wide-157,person_keyword_title,True,,9.071385319999536,1,True
+wide-158,person_keyword_title,True,,8.34309680199658,1,True
+wide-159,person_keyword_title,True,,9.822020093008177,1,True
+wide-160,person_keyword_title,True,,9.166224397995393,1,True
+wide-161,person_keyword_title,True,,10.358729845989728,1,True
+wide-162,person_keyword_title,True,,8.444243013000232,1,True
+wide-163,person_keyword_title,True,,9.713410705997376,1,True
+wide-164,person_keyword_title,True,,8.84475968699553,1,True
+wide-165,person_keyword_title,True,,5.332855019994895,1,True
+wide-166,person_keyword_title,True,,7.285008748003747,1,True
+wide-167,person_keyword_title,True,,7.721775719997822,1,True
+wide-168,person_keyword_title,True,,10.334125888999552,1,True
+wide-169,document_author_category_multi_hop,True,,5.7036590370116755,1,True
+wide-170,document_author_category_multi_hop,True,,9.754529660000117,1,True
+wide-171,document_author_category_multi_hop,True,,4.6746241329965414,1,True
+wide-172,document_author_category_multi_hop,True,,9.619921479999903,1,True
+wide-173,document_author_category_multi_hop,True,,6.214922799001215,1,True
+wide-174,document_author_category_multi_hop,True,,8.83508723799605,1,True
+wide-175,document_author_category_multi_hop,True,,6.034610215006978,1,True
+wide-176,document_author_category_multi_hop,False,InvalidAnswerFormat: two invalid JSON/schema responses,30.148610364005435,2,False
+wide-177,document_author_category_multi_hop,True,,5.49371292599244,1,True
+wide-178,document_author_category_multi_hop,True,,8.342899441995542,1,True
+wide-179,document_author_category_multi_hop,True,,9.640073613001732,1,True
+wide-180,document_author_category_multi_hop,True,,7.465760963008506,1,True
+wide-181,document_author_category_multi_hop,True,,5.794508402002975,1,True
+wide-182,document_author_category_multi_hop,True,,8.31849720000173,1,True
+wide-183,document_author_category_multi_hop,True,,9.857232266003848,1,True
+wide-184,document_author_category_multi_hop,True,,6.444520987992291,1,True
+wide-185,document_author_category_multi_hop,True,,9.522022441000445,1,True
+wide-186,document_author_category_multi_hop,True,,6.413496649009176,1,True
+wide-187,document_author_category_multi_hop,True,,10.410627094010124,1,True
+wide-188,document_author_category_multi_hop,True,,5.231139651994454,1,True
+wide-189,document_author_category_multi_hop,True,,8.794849190002424,1,True
+wide-190,document_author_category_multi_hop,True,,4.944473762996495,1,True
+wide-191,document_author_category_multi_hop,True,,10.042116317999898,1,True
+wide-192,document_author_category_multi_hop,True,,10.5407217900065,1,True
+wide-193,person_multiple_categories,True,,4.792528019999736,1,True
+wide-194,person_multiple_categories,True,,4.782916190000833,1,True
+wide-195,person_multiple_categories,True,,8.615475654994952,1,True
+wide-196,person_multiple_categories,True,,4.323636931003421,1,True
+wide-197,person_multiple_categories,True,,7.403106719997595,1,True
+wide-198,person_multiple_categories,True,,5.580519854003796,1,True
+wide-199,person_multiple_categories,True,,7.428712174994871,1,True
+wide-200,person_multiple_categories,True,,5.628509769987431,1,True
+wide-201,person_multiple_categories,True,,7.0406875039916486,1,True
+wide-202,person_multiple_categories,True,,5.428648525994504,1,True
+wide-203,person_multiple_categories,True,,7.3671833719999995,1,True
+wide-204,person_multiple_categories,True,,6.062737486005062,1,True
+wide-205,person_multiple_categories,True,,7.703495145004126,1,True
+wide-206,person_multiple_categories,True,,6.037194566000835,1,True
+wide-207,person_multiple_categories,True,,6.8626932660117745,1,True
+wide-208,person_multiple_categories,True,,5.614748537002015,1,True
+wide-209,person_multiple_categories,True,,7.410357187996851,1,True
+wide-210,person_multiple_categories,True,,5.773788381004124,1,True
+wide-211,person_multiple_categories,True,,8.462341753009241,1,True
+wide-212,person_multiple_categories,True,,3.9356538349966286,1,True
+wide-213,person_multiple_categories,True,,7.3488670679944335,1,True
+wide-214,person_multiple_categories,True,,5.440327513992088,1,True
+wide-215,person_multiple_categories,True,,7.479325737003819,1,True
+wide-216,person_multiple_categories,True,,5.488911034000921,1,True
+wide-217,candidate_set_aggregation,True,,12.322915113996714,1,True
+wide-218,candidate_set_aggregation,True,,7.939687445003074,1,True
+wide-219,candidate_set_aggregation,True,,15.70125729299616,1,True
+wide-220,candidate_set_aggregation,True,,7.490574616007507,1,True
+wide-221,candidate_set_aggregation,True,,14.266550314001506,1,True
+wide-222,candidate_set_aggregation,True,,9.285332848012331,1,True
+wide-223,candidate_set_aggregation,True,,15.166198047008947,1,True
+wide-224,candidate_set_aggregation,True,,7.299934319002205,1,True
+wide-225,candidate_set_aggregation,True,,12.34221489400079,1,True
+wide-226,candidate_set_aggregation,True,,9.4215394359926,1,True
+wide-227,candidate_set_aggregation,True,,11.556736822007224,1,True
+wide-228,candidate_set_aggregation,True,,9.91855254600523,1,True
+wide-229,candidate_set_aggregation,True,,10.58573554699251,1,True
+wide-230,candidate_set_aggregation,True,,10.693445764001808,1,True
+wide-231,candidate_set_aggregation,True,,11.349713716001133,1,True
+wide-232,candidate_set_aggregation,True,,11.74791878099495,1,True
+wide-233,candidate_set_aggregation,True,,11.048946268012514,1,True
+wide-234,candidate_set_aggregation,True,,11.014590331993531,1,True
+wide-235,candidate_set_aggregation,True,,10.42495377000887,1,True
+wide-236,candidate_set_aggregation,True,,11.615297402007855,1,True
+wide-237,candidate_set_aggregation,True,,11.304762533996836,1,True
+wide-238,candidate_set_aggregation,True,,12.494652763998602,1,True
+wide-239,candidate_set_aggregation,True,,10.859883589990204,1,True
+wide-240,candidate_set_aggregation,True,,11.546406508990913,1,True
+wide-241,candidate_set_aggregation,True,,9.245278398011578,1,True
+wide-242,candidate_set_aggregation,True,,12.081533781005419,1,True
+wide-243,candidate_set_aggregation,True,,9.45886921499914,1,True
+wide-244,candidate_set_aggregation,True,,12.694161742008873,1,True
+wide-245,candidate_set_aggregation,True,,8.572539522006991,1,True
+wide-246,candidate_set_aggregation,True,,12.862463094992563,1,True
+wide-247,candidate_set_aggregation,True,,8.681235329000629,1,True
+wide-248,candidate_set_aggregation,True,,14.288417944990215,1,True
+wide-249,conjunctive_disambiguation,True,,5.564151740996749,1,True
+wide-250,conjunctive_disambiguation,True,,9.57288877099927,1,True
+wide-251,conjunctive_disambiguation,True,,7.981897253994248,1,True
+wide-252,conjunctive_disambiguation,True,,8.615102830008254,1,True
+wide-253,conjunctive_disambiguation,True,,7.343791733001126,1,True
+wide-254,conjunctive_disambiguation,True,,8.36678491201019,1,True
+wide-255,conjunctive_disambiguation,True,,6.50943264500529,1,True
+wide-256,conjunctive_disambiguation,True,,8.855455351003911,1,True
+wide-257,conjunctive_disambiguation,True,,6.625423559002229,1,True
+wide-258,conjunctive_disambiguation,True,,10.785042872012127,1,True
+wide-259,conjunctive_disambiguation,True,,5.227306733999285,1,True
+wide-260,conjunctive_disambiguation,True,,9.602625528001226,1,True
+wide-261,conjunctive_disambiguation,True,,7.348476777013275,1,True
+wide-262,conjunctive_disambiguation,True,,9.600847927998984,1,True
+wide-263,conjunctive_disambiguation,True,,7.57662963400071,1,True
+wide-264,conjunctive_disambiguation,True,,9.053563342007692,1,True
+wide-265,conjunctive_disambiguation,True,,7.701207513004192,1,True
+wide-266,conjunctive_disambiguation,True,,8.352936759009026,1,True
+wide-267,conjunctive_disambiguation,True,,7.6647831270092865,1,True
+wide-268,conjunctive_disambiguation,True,,7.513196407002397,1,True
+wide-269,conjunctive_disambiguation,True,,7.314271451992681,1,True
+wide-270,conjunctive_disambiguation,True,,7.282059941004263,1,True
+wide-271,conjunctive_disambiguation,True,,7.553321383005823,1,True
+wide-272,conjunctive_disambiguation,True,,8.163246180993156,1,True
+wide-273,conjunctive_disambiguation,True,,7.806915986991953,1,True
+wide-274,conjunctive_disambiguation,True,,8.457703224004945,1,True
+wide-275,conjunctive_disambiguation,True,,7.778582415005076,1,True
+wide-276,conjunctive_disambiguation,True,,9.062161419002223,1,True
+wide-277,conjunctive_disambiguation,True,,7.960439574002521,1,True
+wide-278,conjunctive_disambiguation,True,,8.366461051002261,1,True
+wide-279,conjunctive_disambiguation,True,,7.2900457229989115,1,True
+wide-280,conjunctive_disambiguation,True,,7.8031022979994304,1,True
+wide-281,conjunctive_disambiguation,True,,7.834450206995825,1,True
+wide-282,conjunctive_disambiguation,True,,8.4359122379974,1,True
+wide-283,conjunctive_disambiguation,True,,9.275576508007362,1,True
+wide-284,conjunctive_disambiguation,True,,8.009781319007743,1,True
+wide-285,conjunctive_disambiguation,True,,10.937106861994835,1,True
+wide-286,conjunctive_disambiguation,True,,5.5999535669980105,1,True
+wide-287,conjunctive_disambiguation,True,,10.864354767996701,1,True
+wide-288,conjunctive_disambiguation,True,,5.759624956001062,1,True
+wide-289,negative_lookup,True,,7.9427883150056005,1,True
+wide-290,negative_lookup,True,,5.373547496012179,1,True
+wide-291,negative_lookup,True,,7.423920717992587,1,True
+wide-292,negative_lookup,True,,2.929652819002513,1,True
+wide-293,negative_lookup,True,,5.856424662997597,1,True
+wide-294,negative_lookup,True,,7.053081437989022,1,True
+wide-295,negative_lookup,True,,6.665841868001735,1,True
+wide-296,negative_lookup,True,,7.309867624993785,1,True
+wide-297,negative_lookup,True,,6.265542463006568,1,True
+wide-298,negative_lookup,True,,7.0743426569970325,1,True
+wide-299,negative_lookup,True,,5.507314685994061,1,True
+wide-300,negative_lookup,True,,5.693418583992752,1,True
+wide-301,negative_lookup,True,,5.491494399990188,1,True
+wide-302,negative_lookup,True,,4.511676192007144,1,True
+wide-303,negative_lookup,True,,5.679598642003839,1,True
+wide-304,negative_lookup,True,,5.175881937990198,1,True
+wide-305,negative_lookup,True,,7.612728366002557,1,True
+wide-306,negative_lookup,True,,5.382500367995817,1,True
+wide-307,negative_lookup,True,,7.818900088008377,1,True
+wide-308,negative_lookup,True,,5.674700601011864,1,True
+wide-309,negative_lookup,True,,8.50653202299145,1,True
+wide-310,negative_lookup,True,,3.142994189009187,1,True
+wide-311,negative_lookup,True,,7.0121463510004105,1,True
+wide-312,negative_lookup,True,,2.9377342259977013,1,True
+wide-313,negative_lookup,True,,6.331644553996739,1,True
+wide-314,negative_lookup,True,,3.377158279006835,1,True
+wide-315,negative_lookup,True,,5.142885044988361,1,True
+wide-316,negative_lookup,True,,5.371270889008883,1,True
+wide-317,negative_lookup,True,,6.465425841000979,1,True
+wide-318,negative_lookup,True,,2.8997244839993073,1,True
+wide-319,negative_lookup,True,,5.762461207006709,1,True
+wide-320,negative_lookup,True,,5.02917427700595,1,True

tools/graphrag/evaluation/results/answers_v6_regression_2026-10-25/failures.json

@@ -0,0 +1,738 @@
+{
+  "evaluation_kind": "historical_regression",
+  "questions": [
+    {
+      "id": "wide-104",
+      "category": "same_start_year_pair",
+      "question": "Aký spoločný rok začiatku štúdia majú osoby Adrián Remiáš a Martin NaN1?",
+      "graph_context": {
+        "status": "ok",
+        "reason": null,
+        "facts": [
+          {
+            "person": "Adrián Remiáš",
+            "start_year": 2018,
+            "document_path": "pages/students/2018/adrian_remias/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2018/adrian_remias"
+          },
+          {
+            "person": "Martin NaN1",
+            "start_year": 2018,
+            "document_path": "pages/students/2018/NaN1name_buday/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2018/NaN1name_buday"
+          }
+        ],
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "pages/students/2018/NaN1name_buday/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2018/NaN1name_buday",
+            "title": "Martin NaN1"
+          },
+          {
+            "source_id": "S2",
+            "document_path": "pages/students/2018/adrian_remias/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2018/adrian_remias",
+            "title": "Adrián Remiáš"
+          }
+        ],
+        "aggregation": null,
+        "complete": true,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+          "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+          "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+          "embedding_model": "intfloat/multilingual-e5-small",
+          "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+          "publication_policy": "explicit-false-hidden-v1",
+          "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+          "version": "10b78a8e70744e2792ffe4cf11d45571"
+        },
+        "answer_candidates": []
+      },
+      "strict": false,
+      "attempts": [
+        {
+          "response": {
+            "choices": [
+              {
+                "finish_reason": "stop",
+                "index": 0,
+                "message": {
+                  "role": "assistant",
+                  "content": "{\"answer\": \"Spoločný rok začiatku štúdia pre Adriána Remiáša a Martina NaN1 je 2018. Zdroje: https://zp.kemt.fei.tuke.sk/students/2018/adrian_remias, https://zp.kemt.fei.tuke.sk/students/2018/NaN1name_buday\", \"status\": \"ok\", \"claims\": [{\"person\": \"Adrián Remiáš\", \"start_year\": 2018}, {\"person\": \"Martin NaN1\", \"start_year\": 2018}], \"source_urls\": [\"https://zp.kemt.fei.tuke.sk/students/2018/adrian_remias\", \"https://zp.kemt.fei.tuke.sk/students/2018/NaN1name_buday\"], \"count\": 2}"
+                }
+              }
+            ],
+            "created": 1792964273,
+            "model": "Qwen3.5-122B-A10B-Q4_K_M",
+            "system_fingerprint": "b8580-7c203670f",
+            "object": "chat.completion",
+            "usage": {
+              "completion_tokens": 191,
+              "prompt_tokens": 1873,
+              "total_tokens": 2064,
+              "prompt_tokens_details": {
+                "cached_tokens": 1158
+              }
+            },
+            "id": "chatcmpl-2R7dPvfcQNspVFStpBK0TpFUlyP9Vizu",
+            "timings": {
+              "cache_n": 1158,
+              "prompt_n": 715,
+              "prompt_ms": 1023.639,
+              "prompt_per_token_ms": 1.431662937062937,
+              "prompt_per_second": 698.4884319569692,
+              "predicted_n": 191,
+              "predicted_ms": 5221.067,
+              "predicted_per_token_ms": 27.335429319371727,
+              "predicted_per_second": 36.582560614525725
+            }
+          },
+          "answer": {
+            "answer": "Spoločný rok začiatku štúdia pre Adriána Remiáša a Martina NaN1 je 2018. Zdroje: https://zp.kemt.fei.tuke.sk/students/2018/adrian_remias, https://zp.kemt.fei.tuke.sk/students/2018/NaN1name_buday",
+            "status": "ok",
+            "claims": [
+              {
+                "person": "Adrián Remiáš",
+                "start_year": 2018
+              },
+              {
+                "person": "Martin NaN1",
+                "start_year": 2018
+              }
+            ],
+            "source_urls": [
+              "https://zp.kemt.fei.tuke.sk/students/2018/adrian_remias",
+              "https://zp.kemt.fei.tuke.sk/students/2018/NaN1name_buday"
+            ],
+            "count": 2
+          },
+          "format_normalized": false,
+          "context_issues": [],
+          "strict": false
+        }
+      ],
+      "accepted_for_delivery": true,
+      "response": {
+        "choices": [
+          {
+            "finish_reason": "stop",
+            "index": 0,
+            "message": {
+              "role": "assistant",
+              "content": "{\"answer\": \"Spoločný rok začiatku štúdia pre Adriána Remiáša a Martina NaN1 je 2018. Zdroje: https://zp.kemt.fei.tuke.sk/students/2018/adrian_remias, https://zp.kemt.fei.tuke.sk/students/2018/NaN1name_buday\", \"status\": \"ok\", \"claims\": [{\"person\": \"Adrián Remiáš\", \"start_year\": 2018}, {\"person\": \"Martin NaN1\", \"start_year\": 2018}], \"source_urls\": [\"https://zp.kemt.fei.tuke.sk/students/2018/adrian_remias\", \"https://zp.kemt.fei.tuke.sk/students/2018/NaN1name_buday\"], \"count\": 2}"
+            }
+          }
+        ],
+        "created": 1792964273,
+        "model": "Qwen3.5-122B-A10B-Q4_K_M",
+        "system_fingerprint": "b8580-7c203670f",
+        "object": "chat.completion",
+        "usage": {
+          "completion_tokens": 191,
+          "prompt_tokens": 1873,
+          "total_tokens": 2064,
+          "prompt_tokens_details": {
+            "cached_tokens": 1158
+          }
+        },
+        "id": "chatcmpl-2R7dPvfcQNspVFStpBK0TpFUlyP9Vizu",
+        "timings": {
+          "cache_n": 1158,
+          "prompt_n": 715,
+          "prompt_ms": 1023.639,
+          "prompt_per_token_ms": 1.431662937062937,
+          "prompt_per_second": 698.4884319569692,
+          "predicted_n": 191,
+          "predicted_ms": 5221.067,
+          "predicted_per_token_ms": 27.335429319371727,
+          "predicted_per_second": 36.582560614525725
+        }
+      },
+      "answer": {
+        "answer": "Spoločný rok začiatku štúdia pre Adriána Remiáša a Martina NaN1 je 2018. Zdroje: https://zp.kemt.fei.tuke.sk/students/2018/adrian_remias, https://zp.kemt.fei.tuke.sk/students/2018/NaN1name_buday",
+        "status": "ok",
+        "claims": [
+          {
+            "person": "Adrián Remiáš",
+            "start_year": 2018
+          },
+          {
+            "person": "Martin NaN1",
+            "start_year": 2018
+          }
+        ],
+        "source_urls": [
+          "https://zp.kemt.fei.tuke.sk/students/2018/adrian_remias",
+          "https://zp.kemt.fei.tuke.sk/students/2018/NaN1name_buday"
+        ],
+        "count": 2
+      },
+      "bindings_ok": true,
+      "unsupported_claims": [],
+      "sources_ok": true,
+      "status_ok": true,
+      "count_ok": true,
+      "set_ok": true,
+      "citations_ok": true,
+      "unsupported_urls": [],
+      "context_issues": [],
+      "text_values_ok": false,
+      "missing_text_values": [
+        "Martin NaN1"
+      ],
+      "contradictory_thesis_label": false,
+      "latency_seconds": 8.045926626000437
+    },
+    {
+      "id": "wide-119",
+      "category": "person_title_year",
+      "question": "Aký názov diplomového projektu a rok práce sú pri osobe Ján Malinovský uvedené pre rok 2026?",
+      "graph_context": {
+        "status": "ok",
+        "reason": null,
+        "facts": [
+          {
+            "person": "Ján Malinovský",
+            "title": "Grafové vyhľadávanie pre podporu vzdelávania",
+            "year": 2026,
+            "document_path": "pages/students/2022/jan_malinovsky/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2022/jan_malinovsky"
+          }
+        ],
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "pages/students/2022/jan_malinovsky/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2022/jan_malinovsky",
+            "title": "Ján Malinovský"
+          }
+        ],
+        "aggregation": null,
+        "complete": true,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+          "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+          "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+          "embedding_model": "intfloat/multilingual-e5-small",
+          "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+          "publication_policy": "explicit-false-hidden-v1",
+          "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+          "version": "10b78a8e70744e2792ffe4cf11d45571"
+        },
+        "answer_candidates": []
+      },
+      "strict": false,
+      "attempts": [
+        {
+          "response": {
+            "choices": [
+              {
+                "finish_reason": "stop",
+                "index": 0,
+                "message": {
+                  "role": "assistant",
+                  "content": "{\"answer\": \"Pre rok 2026 je pri osobe Ján Malinovský uvedený názov práce „Grafové vyhľadávanie pre podporu vzdelávania\" a rok 2026. Zdroj: https://zp.kemt.fei.tuke.sk/students/2022/jan_malinovsky\", \"status\": \"ok\", \"claims\": [{\"person\": \"Ján Malinovský\", \"title\": \"Grafové vyhľadávanie pre podporu vzdelávania\", \"year\": 2026, \"keyword\": null}], \"source_urls\": [\"https://zp.kemt.fei.tuke.sk/students/2022/jan_malinovsky\"], \"count\": 1}"
+                }
+              }
+            ],
+            "created": 1792964319,
+            "model": "Qwen3.5-122B-A10B-Q4_K_M",
+            "system_fingerprint": "b8580-7c203670f",
+            "object": "chat.completion",
+            "usage": {
+              "completion_tokens": 165,
+              "prompt_tokens": 1758,
+              "total_tokens": 1923,
+              "prompt_tokens_details": {
+                "cached_tokens": 1158
+              }
+            },
+            "id": "chatcmpl-dPYPeGnw7RQOOaX5NDbxccFWLPffOv7n",
+            "timings": {
+              "cache_n": 1158,
+              "prompt_n": 600,
+              "prompt_ms": 941.121,
+              "prompt_per_token_ms": 1.568535,
+              "prompt_per_second": 637.537574870819,
+              "predicted_n": 165,
+              "predicted_ms": 4505.923,
+              "predicted_per_token_ms": 27.30862424242424,
+              "predicted_per_second": 36.61846862451933
+            }
+          },
+          "format_error": "JSONDecodeError",
+          "strict": false
+        },
+        {
+          "response": {
+            "choices": [
+              {
+                "finish_reason": "stop",
+                "index": 0,
+                "message": {
+                  "role": "assistant",
+                  "content": "{\"answer\": \"Pre rok 2026 je pri osobe Ján Malinovský uvedený názov práce „Grafové vyhľadávanie pre podporu vzdelávania\" a rok 2026. Zdroj: https://zp.kemt.fei.tuke.sk/students/2022/jan_malinovsky\", \"status\": \"ok\", \"claims\": [{\"person\": \"Ján Malinovský\", \"title\": \"Grafové vyhľadávanie pre podporu vzdelávania\", \"year\": 2026}], \"source_urls\": [\"https://zp.kemt.fei.tuke.sk/students/2022/jan_malinovsky\"], \"count\": 1}"
+                }
+              }
+            ],
+            "created": 1792964327,
+            "model": "Qwen3.5-122B-A10B-Q4_K_M",
+            "system_fingerprint": "b8580-7c203670f",
+            "object": "chat.completion",
+            "usage": {
+              "completion_tokens": 160,
+              "prompt_tokens": 2372,
+              "total_tokens": 2532,
+              "prompt_tokens_details": {
+                "cached_tokens": 1158
+              }
+            },
+            "id": "chatcmpl-o2tXoEJknRgCWtI8dGVpNaSAln5cygJE",
+            "timings": {
+              "cache_n": 1158,
+              "prompt_n": 1214,
+              "prompt_ms": 1370.831,
+              "prompt_per_token_ms": 1.1291853377265237,
+              "prompt_per_second": 885.5942125615777,
+              "predicted_n": 160,
+              "predicted_ms": 4436.626,
+              "predicted_per_token_ms": 27.7289125,
+              "predicted_per_second": 36.06344100223909
+            }
+          },
+          "format_error": "JSONDecodeError",
+          "strict": false
+        }
+      ],
+      "accepted_for_delivery": false,
+      "response": {
+        "choices": [
+          {
+            "finish_reason": "stop",
+            "index": 0,
+            "message": {
+              "role": "assistant",
+              "content": "{\"answer\": \"Pre rok 2026 je pri osobe Ján Malinovský uvedený názov práce „Grafové vyhľadávanie pre podporu vzdelávania\" a rok 2026. Zdroj: https://zp.kemt.fei.tuke.sk/students/2022/jan_malinovsky\", \"status\": \"ok\", \"claims\": [{\"person\": \"Ján Malinovský\", \"title\": \"Grafové vyhľadávanie pre podporu vzdelávania\", \"year\": 2026}], \"source_urls\": [\"https://zp.kemt.fei.tuke.sk/students/2022/jan_malinovsky\"], \"count\": 1}"
+            }
+          }
+        ],
+        "created": 1792964327,
+        "model": "Qwen3.5-122B-A10B-Q4_K_M",
+        "system_fingerprint": "b8580-7c203670f",
+        "object": "chat.completion",
+        "usage": {
+          "completion_tokens": 160,
+          "prompt_tokens": 2372,
+          "total_tokens": 2532,
+          "prompt_tokens_details": {
+            "cached_tokens": 1158
+          }
+        },
+        "id": "chatcmpl-o2tXoEJknRgCWtI8dGVpNaSAln5cygJE",
+        "timings": {
+          "cache_n": 1158,
+          "prompt_n": 1214,
+          "prompt_ms": 1370.831,
+          "prompt_per_token_ms": 1.1291853377265237,
+          "prompt_per_second": 885.5942125615777,
+          "predicted_n": 160,
+          "predicted_ms": 4436.626,
+          "predicted_per_token_ms": 27.7289125,
+          "predicted_per_second": 36.06344100223909
+        }
+      },
+      "error": "InvalidAnswerFormat: two invalid JSON/schema responses",
+      "latency_seconds": 17.042694887000835
+    },
+    {
+      "id": "wide-143",
+      "category": "person_keyword_title",
+      "question": "Aký názov práce osoby Dominik Nagy v roku 2022 súvisí s kľúčovým slovom seq2seq?",
+      "graph_context": {
+        "status": "ok",
+        "reason": null,
+        "facts": [
+          {
+            "person": "Dominik Nagy",
+            "title": "Neurónový strojový preklad pomocou knižnice Fairseq",
+            "year": 2022,
+            "document_path": "pages/students/2016/dominik_nagy/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy"
+          },
+          {
+            "person": "Dominik Nagy",
+            "keyword": "seq2seq",
+            "document_path": "pages/students/2016/dominik_nagy/timovy_projekt/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy/timovy_projekt"
+          }
+        ],
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "pages/students/2016/dominik_nagy/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy",
+            "title": "Dominik Nagy"
+          },
+          {
+            "source_id": "S2",
+            "document_path": "pages/students/2016/dominik_nagy/timovy_projekt/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy/timovy_projekt",
+            "title": "Sequence-to-sequence"
+          },
+          {
+            "source_id": "S3",
+            "document_path": "pages/topics/seq2seq/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/topics/seq2seq",
+            "title": "Prepis postupností"
+          }
+        ],
+        "aggregation": null,
+        "complete": true,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+          "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+          "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+          "embedding_model": "intfloat/multilingual-e5-small",
+          "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+          "publication_policy": "explicit-false-hidden-v1",
+          "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+          "version": "10b78a8e70744e2792ffe4cf11d45571"
+        },
+        "answer_candidates": [
+          {
+            "person": "Dominik Nagy",
+            "title": "Neurónový strojový preklad pomocou knižnice Fairseq",
+            "year": 2022,
+            "keyword": "seq2seq",
+            "evidence_documents": [
+              "pages/students/2016/dominik_nagy/README.md",
+              "pages/students/2016/dominik_nagy/timovy_projekt/README.md"
+            ],
+            "source_urls": [
+              "https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy",
+              "https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy/timovy_projekt"
+            ]
+          }
+        ]
+      },
+      "strict": false,
+      "attempts": [
+        {
+          "response": {
+            "choices": [
+              {
+                "finish_reason": "stop",
+                "index": 0,
+                "message": {
+                  "role": "assistant",
+                  "content": "{\"answer\": \"Názov práce osoby Dominik Nagy z roku 2022 súvisiacej s kľúčovým slovom seq2seq je „Neurónový strojový preklad pomocou knižnice Fairseq\". Táto práca je doložená na stránke https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy a kľúčové slovo seq2seq je priradené k osobe Dominik Nagy v dokumente https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy/timovy_projekt.\", \"status\": \"ok\", \"claims\": [{\"person\": \"Dominik Nagy\", \"title\": \"Neurónový strojový preklad pomocou knižnice Fairseq\", \"year\": 2022, \"keyword\": \"seq2seq\"}], \"source_urls\": [\"https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy\", \"https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy/timovy_projekt\"], \"count\": 1}"
+                }
+              }
+            ],
+            "created": 1792964392,
+            "model": "Qwen3.5-122B-A10B-Q4_K_M",
+            "system_fingerprint": "b8580-7c203670f",
+            "object": "chat.completion",
+            "usage": {
+              "completion_tokens": 258,
+              "prompt_tokens": 2111,
+              "total_tokens": 2369,
+              "prompt_tokens_details": {
+                "cached_tokens": 1158
+              }
+            },
+            "id": "chatcmpl-rjU4N3OATV0ZzVklvDtgibjRjrtfEOtt",
+            "timings": {
+              "cache_n": 1158,
+              "prompt_n": 953,
+              "prompt_ms": 1158.656,
+              "prompt_per_token_ms": 1.2157985309548793,
+              "prompt_per_second": 822.5046950950066,
+              "predicted_n": 258,
+              "predicted_ms": 8424.558,
+              "predicted_per_token_ms": 32.65332558139535,
+              "predicted_per_second": 30.624752064143898
+            }
+          },
+          "format_error": "JSONDecodeError",
+          "strict": false
+        },
+        {
+          "response": {
+            "choices": [
+              {
+                "finish_reason": "stop",
+                "index": 0,
+                "message": {
+                  "role": "assistant",
+                  "content": "{\"answer\": \"Názov práce osoby Dominik Nagy z roku 2022 súvisiacej s kľúčovým slovom seq2seq je „Neurónový strojový preklad pomocou knižnice Fairseq\". Táto práca je doložená na stránke https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy a kľúčové slovo seq2seq je priradené k osobe Dominik Nagy v dokumente https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy/timovy_projekt.\", \"status\": \"ok\", \"claims\": [{\"person\": \"Dominik Nagy\", \"title\": \"Neurónový strojový preklad pomocou knižnice Fairseq\", \"year\": 2022, \"keyword\": \"seq2seq\"}], \"source_urls\": [\"https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy\", \"https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy/timovy_projekt\"], \"count\": 1}"
+                }
+              }
+            ],
+            "created": 1792964406,
+            "model": "Qwen3.5-122B-A10B-Q4_K_M",
+            "system_fingerprint": "b8580-7c203670f",
+            "object": "chat.completion",
+            "usage": {
+              "completion_tokens": 258,
+              "prompt_tokens": 2818,
+              "total_tokens": 3076,
+              "prompt_tokens_details": {
+                "cached_tokens": 1158
+              }
+            },
+            "id": "chatcmpl-DkJZ2MCjuA8xGFbo7Me8IzZYp8W2DEb2",
+            "timings": {
+              "cache_n": 1158,
+              "prompt_n": 1660,
+              "prompt_ms": 1732.792,
+              "prompt_per_token_ms": 1.0438506024096386,
+              "prompt_per_second": 957.9914958056131,
+              "predicted_n": 258,
+              "predicted_ms": 7834.101,
+              "predicted_per_token_ms": 30.364732558139533,
+              "predicted_per_second": 32.93294278437309
+            }
+          },
+          "format_error": "JSONDecodeError",
+          "strict": false
+        }
+      ],
+      "accepted_for_delivery": false,
+      "response": {
+        "choices": [
+          {
+            "finish_reason": "stop",
+            "index": 0,
+            "message": {
+              "role": "assistant",
+              "content": "{\"answer\": \"Názov práce osoby Dominik Nagy z roku 2022 súvisiacej s kľúčovým slovom seq2seq je „Neurónový strojový preklad pomocou knižnice Fairseq\". Táto práca je doložená na stránke https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy a kľúčové slovo seq2seq je priradené k osobe Dominik Nagy v dokumente https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy/timovy_projekt.\", \"status\": \"ok\", \"claims\": [{\"person\": \"Dominik Nagy\", \"title\": \"Neurónový strojový preklad pomocou knižnice Fairseq\", \"year\": 2022, \"keyword\": \"seq2seq\"}], \"source_urls\": [\"https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy\", \"https://zp.kemt.fei.tuke.sk/students/2016/dominik_nagy/timovy_projekt\"], \"count\": 1}"
+            }
+          }
+        ],
+        "created": 1792964406,
+        "model": "Qwen3.5-122B-A10B-Q4_K_M",
+        "system_fingerprint": "b8580-7c203670f",
+        "object": "chat.completion",
+        "usage": {
+          "completion_tokens": 258,
+          "prompt_tokens": 2818,
+          "total_tokens": 3076,
+          "prompt_tokens_details": {
+            "cached_tokens": 1158
+          }
+        },
+        "id": "chatcmpl-DkJZ2MCjuA8xGFbo7Me8IzZYp8W2DEb2",
+        "timings": {
+          "cache_n": 1158,
+          "prompt_n": 1660,
+          "prompt_ms": 1732.792,
+          "prompt_per_token_ms": 1.0438506024096386,
+          "prompt_per_second": 957.9914958056131,
+          "predicted_n": 258,
+          "predicted_ms": 7834.101,
+          "predicted_per_token_ms": 30.364732558139533,
+          "predicted_per_second": 32.93294278437309
+        }
+      },
+      "error": "InvalidAnswerFormat: two invalid JSON/schema responses",
+      "latency_seconds": 25.41826143900107
+    },
+    {
+      "id": "wide-176",
+      "category": "document_author_category_multi_hop",
+      "question": "Ktorý autor dokumentu je pri osobe Dárius Lindvai spojený s kategóriou tp2020?",
+      "graph_context": {
+        "status": "ok",
+        "reason": null,
+        "facts": [
+          {
+            "person": "Dárius Lindvai",
+            "category": "tp2020",
+            "document_path": "pages/students/2016/darius_lindvai/timovy_projekt/tutorial/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/tutorial"
+          },
+          {
+            "person": "Dárius Lindvai",
+            "document_author": "Dárius Lindvai",
+            "category": "tp2020",
+            "document_path": "pages/students/2016/darius_lindvai/timovy_projekt/resers/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/resers"
+          },
+          {
+            "person": "Dárius Lindvai",
+            "document_author": "Dárius Lindvai",
+            "category": "tp2020",
+            "document_path": "pages/students/2016/darius_lindvai/timovy_projekt/tutorial/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/tutorial"
+          }
+        ],
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "pages/students/2016/darius_lindvai/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai",
+            "title": "Dárius Lindvai"
+          },
+          {
+            "source_id": "S2",
+            "document_path": "pages/students/2016/darius_lindvai/timovy_projekt/resers/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/resers",
+            "title": "Novinky v oblasti “Punctuation Restoration”"
+          },
+          {
+            "source_id": "S3",
+            "document_path": "pages/students/2016/darius_lindvai/timovy_projekt/tutorial/README.md",
+            "source_url": "https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/tutorial",
+            "title": "PYTORCH - LSTM TUTORIÁL"
+          }
+        ],
+        "aggregation": null,
+        "complete": true,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+          "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+          "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+          "embedding_model": "intfloat/multilingual-e5-small",
+          "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+          "publication_policy": "explicit-false-hidden-v1",
+          "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+          "version": "10b78a8e70744e2792ffe4cf11d45571"
+        },
+        "answer_candidates": []
+      },
+      "strict": false,
+      "attempts": [
+        {
+          "response": {
+            "choices": [
+              {
+                "finish_reason": "stop",
+                "index": 0,
+                "message": {
+                  "role": "assistant",
+                  "content": "{\"answer\": \"Pri osobe Dárius Lindvai je s kategóriou tp2020 spojený dokumentový autor Dárius Lindvai, čo je doložené v dokumentoch s názvami „Novinky v oblasti “Punctuation Restoration”“ a „PYTORCH - LSTM TUTORIÁL\" (zdroje: https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/resers, https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/tutorial).\", \"status\": \"ok\", \"claims\": [{\"person\": \"Dárius Lindvai\", \"category\": \"tp2020\", \"document_author\": \"Dárius Lindvai\", \"title\": \"Novinky v oblasti “Punctuation Restoration”\"}, {\"person\": \"Dárius Lindvai\", \"category\": \"tp2020\", \"document_author\": \"Dárius Lindvai\", \"title\": \"PYTORCH - LSTM TUTORIÁL\"}], \"source_urls\": [\"https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/resers\", \"https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/tutorial\"], \"count\": 1}"
+                }
+              }
+            ],
+            "created": 1792964491,
+            "model": "Qwen3.5-122B-A10B-Q4_K_M",
+            "system_fingerprint": "b8580-7c203670f",
+            "object": "chat.completion",
+            "usage": {
+              "completion_tokens": 305,
+              "prompt_tokens": 2107,
+              "total_tokens": 2412,
+              "prompt_tokens_details": {
+                "cached_tokens": 1158
+              }
+            },
+            "id": "chatcmpl-WgV4U2DbFvSrlej5sazW7c1Do7YIgvd5",
+            "timings": {
+              "cache_n": 1158,
+              "prompt_n": 949,
+              "prompt_ms": 1163.485,
+              "prompt_per_token_ms": 1.2260115911485774,
+              "prompt_per_second": 815.6529736094578,
+              "predicted_n": 305,
+              "predicted_ms": 9590.628,
+              "predicted_per_token_ms": 31.444681967213118,
+              "predicted_per_second": 31.80187991860387
+            }
+          },
+          "format_error": "JSONDecodeError",
+          "strict": false
+        },
+        {
+          "response": {
+            "choices": [
+              {
+                "finish_reason": "stop",
+                "index": 0,
+                "message": {
+                  "role": "assistant",
+                  "content": "{\"answer\": \"Pri osobe Dárius Lindvai je s kategóriou tp2020 spojený dokumentový autor Dárius Lindvai, čo je doložené v dokumentoch s názvami „Novinky v oblasti “Punctuation Restoration”“ a „PYTORCH - LSTM TUTORIÁL\" (zdroje: https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/resers, https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/tutorial).\", \"status\": \"ok\", \"claims\": [{\"person\": \"Dárius Lindvai\", \"category\": \"tp2020\", \"document_author\": \"Dárius Lindvai\", \"title\": \"Novinky v oblasti “Punctuation Restoration”\"}, {\"person\": \"Dárius Lindvai\", \"category\": \"tp2020\", \"document_author\": \"Dárius Lindvai\", \"title\": \"PYTORCH - LSTM TUTORIÁL\"}], \"source_urls\": [\"https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/resers\", \"https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/tutorial\"], \"count\": 1}"
+                }
+              }
+            ],
+            "created": 1792964507,
+            "model": "Qwen3.5-122B-A10B-Q4_K_M",
+            "system_fingerprint": "b8580-7c203670f",
+            "object": "chat.completion",
+            "usage": {
+              "completion_tokens": 305,
+              "prompt_tokens": 2861,
+              "total_tokens": 3166,
+              "prompt_tokens_details": {
+                "cached_tokens": 1158
+              }
+            },
+            "id": "chatcmpl-Sh2RZILz46buMfHMXm2pL8Vi7Q8DatLf",
+            "timings": {
+              "cache_n": 1158,
+              "prompt_n": 1703,
+              "prompt_ms": 1698.632,
+              "prompt_per_token_ms": 0.9974351145038168,
+              "prompt_per_second": 1002.5714810506337,
+              "predicted_n": 305,
+              "predicted_ms": 9517.893,
+              "predicted_per_token_ms": 31.206206557377048,
+              "predicted_per_second": 32.04490741805986
+            }
+          },
+          "format_error": "JSONDecodeError",
+          "strict": false
+        }
+      ],
+      "accepted_for_delivery": false,
+      "response": {
+        "choices": [
+          {
+            "finish_reason": "stop",
+            "index": 0,
+            "message": {
+              "role": "assistant",
+              "content": "{\"answer\": \"Pri osobe Dárius Lindvai je s kategóriou tp2020 spojený dokumentový autor Dárius Lindvai, čo je doložené v dokumentoch s názvami „Novinky v oblasti “Punctuation Restoration”“ a „PYTORCH - LSTM TUTORIÁL\" (zdroje: https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/resers, https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/tutorial).\", \"status\": \"ok\", \"claims\": [{\"person\": \"Dárius Lindvai\", \"category\": \"tp2020\", \"document_author\": \"Dárius Lindvai\", \"title\": \"Novinky v oblasti “Punctuation Restoration”\"}, {\"person\": \"Dárius Lindvai\", \"category\": \"tp2020\", \"document_author\": \"Dárius Lindvai\", \"title\": \"PYTORCH - LSTM TUTORIÁL\"}], \"source_urls\": [\"https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/resers\", \"https://zp.kemt.fei.tuke.sk/students/2016/darius_lindvai/timovy_projekt/tutorial\"], \"count\": 1}"
+            }
+          }
+        ],
+        "created": 1792964507,
+        "model": "Qwen3.5-122B-A10B-Q4_K_M",
+        "system_fingerprint": "b8580-7c203670f",
+        "object": "chat.completion",
+        "usage": {
+          "completion_tokens": 305,
+          "prompt_tokens": 2861,
+          "total_tokens": 3166,
+          "prompt_tokens_details": {
+            "cached_tokens": 1158
+          }
+        },
+        "id": "chatcmpl-Sh2RZILz46buMfHMXm2pL8Vi7Q8DatLf",
+        "timings": {
+          "cache_n": 1158,
+          "prompt_n": 1703,
+          "prompt_ms": 1698.632,
+          "prompt_per_token_ms": 0.9974351145038168,
+          "prompt_per_second": 1002.5714810506337,
+          "predicted_n": 305,
+          "predicted_ms": 9517.893,
+          "predicted_per_token_ms": 31.206206557377048,
+          "predicted_per_second": 32.04490741805986
+        }
+      },
+      "error": "InvalidAnswerFormat: two invalid JSON/schema responses",
+      "latency_seconds": 30.148610364005435
+    }
+  ]
+}

tools/graphrag/evaluation/results/answers_v6_regression_2026-10-25/frozen_answers.json

@@ -0,0 +1,712 @@
+{
+  "method": "Previously exposed LLM questions; regression, not independent TEST.",
+  "graph_freeze_sha256": "54946b476fd713f551414b00cf2b843627566f6a333e3b59ce3efbbdf2328629",
+  "audit_sha256": "d5922db25b6d40cb7e5dc7df7990e93a3caee263a6e747ae910ab340da340023",
+  "generation_files": {
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/evaluate_graph_answers.py": "2553a0d17e164922b02046416fdf0d5df4496dc443466d8d9a1f4be86415b318",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/graph_answer_prompt.txt": "8c8274a22bf5b44364d8fc8226a8f659b31c232f5a2aba5916d693b559e9f341",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/answer_format.py": "86a27b4aebc6a5077e13794b81957a1cf063a1f3e070c583120ba53c1788d2f8",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/context_policy.py": "e7b7c511e5e9cafdef04969eb4007a6a61ac4b99bcc60e9e722945a15bec1f26",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/checked_generation.py": "7e017471e6281db62b79f270210db3ab14c7e31a4623bbfa3d9a5fd8911b1c20",
+    "tools/rag/evaluation/rag_runner.py": "031abf7d5b1a026eb199aa2a579bfb8baaa01175279d0018a364bfe50b76c3c1"
+  },
+  "model": "model120-fast",
+  "temperature": 0,
+  "workers": 3,
+  "dev_ids": [
+    "wide-001",
+    "wide-002",
+    "wide-041",
+    "wide-042",
+    "wide-080",
+    "wide-081",
+    "wide-082",
+    "wide-104",
+    "wide-113",
+    "wide-114",
+    "wide-116",
+    "wide-119",
+    "wide-129",
+    "wide-137",
+    "wide-138",
+    "wide-139",
+    "wide-141",
+    "wide-142",
+    "wide-143",
+    "wide-145",
+    "wide-155",
+    "wide-159",
+    "wide-161",
+    "wide-163",
+    "wide-165",
+    "wide-169",
+    "wide-170",
+    "wide-178",
+    "wide-181",
+    "wide-193",
+    "wide-194",
+    "wide-209",
+    "wide-217",
+    "wide-218",
+    "wide-232",
+    "wide-236",
+    "wide-239",
+    "wide-249",
+    "wide-250",
+    "wide-283",
+    "wide-289",
+    "wide-290"
+  ],
+  "test_ids": [
+    "wide-003",
+    "wide-004",
+    "wide-005",
+    "wide-006",
+    "wide-007",
+    "wide-008",
+    "wide-009",
+    "wide-010",
+    "wide-011",
+    "wide-012",
+    "wide-013",
+    "wide-014",
+    "wide-015",
+    "wide-016",
+    "wide-017",
+    "wide-018",
+    "wide-019",
+    "wide-020",
+    "wide-021",
+    "wide-022",
+    "wide-023",
+    "wide-024",
+    "wide-025",
+    "wide-026",
+    "wide-027",
+    "wide-028",
+    "wide-029",
+    "wide-030",
+    "wide-031",
+    "wide-032",
+    "wide-033",
+    "wide-034",
+    "wide-035",
+    "wide-036",
+    "wide-037",
+    "wide-038",
+    "wide-039",
+    "wide-040",
+    "wide-043",
+    "wide-044",
+    "wide-045",
+    "wide-046",
+    "wide-047",
+    "wide-048",
+    "wide-049",
+    "wide-050",
+    "wide-051",
+    "wide-052",
+    "wide-053",
+    "wide-054",
+    "wide-055",
+    "wide-056",
+    "wide-057",
+    "wide-058",
+    "wide-059",
+    "wide-060",
+    "wide-061",
+    "wide-062",
+    "wide-063",
+    "wide-064",
+    "wide-065",
+    "wide-066",
+    "wide-067",
+    "wide-068",
+    "wide-069",
+    "wide-070",
+    "wide-071",
+    "wide-072",
+    "wide-073",
+    "wide-074",
+    "wide-075",
+    "wide-076",
+    "wide-077",
+    "wide-078",
+    "wide-079",
+    "wide-080",
+    "wide-083",
+    "wide-084",
+    "wide-085",
+    "wide-086",
+    "wide-087",
+    "wide-088",
+    "wide-089",
+    "wide-090",
+    "wide-091",
+    "wide-092",
+    "wide-093",
+    "wide-094",
+    "wide-095",
+    "wide-096",
+    "wide-097",
+    "wide-098",
+    "wide-099",
+    "wide-100",
+    "wide-101",
+    "wide-102",
+    "wide-103",
+    "wide-104",
+    "wide-105",
+    "wide-106",
+    "wide-107",
+    "wide-108",
+    "wide-109",
+    "wide-110",
+    "wide-111",
+    "wide-112",
+    "wide-115",
+    "wide-116",
+    "wide-117",
+    "wide-118",
+    "wide-119",
+    "wide-120",
+    "wide-121",
+    "wide-122",
+    "wide-123",
+    "wide-124",
+    "wide-125",
+    "wide-126",
+    "wide-127",
+    "wide-128",
+    "wide-129",
+    "wide-130",
+    "wide-131",
+    "wide-132",
+    "wide-133",
+    "wide-134",
+    "wide-135",
+    "wide-136",
+    "wide-139",
+    "wide-140",
+    "wide-141",
+    "wide-142",
+    "wide-143",
+    "wide-144",
+    "wide-145",
+    "wide-146",
+    "wide-147",
+    "wide-148",
+    "wide-149",
+    "wide-150",
+    "wide-151",
+    "wide-152",
+    "wide-153",
+    "wide-154",
+    "wide-155",
+    "wide-156",
+    "wide-157",
+    "wide-158",
+    "wide-159",
+    "wide-160",
+    "wide-161",
+    "wide-162",
+    "wide-163",
+    "wide-164",
+    "wide-165",
+    "wide-166",
+    "wide-167",
+    "wide-168",
+    "wide-171",
+    "wide-172",
+    "wide-173",
+    "wide-174",
+    "wide-175",
+    "wide-176",
+    "wide-177",
+    "wide-178",
+    "wide-179",
+    "wide-180",
+    "wide-181",
+    "wide-182",
+    "wide-183",
+    "wide-184",
+    "wide-185",
+    "wide-186",
+    "wide-187",
+    "wide-188",
+    "wide-189",
+    "wide-190",
+    "wide-191",
+    "wide-192",
+    "wide-195",
+    "wide-196",
+    "wide-197",
+    "wide-198",
+    "wide-199",
+    "wide-200",
+    "wide-201",
+    "wide-202",
+    "wide-203",
+    "wide-204",
+    "wide-205",
+    "wide-206",
+    "wide-207",
+    "wide-208",
+    "wide-209",
+    "wide-210",
+    "wide-211",
+    "wide-212",
+    "wide-213",
+    "wide-214",
+    "wide-215",
+    "wide-216",
+    "wide-219",
+    "wide-220",
+    "wide-221",
+    "wide-222",
+    "wide-223",
+    "wide-224",
+    "wide-225",
+    "wide-226",
+    "wide-227",
+    "wide-228",
+    "wide-229",
+    "wide-230",
+    "wide-231",
+    "wide-232",
+    "wide-233",
+    "wide-234",
+    "wide-235",
+    "wide-236",
+    "wide-237",
+    "wide-238",
+    "wide-239",
+    "wide-240",
+    "wide-241",
+    "wide-242",
+    "wide-243",
+    "wide-244",
+    "wide-245",
+    "wide-246",
+    "wide-247",
+    "wide-248",
+    "wide-251",
+    "wide-252",
+    "wide-253",
+    "wide-254",
+    "wide-255",
+    "wide-256",
+    "wide-257",
+    "wide-258",
+    "wide-259",
+    "wide-260",
+    "wide-261",
+    "wide-262",
+    "wide-263",
+    "wide-264",
+    "wide-265",
+    "wide-266",
+    "wide-267",
+    "wide-268",
+    "wide-269",
+    "wide-270",
+    "wide-271",
+    "wide-272",
+    "wide-273",
+    "wide-274",
+    "wide-275",
+    "wide-276",
+    "wide-277",
+    "wide-278",
+    "wide-279",
+    "wide-280",
+    "wide-281",
+    "wide-282",
+    "wide-283",
+    "wide-284",
+    "wide-285",
+    "wide-286",
+    "wide-287",
+    "wide-288",
+    "wide-291",
+    "wide-292",
+    "wide-293",
+    "wide-294",
+    "wide-295",
+    "wide-296",
+    "wide-297",
+    "wide-298",
+    "wide-299",
+    "wide-300",
+    "wide-301",
+    "wide-302",
+    "wide-303",
+    "wide-304",
+    "wide-305",
+    "wide-306",
+    "wide-307",
+    "wide-308",
+    "wide-309",
+    "wide-310",
+    "wide-311",
+    "wide-312",
+    "wide-313",
+    "wide-314",
+    "wide-315",
+    "wide-316",
+    "wide-317",
+    "wide-318",
+    "wide-319",
+    "wide-320"
+  ],
+  "dev_count": 42,
+  "test_count": 300,
+  "target_accuracy": 0.92,
+  "selected_split": "regression",
+  "selected_ids": [
+    "wide-001",
+    "wide-002",
+    "wide-003",
+    "wide-004",
+    "wide-005",
+    "wide-006",
+    "wide-007",
+    "wide-008",
+    "wide-009",
+    "wide-010",
+    "wide-011",
+    "wide-012",
+    "wide-013",
+    "wide-014",
+    "wide-015",
+    "wide-016",
+    "wide-017",
+    "wide-018",
+    "wide-019",
+    "wide-020",
+    "wide-021",
+    "wide-022",
+    "wide-023",
+    "wide-024",
+    "wide-025",
+    "wide-026",
+    "wide-027",
+    "wide-028",
+    "wide-029",
+    "wide-030",
+    "wide-031",
+    "wide-032",
+    "wide-033",
+    "wide-034",
+    "wide-035",
+    "wide-036",
+    "wide-037",
+    "wide-038",
+    "wide-039",
+    "wide-040",
+    "wide-041",
+    "wide-042",
+    "wide-043",
+    "wide-044",
+    "wide-045",
+    "wide-046",
+    "wide-047",
+    "wide-048",
+    "wide-049",
+    "wide-050",
+    "wide-051",
+    "wide-052",
+    "wide-053",
+    "wide-054",
+    "wide-055",
+    "wide-056",
+    "wide-057",
+    "wide-058",
+    "wide-059",
+    "wide-060",
+    "wide-061",
+    "wide-062",
+    "wide-063",
+    "wide-064",
+    "wide-065",
+    "wide-066",
+    "wide-067",
+    "wide-068",
+    "wide-069",
+    "wide-070",
+    "wide-071",
+    "wide-072",
+    "wide-073",
+    "wide-074",
+    "wide-075",
+    "wide-076",
+    "wide-077",
+    "wide-078",
+    "wide-079",
+    "wide-080",
+    "wide-081",
+    "wide-082",
+    "wide-083",
+    "wide-084",
+    "wide-085",
+    "wide-086",
+    "wide-087",
+    "wide-088",
+    "wide-089",
+    "wide-090",
+    "wide-091",
+    "wide-092",
+    "wide-093",
+    "wide-094",
+    "wide-095",
+    "wide-096",
+    "wide-097",
+    "wide-098",
+    "wide-099",
+    "wide-100",
+    "wide-101",
+    "wide-102",
+    "wide-103",
+    "wide-104",
+    "wide-105",
+    "wide-106",
+    "wide-107",
+    "wide-108",
+    "wide-109",
+    "wide-110",
+    "wide-111",
+    "wide-112",
+    "wide-113",
+    "wide-114",
+    "wide-115",
+    "wide-116",
+    "wide-117",
+    "wide-118",
+    "wide-119",
+    "wide-120",
+    "wide-121",
+    "wide-122",
+    "wide-123",
+    "wide-124",
+    "wide-125",
+    "wide-126",
+    "wide-127",
+    "wide-128",
+    "wide-129",
+    "wide-130",
+    "wide-131",
+    "wide-132",
+    "wide-133",
+    "wide-134",
+    "wide-135",
+    "wide-136",
+    "wide-137",
+    "wide-138",
+    "wide-139",
+    "wide-140",
+    "wide-141",
+    "wide-142",
+    "wide-143",
+    "wide-144",
+    "wide-145",
+    "wide-146",
+    "wide-147",
+    "wide-148",
+    "wide-149",
+    "wide-150",
+    "wide-151",
+    "wide-152",
+    "wide-153",
+    "wide-154",
+    "wide-155",
+    "wide-156",
+    "wide-157",
+    "wide-158",
+    "wide-159",
+    "wide-160",
+    "wide-161",
+    "wide-162",
+    "wide-163",
+    "wide-164",
+    "wide-165",
+    "wide-166",
+    "wide-167",
+    "wide-168",
+    "wide-169",
+    "wide-170",
+    "wide-171",
+    "wide-172",
+    "wide-173",
+    "wide-174",
+    "wide-175",
+    "wide-176",
+    "wide-177",
+    "wide-178",
+    "wide-179",
+    "wide-180",
+    "wide-181",
+    "wide-182",
+    "wide-183",
+    "wide-184",
+    "wide-185",
+    "wide-186",
+    "wide-187",
+    "wide-188",
+    "wide-189",
+    "wide-190",
+    "wide-191",
+    "wide-192",
+    "wide-193",
+    "wide-194",
+    "wide-195",
+    "wide-196",
+    "wide-197",
+    "wide-198",
+    "wide-199",
+    "wide-200",
+    "wide-201",
+    "wide-202",
+    "wide-203",
+    "wide-204",
+    "wide-205",
+    "wide-206",
+    "wide-207",
+    "wide-208",
+    "wide-209",
+    "wide-210",
+    "wide-211",
+    "wide-212",
+    "wide-213",
+    "wide-214",
+    "wide-215",
+    "wide-216",
+    "wide-217",
+    "wide-218",
+    "wide-219",
+    "wide-220",
+    "wide-221",
+    "wide-222",
+    "wide-223",
+    "wide-224",
+    "wide-225",
+    "wide-226",
+    "wide-227",
+    "wide-228",
+    "wide-229",
+    "wide-230",
+    "wide-231",
+    "wide-232",
+    "wide-233",
+    "wide-234",
+    "wide-235",
+    "wide-236",
+    "wide-237",
+    "wide-238",
+    "wide-239",
+    "wide-240",
+    "wide-241",
+    "wide-242",
+    "wide-243",
+    "wide-244",
+    "wide-245",
+    "wide-246",
+    "wide-247",
+    "wide-248",
+    "wide-249",
+    "wide-250",
+    "wide-251",
+    "wide-252",
+    "wide-253",
+    "wide-254",
+    "wide-255",
+    "wide-256",
+    "wide-257",
+    "wide-258",
+    "wide-259",
+    "wide-260",
+    "wide-261",
+    "wide-262",
+    "wide-263",
+    "wide-264",
+    "wide-265",
+    "wide-266",
+    "wide-267",
+    "wide-268",
+    "wide-269",
+    "wide-270",
+    "wide-271",
+    "wide-272",
+    "wide-273",
+    "wide-274",
+    "wide-275",
+    "wide-276",
+    "wide-277",
+    "wide-278",
+    "wide-279",
+    "wide-280",
+    "wide-281",
+    "wide-282",
+    "wide-283",
+    "wide-284",
+    "wide-285",
+    "wide-286",
+    "wide-287",
+    "wide-288",
+    "wide-289",
+    "wide-290",
+    "wide-291",
+    "wide-292",
+    "wide-293",
+    "wide-294",
+    "wide-295",
+    "wide-296",
+    "wide-297",
+    "wide-298",
+    "wide-299",
+    "wide-300",
+    "wide-301",
+    "wide-302",
+    "wide-303",
+    "wide-304",
+    "wide-305",
+    "wide-306",
+    "wide-307",
+    "wide-308",
+    "wide-309",
+    "wide-310",
+    "wide-311",
+    "wide-312",
+    "wide-313",
+    "wide-314",
+    "wide-315",
+    "wide-316",
+    "wide-317",
+    "wide-318",
+    "wide-319",
+    "wide-320"
+  ],
+  "selected_count": 320,
+  "selected_dev_overlap": 42,
+  "corrective_request_limit": 1,
+  "format_repair_limit": 1,
+  "repair_policy": "At most one total corrective request for format, citations or retrieval-context violations. Same question/context, no gold; retain all attempts.",
+  "dev_results_sha256": "f70834bf4ee1fe106c0849ab56e0758dd50e07dfa9bbc25b05ae4d4e811a4edd",
+  "policy": "Evaluate every declared regression case, including DEV overlap; retain all failures and stop tuning after this freeze.",
+  "selected_categories": {
+    "shared_keyword_pair": 40,
+    "same_academic_year_pair": 40,
+    "same_start_year_pair": 32,
+    "person_title_year": 24,
+    "person_keyword_title": 32,
+    "document_author_category_multi_hop": 24,
+    "person_multiple_categories": 24,
+    "candidate_set_aggregation": 32,
+    "conjunctive_disambiguation": 40,
+    "negative_lookup": 32
+  },
+  "limitations": [
+    "All questions and previous model outputs are known; this is historical regression.",
+    "Harness supplies graph context; native OpenWebUI tool choice and browser UI are outside this experiment.",
+    "Checks structured factual claims, their source support/citations and required prose values; not a general semantic judge of arbitrary prose."
+  ]
+}

tools/graphrag/evaluation/results/final_v6_2026-10-25/REPORT.md

@@ -0,0 +1,139 @@
+# GraphRAG v6 — finálne kontroly bodu 3
+
+Grafový DEV má **722/722** a historická regresia **320/320**, vrátane nezávislej
+zdrojovej kontroly všetkých vrátených typovaných faktov. Generované odpovede
+majú strict **316/320 — 98,75 %**. Cieľ ≥92 % je splnený.
+
+Ide o **historickú regresiu**, nie nový nezávislý TEST. Všetky otázky sú známe;
+42 DEV otázok je zahrnutých aj vo finálnych 320. Nejde o všeobecnú pravdivosť
+agenta ani o meranie natívneho výberu OpenWebUI nástrojov.
+
+## Výsledky
+
+| Meranie | Výsledok | Úspešnosť |
+|---|---:|---:|
+| Graf DEV | 722/722 | 100,00 % |
+| Graf regresia | 320/320 | 100,00 % |
+| Zdrojovo podporené typované fakty v grafových regresných výstupoch | 320/320 | 100,00 % |
+| LLM DEV | 40/42 | 95,24 % |
+| LLM regresia — prvý pokus | 316/320 | 98,75 % |
+| LLM regresia — po deklarovaných kontrolách/opravách | 316/320 | 98,75 % |
+| Parseovateľné odpovede so zdrojovo podporenými štruktúrovanými tvrdeniami | 317/320 | 99,06 % |
+| Syntetické živé funkčné scenáre, mimo benchmarku | 9/9 | 100,00 % |
+| Automatizované testy vrátane všetkých živých testov | 565 | všetky prešli |
+
+Tri otázky použili jediný povolený opravný požiadavok pre neplatný JSON;
+ani druhá odpoveď neprešla schémou. Opravný pokus vyvolaný porušením
+kontextu sa v tejto finálnej regresii nepoužil. Žiadne prvé JSON odpovede
+nepotrebovali odbaľovanie Markdown bloku. Všetky pokusy sú uložené.
+
+| Kategória generovania | Strict správne | Úspešnosť |
+|---|---:|---:|
+| `candidate_set_aggregation` | 32/32 | 100.00 % |
+| `conjunctive_disambiguation` | 40/40 | 100.00 % |
+| `document_author_category_multi_hop` | 23/24 | 95.83 % |
+| `negative_lookup` | 32/32 | 100.00 % |
+| `person_keyword_title` | 31/32 | 96.88 % |
+| `person_multiple_categories` | 24/24 | 100.00 % |
+| `person_title_year` | 23/24 | 95.83 % |
+| `same_academic_year_pair` | 40/40 | 100.00 % |
+| `same_start_year_pair` | 31/32 | 96.88 % |
+| `shared_keyword_pair` | 40/40 | 100.00 % |
+
+## Štyri strict zlyhania
+
+- `wide-104`: text obsahuje skloňované „Martina NaN1“, protokol vyžaduje
+  pôvodné „Martin NaN1“. Štruktúrované väzby a zdroje prešli; ide o
+  konzervatívne odmietnutie povrchového tvaru mena. Neoznačujeme ho za
+  fakticky nesprávny rok. Zmrazený primárny strict výsledok nemeníme.
+- `wide-119`, `wide-143`, `wide-176`: dva neplatné JSON/schema výstupy.
+  `accepted_for_delivery=false`; odpovede neprešli na doručenie. Bez
+  parseovateľného výstupu sa správnosť tvrdení nedá spoľahlivo vyhodnotiť.
+
+Úplné zlyhania aj kontexty sú v
+[failures.json](../answers_v6_regression_2026-10-25/failures.json).
+Kód ani prompt grafového generovania sa podľa tohto finálneho výsledku neladil.
+
+## Zdrojový benchmark
+
+V6 zachováva 1 042 otázok a všetky ID, formulácie, kategórie a rozdelenie.
+Priame čítanie odhalilo generické linkové označenia „rešerš“, ktoré sa
+nesprávne vydávali za akademický názov. Opravených je 15 očakávaní; štyri
+regresné otázky `wide-116`, `wide-142`, `wide-145`, `wide-165` ostali v
+menovateli a očakávajú `no_answer`. Regresia má 284 pozitívnych a 36
+negatívnych očakávaní. Pôvodná v4 má 288/32 a zostáva nezmenená.
+Zdrojový audit všetkých 1 042 aktuálnych otázok má **0 nesúladov**.
+
+Nezávislé priame čítanie vzorky 34 otázok vykonala **AI** oddelene od
+retrievera: 31 podporených, dve opravené a jedna podmienená ročníkom
+adresára. Pri `wide-112` Tomáš Kuchárik nemá explicitne uvedený začiatok
+štúdia; používa sa zavedený fallback `students/2016`. Ide o konvenciu,
+ktorú má človek potvrdiť pred výkladom ako administratívny fakt.
+[Hárok pre človeka](../v6_preparation/source_review.csv) je pripravený;
+**ľudské posúdenie neprebehlo**.
+
+## Kontext, citácie a RAG
+
+Kontext obsahuje úplnosť, orezanie, dôvody a verziu dát. Model má jasné pokyny
+pre neúplné výsledky a presný agregovaný počet s orezanými dôkazmi. Guard
+kontroluje URL v zozname aj v texte oproti zdrojom konkrétneho dotazu,
+štruktúrované hodnoty a spojenie názvu/roku s tagom cez tú istú osobu.
+Nepoužíva gold. Najviac jeden spoločný opravný požiadavok rieši formát
+alebo porušenie kontextu; oracle vstupuje až do následného hodnotenia.
+
+Prepočet 56 archivovaných súborov skontroloval 6 044 uložených odpovedí,
+vrátane opakovaných experimentov. Našiel päť výskytov nepovolenej URL;
+nejde o 6 044 nezávislých otázok. Historický finálny RAG DEV ostáva
+624/629, finálny RAG TEST po prísnej kontrole URL má **506/519 — 97,50 %**,
+pôvodné skóre bolo 507/519. Súbory ani pôvodné metriky sa neprepisovali.
+Pri `q0892` navyše uvedená `http://skner.tukekemt.xyz` v Markdowne reálne
+existuje, ale nie je medzi archivovanými `source_url`; prísne odmietnutie
+odkazu nie je dôkazom nepravdivosti tej adresy. [Celý prepočet](../v6_preparation/citation_regrade.json).
+
+Živá RAG vzorka piatich vopred zvolených otázok mala 5/5 správnych faktov
+aj citácií, ale 5/5 vynechaných upozornení. Vrstva `finalize_rag_answer()`
+preto vie pridať iba pravdivé upozornenie, bez zmeny faktov; uchová
+`model_answer`, pôvodné dôvody a príznak `display_notice_added`. Neobíde
+neznámu citáciu ani nepodložené tvrdenie o úplnom zozname. Prehratie tých
+istých uložených výstupov má 5/5 prijateľných odpovedí. **Nie je to nové
+meranie modelu ani odhad jeho presnosti z piatich prípadov.**
+
+## História a porovnateľnosť
+
+| Verzia | Grafové meranie | Generovanie |
+|---|---|---|
+| v4 | TEST 320/320; dodatočné faktové overenie 319/320 | nový LLM TEST 281/300 |
+| v5.1 | historická regresia 320/320 so všetkými typovanými faktami | regresia 317/320 |
+| v6 | historická regresia 320/320 so všetkými typovanými faktami | strict regresia 316/320 |
+
+V6 opravuje 15 gold očakávaní a pridáva kontroly; rozdiel percent nie je
+kontrolovaný odhad zlepšenia/zhoršenia modelu. Všetky v1–v5.1 výstupy,
+klasický RAG, retrieval baseline aj vybrané experimenty ostávajú zachované.
+Prvý v6 DEV kandidát 41/42 je uložený; po oprave odmietnutého spojenia tagu
+a roku bol finálny DEV 40/42 s dvomi JSON chybami. Všetky iterácie sú
+priznané a žiadne chybné otázky sa nevyberali z finálneho merania.
+
+## Nasadenie a ďalší bod
+
+Nasadené Docker API, SQLite a Neo4j používajú spoločnú verziu
+`10b78a8e70744e2792ffe4cf11d45571`, wiki Git
+`3e892a1d49e9bce118cd03478ce15e01c2bebb4e`.
+Graf má 446 uzlov a 877 vzťahov; index 114 dokumentov a 887 chunkov.
+Predvolene je viditeľných 113 dokumentov: 16 bez published sa zahrnie,
+jediný explicitný published=false sa vylúči. Kontrola porovnáva
+normalizované Neo4j hodnoty s pôvodnými JSON metadátami.
+
+[Overenie hostiteľa](deployment_check_host.json) a
+[overenie v Docker obraze](deployment_check_docker.json) kontrolujú odtlačky,
+pripravenosť RAG/Neo4j/GraphRAG, skutočné RAG API, grafové pozitívne aj
+negatívne dotazy a spoločnú verziu. `frozen_v6.json` fixuje graf,
+`frozen_answers.json` parametre generovania a `frozen_release_v6.json`
+všetky runtime moduly vrátane samostatnej RAG prezentačnej vrstvy.
+
+Bod 4 pridá samostatne inštalovateľné OpenWebUI nástroje a hlavného agenta.
+Musí používať existujúce guardy a rešpektovať odmietnutie doručenia aj
+úplnosť zdrojov. Natívny výber a kombinovanie nástrojov sa v dnešnom
+320-otázkovom grafovom meraní netestovali. Pre nový nezávislý odhad treba
+nové vopred overené otázky; ľudská kontrola pripraveného hárka je odporúčaná.
+
+[Príkazy a presné zmeny](../../../../../docs/graphrag_v6_2026-10-25_worklog.md).

tools/graphrag/evaluation/results/final_v6_2026-10-25/deployment_check_docker.json

@@ -0,0 +1,115 @@
+{
+  "checked_at_utc": "2026-10-25T21:53:22.539832+00:00",
+  "health": {
+    "status": "ok",
+    "ready": true,
+    "database_exists": true,
+    "search_engine": "hybrid_fts5_embeddings",
+    "rag_enabled": true,
+    "zpwiki_exists": true,
+    "security_configured": true,
+    "index_version": {
+      "version": "10b78a8e70744e2792ffe4cf11d45571",
+      "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+      "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+      "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+      "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+      "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+      "embedding_model": "intfloat/multilingual-e5-small",
+      "publication_policy": "explicit-false-hidden-v1"
+    },
+    "tools": {
+      "rag": {
+        "ready": true,
+        "status": "ready",
+        "reason": null,
+        "embedding_model": "intfloat/multilingual-e5-small",
+        "dimensions": 384,
+        "index_version": {
+          "version": "10b78a8e70744e2792ffe4cf11d45571",
+          "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+          "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+          "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+          "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+          "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+          "embedding_model": "intfloat/multilingual-e5-small",
+          "publication_policy": "explicit-false-hidden-v1"
+        }
+      },
+      "neo4j": {
+        "ready": true,
+        "status": "ready",
+        "reason": null
+      },
+      "graphrag": {
+        "ready": true,
+        "status": "ready",
+        "reason": null,
+        "index_version": {
+          "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+          "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+          "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+          "embedding_model": "intfloat/multilingual-e5-small",
+          "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+          "publication_policy": "explicit-false-hidden-v1",
+          "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+          "version": "10b78a8e70744e2792ffe4cf11d45571"
+        },
+        "nodes": 446
+      }
+    }
+  },
+  "index_version": {
+    "version": "10b78a8e70744e2792ffe4cf11d45571",
+    "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+    "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+    "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+    "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+    "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+    "embedding_model": "intfloat/multilingual-e5-small",
+    "publication_policy": "explicit-false-hidden-v1"
+  },
+  "frozen_implementation_inputs_and_graph_match": true,
+  "graph": {
+    "sha256": "6fd45dc58382641072d991a088fcdfcd1c0e81ce912befbae6fca700d630fe01",
+    "nodes": 446,
+    "relationships": 877
+  },
+  "graph_cases": {
+    "known_title": {
+      "status": "ok",
+      "reason": null,
+      "complete": true
+    },
+    "generic_label_not_title": {
+      "status": "no_answer",
+      "reason": "no_matching_facts",
+      "complete": true
+    },
+    "unknown_person": {
+      "status": "no_answer",
+      "reason": "unknown_person",
+      "complete": true
+    }
+  },
+  "rag_live_query": "ok",
+  "release_code_match": true,
+  "rag_context_metadata_matches_contract": true,
+  "publication": {
+    "documents": 114,
+    "explicit_false": 1,
+    "missing_included": 16,
+    "visible_by_policy": 113,
+    "hidden_document_absent_from_rag": true
+  },
+  "runtime": {
+    "python": "3.11.17",
+    "fastapi": "0.136.3",
+    "pydantic": "2.13.4",
+    "neo4j": "6.3.1",
+    "sentence-transformers": "5.7.0",
+    "torch": "2.11.0+cpu"
+  },
+  "scope": "RAG HTTP API plus GraphRAG library/CLI; native OpenWebUI graph-tool routing is point 4.",
+  "docker_image_id": "sha256:a4b97798ff2082bb1e7ea9bd2b1a5666dda4940d118cef3420077b6d38983fea"
+}

tools/graphrag/evaluation/results/final_v6_2026-10-25/deployment_check_host.json

@@ -0,0 +1,114 @@
+{
+  "checked_at_utc": "2026-10-25T21:52:07.547882+00:00",
+  "health": {
+    "status": "ok",
+    "ready": true,
+    "database_exists": true,
+    "search_engine": "hybrid_fts5_embeddings",
+    "rag_enabled": true,
+    "zpwiki_exists": true,
+    "security_configured": true,
+    "index_version": {
+      "version": "10b78a8e70744e2792ffe4cf11d45571",
+      "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+      "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+      "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+      "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+      "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+      "embedding_model": "intfloat/multilingual-e5-small",
+      "publication_policy": "explicit-false-hidden-v1"
+    },
+    "tools": {
+      "rag": {
+        "ready": true,
+        "status": "ready",
+        "reason": null,
+        "embedding_model": "intfloat/multilingual-e5-small",
+        "dimensions": 384,
+        "index_version": {
+          "version": "10b78a8e70744e2792ffe4cf11d45571",
+          "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+          "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+          "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+          "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+          "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+          "embedding_model": "intfloat/multilingual-e5-small",
+          "publication_policy": "explicit-false-hidden-v1"
+        }
+      },
+      "neo4j": {
+        "ready": true,
+        "status": "ready",
+        "reason": null
+      },
+      "graphrag": {
+        "ready": true,
+        "status": "ready",
+        "reason": null,
+        "index_version": {
+          "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+          "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+          "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+          "embedding_model": "intfloat/multilingual-e5-small",
+          "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+          "publication_policy": "explicit-false-hidden-v1",
+          "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+          "version": "10b78a8e70744e2792ffe4cf11d45571"
+        },
+        "nodes": 446
+      }
+    }
+  },
+  "index_version": {
+    "version": "10b78a8e70744e2792ffe4cf11d45571",
+    "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+    "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+    "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+    "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+    "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+    "embedding_model": "intfloat/multilingual-e5-small",
+    "publication_policy": "explicit-false-hidden-v1"
+  },
+  "frozen_implementation_inputs_and_graph_match": true,
+  "graph": {
+    "sha256": "6fd45dc58382641072d991a088fcdfcd1c0e81ce912befbae6fca700d630fe01",
+    "nodes": 446,
+    "relationships": 877
+  },
+  "graph_cases": {
+    "known_title": {
+      "status": "ok",
+      "reason": null,
+      "complete": true
+    },
+    "generic_label_not_title": {
+      "status": "no_answer",
+      "reason": "no_matching_facts",
+      "complete": true
+    },
+    "unknown_person": {
+      "status": "no_answer",
+      "reason": "unknown_person",
+      "complete": true
+    }
+  },
+  "rag_live_query": "ok",
+  "release_code_match": true,
+  "rag_context_metadata_matches_contract": true,
+  "publication": {
+    "documents": 114,
+    "explicit_false": 1,
+    "missing_included": 16,
+    "visible_by_policy": 113,
+    "hidden_document_absent_from_rag": true
+  },
+  "runtime": {
+    "python": "3.10.12",
+    "fastapi": "0.136.3",
+    "pydantic": "2.13.4",
+    "neo4j": "6.3.1",
+    "sentence-transformers": "5.7.0",
+    "torch": "2.11.0+cpu"
+  },
+  "scope": "RAG HTTP API plus GraphRAG library/CLI; native OpenWebUI graph-tool routing is point 4."
+}

tools/graphrag/evaluation/results/final_v6_2026-10-25/summary.json

@@ -0,0 +1,188 @@
+{
+  "evaluation_kind": "historical_regression",
+  "model": "model120-fast",
+  "graph_dev": {
+    "questions": 722,
+    "source_checked_count": 722,
+    "all_facts_supported_count": 722,
+    "strict_count": 722,
+    "strict_accuracy": 1.0,
+    "error_count": 0,
+    "value_score": 1.0,
+    "entity_score": 1.0,
+    "relation_score": 1.0,
+    "property_score": 1.0,
+    "relation_choice_score": 1.0,
+    "document_recall": 1.0,
+    "source_recall": 1.0,
+    "binding_score": 1.0,
+    "path_score": 1.0,
+    "answerability_accuracy": 1.0,
+    "latency_mean_seconds": 0.051461,
+    "latency_median_seconds": 0.04753,
+    "latency_p95_seconds": 0.079209
+  },
+  "graph_regression": {
+    "questions": 320,
+    "source_checked_count": 320,
+    "all_facts_supported_count": 320,
+    "strict_count": 320,
+    "strict_accuracy": 1.0,
+    "error_count": 0,
+    "value_score": 1.0,
+    "entity_score": 1.0,
+    "relation_score": 1.0,
+    "property_score": 1.0,
+    "relation_choice_score": 1.0,
+    "document_recall": 1.0,
+    "source_recall": 1.0,
+    "binding_score": 1.0,
+    "path_score": 1.0,
+    "answerability_accuracy": 1.0,
+    "latency_mean_seconds": 0.053195,
+    "latency_median_seconds": 0.049496,
+    "latency_p95_seconds": 0.075778
+  },
+  "llm_dev": {
+    "questions": 42,
+    "correct": 40,
+    "accuracy": 0.9523809523809523,
+    "errors": 2,
+    "first_pass_correct": 40,
+    "format_repair_attempted": 2,
+    "corrective_request_count": 2,
+    "context_repair_attempted": 0,
+    "normalized_first_response": 0
+  },
+  "llm_regression": {
+    "questions": 320,
+    "correct": 316,
+    "accuracy": 0.9875,
+    "errors": 3,
+    "first_pass_correct": 316,
+    "format_repair_attempted": 3,
+    "corrective_request_count": 3,
+    "context_repair_attempted": 0,
+    "normalized_first_response": 0
+  },
+  "llm_by_category": {
+    "candidate_set_aggregation": {
+      "questions": 32,
+      "correct": 32,
+      "accuracy": 1.0
+    },
+    "conjunctive_disambiguation": {
+      "questions": 40,
+      "correct": 40,
+      "accuracy": 1.0
+    },
+    "document_author_category_multi_hop": {
+      "questions": 24,
+      "correct": 23,
+      "accuracy": 0.9583333333333334
+    },
+    "negative_lookup": {
+      "questions": 32,
+      "correct": 32,
+      "accuracy": 1.0
+    },
+    "person_keyword_title": {
+      "questions": 32,
+      "correct": 31,
+      "accuracy": 0.96875
+    },
+    "person_multiple_categories": {
+      "questions": 24,
+      "correct": 24,
+      "accuracy": 1.0
+    },
+    "person_title_year": {
+      "questions": 24,
+      "correct": 23,
+      "accuracy": 0.9583333333333334
+    },
+    "same_academic_year_pair": {
+      "questions": 40,
+      "correct": 40,
+      "accuracy": 1.0
+    },
+    "same_start_year_pair": {
+      "questions": 32,
+      "correct": 31,
+      "accuracy": 0.96875
+    },
+    "shared_keyword_pair": {
+      "questions": 40,
+      "correct": 40,
+      "accuracy": 1.0
+    }
+  },
+  "valid_json_responses": 317,
+  "source_supported_structured_claims": 317,
+  "tests": {
+    "passed": 565,
+    "skipped": 0,
+    "warning": "Existing Starlette/httpx deprecation warning."
+  },
+  "synthetic_functional_checks": {
+    "passed": 9,
+    "total": 9
+  },
+  "rag_live_smoke": {
+    "total": 5,
+    "factual_and_citation_passes": 5,
+    "raw_delivery_accepted": 0,
+    "missing_notice": 5,
+    "deterministic_delivery_replay_passes": 5,
+    "model_accuracy_estimate": false
+  },
+  "source_review": {
+    "total": 34,
+    "supported": 31,
+    "corrected": 2,
+    "conditional_directory_cohort": 1,
+    "human_review": "not performed; worksheet available"
+  },
+  "gold_corrections": 15,
+  "regression_positive": 284,
+  "regression_negative": 36,
+  "llm_failure_details": [
+    {
+      "id": "wide-104",
+      "error": null,
+      "missing_text_values": [
+        "Martin NaN1"
+      ],
+      "source_supported_claims": true
+    },
+    {
+      "id": "wide-119",
+      "error": "InvalidAnswerFormat: two invalid JSON/schema responses",
+      "missing_text_values": null,
+      "source_supported_claims": false
+    },
+    {
+      "id": "wide-143",
+      "error": "InvalidAnswerFormat: two invalid JSON/schema responses",
+      "missing_text_values": null,
+      "source_supported_claims": false
+    },
+    {
+      "id": "wide-176",
+      "error": "InvalidAnswerFormat: two invalid JSON/schema responses",
+      "missing_text_values": null,
+      "source_supported_claims": false
+    }
+  ],
+  "target_accuracy": 0.92,
+  "target_met": true,
+  "limitations": [
+    "Known questions and overlapping DEV; not independent TEST.",
+    "15 source-gold corrections; accuracy change vs v5.1 is not controlled comparison.",
+    "Direct source review by AI, not a human. One study-start expectation uses directory-cohort fallback.",
+    "Strict lexical name check rejects a grammatical inflection; structured claims remain supported.",
+    "Graph contexts supplied by evaluation client; native OpenWebUI tool routing belongs to point 4.",
+    "URL allowlist is stricter than factual truth of URLs mentioned inside source documents.",
+    "Rules check structured claims and selected prose requirements, not every possible natural-language assertion."
+  ]
+}

tools/graphrag/evaluation/results/final_v6_2026-10-25/verification.json

@@ -0,0 +1,50 @@
+{
+  "full_test_command": "RUN_NEO4J_TESTS=1 RUN_LIVE_TESTS=1 RUN_LIVE_RAG_E2E=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 python -m pytest -q",
+  "passed": 565,
+  "skipped": 0,
+  "logs": [
+    {
+      "name": "zp-v6-release-tests.log",
+      "sha256": "22b0db8cf0e707463f703aa4de9753b5b1dd269c3062bc5f7c9b27b991bc774e",
+      "output": "........................................................................ [ 12%]\n........................................................................ [ 25%]\n........................................................................ [ 38%]\n........................................................................ [ 50%]\n........................................................................ [ 63%]\n........................................................................ [ 76%]\n........................................................................ [ 89%]\n.............................................................            [100%]\n=============================== warnings summary ===============================\n.venv/lib/python3.10/site-packages/fastapi/testclient.py:1\n  /home/janko/DP/zp-agent/.venv/lib/python3.10/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.\n    from starlette.testclient import TestClient as TestClient  # noqa\n\n-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html\n565 passed, 1 warning in 42.49s\n"
+    },
+    {
+      "name": "zp-v6-rag-entry-tests.log",
+      "sha256": "e64b7642f5e893e83001de48f6d0e786cf6e8cfd9afb61c5fba3647ff82173c4",
+      "output": "..................                                                       [100%]\n18 passed in 1.80s\n"
+    },
+    {
+      "name": "zp-v6-live-index-tests.log",
+      "sha256": "06a32f40463e14518133b3b557dd6b2cf45ae490da54e390ddcbd5f19582e8ee",
+      "output": "....                                                                     [100%]\n=============================== warnings summary ===============================\n.venv/lib/python3.10/site-packages/fastapi/testclient.py:1\n  /home/janko/DP/zp-agent/.venv/lib/python3.10/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.\n    from starlette.testclient import TestClient as TestClient  # noqa\n\n-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html\n4 passed, 1 warning in 24.21s\n"
+    }
+  ],
+  "direct_script_help": "python tools/rag/evaluation/evaluate_rag_answers.py --help passed",
+  "sync_response": {
+    "status": "ok",
+    "pull_git": false,
+    "duration_seconds": 153.12,
+    "counts": {
+      "documents": 114,
+      "chunks": 887,
+      "fts_chunks": 887,
+      "embedding_chunks": 887,
+      "tags": 2189,
+      "categories": 1193
+    },
+    "index_version": {
+      "version": "10b78a8e70744e2792ffe4cf11d45571",
+      "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+      "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+      "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+      "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+      "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+      "embedding_model": "intfloat/multilingual-e5-small",
+      "publication_policy": "explicit-false-hidden-v1"
+    },
+    "graph_counts": {
+      "nodes": 446,
+      "relationships": 877
+    }
+  }
+}

tools/graphrag/evaluation/results/v6_preparation/candidate_to_final.patch

@@ -0,0 +1,118 @@
+--- a/tools/answer_evaluation/context_policy.py
++++ b/tools/answer_evaluation/context_policy.py
+@@ -32,17 +32,20 @@
+ 
+ 
+ def joined_candidates(facts: list[dict], intent: dict) -> list[dict]:
+-    """Join title/year and requested tag by person, retaining both real sources."""
+-    if not intent.get('mentions_title') or not (intent.get('mentions_keyword') or intent.get('keyword_phrase')):
++    """Join requested academic fields and tag by person, retaining real sources."""
++    needs_title = intent.get('mentions_title')
++    needs_year = intent.get('mentions_academic_year') or intent.get('years')
++    if not (needs_title or needs_year) or not (intent.get('mentions_keyword') or intent.get('keyword_phrase')):
+         return []
+     result=[]
+     for record in facts:
+-        if not record.get('title'):
++        if (needs_title and not record.get('title')) or (not needs_title and record.get('year') is None):
+             continue
+         for tag in facts:
+             if normalized(tag.get('person'))!=normalized(record.get('person')) or not tag.get('keyword'):
+                 continue
+-            candidate={k:record[k] for k in ['person','title','year'] if record.get(k) is not None}
++            fields = ['person', 'title', 'year'] if needs_title else ['person', 'year']
++            candidate={k:record[k] for k in fields if record.get(k) is not None}
+             candidate['keyword']=tag['keyword']
+             candidate['evidence_documents']=sorted({p for p in [record.get('document_path'),tag.get('document_path')] if p})
+             candidate['source_urls']=sorted({u for u in [record.get('source_url'),tag.get('source_url')] if u})
+--- a/tools/answer_evaluation/checked_generation.py
++++ b/tools/answer_evaluation/checked_generation.py
+@@ -40,7 +40,7 @@
+                 return result
+             feedback=('Odpoveď porušila pravidlá dodaného kontextu: '+', '.join(issues)+'. '
+                 'Oprav jediný JSON objekt výhradne z pôvodných podkladov. Použi iba source_urls zo sources. '
+-                'Typ práce nie je keyword. answer_candidates dokladajú spojenie názvu, roku a tagu cez tú istú osobu. '
++                'Typ práce nie je keyword. answer_candidates dokladajú spojenie požadovaného názvu alebo roku a tagu cez tú istú osobu. '
+                 'Pri neúplnom výstupe uveď: Výpis je neúplný.')
+         messages += [{'role':'assistant','content':text},{'role':'user','content':feedback}]
+     return result
+--- a/tools/answer_evaluation/graph_answer_prompt.txt
++++ b/tools/answer_evaluation/graph_answer_prompt.txt
+@@ -13,4 +13,4 @@
+ Vráť iba jediný platný JSON objekt bez úvodu, vysvetlení mimo JSON a bez Markdown ohraničenia. Reťazce ohraničuj ASCII dvojitými úvodzovkami. Vo vnútri answer názvy neobaľuj ďalšími ASCII dvojitými úvodzovkami; ak ich použiješ, musia byť escapované. Roky a count sú celé čísla. Nevypisuj čiarku za poslednou položkou.
+ Rešpektuj complete, truncated, truncation_reasons a index_version. Ak truncated=true alebo complete=false a uvádzaš výsledky, do answer vlož vetu „Výpis je neúplný.“ Neoznačuj výpis za úplný zoznam ani za všetky existujúce práce. Pri complete=null úplnosť nemožno zaručiť. Ak aggregation.complete=true, počet môže byť presný aj pri orezaných dôkazoch; vtedy uveď „Počet je úplný, dôkazy sú orezané.“ Inak z počtu zobrazených položiek neodvodzuj celkový počet.
+ Všetky URL v answer aj source_urls musia byť presne zo sources. Typ práce (napríklad diplomový projekt) nie je kľúčové slovo; keyword uveď iba ak ho dokladá fakt keyword pre tú istú person.
+-answer_candidates sú doložené spojenia názvu, roku a tagu cez tú istú osobu, nie nové odhady. Ak je taký kandidát poskytnutý a status kontextu je ok, odpovedz podľa neho a neodmietaj ho len preto, že tag pochádza z dcérskeho dokumentu. V claims ponechaj person, title, year a keyword spolu v jednom objekte a cituj všetky jeho podklady.
++answer_candidates sú doložené spojenia požadovaného názvu alebo roku a tagu cez tú istú osobu, nie nové odhady. Ak je taký kandidát poskytnutý a status kontextu je ok, odpovedz podľa neho a neodmietaj ho len preto, že tag pochádza z dcérskeho dokumentu. V claims ponechaj jeho person, year, keyword a prípadne title spolu v jednom objekte a cituj všetky jeho podklady. Aj pri samotnej kombinácii tag + rok práce sa tag viaže na osobu, nemusí byť v dokumente obsahujúcom rok práce.
+--- a/tools/graphrag/evaluation/check_openwebui.py
++++ b/tools/graphrag/evaluation/check_openwebui.py
+@@ -82,6 +82,11 @@
+ 
+ 
+ def main():
++    # Import the common adapter here: it imports grade_claims from this module.
++    from tools.answer_evaluation.evaluate_graph_answers import graph_context, grade_answer, HERE
++    from tools.answer_evaluation.checked_generation import generate_checked_answer
++    from tools.graphrag.build_graphrag import graph_version
++
+     parser = argparse.ArgumentParser(description=__doc__)
+     parser.add_argument('--questions', type=Path, default=BASE / 'graphrag_questions_v4.json')
+     parser.add_argument('--freeze-manifest', type=Path, required=True)
+@@ -100,8 +105,10 @@
+             selected.append(case); counts[case['category']] += 1
+     oracle = load_source_oracle(); manifest = json.loads(args.freeze_manifest.read_text())
+     settings = get_neo4j_settings(); rows = []
+-    payload = {'method': 'OpenWebUI API generation with locally supplied GraphRAG context; no autonomous tool routing.',
+-               'model': args.model, 'selected_ids': [q['id'] for q in selected], 'system_prompt': SYSTEM,
++    prompt = (HERE / 'graph_answer_prompt.txt').read_text()
++    payload = {'method': 'Supplementary known-query check with common context guards; no autonomous tool routing.',
++               'model': args.model, 'selected_ids': [q['id'] for q in selected], 'system_prompt': prompt,
++               'corrective_request_limit': 1,
+                'limitations': ['Small stratified supplementary sample; not the primary 320-case accuracy.',
+                    'Structured claims, source citations and binding checks; free prose requires separate review.'],
+                'questions': rows}
+@@ -110,24 +117,24 @@
+         metadata = run_metadata(args.questions)
+         verify_freeze(manifest, metadata, graph_fingerprint(session), audit=args.audit, **manifest['parameters'])
+         payload['run_metadata'] = metadata
++        version = graph_version(session)
+         for case in selected:
+             started = time.perf_counter()
+             result = search_graph_with_session(session, case['question'], **manifest['parameters'])
+-            context = {k: result[k] for k in ['status', 'reason', 'facts', 'documents', 'aggregation']}
++            result['index_version'] = version
++            context = graph_context(result)
+             row = {'id': case['id'], 'category': case['category'], 'question': case['question'],
+                    'graph_context': context, 'strict': False}
+             try:
+-                response = request_json(OPENWEBUI_URL, method='POST', timeout=90, max_attempts=1,
+-                    headers={'Authorization': f'Bearer {os.environ["OPENWEBUI_API_KEY"]}', 'Content-Type': 'application/json'},
+-                    payload={'model': args.model, 'stream': False, 'response_format': {'type': 'json_object'},
+-                        'messages': [{'role': 'system', 'content': SYSTEM},
+-                            {'role': 'user', 'content': json.dumps({'question': case['question'], 'graph_context': context}, ensure_ascii=False)}]})
+-                row['response'] = response
+-                text = response['choices'][0]['message']['content'].strip()
+-                if text.startswith('```'):
+-                    text = text.split('\n', 1)[1].rsplit('```', 1)[0]
+-                answer = json.loads(text)
+-                row['answer'] = answer; row.update(grade_claims(case, answer, oracle))
++                def complete(messages):
++                    return request_json(OPENWEBUI_URL, method='POST', timeout=90, max_attempts=1,
++                        headers={'Authorization': f'Bearer {os.environ["OPENWEBUI_API_KEY"]}', 'Content-Type': 'application/json'},
++                        payload={'model': args.model, 'stream': False, 'response_format': {'type': 'json_object'},
++                                 'messages': messages})
++                row.update(generate_checked_answer(case['question'], context, prompt, complete))
++                if row.get('answer'):
++                    row.update(grade_answer(case, row['answer'], oracle, context=context))
++                row['strict'] = bool(row['strict'] and row['accepted_for_delivery'])
+             except Exception as error:
+                 row['error'] = f'{type(error).__name__}: {str(error)[:500]}'
+             row['latency_seconds'] = time.perf_counter() - started
+@@ -137,6 +144,7 @@
+             args.output.write_text(json.dumps(payload, ensure_ascii=False, indent=2) + '\n')
+             print(f'{case["id"]}: {"OK" if row["strict"] else "FAIL"} ({row["latency_seconds"]:.1f}s)', flush=True)
+         verify_freeze(manifest, run_metadata(args.questions), graph_fingerprint(session), audit=args.audit, **manifest['parameters'])
++        assert graph_version(session) == version, 'Index version changed during generation.'
+     print(json.dumps(payload['summary']))
+ 
+ 

tools/graphrag/evaluation/results/v6_preparation/frozen_candidate.json

@@ -0,0 +1,241 @@
+{
+  "schema_version": 2,
+  "evaluation_kind": "historical_regression",
+  "partition": {
+    "method": "Previously exposed questions; regression, not independent TEST",
+    "dev_profile_count": 72,
+    "test_profile_count": 72,
+    "source_profile_overlap": 72
+  },
+  "frozen_at_utc": "2026-10-25T21:12:46.534949+00:00",
+  "fingerprints": {
+    "implementation": {
+      "tools/graphrag/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "tools/graphrag/build_graphrag.py": "256292101d6d7d1946e926e2717ac45465b169f4091c51ecfed4f1a779dfd700",
+      "tools/graphrag/evaluation/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "tools/graphrag/evaluation/audit_benchmark.py": "6619c818a52fc284e87ed6c48a503a54a3c9dd95447463ed78f075b7c58bad80",
+      "tools/graphrag/evaluation/benchmark_v2.py": "129dd774faa414f06733e1ee6333e7b411b1c75b5601b895c405778df478a563",
+      "tools/graphrag/evaluation/benchmark_v3.py": "30bae6c605ed26f4b49973acec799087c4e2b97a883ccedcafb965dd909c4c4d",
+      "tools/graphrag/evaluation/benchmark_v4.py": "70a8b08f7c2b2cb3e2bdc77be9efedd23ea24ac564ec1c1eb224a9f6388b3a67",
+      "tools/graphrag/evaluation/benchmark_v6.py": "f955e600d17358fafcb33bab772aef09d787452f6b33352e299fdd42669aaf34",
+      "tools/graphrag/evaluation/check_openwebui.py": "4dc1309ded94325922bdd7bdff5dd3eaa74928229a7aa752bef00abc0d54f160",
+      "tools/graphrag/evaluation/evaluate_graphrag.py": "89c364d15b70c9cb55df39d5970b4d0d79b681ba70eed7530a4e3fe64167812e",
+      "tools/graphrag/evaluation/evidence_contract.py": "78e24264c176e163e1782fef1634b6c45c7f31bd4692726d02a30ab45a06648e",
+      "tools/graphrag/evaluation/freeze_graphrag.py": "793c6d04a13c3217347283c609f647cd0d69b020e44bbde458bdc7ebbab8ec7e",
+      "tools/graphrag/evaluation/graphrag_evaluator.py": "7f99bf0f97974058650b619ea49a8ee32284cb4b324f1ffab9baf54f5c4495f4",
+      "tools/graphrag/evaluation/reproducibility.py": "8db9a0e0f86236b56239f41da11803538dfaa6745bbf74dbd7618e0a73bda0ae",
+      "tools/graphrag/evaluation/source_oracle.py": "f2265d0919a4019f10d2b4a986bd772684fe59c598d7ae2f5cd0daa41f81b87b",
+      "tools/graphrag/evaluation/source_validation.py": "9671b8978623888dbed8b8b2d1b6b01c1ce73cb0a512b03919e48873990f15ed",
+      "tools/graphrag/graph_db.py": "c4b1a2edb765856b807398e4c243a8d17753529a5d2dfdb51791e28bc8de17f7",
+      "tools/graphrag/graph_search.py": "8cc3668b4ad7beabf849e6978ea1d5defc818b142ee8b157b19c319998fb7541",
+      "tools/graphrag/graphrag.py": "a718efbd001301c4a809aa449a2f175ae7aca8f6f3248f539d6f0597faeb3187",
+      "tools/graphrag/neo4j_smoke.py": "fb5244c72c29ef2acf96d1e7f54335ff168eed4119db1692025eb1b99194b7bb",
+      "tools/graphrag/publication.py": "fba1fa0d0bed325fd87f8d3593baae9271de1766b772d7214c7f63fd3f1c37b6",
+      "tools/graphrag/tests/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "tools/graphrag/tests/test_evaluation_pipeline_v2.py": "63d1b3e80ae3c7b563374fd7ab4aa5ffad3c614eafbd15142c577abd7f756bf9",
+      "tools/graphrag/tests/test_evaluation_v5.py": "9387e9d1418ebab87530986b0e29dac31cf33d762777a29a31cff24cc72e552a",
+      "tools/graphrag/tests/test_evaluation_wide.py": "1b28bb74fd6b9457c869186b88606f1f7863d573aa3ab5226162e878f236d99e",
+      "tools/graphrag/tests/test_evaluator_connected_v2.py": "29d7c9838ed1ff37ea3276df6feb1b6142158c43a0525712f490b648181882de",
+      "tools/graphrag/tests/test_graph_publication.py": "96e53c1ae28e36c7a4290e1e60c8b398bf095ceaf883e819e1a477e2672b11c0",
+      "tools/graphrag/tests/test_graph_rebuild_atomic.py": "fe37d4fff0b1f0fb1d565f0a96152b9b4e336837bc78b5977cf3b4879d7fbaf3",
+      "tools/graphrag/tests/test_graph_search.py": "eb58661aef9dafe6bf7188fcdc2d79cef6652d57c64d1b81ae81a52e8e0204d7",
+      "tools/graphrag/tests/test_graph_search_entity_v2.py": "0ef1a8ebd513c5a4507eb38fcf5ae6137c69e9901b5a2975ba513be5902ac693",
+      "tools/graphrag/tests/test_graph_search_neo4j.py": "fec2bf72de8f752f17c9164fee5754bdb2e153828855e47efbf0000dbe82cf3b",
+      "tools/graphrag/tests/test_graph_search_regressions.py": "9db7779aeb8ed9e8c0f92c1657011a81080aff3c928649e811358e3009a18c48",
+      "tools/graphrag/tests/test_graphrag.py": "9ee8629d6b528dd78c2ddfc25cee95e17613f45b454fc0a38d4c591862637a9f",
+      "tools/graphrag/tests/test_graphrag_evaluator.py": "1c6d08aeec9ae56ac4123239a5aed12a0081d66cc3f34b2047d95764579d7298",
+      "tools/graphrag/tests/test_graphrag_freeze.py": "c4b760baf4c7ebf13b16bd4cd3c51874f5512eef3b4c61470dc4d428b0303d9f",
+      "tools/graphrag/tests/test_graphrag_research_v2.py": "711008a91ab8a58ae0854e7f90329bcd42e5189fde2e2001589e97b0c22c5198",
+      "tools/graphrag/tests/test_graphrag_root_title.py": "085651fe6e6081521a5dde9554abfee93456250108cf15978df25378d886a46c",
+      "tools/graphrag/tests/test_source_oracle_v2.py": "0afdbe235e185ffe9f38aa667f90f5a6c20504fe29e9ac572642b39a8eb7faca",
+      "tools/graphrag/tests/test_title_labels.py": "2983ead68ad6e2e346d24af0fc96dc6771c163e9c2a351af426a1e0383998e6f",
+      "requirements.txt": "26a6d097727eba03c1b6c71cd02b06373ef01ce03e567dd2bc4357ae330e5a1a",
+      "tools/publication.py": "f99d445073390a733afc8d849b9fcd4b4b0890bb54d136db67d4015b78436e1a",
+      "tools/retrieval_contract.py": "b8fa272c3babf924a3ce8b937fe7ae437cd959ba9f7667193ac67c18d2c5b58d",
+      "tools/index_state.py": "c33920a63d04009f11ef23ca9e9eee9bf86453b7d0586d027bf3b39156afea0d",
+      "tools/rag/common.py": "8497c46d5c6c27a7c76df1568360ad5e9d00d5e5a7ef186e758960cc79d81d1c",
+      "tools/sync_indexes.py": "5b909b4394c24e5ce22848594b16d9aea45fb06418f5cd92bb6aa6381a0ccf10",
+      "tools/rag/rag_utils.py": "4f898cf0facc2ae9f2ea28a36f48d4ed37bd7fc716e097b319bcf4d36a4a4f03",
+      "tools/answer_evaluation/context_policy.py": "ee3c423772800d3e7368fff9b48bcd464a98a741b87a0c79158398ed22a3dc00",
+      "tools/answer_evaluation/checked_generation.py": "e82b95314b8c0ae9ba82353b98876da7b2a4dce464eb21129ad904bf58b8ef5e",
+      "tools/answer_evaluation/answer_format.py": "86a27b4aebc6a5077e13794b81957a1cf063a1f3e070c583120ba53c1788d2f8",
+      "tools/rag/evaluation/rag_metrics.py": "3e95922df90dd1b757a550040a2d9bd4f857e17d179f60170d7cc881a08f94b6",
+      "tools/rag/evaluation/rag_runner.py": "031abf7d5b1a026eb199aa2a579bfb8baaa01175279d0018a364bfe50b76c3c1"
+    },
+    "dataset_sha256": "a7bb38be01e03603cf2fa2c7137817094dff0a0cbb3f801263609e55b28e9edc",
+    "inputs": {
+      "data/documents.json": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+      "data/chunks.json": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9"
+    },
+    "source_corpus": {
+      "sha256": "440da12a7a991824e923f70cada7c616c3fb3b0f12767ec7d75039a6a3231b10",
+      "document_count": 114,
+      "documents": {
+        "pages/interns/bogdan_paul_chis/README.md": "3b4b946eb3c10e9f9a4092e6cffa01c5080e7579aded2f6ed3120c41ed62b647",
+        "pages/interns/cesar_gutierrez/README.md": "b0be60576eed28d4b1920e9acf198684519373dfc13171567247c853b3ed699e",
+        "pages/interns/oliver_pejic/README.md": "0470f3c8465be8f02204b08e4c75ef47f68c7e4ad1038faa5641cdf1811f33d5",
+        "pages/interns/sevval_bulburu/README.md": "ff2a9d66253802f1af0fdfd74832e40fae41b6df23c5e61dc266f6cd0cc16c4d",
+        "pages/interns/yussef_ressaissi/README.md": "f997a65699da0db3222ef7a3c37652d05dfc3419ad4ec74683bafc2c897bb987",
+        "pages/students/2016/darius_lindvai/README.md": "fcaa47bb9708a6439eec5c4b7f7b4ed6c7359e423e1d940e1aed9f44957e6d62",
+        "pages/students/2016/darius_lindvai/dp2021/README.md": "be3d0fea2a6d592b225eecabd9b19445c2fce62b8289c07ab6af93835db12970",
+        "pages/students/2016/darius_lindvai/timovy_projekt/resers/README.md": "352162de1b1e192228fa5d26506c028ac47e567e90df53b1bbaa6ad736ec4766",
+        "pages/students/2016/darius_lindvai/timovy_projekt/tutorial/README.md": "b4a81ff457624b089d618e4479afb1e3d748f6724f72104ef5518e535ae02fe5",
+        "pages/students/2016/dominik_nagy/README.md": "1c4037a158d84c45fa705a0c38e47142a9d46e226f0c138bc2d56d7c38f6d74d",
+        "pages/students/2016/dominik_nagy/timovy_projekt/README.md": "b81a4dab7316f7b846dcf36515f6aa9d44035c86f46f0e07b7a43308bcd31458",
+        "pages/students/2016/ediz_morochovic/README.md": "045e5eb9a3df2b84140808fbca0c3163266ad0efa144bc42d9d5c597881c6ac6",
+        "pages/students/2016/jakub_maruniak/README.md": "6734c41a95b9d0c1e3d226b393ad109ee4a7b82ebfffa73df2d7e0a743fefa14",
+        "pages/students/2016/jakub_maruniak/dp2021/README.md": "9cd884aec70705cdb2aaa0f514b513f16de263c37b749ced7f92916cd2f91c39",
+        "pages/students/2016/jakub_maruniak/dp2021/annotation/README.md": "debae0c50aea1accef0840912b9b1f37b2138f856002bb50e91d7ea2dbbe7963",
+        "pages/students/2016/jakub_maruniak/dp2021/train_experiments/README.md": "01dff38b37043414fa1db204394ddbe7aade00e14d02d495572b65c85d09b194",
+        "pages/students/2016/jakub_maruniak/timovy_projekt/README.md": "04114b1f3d84664aad5bdba5a0b614c5a6f23392be3c31fd9123d63db1be0a35",
+        "pages/students/2016/jan_holp/README.md": "80d6a8492766e5bfa61673a372e5f059d30c8e335de39842866c51485960e4d3",
+        "pages/students/2016/jan_holp/dp2021/README.md": "a62acb74009376766c7e2f9856ee01d21f84593eb9b3ce85cf2e6032aa412476",
+        "pages/students/2016/jan_holp/dp2021/zdrojove_subory/README.md": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+        "pages/students/2016/jan_holp/timovy_projekt/README.md": "79eebdfd3af7e8d5c279077e285a580731b1cf8cc9b4454f6596796e9915f70c",
+        "pages/students/2016/lukas_pokryvka/README.md": "f279456c5c277ff1ee0dd89a3464ed9f9f67b44bc185a5da9811ef5cfb675002",
+        "pages/students/2016/lukas_pokryvka/dp2021/README.md": "39b61376ce108e4a930f926ab2d6a997207457121bccfcf617129a3b210a15cf",
+        "pages/students/2016/lukas_pokryvka/timovy_projekt/README.md": "4de4b6c8cfed17b6c60090f82df1d55a66be70a79d3013850689b2baebdd801b",
+        "pages/students/2016/mark_feher/README.md": "bec37c0319fe0869a0377d771db23d090dbda93c27328f5697cdaeb3bbccfe9b",
+        "pages/students/2016/maros_harahus/README.md": "0da150e91918632c9e8cd1443f04da403595a372517e254928ce9f76c9d5033d",
+        "pages/students/2016/maros_harahus/timovy_projekt/README.md": "75af19f7c68f4b646fa1dbae75f3f3dfc52633a9f9da1400d21808bb20be0d83",
+        "pages/students/2016/patrik_pavlisin/README.md": "dc1c648d7cd9b0f0e021c57f9b98189e37c100871439e7e304390715913f2763",
+        "pages/students/2016/patrik_pavlisin/dp21/README.md": "ac1eb0fd0360ad34aa21b7c8da5bb010a6edf10be5721f28dedc3986cd1d8c54",
+        "pages/students/2016/patrik_pavlisin/dp22/README.md": "f2a8c619a812b99abab618ae388eb672407950e1f33fb8d7da7a528c4de81d50",
+        "pages/students/2016/patrik_pavlisin/tp20/README.md": "ae9c771f27e7e093bc29c6888c8d0b27ed61fb41519fd0d7e7f4f6cbb4626edc",
+        "pages/students/2016/tomas_kucharik/README.md": "6a1a215433eb1857660177b4cb2ecec03c6a86743d3a04cd6d5c9d9682c9e63b",
+        "pages/students/2017/dmytro_ushatenko/README.md": "65f4ac177d492162b08ca5da3d9eb048bd088a3c117a28a662113b7e4bc3a4ec",
+        "pages/students/2017/martin_jancura/README.md": "6a0a18d5095dcfb12abfdd72c9cfd8025d714fb5e29a7858cd2b285e4cb986b9",
+        "pages/students/2017/stanislav_matsunych/README.md": "90ffa11eff4a4360b0252dbc53a116c3a5777b3d5dd86ce5c89c02bdd2051c73",
+        "pages/students/2017/than_trung_thanh/README.md": "2150519b8580e7337520db4de64d2d1c82d2634380537d019f41c506dbf2979d",
+        "pages/students/2018/NaN1name_buday/README.md": "23c8c58090bdc954347e36f141bea91f43c8d5b1d777e346864a2ebe84c0bcce",
+        "pages/students/2018/adrian_remias/README.md": "e5bfe1d5d4e626beaa636a02d34ec58cf2cff9b0e57833905af35b80bf1465a2",
+        "pages/students/2018/david_ilas/README.md": "96c45ff9c5a821ddcb1a2fc86d487bd8f4729d1b76887914b0f5fbf99af45af0",
+        "pages/students/2018/david_omasta/README.md": "4164051870e1e1cf8949ed18a74196e9ea789dfbe4fee24bdeb736ca42848327",
+        "pages/students/2018/martin_wencel/README.md": "0e6a7a981689f22304f8baa5de97d4315c0695c94c9277de7851943eb012253b",
+        "pages/students/2018/oleh_bilykh/README.md": "4072c62dd8ee753a0c8fa43fdfbbdf566e394c2df13e6a79d08a302e332710b1",
+        "pages/students/2018/ondrej_megela/README.md": "969ff2e1a436b8f957e7168cf5eb678196b99aeda17701c502b016441f644065",
+        "pages/students/2018/samuel_sirotnik/README.md": "cc230ede67b6e8632511c92a5ed6d1e9deb42e70819280222bdfd369ee77fc70",
+        "pages/students/2019/alina_vitko/README.md": "9b071fe2aff470c275ae027be236fe836f741f774cabb9cbbb55304a8d5102cd",
+        "pages/students/2019/artem_yatsenko/README.md": "328b101e46c51c959e792cf7ca226942f81d2ab8fe9648ad6d0ecd07016429e2",
+        "pages/students/2019/david_stromp/README.md": "e46ca8691d5ec894128f6f6f2ed33e2c7bd141f80719bb0449b20f10f30cca09",
+        "pages/students/2019/dmytro_mural/README.md": "e881a47d847c3ef8d680844113fdbfbb87553b28982bf9eef4936f6d30923e93",
+        "pages/students/2019/filip_tomas/README.md": "75a1fe7b921e4bff0fdf9e2a957b4adad4b4a57aa7ebdffbb73dd82c02123f94",
+        "pages/students/2019/kristian_sopkovic/README.md": "3286960fb30f584df35455f9b6dfae81ca4827a1d711d2bcd6e48c3f150c9279",
+        "pages/students/2019/michal_stromko/README.md": "3ae1dd9b6638a6a249a94be3f3846286555666aefa225bb3a2c7288b35900ce7",
+        "pages/students/2019/michal_stromko/vp2021/README.md": "feb2e093fdd6461239e79b1e1f96e65a1c8cd96e8a2b70535ae556a9c198046a",
+        "pages/students/2019/michal_stromko/vp2021/tutorials/README.md": "40e3fa448c7329ef863e705435e23fe6b4788de440710f066fe0eed92d132bad",
+        "pages/students/2019/michal_stromko/vp2021/tutorials/img/README.md": "e8d3ac160b17dfdee2c3e996bf6a03af7a6f16f9140bf5cf7ee343a82dd1865f",
+        "pages/students/2019/patrik_pokrivcak/README.md": "3e3957ad0f7aca59d6eb64c38a3f5e21d8fd953200825d0d0e5cacabfbc35a7d",
+        "pages/students/2019/samuel_horani/README.md": "86e6959ccc612b069687b57996fa3f61660075d6a0d65bfec64f423d39a0a999",
+        "pages/students/2020/david_kostilnik/README.md": "2927524b75da59a64120fc364a0fc374986fa4d364d54e52c36800762c203264",
+        "pages/students/2020/jakub_kristian_lukas/README.md": "f139d9b1ebf5a3f45612f3df5703ecdd935b90701763c27c7c529044c0b0455a",
+        "pages/students/2020/kamil_tomcufcik/README.md": "9d865adf538bc6d7992ee4f4882713447da5c936fca17c43ca0f576072537048",
+        "pages/students/2020/matej_kobylan/README.md": "3d7b289e17b8f25678a07e9ee0e6741688485a0ccb1e50940a6a6acc6312a547",
+        "pages/students/2020/matus_suchanic/README.md": "770ca49538cca22df35f8e7ba736ee97483e4fe5788c221496553592b4e78dc6",
+        "pages/students/2020/pavol_hudak/README.md": "33d06fccc6451e5f19b27b0cad001a34e81635b4ea294a01dec7771c823abc40",
+        "pages/students/2020/vladyslav_krupko/README.md": "4c28e6d38651bb2cafcbf4a18e35210cd369aef2caff6c5b0aa52fac38f0fe7d",
+        "pages/students/2021/artur_hyrenko/README.md": "0568261f4ade4de5472edcfeb1ffed9abae18dc67e4050f4ce2ad088839a2bb3",
+        "pages/students/2021/eduard_matovka/README.md": "de2a7ae38ec87110e347956b37d0605a298ed2847f972961cc9b977da8221d43",
+        "pages/students/2021/manohar_gowdru_shridharu/README.md": "c00ceb620ed7087ead5d9448d476bbb98d273d1d9e7bf1bc4355c367b26c7a6b",
+        "pages/students/2021/martin_sarissky/README.md": "8e97b45a253745faa55f63c844e0b5dfe680aeaf6eb01fb8f9d06ac6e8ee8bcf",
+        "pages/students/2021/matej_novotny/README.md": "b083a7cadde532e3428dc9a57bc61456fcc5905bda91e011b96033d42bf5d9cb",
+        "pages/students/2021/matej_scislak/README.md": "b9a32cce8605e044a4f2828ff2d91a2995a62ef3a841e62f0a225c97221f0cc8",
+        "pages/students/2021/nikita_bodnar/README.md": "2016c0dd923c0f8e561d5cea42ac81da3610fdcc54a4527f56d078e34e53ce90",
+        "pages/students/2021/oleksandr_hryshchenko/README.md": "4b073992b715b6c75b79fd2c8f850a69f940966e3a61a82c625c32c945a93036",
+        "pages/students/2021/vladimir_ferko/README.md": "68e7d311896eda2171e5eec1ec3a8da90d326ed91acaace8058f68664a50d271",
+        "pages/students/2022/andrii_pervashov/README.md": "319dcaa1cc2e29dd7e3a710bab26cb3ac6944ab71b0b97e1de076b1199122334",
+        "pages/students/2022/daniil_huzenko/README.md": "390782fbff6220a9462060ce1a62959f2a2b88f42d06e2c51d1f406b75d73f2c",
+        "pages/students/2022/jakub_schwarc/README.md": "f541a8852c2b0f6f9d7182e01a34c4e3beee9bfa55427a06b331638b11eb8be6",
+        "pages/students/2022/jan_malinovsky/README.md": "8af021bfb131fa005a7aec3e2f2f12fc17fa5dc14e2675311adbbfc445a4bcab",
+        "pages/students/2022/jan_ptak/README.md": "131cf053764fd5423541ebe672c3b4f5da875cfb8e4b821f1335ddb967282282",
+        "pages/students/2022/oleh_poiasnik/README.md": "c2f7dc3f61f8b4a5b43ded1e3ae4c7d3e7f874439ee6b17dd9c8d45dabc25bc8",
+        "pages/students/2022/samuel_vasil/README.md": "67c807049147aacd04f3050f4ff0233a8b3f0a239ab173b430e13dfe113ad0a5",
+        "pages/students/2022/serhii_yemets/README.md": "ffbeac893947acafe855d576322a8826f9d4f87ac344cb31f8ceeefa244b9133",
+        "pages/students/2022/tetiana_mohorian/README.md": "c9efdc632bf1ca89a0bfde0dc00be7aaac8a85da3d92e2ce8506236c73e93cfb",
+        "pages/students/2022/valerii_kutsenko/README.md": "dc764c609ad3c1df59cfea997d4db82a591b1b1e0764453e319d841115507e4f",
+        "pages/students/2022/vladyslav_yanchenko/README.md": "4d1ce216e3b526ee437ddbc658e51d7bb83530087e43628c096a569a3854a47f",
+        "pages/students/2022/yevhenii_leonov/README.md": "b90e41db0cec2451027020a9102d32a24d777e8a38b604ccca87b36cd6b3a292",
+        "pages/students/2023/denis_le_thanh/README.md": "2cb83f942d86a02712b0c58778e46dc2211346d1a6eba606471dfa832245db61",
+        "pages/students/2023/maksym_zatirka/README.md": "7a7d2cbe276c352dacfbc6ba6de5f15303d30ecf34845ff8da8d085f3d67c3dd",
+        "pages/students/2023/oleksandr_dorybohov/README.md": "c24d6f941de0dafb4459cea8259cf147821cafc86ba65be5b528f419f734bc18",
+        "pages/students/2023/ronald_zalacko/README.md": "724b0f8e6e641236dba9bddbd77f4546e09514076c7dfb151807aaf4b882c96f",
+        "pages/students/2023/rostyslav_rodzhuk/README.md": "de356dd7427c8a0716c165a0f88ae47b691a86ee7a388f60e2d2e9306298415c",
+        "pages/students/2023/samuel_dzurina/README.md": "4818191d044b91219c796a35010c95da5cf2093e050d494f30b219b91e13fa0e",
+        "pages/students/2023/simona_bobrovcanova/README.md": "e473a1911dae10ab40e2cf6696a8be616a58c22aaf09af5953fbac475791a35e",
+        "pages/students/2023/yevhenii_medushivskyi/README.md": "5710ebf91c893a12fe6d19f5372311938d1e7c7bbdf203f2dc8ed2fe7c4f5a24",
+        "pages/topics/akopisat/README.md": "c7245e1fbfa4160618502c0baf8cc6bc47cd9a82b8a85bc1c645314829a7b76a",
+        "pages/topics/bert/README.md": "35e507f7ec2ad1d3526e2a15f94ea085ee25f680a02daf6df62fd429bd9c25c9",
+        "pages/topics/chatbot/README.md": "6f2f83ffb80109be47f9750cca9d3036ec3645abef9f2b590bd6825780c7c48a",
+        "pages/topics/faq/README.md": "15cfbd9c2a1649c80a7a58922e2b66dadab5c86a5ef33ef16c82bebd4d2dbf06",
+        "pages/topics/hatespeech/README.md": "364df94e70dd740c63544e2586ba7b957780f714fc0c941d1b04b02233e6cbc5",
+        "pages/topics/legal/README.md": "003bb9bfe1e0ca21652091c2d11496312eb04e0689fe459e14c2d5a738e8eeea",
+        "pages/topics/morphology/README.md": "e242cc5ff1a4cc98e227566c0ead999bbc95937b7c20be62cb2afb773d0d35c0",
+        "pages/topics/named-entity/README.md": "66b2e25f70211d432d596e377d552b3b8f0c525b14c31e3bc9c956c3aeadca84",
+        "pages/topics/named-entity/navod/README.md": "1ff87079a98411de0414754c60241e08d2214ddcd63ddf974e4340cce2ce5b02",
+        "pages/topics/nlp/README.md": "6f90abf9dc3f0ff33475283c13b4a08561abf631425ec2daacb076155a9e02c2",
+        "pages/topics/otvorene/README.md": "cd5544925553fe3840c0ea6f8ef60ab756e26ab6757bf810025c948d8d4f98fc",
+        "pages/topics/prodigy/README.md": "5e4f691f2754c2ec4afbaab2f41e6db77feb5b322dd9344465eb40c35f116499",
+        "pages/topics/python/README.md": "2ef073adba672142c39e9a59fa2e00a98ab52bda54233d46fbe6940ee23f9fcc",
+        "pages/topics/question/README.md": "408078a2a67cea2699e12359e85ab641136231e6104c2afdf874a705879525df",
+        "pages/topics/question/navod/README.md": "061e737c69184b2bd244316ca7ba841f7186fe01ee80ee993742a733dcad3ecd",
+        "pages/topics/question/nezodpovedatelne/README.md": "b6703d9690f4599b973efb27595ce157773ae0fc0d55cf2c0a09d9f50e8e4c92",
+        "pages/topics/question/validacie/README.md": "3216a7127abdfc6da50ed116f44ff2d6785e7da4af70d23aceda774e745ad562",
+        "pages/topics/resources/README.md": "09405397387f00cbcc5680f7d655f2d861e4d92e5df792a62d9dfd04f3534f48",
+        "pages/topics/seq2seq/README.md": "969d15c03a63a3f8d5edc9017fe5a0fb7311fdf4de4a3473a08266241f92c1e2",
+        "pages/topics/spacy/README.md": "f504e5db77cb8694f08a57fb8636e522d59a0e669216f4bc8273bc3c5f375f87",
+        "pages/topics/steel/README.md": "b4068cef2ed197711467ac47630d5b2064fdc8828f898f7af99664e94d833330",
+        "pages/topics/translation/README.md": "a7d4289e72199b307dc5ad8fc69cda39c1a33c27a6c7447af70aa9c3cc6b8635"
+      }
+    },
+    "retrieval_policy": {
+      "published_only": true,
+      "publication_policy": "explicit-false-hidden-v1",
+      "academic_preparation_is_work": false
+    }
+  },
+  "graph": {
+    "sha256": "6fd45dc58382641072d991a088fcdfcd1c0e81ce912befbae6fca700d630fe01",
+    "nodes": 446,
+    "relationships": 877
+  },
+  "parameters": {
+    "limit": 40,
+    "max_entities": 8
+  },
+  "dev_summary": {
+    "questions": 722,
+    "source_checked_count": 722,
+    "all_facts_supported_count": 722,
+    "strict_count": 722,
+    "strict_accuracy": 1.0,
+    "error_count": 0,
+    "value_score": 1.0,
+    "entity_score": 1.0,
+    "relation_score": 1.0,
+    "property_score": 1.0,
+    "relation_choice_score": 1.0,
+    "document_recall": 1.0,
+    "source_recall": 1.0,
+    "binding_score": 1.0,
+    "path_score": 1.0,
+    "answerability_accuracy": 1.0,
+    "latency_mean_seconds": 0.048097,
+    "latency_median_seconds": 0.045389,
+    "latency_p95_seconds": 0.067144
+  },
+  "source_audit_summary": {
+    "dev": {
+      "questions": 722,
+      "source_consistent": 722,
+      "issues": {}
+    },
+    "test": {
+      "questions": 320,
+      "source_consistent": 320,
+      "issues": {}
+    }
+  },
+  "policy": "Report all questions including negatives. No exclusions or holdout tuning.",
+  "dev_results_sha256": "6ffa8ee2c071bb40f97cefe39fe4e5487b8c2a935c9ea829b058d5557d55aad7",
+  "audit_sha256": "d5922db25b6d40cb7e5dc7df7990e93a3caee263a6e747ae910ab340da340023"
+}

tools/graphrag/evaluation/results/v6_preparation/generation_dev_ids.json

@@ -0,0 +1,44 @@
+[
+  "wide-001",
+  "wide-002",
+  "wide-041",
+  "wide-042",
+  "wide-080",
+  "wide-081",
+  "wide-082",
+  "wide-104",
+  "wide-113",
+  "wide-114",
+  "wide-116",
+  "wide-119",
+  "wide-129",
+  "wide-137",
+  "wide-138",
+  "wide-139",
+  "wide-141",
+  "wide-142",
+  "wide-143",
+  "wide-145",
+  "wide-155",
+  "wide-159",
+  "wide-161",
+  "wide-163",
+  "wide-165",
+  "wide-169",
+  "wide-170",
+  "wide-178",
+  "wide-181",
+  "wide-193",
+  "wide-194",
+  "wide-209",
+  "wide-217",
+  "wide-218",
+  "wide-232",
+  "wide-236",
+  "wide-239",
+  "wide-249",
+  "wide-250",
+  "wide-283",
+  "wide-289",
+  "wide-290"
+]

tools/graphrag/evaluation/results/v6_preparation/live_context_scenarios.declaration.json

@@ -0,0 +1,350 @@
+{
+  "method": "Live synthetic functional checks; separate from corpus accuracy metrics.",
+  "model": "model120-fast",
+  "hashes": {
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/check_context_scenarios.py": "786c043c14b5014fa2c10d6d195fb250d81c16c15c449bc72ab7c8f86bdd8b66",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/graph_answer_prompt.txt": "c7104ffb08b5b384f38d48fbdad40ccd40de61beace984d8a118cf12bd229860",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/context_policy.py": "ee3c423772800d3e7368fff9b48bcd464a98a741b87a0c79158398ed22a3dc00",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/checked_generation.py": "e82b95314b8c0ae9ba82353b98876da7b2a4dce464eb21129ad904bf58b8ef5e",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/answer_format.py": "86a27b4aebc6a5077e13794b81957a1cf063a1f3e070c583120ba53c1788d2f8"
+  },
+  "scenarios": [
+    {
+      "id": "complete",
+      "question": "Uveď názov práce osoby Eva Vzorová a rok práce.",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": true,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "aggregation": null,
+        "answer_candidates": []
+      },
+      "expected": {
+        "status": "ok",
+        "claims": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026
+          }
+        ]
+      }
+    },
+    {
+      "id": "incomplete_without_truncated",
+      "question": "Uveď názov práce osoby Eva Vzorová a rok práce.",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": false,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "aggregation": null,
+        "answer_candidates": []
+      },
+      "expected": {
+        "status": "ok",
+        "claims": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026
+          }
+        ]
+      }
+    },
+    {
+      "id": "truncated",
+      "question": "Uveď názov práce osoby Eva Vzorová a rok práce.",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": false,
+        "truncated": true,
+        "truncation_reasons": [
+          "limit"
+        ],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "aggregation": null,
+        "answer_candidates": []
+      },
+      "expected": {
+        "status": "ok",
+        "claims": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026
+          }
+        ]
+      }
+    },
+    {
+      "id": "unknown_completeness",
+      "question": "Uveď názov práce osoby Eva Vzorová a rok práce.",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": null,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "aggregation": null,
+        "answer_candidates": []
+      },
+      "expected": {
+        "status": "ok",
+        "claims": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026
+          }
+        ]
+      }
+    },
+    {
+      "id": "exact_count_partial_evidence",
+      "question": "Aký je počet vyhovujúcich osôb?",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": false,
+        "truncated": true,
+        "truncation_reasons": [
+          "evidence_limit"
+        ],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "aggregation": {
+          "count": 2,
+          "complete": true
+        },
+        "answer_candidates": []
+      },
+      "expected": {
+        "status": "ok",
+        "count": 2
+      }
+    },
+    {
+      "id": "same_person_child_tag",
+      "question": "Uveď názov práce osoby Eva Vzorová v roku 2026 s kľúčovým slovom python.",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": true,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          },
+          {
+            "source_id": "S2",
+            "document_path": "child.md",
+            "source_url": "https://example.test/child"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          },
+          {
+            "person": "Eva Vzorová",
+            "keyword": "python",
+            "document_path": "child.md",
+            "source_url": "https://example.test/child"
+          }
+        ],
+        "aggregation": null,
+        "answer_candidates": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "keyword": "python",
+            "evidence_documents": [
+              "a.md",
+              "child.md"
+            ],
+            "source_urls": [
+              "https://example.test/a",
+              "https://example.test/child"
+            ]
+          }
+        ]
+      },
+      "expected": {
+        "status": "ok",
+        "claims": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "keyword": "python"
+          }
+        ]
+      }
+    },
+    {
+      "id": "type_is_not_keyword",
+      "question": "Uveď názov diplomového projektu osoby Eva Vzorová a rok.",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": true,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "aggregation": null,
+        "answer_candidates": []
+      },
+      "expected": {
+        "status": "ok",
+        "no_keyword": true
+      }
+    },
+    {
+      "id": "no_answer",
+      "question": "Akú prácu má Neexistujúca Osoba?",
+      "context": {
+        "status": "no_answer",
+        "reason": "unknown_person",
+        "complete": true,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [],
+        "facts": [],
+        "aggregation": null,
+        "answer_candidates": []
+      },
+      "expected": {
+        "status": "no_answer"
+      }
+    }
+  ]
+}

tools/graphrag/evaluation/results/v6_preparation/live_context_scenarios_final.declaration.json

@@ -0,0 +1,416 @@
+{
+  "method": "Live synthetic functional checks; separate from corpus accuracy metrics.",
+  "model": "model120-fast",
+  "hashes": {
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/check_context_scenarios.py": "cbb2edf8fdcba61b9e5cc50e28cb1a1dcc25b0e0ab990177b41f40031223ca63",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/graph_answer_prompt.txt": "8c8274a22bf5b44364d8fc8226a8f659b31c232f5a2aba5916d693b559e9f341",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/context_policy.py": "e7b7c511e5e9cafdef04969eb4007a6a61ac4b99bcc60e9e722945a15bec1f26",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/checked_generation.py": "7e017471e6281db62b79f270210db3ab14c7e31a4623bbfa3d9a5fd8911b1c20",
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/answer_format.py": "86a27b4aebc6a5077e13794b81957a1cf063a1f3e070c583120ba53c1788d2f8"
+  },
+  "scenarios": [
+    {
+      "id": "complete",
+      "question": "Uveď názov práce osoby Eva Vzorová a rok práce.",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": true,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "aggregation": null,
+        "answer_candidates": []
+      },
+      "expected": {
+        "status": "ok",
+        "claims": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026
+          }
+        ]
+      }
+    },
+    {
+      "id": "incomplete_without_truncated",
+      "question": "Uveď názov práce osoby Eva Vzorová a rok práce.",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": false,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "aggregation": null,
+        "answer_candidates": []
+      },
+      "expected": {
+        "status": "ok",
+        "claims": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026
+          }
+        ]
+      }
+    },
+    {
+      "id": "truncated",
+      "question": "Uveď názov práce osoby Eva Vzorová a rok práce.",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": false,
+        "truncated": true,
+        "truncation_reasons": [
+          "limit"
+        ],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "aggregation": null,
+        "answer_candidates": []
+      },
+      "expected": {
+        "status": "ok",
+        "claims": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026
+          }
+        ]
+      }
+    },
+    {
+      "id": "unknown_completeness",
+      "question": "Uveď názov práce osoby Eva Vzorová a rok práce.",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": null,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "aggregation": null,
+        "answer_candidates": []
+      },
+      "expected": {
+        "status": "ok",
+        "claims": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026
+          }
+        ]
+      }
+    },
+    {
+      "id": "exact_count_partial_evidence",
+      "question": "Aký je počet vyhovujúcich osôb?",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": false,
+        "truncated": true,
+        "truncation_reasons": [
+          "evidence_limit"
+        ],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "aggregation": {
+          "count": 2,
+          "complete": true
+        },
+        "answer_candidates": []
+      },
+      "expected": {
+        "status": "ok",
+        "count": 2
+      }
+    },
+    {
+      "id": "same_person_child_tag",
+      "question": "Uveď názov práce osoby Eva Vzorová v roku 2026 s kľúčovým slovom python.",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": true,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          },
+          {
+            "source_id": "S2",
+            "document_path": "child.md",
+            "source_url": "https://example.test/child"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          },
+          {
+            "person": "Eva Vzorová",
+            "keyword": "python",
+            "document_path": "child.md",
+            "source_url": "https://example.test/child"
+          }
+        ],
+        "aggregation": null,
+        "answer_candidates": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "keyword": "python",
+            "evidence_documents": [
+              "a.md",
+              "child.md"
+            ],
+            "source_urls": [
+              "https://example.test/a",
+              "https://example.test/child"
+            ]
+          }
+        ]
+      },
+      "expected": {
+        "status": "ok",
+        "claims": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "keyword": "python"
+          }
+        ]
+      }
+    },
+    {
+      "id": "same_person_year_child_tag",
+      "question": "Ktorá osoba spĺňa kľúčové slovo python a rok práce 2026?",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": true,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          },
+          {
+            "source_id": "S2",
+            "document_path": "child.md",
+            "source_url": "https://example.test/child"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          },
+          {
+            "person": "Eva Vzorová",
+            "keyword": "python",
+            "document_path": "child.md",
+            "source_url": "https://example.test/child"
+          }
+        ],
+        "aggregation": null,
+        "answer_candidates": [
+          {
+            "person": "Eva Vzorová",
+            "year": 2026,
+            "keyword": "python",
+            "evidence_documents": [
+              "a.md",
+              "child.md"
+            ],
+            "source_urls": [
+              "https://example.test/a",
+              "https://example.test/child"
+            ]
+          }
+        ]
+      },
+      "expected": {
+        "status": "ok",
+        "claims": [
+          {
+            "person": "Eva Vzorová",
+            "year": 2026,
+            "keyword": "python"
+          }
+        ]
+      }
+    },
+    {
+      "id": "type_is_not_keyword",
+      "question": "Uveď názov diplomového projektu osoby Eva Vzorová a rok.",
+      "context": {
+        "status": "ok",
+        "reason": null,
+        "complete": true,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [
+          {
+            "source_id": "S1",
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "facts": [
+          {
+            "person": "Eva Vzorová",
+            "title": "Vyhľadávanie informácií",
+            "year": 2026,
+            "document_path": "a.md",
+            "source_url": "https://example.test/a"
+          }
+        ],
+        "aggregation": null,
+        "answer_candidates": []
+      },
+      "expected": {
+        "status": "ok",
+        "no_keyword": true
+      }
+    },
+    {
+      "id": "no_answer",
+      "question": "Akú prácu má Neexistujúca Osoba?",
+      "context": {
+        "status": "no_answer",
+        "reason": "unknown_person",
+        "complete": true,
+        "truncated": false,
+        "truncation_reasons": [],
+        "index_version": {
+          "version": "synthetic-functional-fixture"
+        },
+        "sources": [],
+        "facts": [],
+        "aggregation": null,
+        "answer_candidates": []
+      },
+      "expected": {
+        "status": "no_answer"
+      }
+    }
+  ]
+}

tools/graphrag/evaluation/results/v6_preparation/protocol.json

@@ -0,0 +1,86 @@
+{
+  "evaluation_kind": "historical_regression",
+  "dataset_sha256": "a7bb38be01e03603cf2fa2c7137817094dff0a0cbb3f801263609e55b28e9edc",
+  "graph_dev": 722,
+  "graph_regression": 320,
+  "llm_dev_ids": [
+    "wide-001",
+    "wide-002",
+    "wide-041",
+    "wide-042",
+    "wide-080",
+    "wide-081",
+    "wide-082",
+    "wide-104",
+    "wide-113",
+    "wide-114",
+    "wide-116",
+    "wide-119",
+    "wide-129",
+    "wide-137",
+    "wide-138",
+    "wide-139",
+    "wide-141",
+    "wide-142",
+    "wide-143",
+    "wide-145",
+    "wide-155",
+    "wide-159",
+    "wide-161",
+    "wide-163",
+    "wide-165",
+    "wide-169",
+    "wide-170",
+    "wide-178",
+    "wide-181",
+    "wide-193",
+    "wide-194",
+    "wide-209",
+    "wide-217",
+    "wide-218",
+    "wide-232",
+    "wide-236",
+    "wide-239",
+    "wide-249",
+    "wide-250",
+    "wide-283",
+    "wide-289",
+    "wide-290"
+  ],
+  "llm_dev_count": 42,
+  "llm_final_regression": 320,
+  "llm_final_dev_overlap": 42,
+  "test_positive_count": 284,
+  "test_negative_count": 36,
+  "corrective_request_limit": 1,
+  "correction_basis": "Only actual retrieval context or invalid JSON; never oracle, expected answer, gold, or final grade.",
+  "gold_changes": 15,
+  "changed_test_ids": [
+    "wide-116",
+    "wide-142",
+    "wide-145",
+    "wide-165"
+  ],
+  "policy": "Keep all IDs, questions and splits. Do not tune after final freeze. Retain failed deliveries and every raw attempt. No claim of independent TEST or human review.",
+  "limitations": [
+    "All questions previously exposed. Gold corrections and new guards prevent direct controlled comparison of accuracy with v5.1.",
+    "Native OpenWebUI tool routing is point 4."
+  ],
+  "source_review": {
+    "reviewed": 34,
+    "supported": 31,
+    "corrected": 2,
+    "conditional_directory_cohort": 1,
+    "human_review": "not performed; worksheet supplied"
+  },
+  "start_year_convention": "Explicit statement, otherwise students/<year> directory cohort; a proxy, not independent proof of actual enrolment.",
+  "development_iterations": [
+    {
+      "name": "candidate",
+      "questions": 42,
+      "correct": 41,
+      "failure": "wide-283: same-person year/tag join rejected",
+      "generation": "answers_v6_dev_2026-10-25"
+    }
+  ]
+}

tools/graphrag/evaluation/results/v6_preparation/rag_delivery_replay.json

@@ -0,0 +1,478 @@
+{
+  "configuration": {
+    "method": "Deterministic delivery replay on the same saved model responses, not new generation.",
+    "input": "tools/graphrag/evaluation/results/v6_preparation/rag_live_smoke.json",
+    "input_sha256": "674272fd911a5ae04690150ac899531feae90b4d44a1735a530a2838a0d32ff3",
+    "delivery_files": {
+      "/home/janko/DP/zp-agent/tools/answer_evaluation/rag_delivery.py": "b83f93d446f44fdd7d41c12eeeaae69c0b02908076624f4eb3f4f72b9b88dbc8",
+      "tools/rag/evaluation/evaluate_rag_answers.py": "945c2da655537fb37fe82a08e10dc3ed66a2d90980921f61f61efb171a4a26a2",
+      "tools/rag/evaluation/rag_metrics.py": "3e95922df90dd1b757a550040a2d9bd4f857e17d179f60170d7cc881a08f94b6",
+      "tools/rag/evaluation/rag_runner.py": "031abf7d5b1a026eb199aa2a579bfb8baaa01175279d0018a364bfe50b76c3c1",
+      "tools/answer_evaluation/context_policy.py": "e7b7c511e5e9cafdef04969eb4007a6a61ac4b99bcc60e9e722945a15bec1f26"
+    }
+  },
+  "results": [
+    {
+      "id": "q0053",
+      "question": "Aký je názov diplomová práca osoby Lukáš Pokrývka?",
+      "citations_ok": true,
+      "unsupported_urls": [],
+      "retrieval_coverage": [
+        {
+          "complete": false,
+          "truncated": true,
+          "truncation_reasons": [
+            "candidate_limit",
+            "per_document_limit",
+            "result_limit"
+          ],
+          "index_version": {
+            "version": "10b78a8e70744e2792ffe4cf11d45571",
+            "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+            "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+            "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+            "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+            "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+            "embedding_model": "intfloat/multilingual-e5-small",
+            "publication_policy": "explicit-false-hidden-v1"
+          }
+        }
+      ],
+      "context_issues": [],
+      "accepted_for_delivery": true,
+      "answer": "Výpis je neúplný.\n\nNázov diplomovej práce Lukáša Pokrývku je „Paralelné trénovanie neurónových sietí\".\n\nZdroj: https://zp.kemt.fei.tuke.sk/students/2016/lukas_pokryvka",
+      "tool_called": true,
+      "tool_call_count": 1,
+      "tool_calls": [
+        {
+          "name": "retrieve_zpwiki_context",
+          "arguments": {
+            "query": "Aký je názov diplomová práca osoby Lukáš Pokrývka?",
+            "limit": 5,
+            "published_only": true,
+            "max_per_document": 1
+          },
+          "latency_seconds": 8.6679,
+          "source_urls": [
+            "https://zp.kemt.fei.tuke.sk/students/2016/lukas_pokryvka",
+            "https://zp.kemt.fei.tuke.sk/students/2016/tomas_kucharik",
+            "https://zp.kemt.fei.tuke.sk/students/2019/patrik_pokrivcak",
+            "https://zp.kemt.fei.tuke.sk/students/2018/oleh_bilykh",
+            "https://zp.kemt.fei.tuke.sk/students/2016/jakub_maruniak/dp2021"
+          ]
+        }
+      ],
+      "rag_source_urls": [
+        "https://zp.kemt.fei.tuke.sk/students/2016/lukas_pokryvka",
+        "https://zp.kemt.fei.tuke.sk/students/2016/tomas_kucharik",
+        "https://zp.kemt.fei.tuke.sk/students/2019/patrik_pokrivcak",
+        "https://zp.kemt.fei.tuke.sk/students/2018/oleh_bilykh",
+        "https://zp.kemt.fei.tuke.sk/students/2016/jakub_maruniak/dp2021"
+      ],
+      "first_model_latency_seconds": 5.65362,
+      "tool_latency_seconds": 8.6679,
+      "final_model_latency_seconds": 23.053733,
+      "total_latency_seconds": 37.375855,
+      "usage": {
+        "prompt_tokens": 19045,
+        "completion_tokens": 137,
+        "total_tokens": 19182
+      },
+      "response_model": "Qwen3.5-122B-A10B-Q4_K_M",
+      "citations_checked": true,
+      "answer_matches": [
+        true
+      ],
+      "answer_contains_score": 1.0,
+      "source_matches": [
+        true
+      ],
+      "source_match_count": 1,
+      "source_required_count": 1,
+      "source_url_score": 1.0,
+      "should_answer_ok": true,
+      "should_answer_score": 1.0,
+      "returned_no_answer": false,
+      "tool_score": 1.0,
+      "overall_score": 1.0,
+      "strict_pass": true,
+      "model_answer": "Názov diplomovej práce Lukáša Pokrývku je „Paralelné trénovanie neurónových sietí\".\n\nZdroj: https://zp.kemt.fei.tuke.sk/students/2016/lukas_pokryvka",
+      "first_pass_accepted_for_delivery": false,
+      "first_pass_context_issues": [
+        "missing_incompleteness_notice"
+      ],
+      "display_notice_added": true
+    },
+    {
+      "id": "q0060",
+      "question": "Rozlíš rok začiatku štúdia a rok práce osoby Lukáš Pokrývka.",
+      "citations_ok": true,
+      "unsupported_urls": [],
+      "retrieval_coverage": [
+        {
+          "complete": false,
+          "truncated": true,
+          "truncation_reasons": [
+            "candidate_limit",
+            "per_document_limit",
+            "result_limit"
+          ],
+          "index_version": {
+            "version": "10b78a8e70744e2792ffe4cf11d45571",
+            "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+            "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+            "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+            "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+            "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+            "embedding_model": "intfloat/multilingual-e5-small",
+            "publication_policy": "explicit-false-hidden-v1"
+          }
+        }
+      ],
+      "context_issues": [],
+      "accepted_for_delivery": true,
+      "answer": "Výpis je neúplný.\n\nRok začiatku štúdia osoby Lukáš Pokrývka je 2016, zatiaľ čo jeho diplomová práca bola realizovaná v roku 2021.\n\nZdroj:\nhttps://zp.kemt.fei.tuke.sk/students/2016/lukas_pokryvka",
+      "tool_called": true,
+      "tool_call_count": 1,
+      "tool_calls": [
+        {
+          "name": "retrieve_zpwiki_context",
+          "arguments": {
+            "query": "Rozlíš rok začiatku štúdia a rok práce osoby Lukáš Pokrývka.",
+            "limit": 10,
+            "published_only": true,
+            "max_per_document": 5
+          },
+          "latency_seconds": 8.727633,
+          "source_urls": [
+            "https://zp.kemt.fei.tuke.sk/students/2016/lukas_pokryvka",
+            "https://zp.kemt.fei.tuke.sk/students/2020/matej_kobylan",
+            "https://zp.kemt.fei.tuke.sk/students/2020/jakub_kristian_lukas",
+            "https://zp.kemt.fei.tuke.sk/students/2019/michal_stromko",
+            "https://zp.kemt.fei.tuke.sk/students/2018/ondrej_megela"
+          ]
+        }
+      ],
+      "rag_source_urls": [
+        "https://zp.kemt.fei.tuke.sk/students/2016/lukas_pokryvka",
+        "https://zp.kemt.fei.tuke.sk/students/2020/matej_kobylan",
+        "https://zp.kemt.fei.tuke.sk/students/2020/jakub_kristian_lukas",
+        "https://zp.kemt.fei.tuke.sk/students/2019/michal_stromko",
+        "https://zp.kemt.fei.tuke.sk/students/2018/ondrej_megela"
+      ],
+      "first_model_latency_seconds": 5.710692,
+      "tool_latency_seconds": 8.727633,
+      "final_model_latency_seconds": 40.777827,
+      "total_latency_seconds": 55.216681,
+      "usage": {
+        "prompt_tokens": 20160,
+        "completion_tokens": 155,
+        "total_tokens": 20315
+      },
+      "response_model": "Qwen3.5-122B-A10B-Q4_K_M",
+      "citations_checked": true,
+      "answer_matches": [
+        true,
+        true
+      ],
+      "answer_contains_score": 1.0,
+      "source_matches": [
+        true
+      ],
+      "source_match_count": 1,
+      "source_required_count": 1,
+      "source_url_score": 1.0,
+      "should_answer_ok": true,
+      "should_answer_score": 1.0,
+      "returned_no_answer": false,
+      "tool_score": 1.0,
+      "overall_score": 1.0,
+      "strict_pass": true,
+      "model_answer": "Rok začiatku štúdia osoby Lukáš Pokrývka je 2016, zatiaľ čo jeho diplomová práca bola realizovaná v roku 2021.\n\nZdroj:\nhttps://zp.kemt.fei.tuke.sk/students/2016/lukas_pokryvka",
+      "first_pass_accepted_for_delivery": false,
+      "first_pass_context_issues": [
+        "missing_incompleteness_notice"
+      ],
+      "display_notice_added": true
+    },
+    {
+      "id": "q0608",
+      "question": "Aký source_url patrí k stránke „Podpora slovenčiny v knižnici Spacy“?",
+      "citations_ok": true,
+      "unsupported_urls": [],
+      "retrieval_coverage": [
+        {
+          "complete": false,
+          "truncated": true,
+          "truncation_reasons": [
+            "candidate_limit",
+            "per_document_limit",
+            "result_limit"
+          ],
+          "index_version": {
+            "version": "10b78a8e70744e2792ffe4cf11d45571",
+            "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+            "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+            "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+            "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+            "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+            "embedding_model": "intfloat/multilingual-e5-small",
+            "publication_policy": "explicit-false-hidden-v1"
+          }
+        }
+      ],
+      "context_issues": [],
+      "accepted_for_delivery": true,
+      "answer": "Výpis je neúplný.\n\nStránka s názvom „Podpora slovenčiny v knižnici Spacy\" má pridelený zdrojový URL adresu https://zp.kemt.fei.tuke.sk/topics/spacy.\n\nZdroj: https://zp.kemt.fei.tuke.sk/topics/spacy",
+      "tool_called": true,
+      "tool_call_count": 1,
+      "tool_calls": [
+        {
+          "name": "retrieve_zpwiki_context",
+          "arguments": {
+            "query": "Aký source_url patrí k stránke „Podpora slovenčiny v knižnici Spacy“?",
+            "limit": 5,
+            "max_per_document": 1,
+            "published_only": true
+          },
+          "latency_seconds": 0.620349,
+          "source_urls": [
+            "https://zp.kemt.fei.tuke.sk/topics/spacy",
+            "https://zp.kemt.fei.tuke.sk/students/2016/maros_harahus",
+            "https://zp.kemt.fei.tuke.sk/students/2016/jakub_maruniak",
+            "https://zp.kemt.fei.tuke.sk/students/2017/than_trung_thanh",
+            "https://zp.kemt.fei.tuke.sk/topics/resources"
+          ]
+        }
+      ],
+      "rag_source_urls": [
+        "https://zp.kemt.fei.tuke.sk/topics/spacy",
+        "https://zp.kemt.fei.tuke.sk/students/2016/maros_harahus",
+        "https://zp.kemt.fei.tuke.sk/students/2016/jakub_maruniak",
+        "https://zp.kemt.fei.tuke.sk/students/2017/than_trung_thanh",
+        "https://zp.kemt.fei.tuke.sk/topics/resources"
+      ],
+      "first_model_latency_seconds": 28.658617,
+      "tool_latency_seconds": 0.620349,
+      "final_model_latency_seconds": 35.815477,
+      "total_latency_seconds": 65.094973,
+      "usage": {
+        "prompt_tokens": 28510,
+        "completion_tokens": 141,
+        "total_tokens": 28651
+      },
+      "response_model": "Qwen3.5-122B-A10B-Q4_K_M",
+      "citations_checked": true,
+      "answer_matches": [
+        true
+      ],
+      "answer_contains_score": 1.0,
+      "source_matches": [
+        true
+      ],
+      "source_match_count": 1,
+      "source_required_count": 1,
+      "source_url_score": 1.0,
+      "should_answer_ok": true,
+      "should_answer_score": 1.0,
+      "returned_no_answer": false,
+      "tool_score": 1.0,
+      "overall_score": 1.0,
+      "strict_pass": true,
+      "model_answer": "Stránka s názvom „Podpora slovenčiny v knižnici Spacy\" má pridelený zdrojový URL adresu https://zp.kemt.fei.tuke.sk/topics/spacy.\n\nZdroj: https://zp.kemt.fei.tuke.sk/topics/spacy",
+      "first_pass_accepted_for_delivery": false,
+      "first_pass_context_issues": [
+        "missing_incompleteness_notice"
+      ],
+      "display_notice_added": true
+    },
+    {
+      "id": "q1147",
+      "question": "Porovnaj záznamy osôb Ján Holp a Ondrej Megela. Pri osobe Ján Holp použi záznam typu „diplomová práca“ a pri osobe Ondrej Megela záznam typu „bakalárska práca“. Uveď názov každej práce a rok, pri ktorom je na ZP Wiki uvedená.",
+      "citations_ok": true,
+      "unsupported_urls": [],
+      "retrieval_coverage": [
+        {
+          "complete": false,
+          "truncated": true,
+          "truncation_reasons": [
+            "candidate_limit",
+            "per_document_limit",
+            "result_limit"
+          ],
+          "index_version": {
+            "version": "10b78a8e70744e2792ffe4cf11d45571",
+            "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+            "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+            "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+            "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+            "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+            "embedding_model": "intfloat/multilingual-e5-small",
+            "publication_policy": "explicit-false-hidden-v1"
+          }
+        }
+      ],
+      "context_issues": [],
+      "accepted_for_delivery": true,
+      "answer": "Výpis je neúplný.\n\nJán Holp vypracoval diplomovú prácu s názvom „Systém získavania informácií v slovenskom jazyku\" v roku 2021 a Ondrej Megela vypracoval bakalársku prácu s názvom „Neurónové jazykové modelovanie typu BERT\" v roku 2020.\n\nZdroje:\nhttps://zp.kemt.fei.tuke.sk/students/2016/jan_holp\nhttps://zp.kemt.fei.tuke.sk/students/2018/ondrej_megela",
+      "tool_called": true,
+      "tool_call_count": 1,
+      "tool_calls": [
+        {
+          "name": "retrieve_zpwiki_context",
+          "arguments": {
+            "query": "Porovnaj záznamy osôb Ján Holp a Ondrej Megela. Pri osobe Ján Holp použi záznam typu „diplomová práca“ a pri osobe Ondrej Megela záznam typu „bakalárska práca“. Uveď názov každej práce a rok, pri ktorom je na ZP Wiki uvedená.",
+            "limit": 5,
+            "published_only": true,
+            "max_per_document": 1
+          },
+          "latency_seconds": 1.001685,
+          "source_urls": [
+            "https://zp.kemt.fei.tuke.sk/students/2018/ondrej_megela",
+            "https://zp.kemt.fei.tuke.sk/students/2016/jan_holp",
+            "https://zp.kemt.fei.tuke.sk/students/2019/michal_stromko",
+            "https://zp.kemt.fei.tuke.sk/topics/faq",
+            "https://zp.kemt.fei.tuke.sk/topics/bert"
+          ]
+        }
+      ],
+      "rag_source_urls": [
+        "https://zp.kemt.fei.tuke.sk/students/2018/ondrej_megela",
+        "https://zp.kemt.fei.tuke.sk/students/2016/jan_holp",
+        "https://zp.kemt.fei.tuke.sk/students/2019/michal_stromko",
+        "https://zp.kemt.fei.tuke.sk/topics/faq",
+        "https://zp.kemt.fei.tuke.sk/topics/bert"
+      ],
+      "first_model_latency_seconds": 14.535465,
+      "tool_latency_seconds": 1.001685,
+      "final_model_latency_seconds": 56.647723,
+      "total_latency_seconds": 72.185893,
+      "usage": {
+        "prompt_tokens": 25893,
+        "completion_tokens": 208,
+        "total_tokens": 26101
+      },
+      "response_model": "Qwen3.5-122B-A10B-Q4_K_M",
+      "citations_checked": true,
+      "answer_matches": [
+        true,
+        true,
+        true,
+        true
+      ],
+      "answer_contains_score": 1.0,
+      "source_matches": [
+        true,
+        true
+      ],
+      "source_match_count": 2,
+      "source_required_count": 2,
+      "source_url_score": 1.0,
+      "should_answer_ok": true,
+      "should_answer_score": 1.0,
+      "returned_no_answer": false,
+      "tool_score": 1.0,
+      "overall_score": 1.0,
+      "strict_pass": true,
+      "model_answer": "Ján Holp vypracoval diplomovú prácu s názvom „Systém získavania informácií v slovenskom jazyku\" v roku 2021 a Ondrej Megela vypracoval bakalársku prácu s názvom „Neurónové jazykové modelovanie typu BERT\" v roku 2020.\n\nZdroje:\nhttps://zp.kemt.fei.tuke.sk/students/2016/jan_holp\nhttps://zp.kemt.fei.tuke.sk/students/2018/ondrej_megela",
+      "first_pass_accepted_for_delivery": false,
+      "first_pass_context_issues": [
+        "missing_incompleteness_notice"
+      ],
+      "display_notice_added": true
+    },
+    {
+      "id": "q0888",
+      "question": "Aký konkrétny finančný rozpočet bol uvedený pre projekt osoby Tomáš Kuchárik?",
+      "citations_ok": true,
+      "unsupported_urls": [],
+      "retrieval_coverage": [
+        {
+          "complete": false,
+          "truncated": true,
+          "truncation_reasons": [
+            "candidate_limit",
+            "per_document_limit",
+            "result_limit"
+          ],
+          "index_version": {
+            "version": "10b78a8e70744e2792ffe4cf11d45571",
+            "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+            "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+            "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+            "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+            "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+            "embedding_model": "intfloat/multilingual-e5-small",
+            "publication_policy": "explicit-false-hidden-v1"
+          }
+        }
+      ],
+      "context_issues": [],
+      "accepted_for_delivery": true,
+      "answer": "Výpis je neúplný.\n\nV dostupných dokumentoch ZP Wiki sa informácia o konkrétnom finančnom rozpočte pre projekt osoby Tomáš Kuchárik nepodarilo spoľahlivo nájsť.\n\nZdroj:\nhttps://zp.kemt.fei.tuke.sk/students/2016/tomas_kucharik",
+      "tool_called": true,
+      "tool_call_count": 1,
+      "tool_calls": [
+        {
+          "name": "retrieve_zpwiki_context",
+          "arguments": {
+            "query": "Aký konkrétny finančný rozpočet bol uvedený pre projekt osoby Tomáš Kuchárik?",
+            "limit": 5,
+            "published_only": true
+          },
+          "latency_seconds": 0.454629,
+          "source_urls": [
+            "https://zp.kemt.fei.tuke.sk/students/2016/tomas_kucharik",
+            "https://zp.kemt.fei.tuke.sk/students/2020/matej_kobylan",
+            "https://zp.kemt.fei.tuke.sk/students/2019/filip_tomas",
+            "https://zp.kemt.fei.tuke.sk/students/2020/matus_suchanic",
+            "https://zp.kemt.fei.tuke.sk/students/2019/michal_stromko"
+          ]
+        }
+      ],
+      "rag_source_urls": [
+        "https://zp.kemt.fei.tuke.sk/students/2016/tomas_kucharik",
+        "https://zp.kemt.fei.tuke.sk/students/2020/matej_kobylan",
+        "https://zp.kemt.fei.tuke.sk/students/2019/filip_tomas",
+        "https://zp.kemt.fei.tuke.sk/students/2020/matus_suchanic",
+        "https://zp.kemt.fei.tuke.sk/students/2019/michal_stromko"
+      ],
+      "first_model_latency_seconds": 129.609931,
+      "tool_latency_seconds": 0.454629,
+      "final_model_latency_seconds": 20.02263,
+      "total_latency_seconds": 150.087617,
+      "usage": {
+        "prompt_tokens": 14022,
+        "completion_tokens": 136,
+        "total_tokens": 14158
+      },
+      "response_model": "Qwen3.5-122B-A10B-Q4_K_M",
+      "citations_checked": true,
+      "answer_matches": [],
+      "answer_contains_score": 1.0,
+      "source_matches": [
+        true
+      ],
+      "source_match_count": 1,
+      "source_required_count": 0,
+      "source_url_score": 1.0,
+      "should_answer_ok": true,
+      "should_answer_score": 1.0,
+      "returned_no_answer": true,
+      "tool_score": 1.0,
+      "overall_score": 1.0,
+      "strict_pass": true,
+      "model_answer": "V dostupných dokumentoch ZP Wiki sa informácia o konkrétnom finančnom rozpočte pre projekt osoby Tomáš Kuchárik nepodarilo spoľahlivo nájsť.\n\nZdroj:\nhttps://zp.kemt.fei.tuke.sk/students/2016/tomas_kucharik",
+      "first_pass_accepted_for_delivery": false,
+      "first_pass_context_issues": [
+        "missing_incompleteness_notice"
+      ],
+      "display_notice_added": true
+    }
+  ],
+  "summary": {
+    "questions": 5,
+    "strict_pass": 5,
+    "display_notice_added": 5
+  }
+}

tools/graphrag/evaluation/results/v6_preparation/rag_live_smoke.declaration.json

@@ -0,0 +1,19 @@
+{
+  "method": "Five preselected historical RAG smoke cases; not a new accuracy estimate.",
+  "selected_ids": [
+    "q0053",
+    "q0060",
+    "q0608",
+    "q1147",
+    "q0888"
+  ],
+  "model": "model120-fast",
+  "source_file": "tools/rag/evaluation/results/test_classic_rag_frozen/rag_answers_test.json",
+  "source_sha256": "eba03639f8a3c32ee47a4e294d8592c1071fc6fd43e38967e39131b840fefa7e",
+  "generation_files": {
+    "/home/janko/DP/zp-agent/tools/answer_evaluation/check_rag_runtime.py": "37eef74ed8120109ed0c7f2d5a0edba497687a76ab5eaa154bd4626c9edb311c",
+    "tools/rag/evaluation/rag_runner.py": "031abf7d5b1a026eb199aa2a579bfb8baaa01175279d0018a364bfe50b76c3c1",
+    "tools/rag/evaluation/rag_metrics.py": "3e95922df90dd1b757a550040a2d9bd4f857e17d179f60170d7cc881a08f94b6",
+    "tools/answer_evaluation/context_policy.py": "e7b7c511e5e9cafdef04969eb4007a6a61ac4b99bcc60e9e722945a15bec1f26"
+  }
+}

tools/graphrag/evaluation/results/v6_preparation/rag_live_smoke.json

@@ -0,0 +1,464 @@
+{
+  "configuration": {
+    "method": "Five preselected historical RAG smoke cases; not a new accuracy estimate.",
+    "selected_ids": [
+      "q0053",
+      "q0060",
+      "q0608",
+      "q1147",
+      "q0888"
+    ],
+    "model": "model120-fast",
+    "source_file": "tools/rag/evaluation/results/test_classic_rag_frozen/rag_answers_test.json",
+    "source_sha256": "eba03639f8a3c32ee47a4e294d8592c1071fc6fd43e38967e39131b840fefa7e",
+    "generation_files": {
+      "/home/janko/DP/zp-agent/tools/answer_evaluation/check_rag_runtime.py": "37eef74ed8120109ed0c7f2d5a0edba497687a76ab5eaa154bd4626c9edb311c",
+      "tools/rag/evaluation/rag_runner.py": "031abf7d5b1a026eb199aa2a579bfb8baaa01175279d0018a364bfe50b76c3c1",
+      "tools/rag/evaluation/rag_metrics.py": "3e95922df90dd1b757a550040a2d9bd4f857e17d179f60170d7cc881a08f94b6",
+      "tools/answer_evaluation/context_policy.py": "e7b7c511e5e9cafdef04969eb4007a6a61ac4b99bcc60e9e722945a15bec1f26"
+    }
+  },
+  "results": [
+    {
+      "id": "q0053",
+      "question": "Aký je názov diplomová práca osoby Lukáš Pokrývka?",
+      "citations_ok": true,
+      "unsupported_urls": [],
+      "retrieval_coverage": [
+        {
+          "complete": false,
+          "truncated": true,
+          "truncation_reasons": [
+            "candidate_limit",
+            "per_document_limit",
+            "result_limit"
+          ],
+          "index_version": {
+            "version": "10b78a8e70744e2792ffe4cf11d45571",
+            "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+            "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+            "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+            "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+            "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+            "embedding_model": "intfloat/multilingual-e5-small",
+            "publication_policy": "explicit-false-hidden-v1"
+          }
+        }
+      ],
+      "context_issues": [
+        "missing_incompleteness_notice"
+      ],
+      "accepted_for_delivery": false,
+      "answer": "Názov diplomovej práce Lukáša Pokrývku je „Paralelné trénovanie neurónových sietí\".\n\nZdroj: https://zp.kemt.fei.tuke.sk/students/2016/lukas_pokryvka",
+      "tool_called": true,
+      "tool_call_count": 1,
+      "tool_calls": [
+        {
+          "name": "retrieve_zpwiki_context",
+          "arguments": {
+            "query": "Aký je názov diplomová práca osoby Lukáš Pokrývka?",
+            "limit": 5,
+            "published_only": true,
+            "max_per_document": 1
+          },
+          "latency_seconds": 8.6679,
+          "source_urls": [
+            "https://zp.kemt.fei.tuke.sk/students/2016/lukas_pokryvka",
+            "https://zp.kemt.fei.tuke.sk/students/2016/tomas_kucharik",
+            "https://zp.kemt.fei.tuke.sk/students/2019/patrik_pokrivcak",
+            "https://zp.kemt.fei.tuke.sk/students/2018/oleh_bilykh",
+            "https://zp.kemt.fei.tuke.sk/students/2016/jakub_maruniak/dp2021"
+          ]
+        }
+      ],
+      "rag_source_urls": [
+        "https://zp.kemt.fei.tuke.sk/students/2016/lukas_pokryvka",
+        "https://zp.kemt.fei.tuke.sk/students/2016/tomas_kucharik",
+        "https://zp.kemt.fei.tuke.sk/students/2019/patrik_pokrivcak",
+        "https://zp.kemt.fei.tuke.sk/students/2018/oleh_bilykh",
+        "https://zp.kemt.fei.tuke.sk/students/2016/jakub_maruniak/dp2021"
+      ],
+      "first_model_latency_seconds": 5.65362,
+      "tool_latency_seconds": 8.6679,
+      "final_model_latency_seconds": 23.053733,
+      "total_latency_seconds": 37.375855,
+      "usage": {
+        "prompt_tokens": 19045,
+        "completion_tokens": 137,
+        "total_tokens": 19182
+      },
+      "response_model": "Qwen3.5-122B-A10B-Q4_K_M",
+      "citations_checked": true,
+      "answer_matches": [
+        true
+      ],
+      "answer_contains_score": 1.0,
+      "source_matches": [
+        true
+      ],
+      "source_match_count": 1,
+      "source_required_count": 1,
+      "source_url_score": 1.0,
+      "should_answer_ok": true,
+      "should_answer_score": 1.0,
+      "returned_no_answer": false,
+      "tool_score": 1.0,
+      "overall_score": 1.0,
+      "strict_pass": false
+    },
+    {
+      "id": "q0060",
+      "question": "Rozlíš rok začiatku štúdia a rok práce osoby Lukáš Pokrývka.",
+      "citations_ok": true,
+      "unsupported_urls": [],
+      "retrieval_coverage": [
+        {
+          "complete": false,
+          "truncated": true,
+          "truncation_reasons": [
+            "candidate_limit",
+            "per_document_limit",
+            "result_limit"
+          ],
+          "index_version": {
+            "version": "10b78a8e70744e2792ffe4cf11d45571",
+            "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+            "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+            "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+            "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+            "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+            "embedding_model": "intfloat/multilingual-e5-small",
+            "publication_policy": "explicit-false-hidden-v1"
+          }
+        }
+      ],
+      "context_issues": [
+        "missing_incompleteness_notice"
+      ],
+      "accepted_for_delivery": false,
+      "answer": "Rok začiatku štúdia osoby Lukáš Pokrývka je 2016, zatiaľ čo jeho diplomová práca bola realizovaná v roku 2021.\n\nZdroj:\nhttps://zp.kemt.fei.tuke.sk/students/2016/lukas_pokryvka",
+      "tool_called": true,
+      "tool_call_count": 1,
+      "tool_calls": [
+        {
+          "name": "retrieve_zpwiki_context",
+          "arguments": {
+            "query": "Rozlíš rok začiatku štúdia a rok práce osoby Lukáš Pokrývka.",
+            "limit": 10,
+            "published_only": true,
+            "max_per_document": 5
+          },
+          "latency_seconds": 8.727633,
+          "source_urls": [
+            "https://zp.kemt.fei.tuke.sk/students/2016/lukas_pokryvka",
+            "https://zp.kemt.fei.tuke.sk/students/2020/matej_kobylan",
+            "https://zp.kemt.fei.tuke.sk/students/2020/jakub_kristian_lukas",
+            "https://zp.kemt.fei.tuke.sk/students/2019/michal_stromko",
+            "https://zp.kemt.fei.tuke.sk/students/2018/ondrej_megela"
+          ]
+        }
+      ],
+      "rag_source_urls": [
+        "https://zp.kemt.fei.tuke.sk/students/2016/lukas_pokryvka",
+        "https://zp.kemt.fei.tuke.sk/students/2020/matej_kobylan",
+        "https://zp.kemt.fei.tuke.sk/students/2020/jakub_kristian_lukas",
+        "https://zp.kemt.fei.tuke.sk/students/2019/michal_stromko",
+        "https://zp.kemt.fei.tuke.sk/students/2018/ondrej_megela"
+      ],
+      "first_model_latency_seconds": 5.710692,
+      "tool_latency_seconds": 8.727633,
+      "final_model_latency_seconds": 40.777827,
+      "total_latency_seconds": 55.216681,
+      "usage": {
+        "prompt_tokens": 20160,
+        "completion_tokens": 155,
+        "total_tokens": 20315
+      },
+      "response_model": "Qwen3.5-122B-A10B-Q4_K_M",
+      "citations_checked": true,
+      "answer_matches": [
+        true,
+        true
+      ],
+      "answer_contains_score": 1.0,
+      "source_matches": [
+        true
+      ],
+      "source_match_count": 1,
+      "source_required_count": 1,
+      "source_url_score": 1.0,
+      "should_answer_ok": true,
+      "should_answer_score": 1.0,
+      "returned_no_answer": false,
+      "tool_score": 1.0,
+      "overall_score": 1.0,
+      "strict_pass": false
+    },
+    {
+      "id": "q0608",
+      "question": "Aký source_url patrí k stránke „Podpora slovenčiny v knižnici Spacy“?",
+      "citations_ok": true,
+      "unsupported_urls": [],
+      "retrieval_coverage": [
+        {
+          "complete": false,
+          "truncated": true,
+          "truncation_reasons": [
+            "candidate_limit",
+            "per_document_limit",
+            "result_limit"
+          ],
+          "index_version": {
+            "version": "10b78a8e70744e2792ffe4cf11d45571",
+            "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+            "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+            "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+            "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+            "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+            "embedding_model": "intfloat/multilingual-e5-small",
+            "publication_policy": "explicit-false-hidden-v1"
+          }
+        }
+      ],
+      "context_issues": [
+        "missing_incompleteness_notice"
+      ],
+      "accepted_for_delivery": false,
+      "answer": "Stránka s názvom „Podpora slovenčiny v knižnici Spacy\" má pridelený zdrojový URL adresu https://zp.kemt.fei.tuke.sk/topics/spacy.\n\nZdroj: https://zp.kemt.fei.tuke.sk/topics/spacy",
+      "tool_called": true,
+      "tool_call_count": 1,
+      "tool_calls": [
+        {
+          "name": "retrieve_zpwiki_context",
+          "arguments": {
+            "query": "Aký source_url patrí k stránke „Podpora slovenčiny v knižnici Spacy“?",
+            "limit": 5,
+            "max_per_document": 1,
+            "published_only": true
+          },
+          "latency_seconds": 0.620349,
+          "source_urls": [
+            "https://zp.kemt.fei.tuke.sk/topics/spacy",
+            "https://zp.kemt.fei.tuke.sk/students/2016/maros_harahus",
+            "https://zp.kemt.fei.tuke.sk/students/2016/jakub_maruniak",
+            "https://zp.kemt.fei.tuke.sk/students/2017/than_trung_thanh",
+            "https://zp.kemt.fei.tuke.sk/topics/resources"
+          ]
+        }
+      ],
+      "rag_source_urls": [
+        "https://zp.kemt.fei.tuke.sk/topics/spacy",
+        "https://zp.kemt.fei.tuke.sk/students/2016/maros_harahus",
+        "https://zp.kemt.fei.tuke.sk/students/2016/jakub_maruniak",
+        "https://zp.kemt.fei.tuke.sk/students/2017/than_trung_thanh",
+        "https://zp.kemt.fei.tuke.sk/topics/resources"
+      ],
+      "first_model_latency_seconds": 28.658617,
+      "tool_latency_seconds": 0.620349,
+      "final_model_latency_seconds": 35.815477,
+      "total_latency_seconds": 65.094973,
+      "usage": {
+        "prompt_tokens": 28510,
+        "completion_tokens": 141,
+        "total_tokens": 28651
+      },
+      "response_model": "Qwen3.5-122B-A10B-Q4_K_M",
+      "citations_checked": true,
+      "answer_matches": [
+        true
+      ],
+      "answer_contains_score": 1.0,
+      "source_matches": [
+        true
+      ],
+      "source_match_count": 1,
+      "source_required_count": 1,
+      "source_url_score": 1.0,
+      "should_answer_ok": true,
+      "should_answer_score": 1.0,
+      "returned_no_answer": false,
+      "tool_score": 1.0,
+      "overall_score": 1.0,
+      "strict_pass": false
+    },
+    {
+      "id": "q1147",
+      "question": "Porovnaj záznamy osôb Ján Holp a Ondrej Megela. Pri osobe Ján Holp použi záznam typu „diplomová práca“ a pri osobe Ondrej Megela záznam typu „bakalárska práca“. Uveď názov každej práce a rok, pri ktorom je na ZP Wiki uvedená.",
+      "citations_ok": true,
+      "unsupported_urls": [],
+      "retrieval_coverage": [
+        {
+          "complete": false,
+          "truncated": true,
+          "truncation_reasons": [
+            "candidate_limit",
+            "per_document_limit",
+            "result_limit"
+          ],
+          "index_version": {
+            "version": "10b78a8e70744e2792ffe4cf11d45571",
+            "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+            "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+            "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+            "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+            "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+            "embedding_model": "intfloat/multilingual-e5-small",
+            "publication_policy": "explicit-false-hidden-v1"
+          }
+        }
+      ],
+      "context_issues": [
+        "missing_incompleteness_notice"
+      ],
+      "accepted_for_delivery": false,
+      "answer": "Ján Holp vypracoval diplomovú prácu s názvom „Systém získavania informácií v slovenskom jazyku\" v roku 2021 a Ondrej Megela vypracoval bakalársku prácu s názvom „Neurónové jazykové modelovanie typu BERT\" v roku 2020.\n\nZdroje:\nhttps://zp.kemt.fei.tuke.sk/students/2016/jan_holp\nhttps://zp.kemt.fei.tuke.sk/students/2018/ondrej_megela",
+      "tool_called": true,
+      "tool_call_count": 1,
+      "tool_calls": [
+        {
+          "name": "retrieve_zpwiki_context",
+          "arguments": {
+            "query": "Porovnaj záznamy osôb Ján Holp a Ondrej Megela. Pri osobe Ján Holp použi záznam typu „diplomová práca“ a pri osobe Ondrej Megela záznam typu „bakalárska práca“. Uveď názov každej práce a rok, pri ktorom je na ZP Wiki uvedená.",
+            "limit": 5,
+            "published_only": true,
+            "max_per_document": 1
+          },
+          "latency_seconds": 1.001685,
+          "source_urls": [
+            "https://zp.kemt.fei.tuke.sk/students/2018/ondrej_megela",
+            "https://zp.kemt.fei.tuke.sk/students/2016/jan_holp",
+            "https://zp.kemt.fei.tuke.sk/students/2019/michal_stromko",
+            "https://zp.kemt.fei.tuke.sk/topics/faq",
+            "https://zp.kemt.fei.tuke.sk/topics/bert"
+          ]
+        }
+      ],
+      "rag_source_urls": [
+        "https://zp.kemt.fei.tuke.sk/students/2018/ondrej_megela",
+        "https://zp.kemt.fei.tuke.sk/students/2016/jan_holp",
+        "https://zp.kemt.fei.tuke.sk/students/2019/michal_stromko",
+        "https://zp.kemt.fei.tuke.sk/topics/faq",
+        "https://zp.kemt.fei.tuke.sk/topics/bert"
+      ],
+      "first_model_latency_seconds": 14.535465,
+      "tool_latency_seconds": 1.001685,
+      "final_model_latency_seconds": 56.647723,
+      "total_latency_seconds": 72.185893,
+      "usage": {
+        "prompt_tokens": 25893,
+        "completion_tokens": 208,
+        "total_tokens": 26101
+      },
+      "response_model": "Qwen3.5-122B-A10B-Q4_K_M",
+      "citations_checked": true,
+      "answer_matches": [
+        true,
+        true,
+        true,
+        true
+      ],
+      "answer_contains_score": 1.0,
+      "source_matches": [
+        true,
+        true
+      ],
+      "source_match_count": 2,
+      "source_required_count": 2,
+      "source_url_score": 1.0,
+      "should_answer_ok": true,
+      "should_answer_score": 1.0,
+      "returned_no_answer": false,
+      "tool_score": 1.0,
+      "overall_score": 1.0,
+      "strict_pass": false
+    },
+    {
+      "id": "q0888",
+      "question": "Aký konkrétny finančný rozpočet bol uvedený pre projekt osoby Tomáš Kuchárik?",
+      "citations_ok": true,
+      "unsupported_urls": [],
+      "retrieval_coverage": [
+        {
+          "complete": false,
+          "truncated": true,
+          "truncation_reasons": [
+            "candidate_limit",
+            "per_document_limit",
+            "result_limit"
+          ],
+          "index_version": {
+            "version": "10b78a8e70744e2792ffe4cf11d45571",
+            "git_revision": "3e892a1d49e9bce118cd03478ce15e01c2bebb4e",
+            "created_at_utc": "2026-10-25T21:10:39.160542+00:00",
+            "source_sha256": "007792c12f3467dd3203bb4951ccc197fdc0eef669580f1b1f893baf609ca363",
+            "documents_sha256": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+            "chunks_sha256": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9",
+            "embedding_model": "intfloat/multilingual-e5-small",
+            "publication_policy": "explicit-false-hidden-v1"
+          }
+        }
+      ],
+      "context_issues": [
+        "missing_incompleteness_notice"
+      ],
+      "accepted_for_delivery": false,
+      "answer": "V dostupných dokumentoch ZP Wiki sa informácia o konkrétnom finančnom rozpočte pre projekt osoby Tomáš Kuchárik nepodarilo spoľahlivo nájsť.\n\nZdroj:\nhttps://zp.kemt.fei.tuke.sk/students/2016/tomas_kucharik",
+      "tool_called": true,
+      "tool_call_count": 1,
+      "tool_calls": [
+        {
+          "name": "retrieve_zpwiki_context",
+          "arguments": {
+            "query": "Aký konkrétny finančný rozpočet bol uvedený pre projekt osoby Tomáš Kuchárik?",
+            "limit": 5,
+            "published_only": true
+          },
+          "latency_seconds": 0.454629,
+          "source_urls": [
+            "https://zp.kemt.fei.tuke.sk/students/2016/tomas_kucharik",
+            "https://zp.kemt.fei.tuke.sk/students/2020/matej_kobylan",
+            "https://zp.kemt.fei.tuke.sk/students/2019/filip_tomas",
+            "https://zp.kemt.fei.tuke.sk/students/2020/matus_suchanic",
+            "https://zp.kemt.fei.tuke.sk/students/2019/michal_stromko"
+          ]
+        }
+      ],
+      "rag_source_urls": [
+        "https://zp.kemt.fei.tuke.sk/students/2016/tomas_kucharik",
+        "https://zp.kemt.fei.tuke.sk/students/2020/matej_kobylan",
+        "https://zp.kemt.fei.tuke.sk/students/2019/filip_tomas",
+        "https://zp.kemt.fei.tuke.sk/students/2020/matus_suchanic",
+        "https://zp.kemt.fei.tuke.sk/students/2019/michal_stromko"
+      ],
+      "first_model_latency_seconds": 129.609931,
+      "tool_latency_seconds": 0.454629,
+      "final_model_latency_seconds": 20.02263,
+      "total_latency_seconds": 150.087617,
+      "usage": {
+        "prompt_tokens": 14022,
+        "completion_tokens": 136,
+        "total_tokens": 14158
+      },
+      "response_model": "Qwen3.5-122B-A10B-Q4_K_M",
+      "citations_checked": true,
+      "answer_matches": [],
+      "answer_contains_score": 1.0,
+      "source_matches": [
+        true
+      ],
+      "source_match_count": 1,
+      "source_required_count": 0,
+      "source_url_score": 1.0,
+      "should_answer_ok": true,
+      "should_answer_score": 1.0,
+      "returned_no_answer": true,
+      "tool_score": 1.0,
+      "overall_score": 1.0,
+      "strict_pass": false
+    }
+  ],
+  "summary": {
+    "questions": 5,
+    "strict_pass": 0
+  }
+}

tools/graphrag/evaluation/results/v6_preparation/review_selection.json

@@ -0,0 +1,39 @@
+{
+  "method": "Three fixed positions per category plus known semantic risk cases; no current retrieval output used.",
+  "ids": [
+    "wide-217",
+    "wide-233",
+    "wide-248",
+    "wide-249",
+    "wide-269",
+    "wide-288",
+    "wide-169",
+    "wide-181",
+    "wide-192",
+    "wide-289",
+    "wide-305",
+    "wide-320",
+    "wide-137",
+    "wide-153",
+    "wide-168",
+    "wide-193",
+    "wide-205",
+    "wide-216",
+    "wide-113",
+    "wide-125",
+    "wide-136",
+    "wide-041",
+    "wide-061",
+    "wide-080",
+    "wide-081",
+    "wide-097",
+    "wide-112",
+    "wide-001",
+    "wide-021",
+    "wide-040",
+    "wide-275",
+    "wide-129",
+    "wide-142",
+    "wide-145"
+  ]
+}

tools/graphrag/evaluation/results/v6_preparation/source_review.csv

@@ -0,0 +1,35 @@
+id,category,question,ai_decision,ai_notes,source_files,source_sha256,reviewer,human_reviewer,human_decision,human_notes
+wide-217,candidate_set_aggregation,"Z kandidátov Dárius Lindvai, Patrik Pavlišin, Adrián Remiáš vyber osoby s prácou v roku 2020. Koľko ich je?",supported,"Roky v akademických nadpisoch; overené aj vyradené osoby, začiatok štúdia nie je rok práce.",pages/students/2016/darius_lindvai/README.md; pages/students/2016/patrik_pavlisin/README.md; pages/students/2018/adrian_remias/README.md,fcaa47bb9708a6439eec5c4b7f7b4ed6c7359e423e1d940e1aed9f44957e6d62; dc1c648d7cd9b0f0e021c57f9b98189e37c100871439e7e304390715913f2763; e5bfe1d5d4e626beaa636a02d34ec58cf2cff9b0e57833905af35b80bf1465a2,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-233,candidate_set_aggregation,"Z kandidátov Alina Vitko, Jakub Maruniak, Adrián Remiáš vyber osoby s prácou v roku 2021. Aký je ich počet?",supported,"Roky v akademických nadpisoch; overené aj vyradené osoby, začiatok štúdia nie je rok práce.",pages/students/2016/jakub_maruniak/README.md; pages/students/2018/adrian_remias/README.md; pages/students/2019/alina_vitko/README.md,6734c41a95b9d0c1e3d226b393ad109ee4a7b82ebfffa73df2d7e0a743fefa14; e5bfe1d5d4e626beaa636a02d34ec58cf2cff9b0e57833905af35b80bf1465a2; 9b071fe2aff470c275ae027be236fe836f741f774cabb9cbbb55304a8d5102cd,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-248,candidate_set_aggregation,"Z kandidátov Artur Hyrenko, Denis Le Thanh, Adrián Remiáš vyber osoby s prácou v roku 2026. Aký je ich počet?",supported,"Roky v akademických nadpisoch; overené aj vyradené osoby, začiatok štúdia nie je rok práce.",pages/students/2018/adrian_remias/README.md; pages/students/2021/artur_hyrenko/README.md; pages/students/2023/denis_le_thanh/README.md,e5bfe1d5d4e626beaa636a02d34ec58cf2cff9b0e57833905af35b80bf1465a2; 0568261f4ade4de5472edcfeb1ffed9abae18dc67e4050f4ce2ad088839a2bb3; 2cb83f942d86a02712b0c58778e46dc2211346d1a6eba606471dfa832245db61,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-249,conjunctive_disambiguation,Ktorá z osôb Lukáš Pokrývka a Dominik Nagy má kľúčové slovo cloud a jej práca je z roku 2019?,supported,Tag z taxonomy.tag a rok akademického záznamu musia patriť tej istej osobe.,pages/students/2016/dominik_nagy/README.md; pages/students/2016/lukas_pokryvka/README.md,1c4037a158d84c45fa705a0c38e47142a9d46e226f0c138bc2d56d7c38f6d74d; f279456c5c277ff1ee0dd89a3464ed9f9f67b44bc185a5da9811ef5cfb675002,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-269,conjunctive_disambiguation,Ktorá z osôb Oleh Bilykh a Artem Yatsenko spĺňa kľúčové slovo drqa a rok práce 2021?,supported,Tag z taxonomy.tag a rok akademického záznamu musia patriť tej istej osobe.,pages/students/2018/oleh_bilykh/README.md; pages/students/2019/artem_yatsenko/README.md,4072c62dd8ee753a0c8fa43fdfbbdf566e394c2df13e6a79d08a302e332710b1; 328b101e46c51c959e792cf7ca226942f81d2ab8fe9648ad6d0ecd07016429e2,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-288,conjunctive_disambiguation,Ktorá z osôb Matej Ščišľak a Jakub Schwarc má kľúčové slovo rasa a jej práca je z roku 2026?,supported,Tag z taxonomy.tag a rok akademického záznamu musia patriť tej istej osobe.,pages/students/2021/matej_scislak/README.md; pages/students/2022/jakub_schwarc/README.md,b9a32cce8605e044a4f2828ff2d91a2995a62ef3a841e62f0a225c97221f0cc8; f541a8852c2b0f6f9d7182e01a34c4e3beee9bfa55427a06b331638b11eb8be6,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-169,document_author_category_multi_hop,"Kto je autor dokumentu osoby Tetiana Mohorian, ktorý patrí do kategórie bp2025?",supported,Kód taxonomy.category a taxonomy.author sú v tom istom pôvodnom dokumente.,pages/students/2022/tetiana_mohorian/README.md,c9efdc632bf1ca89a0bfde0dc00be7aaac8a85da3d92e2ce8506236c73e93cfb,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-181,document_author_category_multi_hop,"Kto je autor dokumentu osoby Jakub Maruniak, ktorý patrí do kategórie dp2021?",supported,Kód taxonomy.category a taxonomy.author sú v tom istom pôvodnom dokumente.,pages/students/2016/jakub_maruniak/README.md,6734c41a95b9d0c1e3d226b393ad109ee4a7b82ebfffa73df2d7e0a743fefa14,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-192,document_author_category_multi_hop,Ktorý autor dokumentu je pri osobe Oleksandr Hryshchenko spojený s kategóriou dp2026?,supported,Kód taxonomy.category a taxonomy.author sú v tom istom pôvodnom dokumente.,pages/students/2021/oleksandr_hryshchenko/README.md,4b073992b715b6c75b79fd2c8f850a69f940966e3a61a82c625c32c945a93036,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-289,negative_lookup,Aký názov práce má osoba Alina Vitko s kľúčovým slovom anaconda?,supported,"Požadovaná kombinácia chýba; kontrolované metadata a akademické nadpisy, nie iba podobné mená.",pages/students/2019/alina_vitko/README.md,9b071fe2aff470c275ae027be236fe836f741f774cabb9cbbb55304a8d5102cd,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-305,negative_lookup,Aký názov práce má osoba Jakub Schwarc v roku 2018?,supported,"Požadovaná kombinácia chýba; kontrolované metadata a akademické nadpisy, nie iba podobné mená.",pages/students/2022/jakub_schwarc/README.md,f541a8852c2b0f6f9d7182e01a34c4e3beee9bfa55427a06b331638b11eb8be6,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-320,negative_lookup,Aký názov práce má osoba Dávid Iľaš v roku 2018?,supported,"Požadovaná kombinácia chýba; kontrolované metadata a akademické nadpisy, nie iba podobné mená.",pages/students/2018/david_ilas/README.md,96c45ff9c5a821ddcb1a2fc86d487bd8f4729d1b76887914b0f5fbf99af45af0,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-137,person_keyword_title,Aký názov práce osoby Lukáš Pokrývka v roku 2021 súvisí s kľúčovým slovom gpu?,supported,Názov a rok z akademickej sekcie; doslovný tag z rodiny dokumentov tej istej osoby.,pages/students/2016/lukas_pokryvka/README.md; pages/students/2016/lukas_pokryvka/timovy_projekt/README.md,f279456c5c277ff1ee0dd89a3464ed9f9f67b44bc185a5da9811ef5cfb675002; 4de4b6c8cfed17b6c60090f82df1d55a66be70a79d3013850689b2baebdd801b,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-153,person_keyword_title,Uveď názov práce osoby Michal Stromko v roku 2024 s kľúčovým slovom cloud.,supported,Názov a rok z akademickej sekcie; doslovný tag z rodiny dokumentov tej istej osoby.,pages/students/2019/michal_stromko/README.md,3ae1dd9b6638a6a249a94be3f3846286555666aefa225bb3a2c7288b35900ce7,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-168,person_keyword_title,Uveď názov práce osoby Matej Ščišľak v roku 2024 s kľúčovým slovom nlp.,supported,Názov a rok z akademickej sekcie; doslovný tag z rodiny dokumentov tej istej osoby.,pages/students/2021/matej_scislak/README.md,b9a32cce8605e044a4f2828ff2d91a2995a62ef3a841e62f0a225c97221f0cc8,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-193,person_multiple_categories,Vypíš všetky kategórie dokumentov osoby Matej Novotný.,supported,Doslovné taxonomy.category z profilu aj jeho dcérskych dokumentov.,pages/students/2021/matej_novotny/README.md,b083a7cadde532e3428dc9a57bc61456fcc5905bda91e011b96033d42bf5d9cb,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-205,person_multiple_categories,Vypíš všetky kategórie dokumentov osoby Oleksandr Hryshchenko.,supported,Doslovné taxonomy.category z profilu aj jeho dcérskych dokumentov.,pages/students/2021/oleksandr_hryshchenko/README.md,4b073992b715b6c75b79fd2c8f850a69f940966e3a61a82c625c32c945a93036,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-216,person_multiple_categories,Aké všetky kategórie majú dokumenty osoby Denis Le Thanh?,supported,Doslovné taxonomy.category z profilu aj jeho dcérskych dokumentov.,pages/students/2023/denis_le_thanh/README.md,2cb83f942d86a02712b0c58778e46dc2211346d1a6eba606471dfa832245db61,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-113,person_title_year,Aký názov práce a rok práce sú pri osobe Ján Holp uvedené pre rok 2021?,supported,Názov a rok doložené v akademickom zázname; profilový názov nie je názov práce.,pages/students/2016/jan_holp/README.md,80d6a8492766e5bfa61673a372e5f059d30c8e335de39842866c51485960e4d3,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-125,person_title_year,Aký názov práce a rok práce sú pri osobe Márk Fehér uvedené pre rok 2022?,supported,Názov a rok doložené v akademickom zázname; profilový názov nie je názov práce.,pages/students/2016/mark_feher/README.md,bec37c0319fe0869a0377d771db23d090dbda93c27328f5697cdaeb3bbccfe9b,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-136,person_title_year,Uveď názov diplomového projektu osoby Lukáš Pokrývka a rok práce pre rok 2020.,supported,Názov a rok doložené v akademickom zázname; profilový názov nie je názov práce.,pages/students/2016/lukas_pokryvka/README.md,f279456c5c277ff1ee0dd89a3464ed9f9f67b44bc185a5da9811ef5cfb675002,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-041,same_academic_year_pair,Aké spoločné roky práce spájajú osoby Jakub Kristián Lukas a Martin Šarišský?,supported,Prienik rokov akademických nadpisov oboch osôb; príprava na projekt je vylúčená.,pages/students/2020/jakub_kristian_lukas/README.md; pages/students/2021/martin_sarissky/README.md,f139d9b1ebf5a3f45612f3df5703ecdd935b90701763c27c7c529044c0b0455a; 8e97b45a253745faa55f63c844e0b5dfe680aeaf6eb01fb8f9d06ac6e8ee8bcf,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-061,same_academic_year_pair,Porovnaj rovnaké roky práce pri osobách Dávid Stromp a Matúš Suchanič.,supported,Prienik rokov akademických nadpisov oboch osôb; príprava na projekt je vylúčená.,pages/students/2019/david_stromp/README.md; pages/students/2020/matus_suchanic/README.md,e46ca8691d5ec894128f6f6f2ed33e2c7bd141f80719bb0449b20f10f30cca09; 770ca49538cca22df35f8e7ba736ee97483e4fe5788c221496553592b4e78dc6,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-080,same_academic_year_pair,Aké spoločné roky práce spájajú osoby Dávid Stromp a Dominik Nagy?,supported,Prienik rokov akademických nadpisov oboch osôb; príprava na projekt je vylúčená. Rok prípravy 2021 nepatrí do prieniku; potvrdený spoločný rok 2022.,pages/students/2016/dominik_nagy/README.md; pages/students/2019/david_stromp/README.md,1c4037a158d84c45fa705a0c38e47142a9d46e226f0c138bc2d56d7c38f6d74d; e46ca8691d5ec894128f6f6f2ed33e2c7bd141f80719bb0449b20f10f30cca09,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-081,same_start_year_pair,Porovnaj rovnaký rok začiatku štúdia osôb Matej Ščišľak a Nikita Bodnar.,supported,"Explicitný začiatok štúdia v oboch profiloch, nie rok práce.",pages/students/2021/matej_scislak/README.md; pages/students/2021/nikita_bodnar/README.md,b9a32cce8605e044a4f2828ff2d91a2995a62ef3a841e62f0a225c97221f0cc8; 2016c0dd923c0f8e561d5cea42ac81da3610fdcc54a4527f56d078e34e53ce90,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-097,same_start_year_pair,Aký spoločný rok začiatku štúdia majú osoby Martin Šarišský a Matej Ščišľak?,supported,"Explicitný začiatok štúdia v oboch profiloch, nie rok práce.",pages/students/2021/martin_sarissky/README.md; pages/students/2021/matej_scislak/README.md,8e97b45a253745faa55f63c844e0b5dfe680aeaf6eb01fb8f9d06ac6e8ee8bcf; b9a32cce8605e044a4f2828ff2d91a2995a62ef3a841e62f0a225c97221f0cc8,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-112,same_start_year_pair,Aký spoločný rok začiatku štúdia majú osoby Jakub Maruniak a Tomáš Kuchárik?,conditional_directory_cohort,"Tomáš Kuchárik nemá explicitnú vetu o začiatku štúdia. Hodnota 2016 pochádza zo zavedeného fallbacku ročníka v adresári pages/students/2016; Jakub má rok uvedený explicitne. Očakávanie platí pod touto konvenciou, skutočný začiatok štúdia treba potvrdiť človekom.",pages/students/2016/jakub_maruniak/README.md; pages/students/2016/tomas_kucharik/README.md,6734c41a95b9d0c1e3d226b393ad109ee4a7b82ebfffa73df2d7e0a743fefa14; 6a1a215433eb1857660177b4cb2ecec03c6a86743d3a04cd6d5c9d9682c9e63b,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-001,shared_keyword_pair,Over spoločné kľúčové slovo nlp pri osobách Maksym Zatirka a Samuel Džurina.,supported,Prienik doslovných taxonomy.tag z rodín dokumentov oboch osôb.,pages/students/2023/maksym_zatirka/README.md; pages/students/2023/samuel_dzurina/README.md,7a7d2cbe276c352dacfbc6ba6de5f15303d30ecf34845ff8da8d085f3d67c3dd; 4818191d044b91219c796a35010c95da5cf2093e050d494f30b219b91e13fa0e,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-021,shared_keyword_pair,Pre osoby Bogdan Paul Chiș a Samuel Džurina over rovnaké kľúčové slovo nlp.,supported,Prienik doslovných taxonomy.tag z rodín dokumentov oboch osôb.,pages/interns/bogdan_paul_chis/README.md; pages/students/2023/samuel_dzurina/README.md,3b4b946eb3c10e9f9a4092e6cffa01c5080e7579aded2f6ed3120c41ed62b647; 4818191d044b91219c796a35010c95da5cf2093e050d494f30b219b91e13fa0e,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-040,shared_keyword_pair,Over spoločné kľúčové slovo nlp pri osobách Oleksandr Hryshchenko a Serhii Yemets.,supported,Prienik doslovných taxonomy.tag z rodín dokumentov oboch osôb.,pages/students/2021/oleksandr_hryshchenko/README.md; pages/students/2022/serhii_yemets/README.md,4b073992b715b6c75b79fd2c8f850a69f940966e3a61a82c625c32c945a93036; ffbeac893947acafe855d576322a8826f9d4f87ac344cb31f8ceeefa244b9133,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-275,conjunctive_disambiguation,Ktorá z osôb Martin Jancura a Dmytro Ushatenko spĺňa kľúčové slovo opennmt a rok práce 2020?,supported,Tag z taxonomy.tag a rok akademického záznamu musia patriť tej istej osobe. Veta o čase konzultácie nie je názov; bp2021 v metadátach neprepisuje výslovný projekt 2020.,pages/students/2017/dmytro_ushatenko/README.md; pages/students/2017/martin_jancura/README.md,65f4ac177d492162b08ca5da3d9eb048bd088a3c117a28a662113b7e4bc3a4ec; 6a0a18d5095dcfb12abfdd72c9cfd8025d714fb5e29a7858cd2b285e4cb986b9,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-129,person_title_year,Uveď názov diplomového projektu osoby Maroš Harahus a rok práce pre rok 2020.,supported,"Názov a rok doložené v akademickom zázname; profilový názov nie je názov práce. Diplomový projekt je typ práce, nie tag.",pages/students/2016/maros_harahus/README.md,0da150e91918632c9e8cd1443f04da403595a372517e254928ce9f76c9d5033d,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-142,person_keyword_title,Uveď názov práce osoby Dárius Lindvai v roku 2019 s kľúčovým slovom lstm.,gold_corrected,"Pôvodný názov rešerš je generický odkaz na výstup TP 2019, nie doložený názov projektu. Dcérsky dokument má vlastný názov a tp2020; nemožno ho preniesť do roku 2019. Nové očakávanie: no_answer.",pages/students/2016/darius_lindvai/README.md; pages/students/2016/darius_lindvai/dp2021/README.md; pages/students/2016/darius_lindvai/timovy_projekt/resers/README.md; pages/students/2016/darius_lindvai/timovy_projekt/tutorial/README.md,fcaa47bb9708a6439eec5c4b7f7b4ed6c7359e423e1d940e1aed9f44957e6d62; be3d0fea2a6d592b225eecabd9b19445c2fce62b8289c07ab6af93835db12970; 352162de1b1e192228fa5d26506c028ac47e567e90df53b1bbaa6ad736ec4766; b4a81ff457624b089d618e4479afb1e3d748f6724f72104ef5518e535ae02fe5,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,
+wide-145,person_keyword_title,Uveď názov práce osoby Dárius Lindvai v roku 2019 s kľúčovým slovom python.,gold_corrected,"Pôvodný názov rešerš je generický odkaz na výstup TP 2019, nie doložený názov projektu. Dcérsky dokument má vlastný názov a tp2020; nemožno ho preniesť do roku 2019. Nové očakávanie: no_answer.",pages/students/2016/darius_lindvai/README.md; pages/students/2016/darius_lindvai/dp2021/README.md; pages/students/2016/darius_lindvai/timovy_projekt/resers/README.md; pages/students/2016/darius_lindvai/timovy_projekt/tutorial/README.md,fcaa47bb9708a6439eec5c4b7f7b4ed6c7359e423e1d940e1aed9f44957e6d62; be3d0fea2a6d592b225eecabd9b19445c2fce62b8289c07ab6af93835db12970; 352162de1b1e192228fa5d26506c028ac47e567e90df53b1bbaa6ad736ec4766; b4a81ff457624b089d618e4479afb1e3d748f6724f72104ef5518e535ae02fe5,"AI – priame čítanie Markdownov, oddelené od retrievera",,pending,

tools/graphrag/evaluation/source_oracle.py

@@ -76,6 +76,10 @@ def academic_type(heading: str) -> str | None:
 
 def acceptable_title(value: str, *, explicit: bool = False) -> bool:
     text = norm(value)
+    # A bare deliverable label is not evidence of an academic title. Do not
+    # inherit a linked document's title/year into its parent academic record.
+    if text in {'resers', 'tutorial', 'projekt', 'dokument', 'repozitar', 'pisomna praca'}:
+        return False
     if not (len(value) >= (6 if explicit else 10) and len(value) <= 180
             and len(text.split()) >= (1 if explicit else 2)):
         return False

tools/graphrag/graphrag.py

@@ -474,6 +474,10 @@ def is_title_candidate(
         value
     )
 
+    # Generic artifact/link labels identify a deliverable, not its title.
+    if normalized in {"resers", "tutorial", "projekt", "dokument", "repozitar", "pisomna praca"}:
+        return False
+
     if normalized.startswith(
         NON_TITLE_PREFIXES
     ):

tools/graphrag/tests/test_title_labels.py

@@ -0,0 +1,16 @@
+import pytest
+
+from tools.graphrag.graphrag import extract_work_title
+from tools.graphrag.evaluation.source_oracle import acceptable_title
+
+
+@pytest.mark.parametrize('label', ['rešerš', 'tutoriál', 'projekt', 'dokument', 'repozitár'])
+def test_generic_deliverable_is_not_an_academic_title(label):
+    assert extract_work_title([f'- Písomná práca: [{label}](./report)']) is None
+    assert not acceptable_title(label, explicit=True)
+
+
+def test_specific_deliverable_title_remains_supported():
+    title = 'Rešerš na tému prekladu'
+    assert extract_work_title([f'- Písomná práca: [{title}](./report)']) == title
+    assert acceptable_title(title, explicit=True)

tools/rag/evaluation/evaluate_rag_answers.py

@@ -18,6 +18,7 @@ if str(PROJECT_ROOT) not in sys.path:
     )
 
 
+from tools.answer_evaluation.rag_delivery import finalize_rag_answer
 from tools.rag.evaluation.rag_answer_data import (
     DEFAULT_DELAY,
     apply_question_overrides,
@@ -708,6 +709,7 @@ def main() -> int:
                 )
             )
 
+            run_result = finalize_rag_answer(run_result)
             scores = evaluate_answer(
                 question,
                 run_result[
@@ -718,12 +720,14 @@ def main() -> int:
                         "tool_called"
                     ]
                 ),
+                allowed_source_urls=run_result.get('rag_source_urls'),
             )
 
             result = {
                 **base_result,
                 **run_result,
                 **scores,
+                'strict_pass': bool(scores['strict_pass'] and run_result.get('accepted_for_delivery', True)),
                 "error": None,
             }
 

tools/rag/evaluation/rag_metrics.py

@@ -6,6 +6,8 @@ import unicodedata
 from collections import defaultdict
 from typing import Any
 
+from tools.answer_evaluation.context_policy import citation_check
+
 
 NO_ANSWER_TEXT = (
     "V dostupných dokumentoch ZP Wiki sa túto "
@@ -949,6 +951,7 @@ def evaluate_answer(
     answer: str,
     *,
     tool_called: bool,
+    allowed_source_urls: list[str] | None = None,
 ) -> dict[str, Any]:
     normalized_answer = (
         normalize_text(
@@ -1114,6 +1117,8 @@ def evaluate_answer(
         )
     )
 
+    citations = (citation_check({'answer': answer, 'source_urls': []}, set(allowed_source_urls))
+                 if allowed_source_urls is not None else {'citations_ok': None, 'unsupported_urls': []})
     strict_pass = (
         bool(
             answer.strip()
@@ -1124,9 +1129,12 @@ def evaluate_answer(
         and source_ok
         and should_answer_ok
         and tool_called
+        and citations['citations_ok'] is not False
     )
 
     return {
+        **citations,
+        'citations_checked': allowed_source_urls is not None,
         "answer_matches": (
             answer_matches
         ),

tools/rag/evaluation/rag_runner.py

@@ -11,6 +11,8 @@ import urllib.request
 from pathlib import Path
 from typing import Any
 
+from tools.answer_evaluation.context_policy import citation_check, coverage_issues
+
 
 PROJECT_ROOT = Path(__file__).resolve().parents[3]
 
@@ -675,6 +677,7 @@ def run_question(
     rag_source_urls: list[
         str
     ] = []
+    retrieval_coverage = []
 
     messages = build_initial_messages(
         question
@@ -858,11 +861,10 @@ def run_question(
             tool_latency
         )
 
-        current_urls = (
+        current_urls = [source['source_url'] for source in rag_result.get('sources', [])
-            extract_urls_from_object(
+                        if source.get('source_url')]
-                rag_result
+        retrieval_coverage.append({key: rag_result.get(key) for key in
-            )
+                                   ['complete', 'truncated', 'truncation_reasons', 'index_version']})
-        )
 
         for url in current_urls:
             if (
@@ -968,7 +970,14 @@ def run_question(
         - started
     )
 
+    citations = citation_check({'answer': answer, 'source_urls': []}, set(rag_source_urls))
+    policy_issues = sorted({issue for coverage in retrieval_coverage
+                           for issue in coverage_issues({'answer': answer, 'status': 'ok'}, coverage)})
     return {
+        **citations,
+        'retrieval_coverage': retrieval_coverage,
+        'context_issues': policy_issues,
+        'accepted_for_delivery': citations['citations_ok'] and not policy_issues,
         "answer": answer,
         "tool_called": True,
         "tool_call_count": len(

tools/rag/rag_utils.py

@@ -29,6 +29,9 @@ NO_ANSWER_TEXT = (
 
 
 RAG_INSTRUCTIONS = [
+    'Rešpektuj complete, truncated, truncation_reasons a index_version. Pri truncated=true alebo '
+    'complete=false v odpovedi jasne uveď, že výpis je neúplný. Pri complete=null nemožno zaručiť '
+    'úplnosť; neoznačuj výber za všetky práce alebo úplný zoznam. Cituj iba URL zo sources.',
     (
         "Odpovedaj výhradne podľa informácií vrátených "
         "v poli context a sources zo ZP Wiki."
@@ -751,10 +754,14 @@ def build_rag_context(
         for index, result in enumerate(results, start=1)
     ]
     context = build_context_text(sources)
+    coverage={key:response.get(key) for key in ['complete','truncated','truncation_reasons','index_version']}
+    context='METADÁTA VYHĽADANIA\n'+json.dumps(coverage,ensure_ascii=False)+'\n\n'+context
 
     return {
         **result_fields(status='ok' if sources else 'no_answer', sources=sources,
                         reason=None if sources else response.get('reason') or 'no_matching_sources',
+                        complete=response.get('complete'),
+                        index_version=response.get('index_version'),
                         truncation_reasons=response.get('truncation_reasons', [])),
         "query": query,
         "engine": response["engine"],