Oprava GraphRAG evaluácie

README.md

@@ -14,8 +14,9 @@ Stabilná baseline retrieval vetva používa klasický hybridný RAG:
 FTS5/BM25 + embeddingy + RRF. Nad týmto baseline sa vo vetve GraphRAG
 dopĺňa knowledge graph uložený v Neo4j. Aktuálne je implementovaná
 grafová infraštruktúra, deterministický builder, vizualizácia grafu a
-samostatné grafové vyhľadávanie. Grafová vrstva má zmrazenú evaluáciu v1; ďalším krokom je zapojenie
-do OpenWebUI a nová zdrojovo overená verzia benchmarku.
+samostatné grafové vyhľadávanie. Grafová vrstva má zdrojovo overený benchmark a zmrazenú opravenú
+verziu v3. Ďalším krokom je zapojenie do OpenWebUI a širšie nezávislé
+vyhodnotenie na nových otázkach/dátach.
 
 ## Implementované
 
@@ -489,7 +490,7 @@ Historická záloha datasetu pred úpravami nejednoznačností.
 
 Historická záloha datasetu pred validačnými úpravami.
 
-`questions_extended_2000.json`
+`questions_extended.json`
 
 Rozšírený benchmark s presne 2000 otázkami. Je pripravený pre neskoršie
 rozsiahlejšie experimenty nad klasickým retrievalom a RAG.
@@ -843,11 +844,14 @@ grafové vyhľadávanie je implementované v `tools/graphrag/graph_search.py`.
 GraphRAG zatiaľ nezasahuje do stabilného klasického `/rag` toku; ďalšia
 etapa spojí grafové výsledky s klasickým RAG a OpenWebUI nástrojom.
 
-Zmrazená evaluácia GraphRAG v1: DEV **209/286 (73,08 %)**, TEST
-**152/214 (71,03 %)**, bez runtime chýb. Audit pred TESTom odhalil
-137 zdrojových nesúladov v benchmarku; všetky zostávajú zahrnuté
-v primárnom skóre. Podrobné výsledky, doplnkové metriky a postup
-reprodukcie sú v [GraphRAG evaluácii](tools/graphrag/evaluation/README.md).
+Aktuálna opravená evaluácia GraphRAG v3: DEV/regresia **589/589**,
+doplnkový TEST **13/13**, bez runtime chýb. Posledný TEST je malý a
+pokrýva iba dva nové profily; neznamená všeobecnú 100 % správnosť.
+Širší nezávislý TEST v2 pred finálnymi opravami zostáva zaznamenaný ako
+**67/80**. Pôvodných 137 nesúladov má zdrojové opravy a nepravdivé
+predpoklady sa hodnotia cez skutočné no-answer správanie.
+Podrobné výsledky, obmedzenia a reprodukcia sú v
+[GraphRAG evaluácii](tools/graphrag/evaluation/README.md).
 
 ## OpenWebUI
 
@@ -1046,7 +1050,7 @@ na konkrétnom commite.
 Pre ďalšie experimenty sú pripravené:
 
 ```text
-questions_extended_2000.json
+questions_extended.json
 graphrag_questions.json
 robustness_questions.json
 performance_queries.json
@@ -1070,10 +1074,11 @@ spoľahlivého zdroja.
 
 ### 2. GraphRAG evaluácia
 
-V1 je zmrazená a vyhodnotená. [Finálna správa](tools/graphrag/evaluation/results/final_2026-10-24/REPORT.md)
-uvádza všetky výsledky aj zostávajúce limity. Pred formálnym porovnaním
-s klasickým RAG treba pripraviť novú zdrojovo overenú verziu benchmarku.
-Zistenia z finálneho TESTu patria do ďalšej verzie s novým holdoutom.
+V3 je opravená, zmrazená a vyhodnotená. [Finálna správa](tools/graphrag/evaluation/results/final_v3_2026-10-25/REPORT.md)
+obsahuje aj históriu v1/v2 a obmedzenia posledného malého holdoutu.
+Pred formálnym porovnaním s klasickým RAG treba širšie nezávislé
+otázky/dáta a spoločnú metodiku. Vystavené TEST otázky sú pri ďalšom
+ladení iba regresiou.
 
 ### 3. OpenWebUI nástroje
 

docs/cleanup_2026-10-24.json

@@ -103,5 +103,51 @@
       "to": "tools/graphrag/evaluation/results/archive_pre_v1/graphrag_dev.csv",
       "reason": "retain historical full DEV run without confusing it with final results"
     }
+  ],
+  "cache_cleanup": [
+    {
+      "path": ".pytest_cache",
+      "files": 5
+    },
+    {
+      "path": "test/__pycache__",
+      "files": 38
+    },
+    {
+      "path": "app/__pycache__",
+      "files": 4
+    },
+    {
+      "path": "app/.pytest_cache",
+      "files": 4
+    },
+    {
+      "path": "tools/__pycache__",
+      "files": 1
+    },
+    {
+      "path": "tools/rag/__pycache__",
+      "files": 14
+    },
+    {
+      "path": "tools/rag/evaluation/__pycache__",
+      "files": 9
+    },
+    {
+      "path": "tools/rag/tests/__pycache__",
+      "files": 41
+    },
+    {
+      "path": "tools/graphrag/__pycache__",
+      "files": 6
+    },
+    {
+      "path": "tools/graphrag/evaluation/__pycache__",
+      "files": 10
+    },
+    {
+      "path": "tools/graphrag/tests/__pycache__",
+      "files": 17
+    }
   ]
 }

docs/graphrag_2026-10-25_worklog.md

@@ -0,0 +1,58 @@
+# Úpravy GraphRAG a upratanie projektu — 25. 10. 2026
+
+Výsledky: [finálna správa v3](../tools/graphrag/evaluation/results/final_v3_2026-10-25/REPORT.md).
+Upravené je grafové vyhľadávanie a jeho vyhodnotenie. Klasický RAG kód ani index sa funkčne nemenili.
+
+## Zálohy a upratanie
+
+Pred úpravami vznikla záloha `/tmp/zp-agent-graphrag-v2-backup-20261024T193749Z/project-before.tar.gz`, stav Gitu a neskôr export Neo4j `neo4j_before_v2.json`. Git tagy uchovávajú presné publikované implementácie.
+
+[Cleanup manifest](cleanup_2026-10-24.json) zaznamenáva odstránené súbory a ponechané protikusy s SHA-256. Odstránených bolo päť presných kópií retrieval výsledkov, tri dočasné terminálové výpisy, štyri smoke/selected výstupy, historická kópia benchmarku už prítomná v Gite a osirelá cache pôvodného `scripts/`. Plný starý GraphRAG DEV sa presunul do označeného archívu. Final JSON/CSV, baseline, zmrazený RAG DEV/TEST a neidentické historické experimenty zostali zachované. CSV a JSON sa nepovažujú za duplicity rôznych formátov.
+
+## Implementácia
+
+- Celé mená, presná zhoda, kontrolované skloňovanie a bezpečné neznáme/nejednoznačné osoby; neznámy kandidát nezmení vyhľadávanie na globálne.
+- Autor dokumentu vs. opisovaná osoba; celá fráza požadovaného tagu a oddelenie následnej vety/osôb.
+- Typované prepojené dôkazy: osoba, konkrétna práca, jej názov/rok a správny dokument. Konfliktné údaje toho istého uzla sa odmietajú.
+- Rozlíšenie roku štúdia a práce, nulového počtu a no-answer, presného počtu/množiny, platných alternatív zdrojov a URL.
+- Nezávislý oracle pôvodných Markdown súborov; opravených 128 tagových predpokladov a 9 množín kategórií. Neskôr zdrojovo rozlíšený bakalársky projekt a záverečná práca.
+- Doplnené vedecké a bakalárske projekty, akademické rozsahy rokov a regresné/negatívne/integritné testy.
+- Freeze viaže kód, benchmark, raw Markdown, JSON vstupy, graf, audit, parametre a rozdelenie profilov; chráni pred prepísaním archívov.
+
+## Podstatné vykonané príkazy
+
+Použitý bol `.venv/bin/python`; pri pripojení sa načítala lokálna `.env` bez výpisu tajomstiev.
+
+```bash
+python -m tools.graphrag.evaluation.benchmark_v2
+python -m tools.graphrag.evaluation.benchmark_v3
+python -m tools.graphrag.build_graphrag
+RUN_NEO4J_TESTS=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 pytest -q -p no:cacheprovider
+
+python -m tools.graphrag.evaluation.evaluate_graphrag \
+  --split dev \
+  --audit tools/graphrag/evaluation/results/v3_preparation/source_audit.json \
+  --output-dir tools/graphrag/evaluation/results/final_v3_2026-10-25
+
+python -m tools.graphrag.evaluation.freeze_graphrag \
+  --dev-results tools/graphrag/evaluation/results/final_v3_2026-10-25/graphrag_dev.json \
+  --audit tools/graphrag/evaluation/results/v3_preparation/source_audit.json \
+  --output tools/graphrag/evaluation/frozen_v3.json
+
+python -m tools.graphrag.evaluation.evaluate_graphrag \
+  --split test \
+  --audit tools/graphrag/evaluation/results/v3_preparation/source_audit.json \
+  --freeze-manifest tools/graphrag/evaluation/frozen_v3.json \
+  --output-dir tools/graphrag/evaluation/results/final_v3_2026-10-25
+
+docker compose build zp-agent-api
+docker compose up -d --no-deps zp-agent-api
+```
+
+Príkazy na tvorbu datasetu a freeze boli použité pred archivovaním; existujúce zmrazené súbory neregeneruj na rovnaké miesto. Reprodukcia používa nový výstupný adresár.
+
+## Výsledok
+
+430 automatizovaných testov prešlo, 4 boli preskočené. Finálny DEV v3 589/589 a posledný doplnkový TEST 13/13 majú 0 runtime chýb. V2 malo prvý širší nezávislý TEST 67/80; tieto výsledky ani neskôr odhalené anotačné nedostatky nie sú skryté. V3 už obsahuje vystavených 80 otázok ako DEV. Nový malý TEST je profilovo oddelený, ale nie je reprezentatívnym dôkazom všeobecnej 100 % presnosti. Kvalita finálnej LLM odpovede a kompletný OpenWebUI GraphRAG agent patria do ďalších blokov projektu.
+
+Docker overenie po zostavení finálneho obrazu: obsahové odtlačky Python kódu a datasetu v kontajneri sa zhodujú s manifestom v3, grafový DEV dotaz prešiel a `/health` má `ready=true`. Po testoch sa odstránila regenerovateľná Python/pytest cache mimo `.venv`; zoznam je v cleanup manifeste.

tools/graphrag/evaluation/README.md

@@ -1,62 +1,61 @@
 # GraphRAG evaluácia
 
-Aktuálny výsledok v1 a obmedzenia: [finálna správa](results/final_2026-10-24/REPORT.md).
-Primárne skóre: DEV **209/286 (73,08 %)**, TEST **152/214 (71,03 %)**, bez runtime chýb.
+Aktuálna opravená verzia je **v3**. [Finálna správa](results/final_v3_2026-10-25/REPORT.md) uvádza výsledky aj ich obmedzenia.
 
-Benchmark má 500 otázok (286 DEV, 214 TEST), schema v2. Statický audit pred finálnym TESTom odhalil 137 nesúladov očakávaní so zdrojmi. Všetky zostávajú zahrnuté v primárnom skóre. Doplnková podmnožina bez zistených nesúladov má DEV 209/209 a TEST 152/154; podrobnosti a dve zostávajúce zlyhania sú v správe.
+- DEV/regresia: **589/589**.
+- Posledný doplnkový TEST: **13/13**, iba dva úplne nové profily a obmedzené kategórie.
+- Širší nezávislý TEST v2 pred finálnymi opravami: **67/80**. Je archivovaný a jeho otázky sa po odhalení chýb stali DEV.
+- Celá testovacia sada: **430 passed, 4 skipped**.
 
-## Reprodukcia zmrazenej evaluácie
+Tieto čísla neznamenajú 100 % všeobecnú správnosť kódu. Vyhľadávanie sa hodnotí bez generovania odpovede LLM.
 
-Použi rovnaké `data/documents.json`, `data/chunks.json` a rovnaký obsah Neo4j. Odtlačky sú v `frozen_v1.json`; Git revízia wiki je vo výsledkovom JSON. Načítaj vlastné lokálne tajomstvá bez ich výpisu:
+## Reprodukcia aktuálnej zmrazenej verzie
+
+Použi zodpovedajúci kód, rovnaké pôvodné wiki súbory, vstupné JSON a obsah Neo4j podľa manifestu. Súkromné kľúče ostávajú v `.env`.
 
 ```bash
 set -a
 source .env
 set +a
-
 python -m tools.graphrag.evaluation.evaluate_graphrag \
   --split test \
-  --audit tools/graphrag/evaluation/results/final_2026-10-24/source_audit.json \
-  --freeze-manifest tools/graphrag/evaluation/frozen_v1.json \
-  --output-dir /tmp/graphrag-v1-reproduction
+  --audit tools/graphrag/evaluation/results/v3_preparation/source_audit.json \
+  --freeze-manifest tools/graphrag/evaluation/frozen_v3.json \
+  --output-dir /tmp/graphrag-v3-reproduction
 ```
 
-TEST/all bez manifestu sa odmietne. Overuje sa kód, benchmark, vstupné JSON súbory, obsah grafu, audit aj parametre. Zmenený obsah, neúplný holdout alebo iné parametre beh zablokujú. Výsledky neregeneruj do adresára archivovaného finálneho behu.
+V1 a v2 sú zachované v pôvodných adresároch s manifestmi a Git tagmi `graphrag-v1-frozen` a `graphrag-v2-frozen`. Na ich reprodukciu treba zodpovedajúcu verziu kódu a grafu; aktuálny upravený kód úmyselne neprejde kontrolou starého manifestu. Originálny klasický RAG ostáva samostatným referenčným experimentom.
 
-## Nová vývojová verzia
+## Referenčné údaje a ďalší vývoj
 
-Ďalší vývoj musí mať vlastné výsledky a nový manifest. Už vyhodnotený TEST nepoužívaj na dolaďovanie deklarované ako nový nezávislý experiment.
+`graphrag_questions.json` je pôvodných 500 otázok. V2 opravila 137 nesúladov: 128 nepravdivých predpokladov o tagoch a 9 množín kategórií. Pravidlá a každá zmena majú zdrojový záznam v `results/v2_preparation/legacy_corrections.json`.
+
+`source_oracle.py` číta pôvodný Markdown nezávisle od grafového buildera. `benchmark_v2.py` vytvára opravenú regresiu a pôvodný profilovo oddelený v2 holdout. `benchmark_v3.py` presúva všetky vystavené v2 prípady do DEV, opravuje projekt/práca anotácie a rezervuje posledné nepoužité profily. Nový vývoj musí mať novú verziu, nové výstupy a primerane nezávislý holdout. Archivované datasety neregeneruj na mieste.
 
 ```bash
-python -m tools.graphrag.evaluation.audit_benchmark \
-  --output /tmp/graphrag-next/source_audit.json
-
+# Príklad vývoja na existujúcom datasete do nového adresára:
 python -m tools.graphrag.evaluation.evaluate_graphrag \
-  --split dev \
-  --audit /tmp/graphrag-next/source_audit.json \
+  --split dev --audit tools/graphrag/evaluation/results/v3_preparation/source_audit.json \
   --output-dir /tmp/graphrag-next
 
 python -m tools.graphrag.evaluation.freeze_graphrag \
   --dev-results /tmp/graphrag-next/graphrag_dev.json \
-  --audit /tmp/graphrag-next/source_audit.json \
+  --audit tools/graphrag/evaluation/results/v3_preparation/source_audit.json \
   --output /tmp/graphrag-next/frozen.json
 ```
 
-Zmrazenie vyžaduje kompletný DEV bez runtime chýb a súlad s aktuálnymi odtlačkami. Nevyžaduje predstieraných 100 % na chybne anotovanom datasete. Obsahový manifest platí aj pre zmeny, ktoré ešte nemajú commit; Git tag označuje publikovaný checkpoint.
+Samotné ďalšie spustenie toho istého TESTu je reprodukcia, nie nové nezávislé vyhodnotenie. Existujúce freeze manifesty a finálne výstupy sa neprepisujú.
 
-## Metriky a dôkazy
+## Čo hodnotí strict
 
-Strict kontroluje očakávané hodnoty, typované entity, relácie, vlastnosti, zdrojové dokumenty a URL, presnú množinu pri definovaných prípadoch, správny počet pri agregácii a neorezaný výsledok. Samotné entity rozpoznané z otázky nie sú odpoveďou. `expected_paths` sú opisné anotácie a nehodnotia sa doslovným porovnaním reťazcov.
-
-Výstup obsahuje JSON aj CSV, metriky podľa kategórie a obtiažnosti, priemer/medián/p95 latencie, čas behu, verziu Pythonu, Git revízie a obsahové odtlačky. Latencia je iba čas vyhľadávania v grafe.
+Správne hodnoty musia patriť tej istej osobe/práci a mať prepojené typované dôkazy. Overuje sa význam roku, názov a typ práce, relácie, zdrojové alternatívy a URL, presné množiny a počet, explicitné no-answer/ambiguous a neorezaný výsledok. Rozpoznaná hodnota z otázky sama nie je dôkazom. Schema3 používa strojovo overované väzby a vzory ciest; textové `expected_paths` pôvodnej schema2 sú historické anotácie.
 
 ## Testy
 
 ```bash
 pytest -q
-
-# Po načítaní Neo4j konfigurácie z .env:
-RUN_NEO4J_TESTS=1 pytest -q tools/graphrag/tests/test_graph_search_neo4j.py
+# Po načítaní .env:
+RUN_NEO4J_TESTS=1 pytest -q
 ```
 
-Neo4j testy používajú unikátny oddelený scope a rollback transakcie. Existujúce wiki dáta nemenia.
+Neo4j integračné testy používajú izolovaný scope a rollback. Výsledky obsahujú JSON aj CSV, kategórie, obtiažnosti, priemer/medián/p95, Git revízie, Python a obsahové odtlačky.

tools/graphrag/evaluation/benchmark_v3.py

@@ -0,0 +1,153 @@
+"""Continue after exposed v2: all old cases become DEV; reserve the last unseen profiles.
+
+The small supplementary holdout cannot replace the broader independent v2
+measurement. It is kept profile-disjoint and must be reported with its size.
+"""
+from __future__ import annotations
+
+import argparse
+import copy
+import hashlib
+import json
+import re
+from collections import Counter
+from pathlib import Path
+
+from .benchmark_v2 import (BASE, make_case, correct_legacy, people_for, academics, has_tag,
+                           academic_binding, validate_case, audit_dataset, write_json)
+from .source_oracle import load_source_oracle, supporting_documents, norm
+
+
+def reannotate(case, oracle, original):
+    if case['id'] in original:
+        return correct_legacy(original[case['id']], oracle)
+    category = case['category']; graph = case['graph']; names = people_for(case, oracle)
+    bindings = []; negative = None; expected_set = None; field = None; count = None; constraints = {}
+    old = graph.get('expected_bindings', [])
+    years = [int(y) for y in re.findall(r'\b20\d{2}\b', case['question'])]
+    keyword = next((row.get('keyword') for row in old if row.get('keyword')), None)
+    if category == 'negative_lookup':
+        corrected = copy.deepcopy(case); corrected['split'] = 'dev'
+        return corrected
+    if category == 'shared_keyword_pair':
+        if all(has_tag(oracle, name, keyword) for name in names):
+            bindings = [{'person': name, 'keyword': keyword} for name in names]
+        else:
+            negative = 'No common literal frontmatter tag.'; constraints = {'keyword': keyword, 'require_all_people': True}
+    elif category == 'same_academic_year_pair':
+        common = set.intersection(*[{r['year'] for r in academics(oracle, name) if r['year']} for name in names])
+        bindings = [{'person': name, 'year': year} for year in sorted(common) for name in names]
+        if not common:
+            negative = 'No shared academic year in original sources.'; constraints = {'shared_academic_year': True}
+    elif category == 'same_start_year_pair':
+        bindings = [{'person': name, 'start_year': oracle['people'][name]['start_year']} for name in names]
+    elif category in {'person_title_year', 'person_keyword_title'}:
+        records = [r for r in academics(oracle, names[0], year=years[-1]) if r.get('title')]
+        if 'diplomoveho projektu' in norm(case['question']):
+            records = [r for r in records if r['academic_type'] == 'diploma_project']
+        if category == 'person_keyword_title' and not has_tag(oracle, names[0], keyword):
+            records = []
+        bindings = [academic_binding(names[0], row, title=True, keyword=keyword) for row in records]
+        if not bindings:
+            negative = 'Requested year/type/title is absent; a project title is not a future thesis title.'
+            constraints = {'year': years[-1], **({'keyword': keyword} if keyword else {})}
+    elif category == 'document_author_category_multi_hop':
+        for row in old:
+            for path in supporting_documents(oracle, names[0], category=row['category']):
+                author = oracle['documents'][path]['author']
+                if author:
+                    bindings.append({'person': names[0], 'category': row['category'], 'document_author': author,
+                                     'document_path': path, 'same_document': True})
+    elif category == 'person_multiple_categories':
+        expected_set = oracle['people'][names[0]]['categories']; field = 'category'
+        bindings = [{'person': names[0], 'category': value} for value in expected_set]
+    elif category == 'candidate_set_aggregation':
+        expected_set = [name for name in names if academics(oracle, name, year=years[-1])]
+        count = len(expected_set); field = 'person'
+        bindings = [{'person': name, 'year': years[-1]} for name in expected_set]
+    elif category == 'conjunctive_disambiguation':
+        matches = [name for name in names if has_tag(oracle, name, keyword) and academics(oracle, name, year=years[-1])]
+        bindings = [{'person': name, 'keyword': keyword, 'year': years[-1]} for name in matches]
+        if not matches:
+            negative = 'No candidate satisfies both source conditions.'
+            constraints = {'keyword': keyword, 'year': years[-1]}
+    else:
+        raise ValueError(category)
+    result = make_case(case['id'], 'dev', category, case['question'], names, oracle, bindings,
+                       no_answer=negative is not None, reason=negative, negative_constraints=constraints,
+                       expected_set=expected_set, set_field=field, count=count)
+    result['legacy_role'] = 'exposed_v2_case_development_only'
+    return result
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--output', type=Path, default=BASE / 'graphrag_questions_v3.json')
+    parser.add_argument('--artifacts', type=Path, default=BASE / 'results/v3_preparation')
+    args = parser.parse_args()
+    oracle = load_source_oracle()
+    previous = json.loads((BASE / 'graphrag_questions_v2.json').read_text())
+    original = {q['id']: q for q in json.loads((BASE / 'graphrag_questions.json').read_text())}
+    dev = [reannotate(case, oracle, original) for case in previous]
+    used = {p for case in previous for p in case['graph']['source_profiles']}
+    used.update(json.loads((BASE / 'results/v2_preparation/partition.json').read_text())[
+        'previously_exposed_profiles_excluded_from_test'])
+    reserved = sorted([name for name, person in oracle['people'].items()
+                       if person['profile'].startswith('pages/students/') and person['profile'] not in used], key=norm)
+    if len(reserved) != 2:
+        raise RuntimeError('Unexpected remaining unseen profile population; review partition policy.')
+    holdout = []
+    def add(category, text, names, bindings, **kwargs):
+        holdout.append(make_case(f'fresh-{len(holdout)+1:03d}', 'test', category, text, names,
+                                 oracle, bindings, **kwargs))
+    for name in reserved:
+        person = oracle['people'][name]
+        add('person_multiple_categories', f'Vypíš všetky kategórie dokumentov osoby {name}.', [name],
+            [{'person': name, 'category': category} for category in person['categories']],
+            expected_set=person['categories'], set_field='category')
+        category = person['categories'][0]
+        for path in supporting_documents(oracle, name, category=category)[:1]:
+            author = oracle['documents'][path]['author']
+            if author:
+                add('document_author_category_multi_hop',
+                    f'Ktorý autor dokumentu je pri osobe {name} spojený s kategóriou {category}?', [name],
+                    [{'person': name, 'category': category, 'document_author': author,
+                      'document_path': path, 'same_document': True}])
+        add('person_start_year', f'Aký rok začiatku štúdia je uvedený pri osobe {name}?', [name],
+            [{'person': name, 'start_year': person['start_year']}])
+        add('negative_lookup', f'Aký názov práce má osoba {name}, ak jej dokument obsahuje kľúčové slovo nenajdenyoverovacitag?',
+            [name], [], no_answer=True, reason='Tag absent from raw frontmatter.',
+            negative_constraints={'keyword': 'nenajdenyoverovacitag'})
+        add('negative_lookup', f'Aký názov práce má osoba {name} v roku 2098?', [name], [],
+            no_answer=True, reason='No academic record for requested year.', negative_constraints={'year': 2098})
+    for name in ['Diana Neznáma Overovacia', 'Milan Neprítomný Vzorový']:
+        add('negative_lookup', f'Aký názov diplomovej práce má osoba {name}?', [name], [],
+            no_answer=True, reason='No profile with requested full name.', negative_constraints={'unknown_person': name})
+    add('candidate_set_aggregation', f'Z kandidátov {", ".join(reserved)} vyber osoby s prácou v roku 2098. Koľko ich je?',
+        reserved, [], expected_set=[], set_field='person', count=0)
+    questions = dev + holdout
+    issues = {q['id']: validate_case(q, oracle) for q in questions if validate_case(q, oracle)}
+    if issues:
+        raise RuntimeError(f'Independent source audit failed: {issues}')
+    dev_profiles = {p for q in dev for p in q['graph']['source_profiles']}
+    test_profiles = {p for q in holdout for p in q['graph']['source_profiles']}
+    assert not dev_profiles & test_profiles
+    write_json(args.output, questions)
+    write_json(args.artifacts / 'source_audit.json', audit_dataset(questions, oracle, args.output))
+    changes = [{'id': old['id'], 'old_bindings': old['graph']['expected_bindings'],
+                'new_bindings': new['graph']['expected_bindings'], 'old_should_answer': old['should_answer'],
+                'new_should_answer': new['should_answer'], 'source_evidence': new['source_evidence']}
+               for old, new in zip(previous, dev) if old['graph']['expected_bindings'] != new['graph']['expected_bindings']
+               or old['should_answer'] != new['should_answer']]
+    write_json(args.artifacts / 'source_adjudications.json', {'method': 'Raw heading project/thesis distinction; no answer-output based relabeling.', 'changes': changes})
+    partition = {'dev_count': len(dev), 'test_count': len(holdout), 'dev_profiles': sorted(dev_profiles),
+                 'test_profiles': sorted(test_profiles), 'source_profile_overlap': 0,
+                 'limitations': ['Only two entirely unseen profiles remain; supplementary holdout has limited category coverage.',
+                                 'The broader v2 independent TEST remains 67/80; v2 cases are now DEV, never fresh TEST.']}
+    write_json(args.artifacts / 'partition.json', partition)
+    print(json.dumps({'dev': len(dev), 'test': len(holdout), 'unseen_profiles': len(reserved),
+                      'source_adjudications': len(changes), 'source_issues': 0}, indent=2))
+
+
+if __name__ == '__main__':
+    main()

tools/graphrag/evaluation/evaluate_graphrag.py

@@ -33,7 +33,7 @@ EVALUATION_DIR = Path(
 
 QUESTIONS_PATH = (
     EVALUATION_DIR
-    / "graphrag_questions_v2.json"
+    / "graphrag_questions_v3.json"
 )
 
 RESULTS_DIR = (
@@ -59,7 +59,7 @@ def parse_args() -> argparse.Namespace:
     parser.add_argument(
         "--output-dir",
         type=Path,
-        default=RESULTS_DIR / 'development_v2',
+        default=RESULTS_DIR / 'development_v3',
     )
 
     parser.add_argument(

tools/graphrag/evaluation/freeze_graphrag.py

@@ -50,7 +50,7 @@ def make_freeze(dev: dict, questions: list[dict], fingerprints: dict, audit: dic
 def main() -> None:
     parser = argparse.ArgumentParser(description=__doc__)
     parser.add_argument('--dev-results', type=Path, required=True)
-    parser.add_argument('--questions', type=Path, default=Path(__file__).parent / 'graphrag_questions_v2.json')
+    parser.add_argument('--questions', type=Path, default=Path(__file__).parent / 'graphrag_questions_v3.json')
     parser.add_argument('--audit', type=Path, required=True)
     parser.add_argument('--output', type=Path, required=True)
     args = parser.parse_args()

tools/graphrag/evaluation/frozen_v3.json

@@ -0,0 +1,212 @@
+{
+  "schema_version": 2,
+  "partition": {
+    "method": "disjoint source profiles",
+    "dev_profile_count": 69,
+    "test_profile_count": 2,
+    "overlap": 0
+  },
+  "frozen_at_utc": "2026-10-25T01:13:16.783627+00:00",
+  "fingerprints": {
+    "implementation": {
+      "tools/graphrag/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "tools/graphrag/build_graphrag.py": "81f6e9f5c9e71b588f8bed1968c1e900e29e484ba81d23b6a3a68e7220ad75a5",
+      "tools/graphrag/evaluation/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "tools/graphrag/evaluation/audit_benchmark.py": "6619c818a52fc284e87ed6c48a503a54a3c9dd95447463ed78f075b7c58bad80",
+      "tools/graphrag/evaluation/benchmark_v2.py": "3e66d8e20d049758ac6df393355c0b5038f09c6ba97c96acc2087d61cfb4ad26",
+      "tools/graphrag/evaluation/benchmark_v3.py": "30bae6c605ed26f4b49973acec799087c4e2b97a883ccedcafb965dd909c4c4d",
+      "tools/graphrag/evaluation/evaluate_graphrag.py": "313882233f5e62f700b512091e8166fc072f42add7a53e2a17af42f785d1fa6b",
+      "tools/graphrag/evaluation/evidence_contract.py": "78e24264c176e163e1782fef1634b6c45c7f31bd4692726d02a30ab45a06648e",
+      "tools/graphrag/evaluation/freeze_graphrag.py": "b850483db6e8dc94f2c8729d1e40b3e9363628a1181a47863f137a3390c69166",
+      "tools/graphrag/evaluation/graphrag_evaluator.py": "3afe014e126b6ad90c8e408284d2c1979f54d6cf5dd19fd67f9249708fbed2d3",
+      "tools/graphrag/evaluation/reproducibility.py": "b707d4c410cbf7d6769a6afdadfe79ef3bbd699a0323f0e0d2ab180ef7498680",
+      "tools/graphrag/evaluation/source_oracle.py": "d9d1b5a2a5811b18fe579528c0bdacba46bddda8a196c4dfa000288033937094",
+      "tools/graphrag/graph_db.py": "a6fce68000065d328139829c5ca894623f6ed5828c74a869663366c2ad7b5ea2",
+      "tools/graphrag/graph_search.py": "296560e77a5c4cefe23430bb8197cc86b5680a7539508a6f63285ddd099a94ed",
+      "tools/graphrag/graphrag.py": "61597bbc76d47dc6e4c3a4517eabb462f803aea450c85b42c1c823a34d4f1397",
+      "tools/graphrag/neo4j_smoke.py": "fb5244c72c29ef2acf96d1e7f54335ff168eed4119db1692025eb1b99194b7bb",
+      "tools/graphrag/tests/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "tools/graphrag/tests/test_evaluation_pipeline_v2.py": "63d1b3e80ae3c7b563374fd7ab4aa5ffad3c614eafbd15142c577abd7f756bf9",
+      "tools/graphrag/tests/test_evaluator_connected_v2.py": "29d7c9838ed1ff37ea3276df6feb1b6142158c43a0525712f490b648181882de",
+      "tools/graphrag/tests/test_graph_search.py": "d5c95b5de2a0962857331929522764aa46a1222e550a3eb1aaca2148f9d58748",
+      "tools/graphrag/tests/test_graph_search_entity_v2.py": "0ef1a8ebd513c5a4507eb38fcf5ae6137c69e9901b5a2975ba513be5902ac693",
+      "tools/graphrag/tests/test_graph_search_neo4j.py": "fec2bf72de8f752f17c9164fee5754bdb2e153828855e47efbf0000dbe82cf3b",
+      "tools/graphrag/tests/test_graph_search_regressions.py": "9db7779aeb8ed9e8c0f92c1657011a81080aff3c928649e811358e3009a18c48",
+      "tools/graphrag/tests/test_graphrag.py": "6379335110f05668beca87ca7e486442071b2f666dbd651408d5f3a8d5723f0e",
+      "tools/graphrag/tests/test_graphrag_evaluator.py": "1c6d08aeec9ae56ac4123239a5aed12a0081d66cc3f34b2047d95764579d7298",
+      "tools/graphrag/tests/test_graphrag_freeze.py": "c4b760baf4c7ebf13b16bd4cd3c51874f5512eef3b4c61470dc4d428b0303d9f",
+      "tools/graphrag/tests/test_graphrag_research_v2.py": "711008a91ab8a58ae0854e7f90329bcd42e5189fde2e2001589e97b0c22c5198",
+      "tools/graphrag/tests/test_graphrag_root_title.py": "085651fe6e6081521a5dde9554abfee93456250108cf15978df25378d886a46c",
+      "tools/graphrag/tests/test_source_oracle_v2.py": "0afdbe235e185ffe9f38aa667f90f5a6c20504fe29e9ac572642b39a8eb7faca",
+      "requirements.txt": "26a6d097727eba03c1b6c71cd02b06373ef01ce03e567dd2bc4357ae330e5a1a"
+    },
+    "dataset_sha256": "3094100b9b6cfe4bb04f7479bddce07f392812ce7bdad97703d74db54b6a54ad",
+    "inputs": {
+      "data/documents.json": "0542af2a373771cf7c4a95f93d14f4f687f6a0187846892c959aa6efc4bade52",
+      "data/chunks.json": "437dde589be46e0c30842dd330de22d96a0173d2dbaff9c5f129568cb2bca4c9"
+    },
+    "source_corpus": {
+      "sha256": "440da12a7a991824e923f70cada7c616c3fb3b0f12767ec7d75039a6a3231b10",
+      "document_count": 114,
+      "documents": {
+        "pages/interns/bogdan_paul_chis/README.md": "3b4b946eb3c10e9f9a4092e6cffa01c5080e7579aded2f6ed3120c41ed62b647",
+        "pages/interns/cesar_gutierrez/README.md": "b0be60576eed28d4b1920e9acf198684519373dfc13171567247c853b3ed699e",
+        "pages/interns/oliver_pejic/README.md": "0470f3c8465be8f02204b08e4c75ef47f68c7e4ad1038faa5641cdf1811f33d5",
+        "pages/interns/sevval_bulburu/README.md": "ff2a9d66253802f1af0fdfd74832e40fae41b6df23c5e61dc266f6cd0cc16c4d",
+        "pages/interns/yussef_ressaissi/README.md": "f997a65699da0db3222ef7a3c37652d05dfc3419ad4ec74683bafc2c897bb987",
+        "pages/students/2016/darius_lindvai/README.md": "fcaa47bb9708a6439eec5c4b7f7b4ed6c7359e423e1d940e1aed9f44957e6d62",
+        "pages/students/2016/darius_lindvai/dp2021/README.md": "be3d0fea2a6d592b225eecabd9b19445c2fce62b8289c07ab6af93835db12970",
+        "pages/students/2016/darius_lindvai/timovy_projekt/resers/README.md": "352162de1b1e192228fa5d26506c028ac47e567e90df53b1bbaa6ad736ec4766",
+        "pages/students/2016/darius_lindvai/timovy_projekt/tutorial/README.md": "b4a81ff457624b089d618e4479afb1e3d748f6724f72104ef5518e535ae02fe5",
+        "pages/students/2016/dominik_nagy/README.md": "1c4037a158d84c45fa705a0c38e47142a9d46e226f0c138bc2d56d7c38f6d74d",
+        "pages/students/2016/dominik_nagy/timovy_projekt/README.md": "b81a4dab7316f7b846dcf36515f6aa9d44035c86f46f0e07b7a43308bcd31458",
+        "pages/students/2016/ediz_morochovic/README.md": "045e5eb9a3df2b84140808fbca0c3163266ad0efa144bc42d9d5c597881c6ac6",
+        "pages/students/2016/jakub_maruniak/README.md": "6734c41a95b9d0c1e3d226b393ad109ee4a7b82ebfffa73df2d7e0a743fefa14",
+        "pages/students/2016/jakub_maruniak/dp2021/README.md": "9cd884aec70705cdb2aaa0f514b513f16de263c37b749ced7f92916cd2f91c39",
+        "pages/students/2016/jakub_maruniak/dp2021/annotation/README.md": "debae0c50aea1accef0840912b9b1f37b2138f856002bb50e91d7ea2dbbe7963",
+        "pages/students/2016/jakub_maruniak/dp2021/train_experiments/README.md": "01dff38b37043414fa1db204394ddbe7aade00e14d02d495572b65c85d09b194",
+        "pages/students/2016/jakub_maruniak/timovy_projekt/README.md": "04114b1f3d84664aad5bdba5a0b614c5a6f23392be3c31fd9123d63db1be0a35",
+        "pages/students/2016/jan_holp/README.md": "80d6a8492766e5bfa61673a372e5f059d30c8e335de39842866c51485960e4d3",
+        "pages/students/2016/jan_holp/dp2021/README.md": "a62acb74009376766c7e2f9856ee01d21f84593eb9b3ce85cf2e6032aa412476",
+        "pages/students/2016/jan_holp/dp2021/zdrojove_subory/README.md": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+        "pages/students/2016/jan_holp/timovy_projekt/README.md": "79eebdfd3af7e8d5c279077e285a580731b1cf8cc9b4454f6596796e9915f70c",
+        "pages/students/2016/lukas_pokryvka/README.md": "f279456c5c277ff1ee0dd89a3464ed9f9f67b44bc185a5da9811ef5cfb675002",
+        "pages/students/2016/lukas_pokryvka/dp2021/README.md": "39b61376ce108e4a930f926ab2d6a997207457121bccfcf617129a3b210a15cf",
+        "pages/students/2016/lukas_pokryvka/timovy_projekt/README.md": "4de4b6c8cfed17b6c60090f82df1d55a66be70a79d3013850689b2baebdd801b",
+        "pages/students/2016/mark_feher/README.md": "bec37c0319fe0869a0377d771db23d090dbda93c27328f5697cdaeb3bbccfe9b",
+        "pages/students/2016/maros_harahus/README.md": "0da150e91918632c9e8cd1443f04da403595a372517e254928ce9f76c9d5033d",
+        "pages/students/2016/maros_harahus/timovy_projekt/README.md": "75af19f7c68f4b646fa1dbae75f3f3dfc52633a9f9da1400d21808bb20be0d83",
+        "pages/students/2016/patrik_pavlisin/README.md": "dc1c648d7cd9b0f0e021c57f9b98189e37c100871439e7e304390715913f2763",
+        "pages/students/2016/patrik_pavlisin/dp21/README.md": "ac1eb0fd0360ad34aa21b7c8da5bb010a6edf10be5721f28dedc3986cd1d8c54",
+        "pages/students/2016/patrik_pavlisin/dp22/README.md": "f2a8c619a812b99abab618ae388eb672407950e1f33fb8d7da7a528c4de81d50",
+        "pages/students/2016/patrik_pavlisin/tp20/README.md": "ae9c771f27e7e093bc29c6888c8d0b27ed61fb41519fd0d7e7f4f6cbb4626edc",
+        "pages/students/2016/tomas_kucharik/README.md": "6a1a215433eb1857660177b4cb2ecec03c6a86743d3a04cd6d5c9d9682c9e63b",
+        "pages/students/2017/dmytro_ushatenko/README.md": "65f4ac177d492162b08ca5da3d9eb048bd088a3c117a28a662113b7e4bc3a4ec",
+        "pages/students/2017/martin_jancura/README.md": "6a0a18d5095dcfb12abfdd72c9cfd8025d714fb5e29a7858cd2b285e4cb986b9",
+        "pages/students/2017/stanislav_matsunych/README.md": "90ffa11eff4a4360b0252dbc53a116c3a5777b3d5dd86ce5c89c02bdd2051c73",
+        "pages/students/2017/than_trung_thanh/README.md": "2150519b8580e7337520db4de64d2d1c82d2634380537d019f41c506dbf2979d",
+        "pages/students/2018/NaN1name_buday/README.md": "23c8c58090bdc954347e36f141bea91f43c8d5b1d777e346864a2ebe84c0bcce",
+        "pages/students/2018/adrian_remias/README.md": "e5bfe1d5d4e626beaa636a02d34ec58cf2cff9b0e57833905af35b80bf1465a2",
+        "pages/students/2018/david_ilas/README.md": "96c45ff9c5a821ddcb1a2fc86d487bd8f4729d1b76887914b0f5fbf99af45af0",
+        "pages/students/2018/david_omasta/README.md": "4164051870e1e1cf8949ed18a74196e9ea789dfbe4fee24bdeb736ca42848327",
+        "pages/students/2018/martin_wencel/README.md": "0e6a7a981689f22304f8baa5de97d4315c0695c94c9277de7851943eb012253b",
+        "pages/students/2018/oleh_bilykh/README.md": "4072c62dd8ee753a0c8fa43fdfbbdf566e394c2df13e6a79d08a302e332710b1",
+        "pages/students/2018/ondrej_megela/README.md": "969ff2e1a436b8f957e7168cf5eb678196b99aeda17701c502b016441f644065",
+        "pages/students/2018/samuel_sirotnik/README.md": "cc230ede67b6e8632511c92a5ed6d1e9deb42e70819280222bdfd369ee77fc70",
+        "pages/students/2019/alina_vitko/README.md": "9b071fe2aff470c275ae027be236fe836f741f774cabb9cbbb55304a8d5102cd",
+        "pages/students/2019/artem_yatsenko/README.md": "328b101e46c51c959e792cf7ca226942f81d2ab8fe9648ad6d0ecd07016429e2",
+        "pages/students/2019/david_stromp/README.md": "e46ca8691d5ec894128f6f6f2ed33e2c7bd141f80719bb0449b20f10f30cca09",
+        "pages/students/2019/dmytro_mural/README.md": "e881a47d847c3ef8d680844113fdbfbb87553b28982bf9eef4936f6d30923e93",
+        "pages/students/2019/filip_tomas/README.md": "75a1fe7b921e4bff0fdf9e2a957b4adad4b4a57aa7ebdffbb73dd82c02123f94",
+        "pages/students/2019/kristian_sopkovic/README.md": "3286960fb30f584df35455f9b6dfae81ca4827a1d711d2bcd6e48c3f150c9279",
+        "pages/students/2019/michal_stromko/README.md": "3ae1dd9b6638a6a249a94be3f3846286555666aefa225bb3a2c7288b35900ce7",
+        "pages/students/2019/michal_stromko/vp2021/README.md": "feb2e093fdd6461239e79b1e1f96e65a1c8cd96e8a2b70535ae556a9c198046a",
+        "pages/students/2019/michal_stromko/vp2021/tutorials/README.md": "40e3fa448c7329ef863e705435e23fe6b4788de440710f066fe0eed92d132bad",
+        "pages/students/2019/michal_stromko/vp2021/tutorials/img/README.md": "e8d3ac160b17dfdee2c3e996bf6a03af7a6f16f9140bf5cf7ee343a82dd1865f",
+        "pages/students/2019/patrik_pokrivcak/README.md": "3e3957ad0f7aca59d6eb64c38a3f5e21d8fd953200825d0d0e5cacabfbc35a7d",
+        "pages/students/2019/samuel_horani/README.md": "86e6959ccc612b069687b57996fa3f61660075d6a0d65bfec64f423d39a0a999",
+        "pages/students/2020/david_kostilnik/README.md": "2927524b75da59a64120fc364a0fc374986fa4d364d54e52c36800762c203264",
+        "pages/students/2020/jakub_kristian_lukas/README.md": "f139d9b1ebf5a3f45612f3df5703ecdd935b90701763c27c7c529044c0b0455a",
+        "pages/students/2020/kamil_tomcufcik/README.md": "9d865adf538bc6d7992ee4f4882713447da5c936fca17c43ca0f576072537048",
+        "pages/students/2020/matej_kobylan/README.md": "3d7b289e17b8f25678a07e9ee0e6741688485a0ccb1e50940a6a6acc6312a547",
+        "pages/students/2020/matus_suchanic/README.md": "770ca49538cca22df35f8e7ba736ee97483e4fe5788c221496553592b4e78dc6",
+        "pages/students/2020/pavol_hudak/README.md": "33d06fccc6451e5f19b27b0cad001a34e81635b4ea294a01dec7771c823abc40",
+        "pages/students/2020/vladyslav_krupko/README.md": "4c28e6d38651bb2cafcbf4a18e35210cd369aef2caff6c5b0aa52fac38f0fe7d",
+        "pages/students/2021/artur_hyrenko/README.md": "0568261f4ade4de5472edcfeb1ffed9abae18dc67e4050f4ce2ad088839a2bb3",
+        "pages/students/2021/eduard_matovka/README.md": "de2a7ae38ec87110e347956b37d0605a298ed2847f972961cc9b977da8221d43",
+        "pages/students/2021/manohar_gowdru_shridharu/README.md": "c00ceb620ed7087ead5d9448d476bbb98d273d1d9e7bf1bc4355c367b26c7a6b",
+        "pages/students/2021/martin_sarissky/README.md": "8e97b45a253745faa55f63c844e0b5dfe680aeaf6eb01fb8f9d06ac6e8ee8bcf",
+        "pages/students/2021/matej_novotny/README.md": "b083a7cadde532e3428dc9a57bc61456fcc5905bda91e011b96033d42bf5d9cb",
+        "pages/students/2021/matej_scislak/README.md": "b9a32cce8605e044a4f2828ff2d91a2995a62ef3a841e62f0a225c97221f0cc8",
+        "pages/students/2021/nikita_bodnar/README.md": "2016c0dd923c0f8e561d5cea42ac81da3610fdcc54a4527f56d078e34e53ce90",
+        "pages/students/2021/oleksandr_hryshchenko/README.md": "4b073992b715b6c75b79fd2c8f850a69f940966e3a61a82c625c32c945a93036",
+        "pages/students/2021/vladimir_ferko/README.md": "68e7d311896eda2171e5eec1ec3a8da90d326ed91acaace8058f68664a50d271",
+        "pages/students/2022/andrii_pervashov/README.md": "319dcaa1cc2e29dd7e3a710bab26cb3ac6944ab71b0b97e1de076b1199122334",
+        "pages/students/2022/daniil_huzenko/README.md": "390782fbff6220a9462060ce1a62959f2a2b88f42d06e2c51d1f406b75d73f2c",
+        "pages/students/2022/jakub_schwarc/README.md": "f541a8852c2b0f6f9d7182e01a34c4e3beee9bfa55427a06b331638b11eb8be6",
+        "pages/students/2022/jan_malinovsky/README.md": "8af021bfb131fa005a7aec3e2f2f12fc17fa5dc14e2675311adbbfc445a4bcab",
+        "pages/students/2022/jan_ptak/README.md": "131cf053764fd5423541ebe672c3b4f5da875cfb8e4b821f1335ddb967282282",
+        "pages/students/2022/oleh_poiasnik/README.md": "c2f7dc3f61f8b4a5b43ded1e3ae4c7d3e7f874439ee6b17dd9c8d45dabc25bc8",
+        "pages/students/2022/samuel_vasil/README.md": "67c807049147aacd04f3050f4ff0233a8b3f0a239ab173b430e13dfe113ad0a5",
+        "pages/students/2022/serhii_yemets/README.md": "ffbeac893947acafe855d576322a8826f9d4f87ac344cb31f8ceeefa244b9133",
+        "pages/students/2022/tetiana_mohorian/README.md": "c9efdc632bf1ca89a0bfde0dc00be7aaac8a85da3d92e2ce8506236c73e93cfb",
+        "pages/students/2022/valerii_kutsenko/README.md": "dc764c609ad3c1df59cfea997d4db82a591b1b1e0764453e319d841115507e4f",
+        "pages/students/2022/vladyslav_yanchenko/README.md": "4d1ce216e3b526ee437ddbc658e51d7bb83530087e43628c096a569a3854a47f",
+        "pages/students/2022/yevhenii_leonov/README.md": "b90e41db0cec2451027020a9102d32a24d777e8a38b604ccca87b36cd6b3a292",
+        "pages/students/2023/denis_le_thanh/README.md": "2cb83f942d86a02712b0c58778e46dc2211346d1a6eba606471dfa832245db61",
+        "pages/students/2023/maksym_zatirka/README.md": "7a7d2cbe276c352dacfbc6ba6de5f15303d30ecf34845ff8da8d085f3d67c3dd",
+        "pages/students/2023/oleksandr_dorybohov/README.md": "c24d6f941de0dafb4459cea8259cf147821cafc86ba65be5b528f419f734bc18",
+        "pages/students/2023/ronald_zalacko/README.md": "724b0f8e6e641236dba9bddbd77f4546e09514076c7dfb151807aaf4b882c96f",
+        "pages/students/2023/rostyslav_rodzhuk/README.md": "de356dd7427c8a0716c165a0f88ae47b691a86ee7a388f60e2d2e9306298415c",
+        "pages/students/2023/samuel_dzurina/README.md": "4818191d044b91219c796a35010c95da5cf2093e050d494f30b219b91e13fa0e",
+        "pages/students/2023/simona_bobrovcanova/README.md": "e473a1911dae10ab40e2cf6696a8be616a58c22aaf09af5953fbac475791a35e",
+        "pages/students/2023/yevhenii_medushivskyi/README.md": "5710ebf91c893a12fe6d19f5372311938d1e7c7bbdf203f2dc8ed2fe7c4f5a24",
+        "pages/topics/akopisat/README.md": "c7245e1fbfa4160618502c0baf8cc6bc47cd9a82b8a85bc1c645314829a7b76a",
+        "pages/topics/bert/README.md": "35e507f7ec2ad1d3526e2a15f94ea085ee25f680a02daf6df62fd429bd9c25c9",
+        "pages/topics/chatbot/README.md": "6f2f83ffb80109be47f9750cca9d3036ec3645abef9f2b590bd6825780c7c48a",
+        "pages/topics/faq/README.md": "15cfbd9c2a1649c80a7a58922e2b66dadab5c86a5ef33ef16c82bebd4d2dbf06",
+        "pages/topics/hatespeech/README.md": "364df94e70dd740c63544e2586ba7b957780f714fc0c941d1b04b02233e6cbc5",
+        "pages/topics/legal/README.md": "003bb9bfe1e0ca21652091c2d11496312eb04e0689fe459e14c2d5a738e8eeea",
+        "pages/topics/morphology/README.md": "e242cc5ff1a4cc98e227566c0ead999bbc95937b7c20be62cb2afb773d0d35c0",
+        "pages/topics/named-entity/README.md": "66b2e25f70211d432d596e377d552b3b8f0c525b14c31e3bc9c956c3aeadca84",
+        "pages/topics/named-entity/navod/README.md": "1ff87079a98411de0414754c60241e08d2214ddcd63ddf974e4340cce2ce5b02",
+        "pages/topics/nlp/README.md": "6f90abf9dc3f0ff33475283c13b4a08561abf631425ec2daacb076155a9e02c2",
+        "pages/topics/otvorene/README.md": "cd5544925553fe3840c0ea6f8ef60ab756e26ab6757bf810025c948d8d4f98fc",
+        "pages/topics/prodigy/README.md": "5e4f691f2754c2ec4afbaab2f41e6db77feb5b322dd9344465eb40c35f116499",
+        "pages/topics/python/README.md": "2ef073adba672142c39e9a59fa2e00a98ab52bda54233d46fbe6940ee23f9fcc",
+        "pages/topics/question/README.md": "408078a2a67cea2699e12359e85ab641136231e6104c2afdf874a705879525df",
+        "pages/topics/question/navod/README.md": "061e737c69184b2bd244316ca7ba841f7186fe01ee80ee993742a733dcad3ecd",
+        "pages/topics/question/nezodpovedatelne/README.md": "b6703d9690f4599b973efb27595ce157773ae0fc0d55cf2c0a09d9f50e8e4c92",
+        "pages/topics/question/validacie/README.md": "3216a7127abdfc6da50ed116f44ff2d6785e7da4af70d23aceda774e745ad562",
+        "pages/topics/resources/README.md": "09405397387f00cbcc5680f7d655f2d861e4d92e5df792a62d9dfd04f3534f48",
+        "pages/topics/seq2seq/README.md": "969d15c03a63a3f8d5edc9017fe5a0fb7311fdf4de4a3473a08266241f92c1e2",
+        "pages/topics/spacy/README.md": "f504e5db77cb8694f08a57fb8636e522d59a0e669216f4bc8273bc3c5f375f87",
+        "pages/topics/steel/README.md": "b4068cef2ed197711467ac47630d5b2064fdc8828f898f7af99664e94d833330",
+        "pages/topics/translation/README.md": "a7d4289e72199b307dc5ad8fc69cda39c1a33c27a6c7447af70aa9c3cc6b8635"
+      }
+    }
+  },
+  "graph": {
+    "sha256": "3f799a1ff8cd86cca24515d742430a111734ad3c65425587b64ae4ffe891701f",
+    "nodes": 447,
+    "relationships": 879
+  },
+  "parameters": {
+    "limit": 40,
+    "max_entities": 8
+  },
+  "dev_summary": {
+    "questions": 589,
+    "strict_count": 589,
+    "strict_accuracy": 1.0,
+    "error_count": 0,
+    "value_score": 1.0,
+    "entity_score": 1.0,
+    "relation_score": 1.0,
+    "property_score": 1.0,
+    "relation_choice_score": 1.0,
+    "document_recall": 1.0,
+    "source_recall": 1.0,
+    "binding_score": 1.0,
+    "path_score": 1.0,
+    "answerability_accuracy": 1.0,
+    "latency_mean_seconds": 0.041531,
+    "latency_median_seconds": 0.040416,
+    "latency_p95_seconds": 0.056271
+  },
+  "source_audit_summary": {
+    "dev": {
+      "questions": 589,
+      "source_consistent": 589,
+      "issues": {}
+    },
+    "test": {
+      "questions": 13,
+      "source_consistent": 13,
+      "issues": {}
+    }
+  },
+  "policy": "Report all questions including negatives. No exclusions or holdout tuning.",
+  "dev_results_sha256": "9334783e76158987f538f103a3ea90691a4c4115c4f2e6cb6327b5d9eda9f478",
+  "audit_sha256": "9b03a6c5914133bb5c1ce056ce0f570f3c43243968a6f8cbbfbbee584461e5ac"
+}

tools/graphrag/evaluation/results/final_v3_2026-10-25/REPORT.md

@@ -0,0 +1,64 @@
+# GraphRAG: finálna opravená verzia v3 — 25. 10. 2026
+
+Upravovalo sa grafové vyhľadávanie a jeho evaluácia. Klasický RAG backend, SQLite index a jeho zmrazené odpovede zostali zachované.
+
+## Výsledky a ich význam
+
+| Meranie | Strict | Úspešnosť | Runtime chyby | Priemer | Medián | p95 |
+|---|---:|---:|---:|---:|---:|---:|
+| DEV/regresia v3 | 589/589 | 100.00 % | 0 | 41.53 ms | 40.42 ms | 56.27 ms |
+| Doplnkový nový TEST v3 | 13/13 | 100.00 % | 0 | 54.83 ms | 43.19 ms | 109.37 ms |
+| Širší nezávislý TEST v2 pred finálnymi opravami | 67/80 | 83.75 % | 0 | 46.29 ms | 37.56 ms | 54.18 ms |
+
+**13/13 nie je dôkaz 100 % správnosti celého programu.** Posledný holdout používa len dva zvyšné doteraz nepoužité profily. Obsahuje kategórie, autora dokumentu, začiatok štúdia, nulový počet a negatívne prípady. Nemá reprezentatívne pokrytie všetkých deviatich kladných reasoning kategórií. Najširšie nezávislé meranie v tejto iterácii je stále zaznamenané v2 67/80; po zistení chýb sa tieto otázky stali DEV/regresiou.
+
+Všetky pôvodné prípady ostali zachované: DEV v3 obsahuje celý súbor 589 už vystavených otázok z v2 vrátane negatívnych otázok a nulového počtu. Nedochádza k vynechávaniu neúspešných prípadov. Starý TEST sa po oprave nevydáva za nový nezávislý TEST. Latencie merajú samotné grafové vyhľadávanie, bez LLM a OpenWebUI.
+
+## Opravy benchmarku a referenčných údajov
+
+- Pôvodný súbor 500 otázok a výsledky v1 zostali nezmenené; pôvodný v1 je reprodukovateľný z tagu `classic-rag-e5-small-frozen` pre RAG a `graphrag-v1-frozen` pre grafovú verziu.
+- 128 otázok malo nepodložený predpoklad o doslovnom frontmatter tagu. Ich opravená anotácia vyžaduje explicitné `no_answer`, dôvod a neprítomnosť kladného vymysleného faktu. Audit sám nepripočítava úspech; hodnotí sa skutočný výsledok vyhľadávania.
+- 9 otázok má opravenú presnú množinu kategórií vrátane poddokumentov.
+- Oracle číta pôvodné Markdown súbory, YAML metadata, akademické nadpisy a označené názvy. Nepoužíva GraphRAG builder ani výsledky retrievalu ako zdroj odpovedí. Proveniencia obsahuje cesty, SHA-256, zdrojové riadky a základ určenia roku.
+- Pri v2 sa odhalila ďalšia chyba anotácie: názov pod `Bakalársky projekt 2020` sa nesmie premeniť na názov záverečnej práce 2021 len podľa kategórie `bp2021`. Opravené je rozlíšenie projektu a práce v oboch nezávislých parseroch. Záznam všetkých 24 zmien/rozšírení väzieb, vrátane štyroch nových opráv kladného predpokladu na záporný, je v `../v3_preparation/source_adjudications.json`.
+- Zdrojové validátory hlásia 0 zostávajúcich nesúladov. Ide o pravidlovú kontrolu s dohľadateľným zdrojom, nie záruku bezchybnej ľudskej anotácie celého korpusu.
+
+## Vyhľadávanie a evaluator
+
+Celé mená sa porovnávajú v súvislom poradí s kontrolovaným skloňovaním. Presná zhoda má prednosť; neznáme celé aj jednoslovné meno alebo neznámy kandidát nezruší obmedzenie na osoby. Rovnomenné profily vedú k `ambiguous`. Rola autora dokumentu sa rozlišuje od opisovaného študenta.
+
+Tag sa číta ako celý požadovaný výraz vrátane formulácie pred menami. Neznámy viacslovný tag sa neskracuje na známy prefix. Výsledky majú stav, dôvod, kontrolované entity a dôkazy zamietnutia.
+
+Evaluator skladá typovaný graf dôkazov z konkrétnych ciest a kontraktov pripravených Cypher dotazov. Kontroluje osobu, konkrétnu prácu, hodnotu jej roku/názvu, správnu reláciu, kategóriu a autora toho istého dokumentu, presnú množinu a počet. Odmieta zmiešanie rôznych osôb/prác a protichodných vlastností rovnakého uzla. Začiatok štúdia sa neráta ako rok práce. Štruktúrované vzory ciest sú skutočne overované. Alternatívne dokumenty dokazujúce rovnaký fakt sa hodnotia ako alternatívy a kontroluje sa URL konkrétneho dôkazu.
+
+Builder navyše rozpoznáva `Vedecký projekt` a `Bakalársky projekt`; akademický rozsah `2023/24` sa rozlišuje podľa jeho rokov a podporujúcej kategórie. Graf bol pred obnovou zálohovaný.
+
+## Testy a zmrazenie
+
+Celý projekt: **430 passed, 4 skipped**, vrátane desiatich skutočných Neo4j prípadov v izolovaných rollback transakciách. Nové testy cielia aj úmyselne nesprávne výsledky a záporné prípady, nie iba kladné ukážky.
+
+Pred TESTom v3 bol celý DEV úspešný a vznikol `../../frozen_v3.json`. Manifest overuje kód, dataset, pôvodný Markdown, JSON vstupy, graf, audit a parametre. Kontroluje nulový prekryv profilov DEV/TEST. Existujúce manifesty a finálne výsledky sú chránené pred prepísaním. Po TESTe v3 sa Python kód ani dataset nemenili.
+
+| Kategória | DEV v3 strict | TEST v3 strict |
+|---|---:|---:|
+| `candidate_set_aggregation` | 25/25 | 1/1 |
+| `conjunctive_disambiguation` | 28/28 | nepokryté |
+| `document_author_category_multi_hop` | 68/68 | 2/2 |
+| `negative_lookup` | 16/16 | 6/6 |
+| `person_keyword_title` | 88/88 | nepokryté |
+| `person_multiple_categories` | 38/38 | 2/2 |
+| `person_title_year` | 88/88 | nepokryté |
+| `same_academic_year_pair` | 108/108 | nepokryté |
+| `same_start_year_pair` | 68/68 | nepokryté |
+| `shared_keyword_pair` | 62/62 | nepokryté |
+| `person_start_year` | samostatné jednotkové pokrytie | 2/2 |
+
+## Zostávajúce obmedzenia
+
+- Na silný všeobecný odhad presnosti treba ďalší nezávislý súbor otázok/dát od človeka alebo nový korpus. Opakované dolaďovanie na už odhalenom TESTe by z neho urobilo DEV.
+- Riešenie používa pravidlá pre slovenské formulácie a kontrolované skloňovanie; nepokrýva ľubovoľné preklepy a všetky možné otázky.
+- Pri nejednoznačných rovnomenných profiloch sa bezpečne odmieta výber. Všeobecné počítanie iných domén a univerzálny prienik pre ľubovoľný počet ľudí nie sú týmto benchmarkom garantované.
+- Ide o retrieval vrstvu. GraphRAG ešte nie je kompletný OpenWebUI agent a toto meranie nehodnotí kvalitu odpovedí LLM.
+- V1, v2 a v3 majú rôzne benchmarky a prísnosť evaluatora. Ich percentá nemožno bez kontrolovaného experimentu priamo odčítať ako zlepšenie modelu.
+
+Zadané chyby bodu 3 sú opravené a meranie aktuálnej verzie uzavreté s uvedenými limitmi. Nasleduje integrácia nástroja a neskôr reprezentatívne porovnanie RAG/GraphRAG.

tools/graphrag/evaluation/results/final_v3_2026-10-25/graphrag_test.csv

@@ -0,0 +1,14 @@
+complete,answerability_ok,bindings_ok,binding_score,paths_ok,path_score,count_ok,expected_count,actual_count,id,split,category,difficulty,strict,value_score,entity_score,relation_score,property_score,relation_choice_score,document_recall,source_recall,entity_set_ok,latency_seconds,error,question,expected_values,matched_values,expected_relations,actual_relations,expected_documents,actual_documents,expected_source_urls,actual_source_urls,fact_types,strategies,binding_results,path_results
+True,True,True,1.0,True,1.0,True,,,fresh-001,test,person_multiple_categories,hard,True,1.0,1.0,1.0,1.0,1.0,1.0,1.0,True,0.047946,,Vypíš všetky kategórie dokumentov osoby Samuel Širotník.,"[""Samuel Širotník"", ""bp2021""]","[""Samuel Širotník"", ""bp2021""]",[],"[""HAS_DOCUMENT"", ""IN_CATEGORY""]","[""pages/students/2018/samuel_sirotnik/README.md""]","[""pages/students/2018/samuel_sirotnik/README.md""]","[""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""]","[""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""]","[""person_category""]","[""entity_resolution"", ""person_category_constraints""]","[{""binding"": {""person"": ""Samuel Širotník"", ""category"": ""bp2021""}, ""matched"": true}]","[{""pattern"": {""binding_index"": 0, ""node_kinds"": [""Person"", ""Document"", ""Category""], ""relationships"": [""HAS_DOCUMENT"", ""IN_CATEGORY""]}, ""matched"": true, ""witness"": [{""kind"": ""Person"", ""name"": ""Samuel Širotník"", ""element_id"": ""4:61c98d29-d5f0-43f4-879d-a5d65f4be67a:732""}, {""kind"": ""Document"", ""path"": ""pages/students/2018/samuel_sirotnik/README.md"", ""source_url"": ""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""}, {""kind"": ""Category"", ""name"": ""bp2021"", ""element_id"": ""4:61c98d29-d5f0-43f4-879d-a5d65f4be67a:70""}]}]"
+True,True,True,1.0,True,1.0,True,,,fresh-002,test,document_author_category_multi_hop,hard,True,1.0,1.0,1.0,1.0,1.0,1.0,1.0,True,0.04332,,Ktorý autor dokumentu je pri osobe Samuel Širotník spojený s kategóriou bp2021?,"[""Samuel Širotník"", ""bp2021"", ""Daniel Hladek""]","[""Samuel Širotník"", ""bp2021"", ""Daniel Hladek""]",[],"[""AUTHORED_DOCUMENT"", ""HAS_DOCUMENT"", ""IN_CATEGORY""]","[""pages/students/2018/samuel_sirotnik/README.md""]","[""pages/students/2018/samuel_sirotnik/README.md""]","[""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""]","[""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""]","[""person_category"", ""person_document_author""]","[""entity_resolution"", ""person_category_constraints"", ""person_document_author_constraints""]","[{""binding"": {""person"": ""Samuel Širotník"", ""category"": ""bp2021"", ""document_author"": ""Daniel Hladek"", ""document_path"": ""pages/students/2018/samuel_sirotnik/README.md"", ""same_document"": true}, ""matched"": true}]","[{""pattern"": {""binding_index"": 0, ""node_kinds"": [""Person"", ""Document"", ""Category""], ""relationships"": [""HAS_DOCUMENT"", ""IN_CATEGORY""]}, ""matched"": true, ""witness"": [{""kind"": ""Person"", ""name"": ""Samuel Širotník"", ""element_id"": ""4:61c98d29-d5f0-43f4-879d-a5d65f4be67a:732""}, {""kind"": ""Document"", ""path"": ""pages/students/2018/samuel_sirotnik/README.md"", ""source_url"": ""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""}, {""kind"": ""Category"", ""name"": ""bp2021"", ""element_id"": ""4:61c98d29-d5f0-43f4-879d-a5d65f4be67a:70""}]}, {""pattern"": {""binding_index"": 0, ""node_kinds"": [""Person"", ""Document"", ""DocumentAuthor""], ""relationships"": [""HAS_DOCUMENT"", ""AUTHORED_DOCUMENT""]}, ""matched"": true, ""witness"": [{""kind"": ""Person"", ""name"": ""Samuel Širotník"", ""element_id"": ""4:61c98d29-d5f0-43f4-879d-a5d65f4be67a:732""}, {""kind"": ""Document"", ""path"": ""pages/students/2018/samuel_sirotnik/README.md"", ""source_url"": ""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""}, {""kind"": ""DocumentAuthor"", ""name"": ""Daniel Hladek"", ""element_id"": ""4:61c98d29-d5f0-43f4-879d-a5d65f4be67a:779""}]}]"
+True,True,True,1.0,True,1.0,True,,,fresh-003,test,person_start_year,hard,True,1.0,1.0,1.0,1.0,1.0,1.0,1.0,True,0.184617,,Aký rok začiatku štúdia je uvedený pri osobe Samuel Širotník?,"[""Samuel Širotník"", ""2018""]","[""Samuel Širotník"", ""2018""]",[],[],"[""pages/students/2018/samuel_sirotnik/README.md""]","[""pages/students/2018/samuel_sirotnik/README.md""]","[""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""]","[""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""]","[""person_start_year""]","[""entity_resolution"", ""start_year_constraints""]","[{""binding"": {""person"": ""Samuel Širotník"", ""start_year"": 2018}, ""matched"": true}]","[{""pattern"": {""binding_index"": 0, ""node_kinds"": [""Person"", ""Document""], ""relationships"": [""HAS_DOCUMENT""]}, ""matched"": true, ""witness"": [{""kind"": ""Person"", ""name"": ""Samuel Širotník"", ""element_id"": ""4:61c98d29-d5f0-43f4-879d-a5d65f4be67a:732"", ""start_year"": 2018}, {""kind"": ""Document"", ""path"": ""pages/students/2018/samuel_sirotnik/README.md"", ""source_url"": ""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""}]}]"
+True,True,True,1.0,True,1.0,True,,,fresh-004,test,negative_lookup,hard,True,1.0,1.0,1.0,1.0,1.0,1.0,1.0,True,0.048377,,"Aký názov práce má osoba Samuel Širotník, ak jej dokument obsahuje kľúčové slovo nenajdenyoverovacitag?",[],[],[],[],"[""pages/students/2018/samuel_sirotnik/README.md""]","[""pages/students/2018/samuel_sirotnik/README.md""]","[""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""]","[""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""]",[],"[""entity_resolution""]",[],[]
+True,True,True,1.0,True,1.0,True,,,fresh-005,test,negative_lookup,hard,True,1.0,1.0,1.0,1.0,1.0,1.0,1.0,True,0.051157,,Aký názov práce má osoba Samuel Širotník v roku 2098?,[],[],[],[],"[""pages/students/2018/samuel_sirotnik/README.md""]","[""pages/students/2018/samuel_sirotnik/README.md""]","[""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""]","[""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""]",[],"[""entity_resolution""]",[],[]
+True,True,True,1.0,True,1.0,True,,,fresh-006,test,person_multiple_categories,hard,True,1.0,1.0,1.0,1.0,1.0,1.0,1.0,True,0.059201,,Vypíš všetky kategórie dokumentov osoby Stanislav Matsunych.,"[""Stanislav Matsunych"", ""bp2020""]","[""Stanislav Matsunych"", ""bp2020""]",[],"[""HAS_DOCUMENT"", ""IN_CATEGORY""]","[""pages/students/2017/stanislav_matsunych/README.md""]","[""pages/students/2017/stanislav_matsunych/README.md""]","[""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych""]","[""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych""]","[""person_category""]","[""entity_resolution"", ""person_category_constraints""]","[{""binding"": {""person"": ""Stanislav Matsunych"", ""category"": ""bp2020""}, ""matched"": true}]","[{""pattern"": {""binding_index"": 0, ""node_kinds"": [""Person"", ""Document"", ""Category""], ""relationships"": [""HAS_DOCUMENT"", ""IN_CATEGORY""]}, ""matched"": true, ""witness"": [{""kind"": ""Person"", ""name"": ""Stanislav Matsunych"", ""element_id"": ""4:61c98d29-d5f0-43f4-879d-a5d65f4be67a:723""}, {""kind"": ""Document"", ""path"": ""pages/students/2017/stanislav_matsunych/README.md"", ""source_url"": ""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych""}, {""kind"": ""Category"", ""name"": ""bp2020"", ""element_id"": ""4:61c98d29-d5f0-43f4-879d-a5d65f4be67a:69""}]}]"
+True,True,True,1.0,True,1.0,True,,,fresh-007,test,document_author_category_multi_hop,hard,True,1.0,1.0,1.0,1.0,1.0,1.0,1.0,True,0.038137,,Ktorý autor dokumentu je pri osobe Stanislav Matsunych spojený s kategóriou bp2020?,"[""Stanislav Matsunych"", ""bp2020"", ""Daniel Hladek""]","[""Stanislav Matsunych"", ""bp2020"", ""Daniel Hladek""]",[],"[""AUTHORED_DOCUMENT"", ""HAS_DOCUMENT"", ""IN_CATEGORY""]","[""pages/students/2017/stanislav_matsunych/README.md""]","[""pages/students/2017/stanislav_matsunych/README.md""]","[""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych""]","[""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych""]","[""person_category"", ""person_document_author""]","[""entity_resolution"", ""person_category_constraints"", ""person_document_author_constraints""]","[{""binding"": {""person"": ""Stanislav Matsunych"", ""category"": ""bp2020"", ""document_author"": ""Daniel Hladek"", ""document_path"": ""pages/students/2017/stanislav_matsunych/README.md"", ""same_document"": true}, ""matched"": true}]","[{""pattern"": {""binding_index"": 0, ""node_kinds"": [""Person"", ""Document"", ""Category""], ""relationships"": [""HAS_DOCUMENT"", ""IN_CATEGORY""]}, ""matched"": true, ""witness"": [{""kind"": ""Person"", ""name"": ""Stanislav Matsunych"", ""element_id"": ""4:61c98d29-d5f0-43f4-879d-a5d65f4be67a:723""}, {""kind"": ""Document"", ""path"": ""pages/students/2017/stanislav_matsunych/README.md"", ""source_url"": ""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych""}, {""kind"": ""Category"", ""name"": ""bp2020"", ""element_id"": ""4:61c98d29-d5f0-43f4-879d-a5d65f4be67a:69""}]}, {""pattern"": {""binding_index"": 0, ""node_kinds"": [""Person"", ""Document"", ""DocumentAuthor""], ""relationships"": [""HAS_DOCUMENT"", ""AUTHORED_DOCUMENT""]}, ""matched"": true, ""witness"": [{""kind"": ""Person"", ""name"": ""Stanislav Matsunych"", ""element_id"": ""4:61c98d29-d5f0-43f4-879d-a5d65f4be67a:723""}, {""kind"": ""Document"", ""path"": ""pages/students/2017/stanislav_matsunych/README.md"", ""source_url"": ""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych""}, {""kind"": ""DocumentAuthor"", ""name"": ""Daniel Hladek"", ""element_id"": ""4:61c98d29-d5f0-43f4-879d-a5d65f4be67a:779""}]}]"
+True,True,True,1.0,True,1.0,True,,,fresh-008,test,person_start_year,hard,True,1.0,1.0,1.0,1.0,1.0,1.0,1.0,True,0.037406,,Aký rok začiatku štúdia je uvedený pri osobe Stanislav Matsunych?,"[""Stanislav Matsunych"", ""2017""]","[""Stanislav Matsunych"", ""2017""]",[],[],"[""pages/students/2017/stanislav_matsunych/README.md""]","[""pages/students/2017/stanislav_matsunych/README.md""]","[""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych""]","[""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych""]","[""person_start_year""]","[""entity_resolution"", ""start_year_constraints""]","[{""binding"": {""person"": ""Stanislav Matsunych"", ""start_year"": 2017}, ""matched"": true}]","[{""pattern"": {""binding_index"": 0, ""node_kinds"": [""Person"", ""Document""], ""relationships"": [""HAS_DOCUMENT""]}, ""matched"": true, ""witness"": [{""kind"": ""Person"", ""name"": ""Stanislav Matsunych"", ""element_id"": ""4:61c98d29-d5f0-43f4-879d-a5d65f4be67a:723"", ""start_year"": 2017}, {""kind"": ""Document"", ""path"": ""pages/students/2017/stanislav_matsunych/README.md"", ""source_url"": ""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych""}]}]"
+True,True,True,1.0,True,1.0,True,,,fresh-009,test,negative_lookup,hard,True,1.0,1.0,1.0,1.0,1.0,1.0,1.0,True,0.043193,,"Aký názov práce má osoba Stanislav Matsunych, ak jej dokument obsahuje kľúčové slovo nenajdenyoverovacitag?",[],[],[],[],"[""pages/students/2017/stanislav_matsunych/README.md""]","[""pages/students/2017/stanislav_matsunych/README.md""]","[""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych""]","[""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych""]",[],"[""entity_resolution""]",[],[]
+True,True,True,1.0,True,1.0,True,,,fresh-010,test,negative_lookup,hard,True,1.0,1.0,1.0,1.0,1.0,1.0,1.0,True,0.039434,,Aký názov práce má osoba Stanislav Matsunych v roku 2098?,[],[],[],[],"[""pages/students/2017/stanislav_matsunych/README.md""]","[""pages/students/2017/stanislav_matsunych/README.md""]","[""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych""]","[""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych""]",[],"[""entity_resolution""]",[],[]
+True,True,True,1.0,True,1.0,True,,,fresh-011,test,negative_lookup,hard,True,1.0,1.0,1.0,1.0,1.0,1.0,1.0,True,0.040584,,Aký názov diplomovej práce má osoba Diana Neznáma Overovacia?,[],[],[],[],[],[],[],[],[],[],[],[]
+True,True,True,1.0,True,1.0,True,,,fresh-012,test,negative_lookup,hard,True,1.0,1.0,1.0,1.0,1.0,1.0,1.0,True,0.038062,,Aký názov diplomovej práce má osoba Milan Neprítomný Vzorový?,[],[],[],[],[],[],[],[],[],[],[],[]
+True,True,True,1.0,True,1.0,True,0,0,fresh-013,test,candidate_set_aggregation,hard,True,1.0,1.0,1.0,1.0,1.0,1.0,1.0,True,0.041371,,"Z kandidátov Samuel Širotník, Stanislav Matsunych vyber osoby s prácou v roku 2098. Koľko ich je?","[""0""]","[""0""]",[],[],"[""pages/students/2017/stanislav_matsunych/README.md"", ""pages/students/2018/samuel_sirotnik/README.md""]","[""pages/students/2017/stanislav_matsunych/README.md"", ""pages/students/2018/samuel_sirotnik/README.md""]","[""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych"", ""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""]","[""https://zp.kemt.fei.tuke.sk/students/2017/stanislav_matsunych"", ""https://zp.kemt.fei.tuke.sk/students/2018/samuel_sirotnik""]",[],"[""entity_resolution""]",[],[]

tools/graphrag/evaluation/results/final_v3_2026-10-25/holdout_failures.json

@@ -0,0 +1,3 @@
+{
+  "questions": []
+}

tools/graphrag/evaluation/results/final_v3_2026-10-25/summary.json

@@ -0,0 +1,78 @@
+{
+  "version": "v3",
+  "dev": {
+    "questions": 589,
+    "strict_count": 589,
+    "strict_accuracy": 1.0,
+    "error_count": 0,
+    "value_score": 1.0,
+    "entity_score": 1.0,
+    "relation_score": 1.0,
+    "property_score": 1.0,
+    "relation_choice_score": 1.0,
+    "document_recall": 1.0,
+    "source_recall": 1.0,
+    "binding_score": 1.0,
+    "path_score": 1.0,
+    "answerability_accuracy": 1.0,
+    "latency_mean_seconds": 0.041531,
+    "latency_median_seconds": 0.040416,
+    "latency_p95_seconds": 0.056271
+  },
+  "test": {
+    "questions": 13,
+    "strict_count": 13,
+    "strict_accuracy": 1.0,
+    "error_count": 0,
+    "value_score": 1.0,
+    "entity_score": 1.0,
+    "relation_score": 1.0,
+    "property_score": 1.0,
+    "relation_choice_score": 1.0,
+    "document_recall": 1.0,
+    "source_recall": 1.0,
+    "binding_score": 1.0,
+    "path_score": 1.0,
+    "answerability_accuracy": 1.0,
+    "latency_mean_seconds": 0.054831,
+    "latency_median_seconds": 0.043193,
+    "latency_p95_seconds": 0.109367
+  },
+  "tests": {
+    "passed": 430,
+    "skipped": 4,
+    "neo4j_rollback_cases": 10
+  },
+  "freeze_sha256": "89731e4cc375e6c2a84759a4d7ec1db3f01f4a406be42c7a8d6df7ce27780ab1",
+  "independence": {
+    "source_profile_overlap": 0,
+    "test_profiles": 2,
+    "test_questions": 13,
+    "limitation": "Supplementary small holdout with limited category coverage; no universal 100% correctness claim."
+  },
+  "broader_v2_independent_test": {
+    "questions": 80,
+    "strict_count": 67,
+    "strict_accuracy": 0.8375,
+    "error_count": 0,
+    "value_score": 0.845833,
+    "entity_score": 0.84375,
+    "relation_score": 1.0,
+    "property_score": 1.0,
+    "relation_choice_score": 1.0,
+    "document_recall": 1.0,
+    "source_recall": 1.0,
+    "binding_score": 0.8375,
+    "path_score": 0.8375,
+    "answerability_accuracy": 0.8375,
+    "latency_mean_seconds": 0.046289,
+    "latency_median_seconds": 0.037556,
+    "latency_p95_seconds": 0.054178
+  },
+  "v2_test_now_development_only": true,
+  "legacy_137_resolution": {
+    "literal_tag_false_premises": 128,
+    "exact_category_corrections": 9
+  },
+  "post_test_python_changes": false
+}

tools/graphrag/evaluation/results/v3_preparation/partition.json

@@ -0,0 +1,84 @@
+{
+  "dev_count": 589,
+  "test_count": 13,
+  "dev_profiles": [
+    "pages/interns/bogdan_paul_chis/README.md",
+    "pages/interns/cesar_gutierrez/README.md",
+    "pages/interns/oliver_pejic/README.md",
+    "pages/interns/sevval_bulburu/README.md",
+    "pages/interns/yussef_ressaissi/README.md",
+    "pages/students/2016/darius_lindvai/README.md",
+    "pages/students/2016/dominik_nagy/README.md",
+    "pages/students/2016/ediz_morochovic/README.md",
+    "pages/students/2016/jakub_maruniak/README.md",
+    "pages/students/2016/jan_holp/README.md",
+    "pages/students/2016/lukas_pokryvka/README.md",
+    "pages/students/2016/mark_feher/README.md",
+    "pages/students/2016/maros_harahus/README.md",
+    "pages/students/2016/patrik_pavlisin/README.md",
+    "pages/students/2016/tomas_kucharik/README.md",
+    "pages/students/2017/dmytro_ushatenko/README.md",
+    "pages/students/2017/than_trung_thanh/README.md",
+    "pages/students/2018/NaN1name_buday/README.md",
+    "pages/students/2018/adrian_remias/README.md",
+    "pages/students/2018/david_ilas/README.md",
+    "pages/students/2018/david_omasta/README.md",
+    "pages/students/2018/martin_wencel/README.md",
+    "pages/students/2018/oleh_bilykh/README.md",
+    "pages/students/2018/ondrej_megela/README.md",
+    "pages/students/2019/alina_vitko/README.md",
+    "pages/students/2019/artem_yatsenko/README.md",
+    "pages/students/2019/david_stromp/README.md",
+    "pages/students/2019/dmytro_mural/README.md",
+    "pages/students/2019/filip_tomas/README.md",
+    "pages/students/2019/kristian_sopkovic/README.md",
+    "pages/students/2019/michal_stromko/README.md",
+    "pages/students/2019/patrik_pokrivcak/README.md",
+    "pages/students/2019/samuel_horani/README.md",
+    "pages/students/2020/david_kostilnik/README.md",
+    "pages/students/2020/jakub_kristian_lukas/README.md",
+    "pages/students/2020/kamil_tomcufcik/README.md",
+    "pages/students/2020/matej_kobylan/README.md",
+    "pages/students/2020/matus_suchanic/README.md",
+    "pages/students/2020/pavol_hudak/README.md",
+    "pages/students/2020/vladyslav_krupko/README.md",
+    "pages/students/2021/artur_hyrenko/README.md",
+    "pages/students/2021/eduard_matovka/README.md",
+    "pages/students/2021/manohar_gowdru_shridharu/README.md",
+    "pages/students/2021/martin_sarissky/README.md",
+    "pages/students/2021/matej_novotny/README.md",
+    "pages/students/2021/matej_scislak/README.md",
+    "pages/students/2021/nikita_bodnar/README.md",
+    "pages/students/2021/oleksandr_hryshchenko/README.md",
+    "pages/students/2021/vladimir_ferko/README.md",
+    "pages/students/2022/andrii_pervashov/README.md",
+    "pages/students/2022/daniil_huzenko/README.md",
+    "pages/students/2022/jakub_schwarc/README.md",
+    "pages/students/2022/jan_malinovsky/README.md",
+    "pages/students/2022/jan_ptak/README.md",
+    "pages/students/2022/oleh_poiasnik/README.md",
+    "pages/students/2022/samuel_vasil/README.md",
+    "pages/students/2022/serhii_yemets/README.md",
+    "pages/students/2022/tetiana_mohorian/README.md",
+    "pages/students/2022/valerii_kutsenko/README.md",
+    "pages/students/2022/vladyslav_yanchenko/README.md",
+    "pages/students/2022/yevhenii_leonov/README.md",
+    "pages/students/2023/denis_le_thanh/README.md",
+    "pages/students/2023/maksym_zatirka/README.md",
+    "pages/students/2023/oleksandr_dorybohov/README.md",
+    "pages/students/2023/ronald_zalacko/README.md",
+    "pages/students/2023/rostyslav_rodzhuk/README.md",
+    "pages/students/2023/samuel_dzurina/README.md",
+    "pages/students/2023/simona_bobrovcanova/README.md",
+    "pages/students/2023/yevhenii_medushivskyi/README.md"
+  ],
+  "test_profiles": [
+    "pages/students/2017/stanislav_matsunych/README.md",
+    "pages/students/2018/samuel_sirotnik/README.md"
+  ],
+  "source_profile_overlap": 0,
+  "limitations": [
+    "Only two entirely unseen profiles remain; supplementary holdout has limited category coverage.",
+    "The broader v2 independent TEST remains 67/80; v2 cases are now DEV, never fresh TEST."
+  ]
+}

tools/graphrag/evaluation/source_oracle.py

@@ -59,6 +59,7 @@ def category_type(category: str) -> tuple[str | None, int | None]:
 def academic_type(heading: str) -> str | None:
     text = norm(heading)
     for prefix, kind in [('bakalarska praca', 'bachelor_thesis'),
+                         ('bakalarsky projekt', 'bachelor_project'),
                          ('diplomova praca', 'diploma_thesis'),
                          ('dizertacna praca', 'doctoral_thesis'),
                          ('diplomovy projekt', 'diploma_project'),

tools/graphrag/graph_search.py

@@ -729,7 +729,7 @@ def explicit_keyword_phrase(query: str) -> str | None:
     if not match:
         return None
     phrase = re.split(
-        r"\s+(?:a\s+(?:jej|jeho|ich|rok|pomenuj|uved\w*)|ak\s+|uved\w*\s+|"
+        r"\s+(?:pri\s+osob\w*|u\s+osob\w*|a\s+(?:jej|jeho|ich|rok|pomenuj|uved\w*)|ak\s+|uved\w*\s+|"
         r"pomenuj\s+|maju\s+|ma\s+)",
         match.group(1),
         maxsplit=1,
@@ -814,6 +814,9 @@ def extract_query_constraints(
             "doctoral_thesis"
         )
 
+    if has_stem(tokens, 'bakalar') and has_project_word:
+        work_types.append('bachelor_project')
+
     if (
         has_stem(
             tokens,

tools/graphrag/graphrag.py

@@ -16,6 +16,7 @@ THESIS_TYPES = {
 }
 
 WORK_TYPE_LABELS = {
+    "bachelor_project": "Bakalársky projekt",
     "bachelor_thesis": "Bakalárska práca",
     "diploma_thesis": "Diplomová práca",
     "diploma_project": "Diplomový projekt",
@@ -25,6 +26,7 @@ WORK_TYPE_LABELS = {
 }
 
 WORK_HEADING_PATTERNS = (
+    ("bachelor_project", "bakalarsky projekt", None),
     ("bachelor_thesis", "bakalarska praca", "bp"),
     ("diploma_thesis", "diplomova praca", "dp"),
     ("diploma_project", "diplomovy projekt", None),

tools/graphrag/tests/test_graph_search_entity_v2.py

@@ -157,3 +157,16 @@ def test_document_subject_is_not_misread_as_the_named_document_author():
     assert gs.named_author_phrases(query) == []
     result = gs.search_graph_with_session(FakeSession({gs.CATALOG_QUERY: catalog()}), query)
     assert not result['constraints']['unresolved_document_author']
+
+
+def test_tag_before_named_people_is_not_extended_with_person_clause():
+    query = 'Over spoločné kľúčové slovo RAG pri osobách Alena Nová a Boris Malý.'
+    assert gs.explicit_keyword_phrase(query) == 'rag'
+    result = gs.search_graph_with_session(FakeSession({gs.CATALOG_QUERY: catalog()}), query)
+    assert not result['constraints']['unresolved_keyword']
+
+
+def test_explicit_bachelor_project_is_not_a_thesis_query():
+    constraints = gs.extract_query_constraints('Aký bakalársky projekt mal Peter Vzor v roku 2024?')
+    assert constraints['work_types'] == ['bachelor_project']
+    assert not constraints['thesis_types']

tools/graphrag/tests/test_graphrag_research_v2.py

@@ -9,3 +9,10 @@ def test_scientific_project_heading_has_research_project_semantics():
     assert len(works) == 1
     assert works[0]['work_type'] == 'research_project'
     assert works[0]['year'] == 2024
+
+
+def test_bachelor_project_is_separate_from_thesis_category_year():
+    works = extract_works(make_document(categories=['bp2025']), [make_chunk(
+        0, ['Bakalársky projekt 2024'],
+        'Sekcia: Bakalársky projekt 2024\nNázov: Overovanie automatických odpovedí\n')])
+    assert [(w['work_type'], w['year']) for w in works] == [('bachelor_project', 2024)]

tools/graphrag/tests/test_source_oracle_v2.py

@@ -93,3 +93,19 @@ def test_academic_range_uses_supported_category_end_year(tmp_path):
     person = load_source_oracle(Path(oracle['wiki_root']))['people']['Eva Nová']
     thesis = next(row for row in person['academics'] if row['academic_kind'] == 'Thesis')
     assert thesis['year'] == 2026
+
+
+def test_bachelor_project_title_does_not_become_next_year_thesis(tmp_path):
+    root = tmp_path / 'wiki'
+    path = root / 'pages/students/2020/peter/README.md'
+    path.parent.mkdir(parents=True)
+    path.write_text('''---
+title: Peter Vzor
+taxonomy:
+  category: [bp2025]
+---
+## Bakalársky projekt 2024
+Názov: Overovanie automatických odpovedí
+''')
+    records = load_source_oracle(root)['people']['Peter Vzor']['academics']
+    assert [(r['academic_kind'], r['academic_type'], r['year']) for r in records] == [('Work', 'bachelor_project', 2024)]