Stručnejšie výstupy nástrojov

This commit is contained in:
Ján Pták 2026-11-04 03:53:14 +01:00
parent 8335c69a4a
commit 23b97401c9
10 changed files with 91 additions and 24 deletions

View File

@ -1160,7 +1160,16 @@ upozornenie na neúplnosť, nezamieňať autora dokumentu s autorom práce a pri
sa nevysvetľujú odhadom; verzia dát neoznačuje verziu programu. sa nevysvetľujú odhadom; verzia dát neoznačuje verziu programu.
Tieto pokyny ovplyvňujú správanie Claude, jeho finálny text však náš backend Tieto pokyny ovplyvňujú správanie Claude, jeho finálny text však náš backend
nemôže programovo vynútiť. Po aktualizácii už nahratých Skills nahraď ich nemôže programovo vynútiť. Po aktualizácii už nahratých Skills nahraď ich
novými ZIPmi z `packages/claude/dist/` (verzia 1.5.0) a over správanie v novom chate. novými ZIPmi z `packages/claude/dist/` (verzia 1.5.1) a over správanie v novom chate.
Predvolený výstup Claude nástrojov obsahuje overenú odpoveď, citácie, identitu
otázky/nástroja, stav doručenia a verziu dát. Technické údaje o výbere kandidátov,
orezaní retrievalu, `answer_coverage`, jednotlivé claims a pomocné počty sa do
prezentačného výsledku nekopírujú. Plný validovaný kontrakt zostáva v `/answer`
a internom klientovi; diagnostika je dostupná cez `--mode retrieve` a `--health`.
Samotný text odpovede sa nemení: konkrétne chýbajúce údaje aj upozornenia pri
skutočne skrátenom výpise zostávajú zachované. Neprítomnosť technických polí
neznamená úplnosť vyhľadania.
### Nahratie Skills do webového Claude ### Nahratie Skills do webového Claude

View File

@ -231,6 +231,18 @@ def backend_health(config=None):
return {**error_result(str(exc)), 'ready': False} return {**error_result(str(exc)), 'ready': False}
def answer_for_presentation(result: dict) -> dict:
"""Keep approved prose/caveats intact; retrieval telemetry is not answer prose.
Called after query_backend validates the complete backend contract. Full
metadata remains available there, in /answer and in diagnostic retrieval.
Never change an incomplete result into a claim of completeness.
"""
fields = {'status', 'reason', 'query', 'engine', 'output_kind',
'accepted_for_delivery', 'answer', 'source_urls', 'index_version'}
return {key: value for key, value in result.items() if key in fields}
def main(engine=None): def main(engine=None):
parser = argparse.ArgumentParser(description=__doc__) parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('query', nargs='?') parser.add_argument('query', nargs='?')
@ -242,6 +254,8 @@ def main(engine=None):
args = parser.parse_args() args = parser.parse_args()
result = backend_health() if args.health else query_backend(args.query, engine or args.engine, result = backend_health() if args.health else query_backend(args.query, engine or args.engine,
mode=args.mode, limit=args.limit) mode=args.mode, limit=args.limit)
if not args.health and args.mode == 'answer':
result = answer_for_presentation(result)
print(json.dumps(result, ensure_ascii=False, separators=(',', ':'))) print(json.dumps(result, ensure_ascii=False, separators=(',', ':')))
return 1 if result.get('status') == 'error' else 0 return 1 if result.get('status') == 'error' else 0

View File

@ -3,12 +3,12 @@
{ {
"archive": "zpwiki-rag.zip", "archive": "zpwiki-rag.zip",
"name": "rag", "name": "rag",
"sha256": "8e5ab92284ee6f53d10e468cb4edee6cfd354ae869583673cbb6aea8ea9ca437" "sha256": "f439237795a770473bf5efd0760667f9ef75fa4882086133996d4a928a9d0252"
}, },
{ {
"archive": "zpwiki-graphrag.zip", "archive": "zpwiki-graphrag.zip",
"name": "graphrag", "name": "graphrag",
"sha256": "99a9ea36b439d6b14edc35c2d4d0c1c2cf97139f78e8fd73b8ce2347d82b1092" "sha256": "cd748dc0035bf932a72f6d178f986b18609728a3478fdf9a49ab6247a37a431d"
} }
], ],
"schema_version": 1 "schema_version": 1

Binary file not shown.

Binary file not shown.

View File

@ -34,15 +34,16 @@ keep all conditions belonging to each question together.
4. For `status=error` or a rejected answer, report unavailability without showing rejected 4. For `status=error` or a rejected answer, report unavailability without showing rejected
text. For `ambiguous`, ask for clarification. For `no_answer`, say no supporting text. For `ambiguous`, ask for clarification. For `no_answer`, say no supporting
relationship was found; do not substitute unrelated people or global results. relationship was found; do not substitute unrelated people or global results.
5. Respect `complete`, `truncated`, `truncation_reasons` and `index_version`. 5. The default verified-answer result contains presentation fields, not retrieval telemetry.
An incomplete or unknown-completeness listing is not exhaustive. An exact total is Preserve every specific limitation or missing-information notice already in `answer`.
permitted only when the aggregation itself is complete. Do not merge index versions. Do not append commentary about backend checks, coverage labels or retrieval limits.
Absence of technical metadata never establishes an exhaustive list, uniqueness or a total.
Use only the checked totals already stated in `answer`. Do not merge `index_version` values.
Presentation boundaries: Presentation boundaries:
- Use the current tool response for the current question. Do not append facts, records - Use the current tool response for the current question. Do not append facts, records
or explanations from previous chat answers. or explanations from previous chat answers.
- `complete` describes retrieval coverage, not universal truth or the existence of a person. - An incomplete listing can contain supported facts but cannot establish uniqueness or the
An incomplete listing can contain supported facts but cannot establish uniqueness or the
absence of other records. Preserve its caveat without dismissing it using earlier answers. absence of other records. Preserve its caveat without dismissing it using earlier answers.
- `document_author` identifies the source document's author. Do not infer thesis authorship, - `document_author` identifies the source document's author. Do not infer thesis authorship,
supervision, or a maintainer role from that metadata. supervision, or a maintainer role from that metadata.

View File

@ -11,7 +11,7 @@ from pathlib import Path
from urllib.parse import urlsplit from urllib.parse import urlsplit
sys.path.insert(0, str(Path(__file__).resolve().parent)) sys.path.insert(0, str(Path(__file__).resolve().parent))
from client import Config, backend_health, query_backend from client import Config, backend_health, query_backend, answer_for_presentation
from mcp.server.fastmcp import FastMCP from mcp.server.fastmcp import FastMCP
from mcp.server.transport_security import TransportSecuritySettings from mcp.server.transport_security import TransportSecuritySettings
from mcp.types import CallToolResult, TextContent, ToolAnnotations from mcp.types import CallToolResult, TextContent, ToolAnnotations
@ -38,11 +38,10 @@ ANSWER_PRESENTATION_POLICY = (
'Do not add factual claims, guesses or commentary from earlier chat answers. ' 'Do not add factual claims, guesses or commentary from earlier chat answers. '
'document_author means author of the source document; it does not establish thesis authorship, ' 'document_author means author of the source document; it does not establish thesis authorship, '
'supervision or a maintainer role. Do not guess those roles. ' 'supervision or a maintainer role. Do not guess those roles. '
'complete and truncated describe retrieval coverage, not the truth of each returned fact. ' 'The answer already includes the specific missing-information and completeness notices checked by the backend. '
'For RAG, answer_coverage separately describes the delivered answer scope. supported means ' 'Do not append a paragraph about backend validation, partial coverage, candidate/result limits or retrieval diagnostics. '
'evidence-backed, not exhaustive. Do not append a generic incompleteness warning merely ' 'Do not infer that results are exhaustive from the absence of technical metadata. '
'because retrieval selected top candidates/results. Preserve the specific notices already ' 'Preserve every specific notice already included in answer for limited lists or clipped source text. '
'included in answer for limited lists or clipped source text. '
'An incomplete listing can contain supported facts, but cannot establish uniqueness or absence ' 'An incomplete listing can contain supported facts, but cannot establish uniqueness or absence '
'of other records. Retain its caveat; do not dismiss it using earlier answers. ' 'of other records. Retain its caveat; do not dismiss it using earlier answers. '
'For no_answer, return only the supplied absence-of-evidence answer and any supplied citations. ' 'For no_answer, return only the supplied absence-of-evidence answer and any supplied citations. '
@ -66,7 +65,7 @@ async def zpwiki_rag(query: str, limit: int = 5) -> CallToolResult:
Present the accepted answer with its citations; add no guesses or history commentary. Present the accepted answer with its citations; add no guesses or history commentary.
""" """
return tool_result(await asyncio.to_thread(query_backend, query, 'rag', limit=limit)) return tool_result(answer_for_presentation(await asyncio.to_thread(query_backend, query, 'rag', limit=limit)))
async def zpwiki_graphrag(query: str, limit: int = 5) -> CallToolResult: async def zpwiki_graphrag(query: str, limit: int = 5) -> CallToolResult:
@ -74,7 +73,7 @@ async def zpwiki_graphrag(query: str, limit: int = 5) -> CallToolResult:
Present the accepted answer with its citations; no_answer means no supporting evidence. Present the accepted answer with its citations; no_answer means no supporting evidence.
""" """
return tool_result(await asyncio.to_thread(query_backend, query, 'graphrag', limit=limit)) return tool_result(answer_for_presentation(await asyncio.to_thread(query_backend, query, 'graphrag', limit=limit)))
async def zpwiki_health() -> CallToolResult: async def zpwiki_health() -> CallToolResult:

View File

@ -36,13 +36,12 @@ keep all conditions belonging to each question together.
4. For `status=error` or rejected delivery, explain that verified output is unavailable; 4. For `status=error` or rejected delivery, explain that verified output is unavailable;
never reproduce rejected text or invent a replacement. For `ambiguous`, ask the user never reproduce rejected text or invent a replacement. For `ambiguous`, ask the user
to clarify. For `no_answer`, report the absence of evidence without claiming the fact is false. to clarify. For `no_answer`, report the absence of evidence without claiming the fact is false.
5. Respect `complete`, `truncated`, `truncation_reasons` and `index_version`. 5. The default verified-answer result contains presentation fields, not retrieval telemetry.
These describe retrieval, not whether the selected answer is supported. `answer_coverage` Missing aspects and actual limitations are already stated specifically in `answer`.
separately describes the answer scope: `supported` means evidence-backed, never exhaustive. Preserve those notices unchanged. Do not append commentary about backend checks,
Present the backend's specific notices unchanged. Do not append a generic incompleteness `partial`, candidate/result limits, excerpt limits or general incompleteness.
warning to a supported fact or short description just because retrieval selected top results. Absence of technical metadata never establishes an exhaustive list or uniqueness.
Never describe an incomplete or unknown-completeness result as an exhaustive list. Do not combine different `index_version` values as if they represented one snapshot.
Do not combine different index versions as if they represented one snapshot.
Presentation boundaries: Presentation boundaries:
- Use the current tool response for the current question. Do not append factual material - Use the current tool response for the current question. Do not append factual material

View File

@ -11,7 +11,7 @@ import zipfile
ROOT = Path(__file__).resolve().parents[1] ROOT = Path(__file__).resolve().parents[1]
BASE = ROOT / 'packages/claude' BASE = ROOT / 'packages/claude'
ENGINES = ('rag', 'graphrag') ENGINES = ('rag', 'graphrag')
PACKAGE_VERSION = '1.5.0' PACKAGE_VERSION = '1.5.1'
def json_bytes(value): def json_bytes(value):

View File

@ -141,6 +141,37 @@ def test_answer_coverage_is_forwarded_without_overwriting_retrieval_limits(backe
assert 'presentation_instructions' not in result assert 'presentation_instructions' not in result
@pytest.mark.parametrize('engine', ['rag', 'graphrag'])
@pytest.mark.parametrize('text', [
'V podkladoch chýbajú údaje o riešených problémoch.',
'Doložené ciele práce. Časť podkladov k zadaniu bola skrátená; uvedené body nemusia tvoriť celé zadanie.',
'Doložené práce. Výpis je neúplný. Výpis môžeš zúžiť uvedením osoby alebo roku.',
])
def test_presentation_preserves_specific_caveats_without_retrieval_telemetry(backend, engine, text):
backend['checked'].update(answer=text, answer_coverage={
'scope':'fact', 'status':'partial', 'warnings':['partial_requested_aspects'],
'limitations':['candidate_limit','result_limit','per_document_limit']})
raw=client.query_backend('Práca?',engine)
original=copy.deepcopy(raw)
shown=client.answer_for_presentation(raw)
assert shown['answer']==text and shown['source_urls']==raw['source_urls']
assert shown['query']==raw['query'] and shown['engine']==engine
assert shown['index_version']==raw['index_version'] and shown['accepted_for_delivery'] is True
hidden={'complete','truncated','truncation_reasons','answer_coverage','claims','count'}
assert not hidden.intersection(shown)
assert not any(token in json.dumps(shown) for token in ['candidate_limit','per_document_limit','partial_requested_aspects'])
# The full API/client contract still reports the actual incomplete retrieval.
assert raw==original and raw['complete'] is False and raw['truncated'] is True
assert raw['answer_coverage']['status']=='partial'
def test_presentation_cannot_turn_a_rejected_response_into_an_approved_answer(backend):
backend['checked'].update(status='error', answer='PRIVATE REJECTED ANSWER')
shown=client.answer_for_presentation(client.query_backend('Práca?','rag'))
assert shown['status']=='error' and shown['accepted_for_delivery'] is False
assert 'PRIVATE' not in json.dumps(shown)
@pytest.mark.parametrize('coverage', [None, [], {'status': 'complete'}, @pytest.mark.parametrize('coverage', [None, [], {'status': 'complete'},
{'scope': 'fact', 'status': 'supported', 'warnings': 'none', 'limitations': []}, {'scope': 'fact', 'status': 'supported', 'warnings': 'none', 'limitations': []},
{'scope': [], 'status': 'supported', 'warnings': [], 'limitations': []}, {'scope': [], 'status': 'supported', 'warnings': [], 'limitations': []},
@ -244,6 +275,13 @@ def test_zip_is_reproducible_and_runs_without_repository(tmp_path, backend, engi
assert result.returncode == 0, result.stderr assert result.returncode == 0, result.stderr
assert json.loads(result.stdout)['answer'] == backend['checked']['answer'] assert json.loads(result.stdout)['answer'] == backend['checked']['answer']
assert 'presentation_instructions' not in json.loads(result.stdout) assert 'presentation_instructions' not in json.loads(result.stdout)
assert not {'complete','truncated','truncation_reasons','answer_coverage','claims'}.intersection(json.loads(result.stdout))
diagnostic=subprocess.run([*command,'--mode','retrieve'],cwd=folder,env=environment,
capture_output=True,text=True,timeout=15)
assert diagnostic.returncode==0,diagnostic.stderr
raw=json.loads(diagnostic.stdout)
assert raw['output_kind']=='retrieval_context' and raw['requires_final_answer_verification'] is True
assert raw['complete'] is False and raw['truncated'] is True
def test_committed_archives_match_current_sources(tmp_path): def test_committed_archives_match_current_sources(tmp_path):
@ -259,6 +297,8 @@ def test_mcp_stdio_initialization_discovery_call_and_rejection(backend):
from mcp import ClientSession, StdioServerParameters from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client from mcp.client.stdio import stdio_client
from packages.claude.mcp_server import ANSWER_PRESENTATION_POLICY from packages.claude.mcp_server import ANSWER_PRESENTATION_POLICY
backend['checked']['answer_coverage']={'scope':'fact','status':'partial',
'warnings':['partial_requested_aspects'],'limitations':['candidate_limit','per_document_limit']}
async def run(): async def run():
params = StdioServerParameters(command=sys.executable, params = StdioServerParameters(command=sys.executable,
@ -275,7 +315,12 @@ def test_mcp_stdio_initialization_discovery_call_and_rejection(backend):
result = await session.call_tool('zpwiki_' + engine, {'query': 'Práca Evy Novej?'}) result = await session.call_tool('zpwiki_' + engine, {'query': 'Práca Evy Novej?'})
assert not result.isError assert not result.isError
assert result.structuredContent['answer'] == backend['checked']['answer'] assert result.structuredContent['answer'] == backend['checked']['answer']
assert not {'complete','truncated','truncation_reasons','answer_coverage','claims'}.intersection(result.structuredContent)
assert 'presentation_instructions' not in result.structuredContent assert 'presentation_instructions' not in result.structuredContent
assert all(json.loads(block.text)==result.structuredContent
for block in result.content if block.type=='text')
assert 'candidate_limit' not in json.dumps(result.model_dump())
assert 'partial_requested_aspects' not in json.dumps(result.model_dump())
assert all('presentation_instructions' not in json.loads(block.text) assert all('presentation_instructions' not in json.loads(block.text)
for block in result.content if block.type == 'text') for block in result.content if block.type == 'text')
rejected = await session.call_tool('zpwiki_rag', {'query': ''}) rejected = await session.call_tool('zpwiki_rag', {'query': ''})