KEMT/zpwiki
31
1
Fork 9
Code Issues Pull Requests Releases Wiki Activity

Update 'pages/topics/question/README.md'

Browse Source
This commit is contained in:
dano 2021-09-03 06:52:26 +00:00
parent 89ef614a6b
commit 0868ef694e
1 changed files with 58 additions and 44 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

102
pages/topics/question/README.md
View File

@ -11,6 +11,7 @@ taxonomy:
- [Project repository](https://git.kemt.fei.tuke.sk/dano/annotation) (private) - [Project repository](https://git.kemt.fei.tuke.sk/dano/annotation) (private)
- [Annotation Manual for question annotation](navod) - [Annotation Manual for question annotation](navod)
- [Annotation Manual for validations](validacie) - [Annotation Manual for validations](validacie)
- [Annotation Manual for unanswerable questions](nezodpovedatelne)
- [Summary database application](https://app.question.tukekemt,xyz) - [Summary database application](https://app.question.tukekemt,xyz)
@ -63,6 +64,19 @@ Notes:
- [167 good articles](https://sk.wikipedia.org/wiki/Wikip%C3%A9dia:Zoznam_dobr%C3%BDch_%C4%8Dl%C3%A1nkov) - [167 good articles](https://sk.wikipedia.org/wiki/Wikip%C3%A9dia:Zoznam_dobr%C3%BDch_%C4%8Dl%C3%A1nkov)
- [Wiki Facts](https://sk.wikipedia.org/wiki/Wikip%C3%A9dia:Zauj%C3%ADmavosti) - [Wiki Facts](https://sk.wikipedia.org/wiki/Wikip%C3%A9dia:Zauj%C3%ADmavosti)
## Finished Tasks
### Annotation Manual
Output: Recommendations for annotators
Done:
- Web Page for annotators (Daniel Hládek)
- Modivation video (Daniel Hládek)
- Video with instructions (Daniel Hládek)
bn application?
### Question Annotation ### Question Annotation
An annotation recipe for Prodigy An annotation recipe for Prodigy
@ -79,15 +93,6 @@ Done:
- answer annotation together with question (Daniel Hládek) - answer annotation together with question (Daniel Hládek)
- prepare final input paragraphs (dataset) - prepare final input paragraphs (dataset)
In progress:
- More annotations (volunteers and workers).
To be done:
- Prepare development set
### Annotation Web Application ### Annotation Web Application
Annotation work summary, web applicatiobn Annotation work summary, web applicatiobn
@ -104,11 +109,6 @@ Done:
- application deployment (Daniel Hládek) - application deployment (Daniel Hládek)
- extract annotations from question annotation in squad format (Daniel Hladek) - extract annotations from question annotation in squad format (Daniel Hladek)
To be done:
- review of validations
### Annotation Validation ### Annotation Validation
Input: annnotated questions and paragraph Input: annnotated questions and paragraph
@ -120,60 +120,53 @@ Done:
- Recipe for validations (binary annotation for paragraphs, question and answers, text fields for correction of question and answer). (Daniel Hládek) - Recipe for validations (binary annotation for paragraphs, question and answers, text fields for correction of question and answer). (Daniel Hládek)
- Deployment - Deployment
To be done:
- Prepare for production ## Tasks in progress
### Annotation Manual ### Unanswerable question annotation
Output: Recommendations for annotators Input: validated questions and answers
Output: Unanswerable questions and answers
Done: Done:
- Web Page for annotators (Daniel Hládek) - Annotation manual
- Modivation video (Daniel Hládek) - Annotation interface
- Video with instructions (Daniel Hládek) - Database schema modifications
- Modification of the database application
- Export of validations
In progress: In progress:
- Should be instructions a part of the annotation webn application? - Annotaion process optimization
### Question Answering Model ### Final Data Export
Training the model with annotated data Input: Validations and unanswerable questions
Input: An annotated QA database Output: Final database in SQUAD format
Output: An evaluated model for QA Done:
- Preliminary export script
To be done: To be done:
- Selecting existing modelling approach - Final export script
- Evaluation set selection - Database web visualization
- Model evaluation - Prepare development set
- Supporting the annotation with the model (pre-selecting answers)
In progress: ## Resources
- Preliminary model (Ján Staš and Matej Čarňanský) ### Bibligraphy
## Existing implementations
- https://github.com/facebookresearch/DrQA
- https://github.com/brmson/yodaqa
- https://github.com/5hirish/adam_qas
- https://github.com/WDAqua/Qanary - metodológia a implementácia QA
## Bibligraphy
- Reading Wikipedia to Answer Open-Domain Questions, Danqi Chen, Adam Fisch, Jason Weston, Antoine Bordes - Reading Wikipedia to Answer Open-Domain Questions, Danqi Chen, Adam Fisch, Jason Weston, Antoine Bordes
Facebook Research Facebook Research
- SQuAD: 100,000+ Questions for Machine Comprehension of Text https://arxiv.org/abs/1606.05250 - SQuAD: 100,000+ Questions for Machine Comprehension of Text https://arxiv.org/abs/1606.05250
- [WDaqua](https://wdaqua.eu/our-work/) publications - [WDaqua](https://wdaqua.eu/our-work/) publications
## Existing Datasets ### Existing Datasets
- [Squad](https://rajpurkar.github.io/SQuAD-explorer/) The Stanford Question Answering Dataset(SQuAD) (Rajpurkar et al., 2016) - [Squad](https://rajpurkar.github.io/SQuAD-explorer/) The Stanford Question Answering Dataset(SQuAD) (Rajpurkar et al., 2016)
- [WebQuestions](https://github.com/brmson/dataset-factoid-webquestions) - [WebQuestions](https://github.com/brmson/dataset-factoid-webquestions)
@ -210,3 +203,24 @@ Output:
- a trained model - a trained model
- evaluation of the model (if possible) - evaluation of the model (if possible)
### Question Answering Model
Training the model with annotated data
Input: An annotated QA database
Output: An evaluated model for QA
To be done:
- Selecting existing modelling approach
- Evaluation set selection
- Model evaluation
- Supporting the annotation with the model (pre-selecting answers)
In progress:
- Preliminary model (Ján Staš and Matej Čarňanský)