tk634rv/dp2022
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

added readme file with instructions

Browse Source
This commit is contained in:
Tomas Kucharik 2022-02-21 11:15:32 +01:00
parent a883488ffa
commit adb8f32fb4
1 changed files with 27 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
README.md Normal file
View File

@ -0,0 +1,27 @@
# Tvorba korpusu otázok a odpovedí vo viacerých jazykoch pomocou strojového prekladu
1. Vypracujte prehľad jazykových mutácii overovacej množiny SQUAD a opíšte spôsob ich tvorby.
2. Vypracujte prehľad aktuálnych systémov pre generovanie odpovede na otázku v prirodzenom jazyku.
3. Navrhnite a vykonajte postup pre vytvorenie korpusu otázok a odpovedí v inom jazyku pomocou strojového prekladu z anglického jazyka,
4. Natrénujte systém pre generovanie odpovedí na viacerých jazykových verziách SQUAD a porovnajte ich presnosť.
## Prerequisites
1. Download and unpack `google-cloud-sdk` from [here](https://cloud.google.com/sdk/docs/install).
2. Create account, project, service account and keys in google cloud following [this documentation](https://cloud.google.com/translate/docs/setup).
3. Create a file named `google_api_key.json` in root directory and copy the contents of the downloaded service account keys file inside.
4. Create a new conda environment and install required packages with `pip install -r requirements.txt`
## Functionality
### `squad_transform.py`
1. Takes `squad-v2-dev-small.json` and adds special characters around every answer in context and calculates new indexes of the answer positions.
2. Translates every context and every question and puts the translated sentences in new fields.
4. Saves the intermediate file as `squad-v2-dev-small-transformed.py`
5. Removes the special characters from contexts and recalculates the indexes of the answer positions.
6. Saves the final file as `squad-v2-dev-small-translated.py`
### `squad_utils.py`
Utility functions for working with squad files
### `translate_utils.py`
Utility fuctions for working with Google's Translate API