|
…
|
||
|---|---|---|
| .. | ||
| README.md | ||
| title | published | taxonomy | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Oliver Pejic | true |
|
Oliver Pejic
IAESTE Intern Summer 2024, 12 weeks in August, September and October.
Goal:
- Help with the Hate Speech Project
- Help with evaluation of sentence transformer models using toolkit MTEB
Final Tasks:
- Prepare an MTEB evaluation task for Slovak HATE speech.
- Prepare an MTEB evaluation task for Slovak question answering.
- Machine translate an SBERT evaluation set for multiple slavic languages.
- Write a short scientific paper with results.
Meeting 3.10.:
State:
- Prepared a pull request for Retrieval SK Quad.
- Prepared a pull request for Hate Speech Slovak.
Tasks:
- Make the pull request compatible with the MTEB Contribution guidelines. Discuss it when it is done.
- Submit pull requests to MTEB project.
- Machine Translate a database (HotpotQA, DB Pedia, FEVER) . Pick a database that is short, because translation might be slow.
Non priority tasks:
- Prepare databse and subnit it to HuggingFace Hub.
- Prepare a MTEB PR for the databse.
Meeting 3.9:
State: Studied MTEB framework and transformers.
Tasks:
- Prepare and try MTEB evaluation tasks for the database. For evaluation you can try me5-base model.
- Make a fork of MTEB and do necessary modification, including the documentation references for the task.
- Prepare 2 GITHUB pull requests for the databases, preliminary BEIR script given.
Future tasks:
- Prepare a machine translation system to create another slovak/multilingual evaluation task from English task.
Preparation (7.8.2024):
- Get familiar with SentenceTransformer framework, study fundamental papers and write down notes.
- Get familiar with MTEB evaluation framework.
- Prepare a working environment on Google Colab or on school server or Anaconda.
- Get familiar with existing finetuning scripts.