ScandiQA

Scandinavian question-answering models and datasets.

Developers:

• Dan Saattrup Nielsen (dan.nielsen@alexandra.dk)

---

Setup

Set up the environment

1. Run make install, which installs Poetry (if it isn't already installed), sets up a virtual environment and all Python dependencies therein.
2. Run source .venv/bin/activate to activate the virtual environment.

Install new packages

To install new PyPI packages, run:

poetry add <package-name>

Auto-generate API documentation

To auto-generate API document for your project, run:

make docs

To view the documentation, run:

make view-docs

Tools used in this project

• Poetry: Dependency management
• hydra: Manage configuration files
• pre-commit plugins: Automate code reviewing formatting
• pdoc: Automatically create an API documentation for your project

Project structure

.
├── .flake8
├── .gitignore
├── .pre-commit-config.yaml
├── LICENSE
├── README.md
├── config
├── data
│   ├── final
│   ├── processed
│   └── raw
├── makefile
├── models
├── notebooks
│   └── data_eda.ipynb
├── poetry.toml
├── pyproject.toml
├── scripts
│   └── build_dataset.py
├── src
│   ├── scandi_qa
│   │   ├── __init__.py
│   │   ├── answer_extraction.py
│   │   ├── builder.py
│   │   ├── cleaning.py
│   │   ├── embedder.py
│   │   ├── merger.py
│   │   ├── translation.py
│   │   └── utils.py
│   └── scripts
│       ├── fix_dot_env_file.py
│       └── versioning.py
└── tests
    └── __init__.py