Skip to content

OpenBench is a Distributed SPRT Testing Framework for Chess Engines

License

Notifications You must be signed in to change notification settings

lynx-chess/OpenBench

 
 

Repository files navigation

OpenBench

Fork from AndyGrant/OpenBench with the purpose of testing Lynx chess engine.

Run

Using docker compose.

docker compose up -d --build --force-recreate
# or
sudo docker compose -f docker-compose.prod.yaml up -d --build && sudo chown -R $(sudo docker compose exec web id -u) ./db && sudo chown -R $(sudo docker compose exec web id -u) ./media && sudo chown -R $(sudo docker compose exec web id -u) ./OpenBench/migrations
# Includes workaround of changing owner of media/ so that the app can save files there

# To stop it:
docker compose down -v --remove-orphans --rmi local
#or
sudo docker compose -f docker-compose.prod.yaml down --remove-orphans --rmi local

PS: don't forget to create an .env or .env.prod file or manually set the environment variables for those values where os.environ.get() is used in settings.py, i.e:

DJANGO_SECRET_KEY='<your_secret>'
DJANGO_DEBUG=1
DJANGO_SECURE_SSL_REDIRECT=0
DJANGO_SESSION_COOKIE_SECURE=1
DJANGO_CSRF_COOKIE_SECURE=1

First time setup

Entering the container created using docker compose up.

docker exec -it <container-id> bash
python3 manage.py migrate
python3 manage.py makemigrations
python3 manage.py migrate --run-syncdb

Update

Quoting Andrew

Backup

python3 manage.py dumpdata > backup.datetime.json

Update to latest

git pull
python3 manage.py makemigrations
python3 manage.py migrate

Removing old logs/events

Info from here

you can start by deleting any error/event text file in Media/, then you can do this:

$ python3 manage.py shell
\>>> from OpenBench.models import LogEvent
\>>> LogEvent.objects.all().delete()

If you are using an sqlite3 database by default, and the size does not change, I'll provide more steps.

Please make a copy of your databse before you do this

$ python3 manage.py shell

>>> from OpenBench.models import LogEvent
>>> LogEvent.objects.all().delete()
>>> from django.db import connection
>>> with connection.cursor() as cursor:
>>>    cursor.execute("VACUUM;")

Test/experiment

Using python docker image.

⚠ db/ directory is excluded by default, to avoid messing up with local db, comment if you want to test stuff and persist the db, but probably first edit DB name in settings in settings.py.

# Starts a container, mounts the whole dir as volume (excluding db/) and exposes OB server in localhost:8001
.\run.sh

# To be run inside of the container
pip install -r requirements.txt
python manage.py check --deploy

python3 manage.py migrate
python3 manage.py makemigrations
python3 manage.py migrate --run-syncdb

python3 manage.py runserver 0.0.0.0:8000

python3 manage.py createsuperuser

A few useful/interesting links:


OpenBench

OpenBench is an open-source Chess Engine Testing Framework for UCI engines. OpenBench provides a lightweight interface and client to facilitate running fixed-game tests as well as SPRT tests to benchmark changes to engines for performance and stability. OpenBench supports Fischer Random Chess.

OpenBench is the primary testing framework used for the development of Ethereal. The primary instance of OpenBench can be found at http://chess.grantnet.us. The Primary instance of OpenBench supports development for Berserk, Bit-Genie, BlackMarlin, Demolito, Drofa, Ethereal, FabChess, Halogen, Igel, Koivisto, Laser, RubiChess, Seer, Stash, Weiss, Winter, and Zahak. A dozen or more engines are using their own private, local instances of OpenBench.

You can join OpenBench's Discord server to join the discussion, see what developers are working on and talking about, or to find out how you can contribute to the project and become a part of it. OpenBench is heavily inspired by Fishtest. The project is powered by the Django Web Framework and Cutechess.

Documentation for OpenBench is available in the Wiki

About

OpenBench is a Distributed SPRT Testing Framework for Chess Engines

Resources

License

Stars

Watchers

Forks

Languages

  • Python 71.8%
  • HTML 18.8%
  • CSS 5.3%
  • JavaScript 2.8%
  • Shell 0.9%
  • Dockerfile 0.3%
  • PowerShell 0.1%