Skip to content

Home

Jump to bottom Edit New page
Razvan Deaconescu edited this page Mar 1, 2014 · 12 revisions

vmchecker is a grading platform built by students from University POLITEHNICA of Bucharest. vmchecker requires two systems to run:

  • A storer system: where assignments are uploaded, stored, submitted for testing and graded by teachers.
  • A tester system: where assignments are tested/checked using a virtual machine infrastructure.

Contributing to vmchecker

We welcome contributors. If you know a bit of Python and/or JavaScript and a bit on virtualization, please join the mailing list by visiting the mailman page for discussion on vmchecker.

To make contributions fork the vmchecker repository on GitHub.

Make your changes and check Python source code using pylint then push your code. You may request code review on the mailing list or create a pull request. Wait for your code to be pulled into the main repository and a thanks message from the maintainers. Grab yourself a beer or a cup of coffee and celebrate.

We recommend using our virtual machine in order to test your code before submitting a pull request.

Behind the Scenes

vmchecker Folder Structure

  • bin: A directory that should be contained by the storer machine. It stores the following Python scripts :
    • vmchecker-download-external-files: A script to download files (that are not stored on the storer) required for testing from another system through SSH.
    • vmchecker-init-course storer: Initializes the directory path for a class storer machine. Its actions are:
      • Creates a sample config file named config in the current directory. It gets the sample config from examples/config-template.
    • Creates the repo for the assignments on the storer.
    • Create the implicit db if it does not exist.
    • Copy the example auth file from examples/auth_file.json.
    • Copy build and run example scripts into the root path
    • Run initialization tasks for the tester machine.
    • Run initialization tasks for the storer machine.
    • vmchecker-init-course tester: Initializes the directory path for a course tester machine.
    • vmchecker-queue-manager: Waits for assignments to appear and invoke vmchecker-vm-executor to handle them.
    • vmchecker-submit: Submit assignments from the command line.
    • vmchecker-update-db: Manually update the db storing info from the repo.
    • vmchecker-view-grades: Generates a HTML table containing student grades.
    • vmchecker-vm-executor: Starts a vm, reverts it to a known snapshot, tests a submission bundle (submission + tests), and closes the vm.

The bin folder must also reside on the tester machine, where vmchecker-queue-manager runs as a deamon. For each submission vmchecker-vm-executor is started.

Authentication

The vmchecker WebUI authentication is achieved through the university LDAP server. If the user isn't on the LDAP server, then the user is searched in a plain text file, where the user names and the passwords are saved.

Configuring a New Class User on a Storer System

In order to add a new class on vmchecker, you need to add a corresponding user on the tester system and configure vmchecker for that user. You may then configure assignments, request students to submit assignments and then grade assignments.

We'll use pc (for "Programarea Calculatoarelor") as the username to describe the steps to follow. The steps are show below. We assume the tester system is elf.cs.pub.ro.

  1. First you'll need to create the actual user:
   $ ssh root@elf.cs.pub.ro
   elf:~# adduser pc # don't set a password; use ssh keys
   elf:~# su - pc
   pc@elf:~$ ssh pc@localhost # creates .ssh directory with default permissions
   pc@elf:~$ emacs .ssh/authorized_keys # put your id_rsa.pub key here and log-out
  1. Create and initialize the vmchecker class folder:
   $ ssh pc@elf.cs.pub.ro
   pc@elf:~$ mkdir vmchecker-storer
   pc@elf:~$ cd vmchecker-storer/
   pc@elf:~/vmchecker-storer$ vmchecker-init-course storer
  1. Create ssh key to connect to tester system:
   pc@elf:~/vmchecker-storer$ ssh-keygen -f ~/.ssh/id_rsa-vmchecker -N '' # creates a key with no password
  1. Add the SSH key to the corresponding user on the tester system (we'll assume it is calleed checker.cs.pub.ro; you will use whatever you have configured):
   youraccount@yourhostname:~$ ssh pc@checker.cs.pub.ro
   pc@sanctuary:~$ emacs .ssh/authorized_keys
  1. Test connection and also save the fingerprint to .ssh/known_hosts:
   pc@elf:~$ ssh -i .ssh/id_rsa-vmchecker pc@checker.cs.pub.ro
   The authenticity of host 'checker.cs.pub.ro (141.85.227.120)' can't be established.
   RSA key fingerprint is 8a:0a:51:86:46:85:17:85:b6:84:3d:e1:b5:68:45:01.
   Are you sure you want to continue connecting (yes/no)? yes
  1. Add pc@checker.cs.pub.ro's public key in the ~.ssh/authorized_keys file:
   pc@elf:~$ emacs .ssh/authorized_keys
   
   # paste-in pc@checker.cs.pub.ro's password
  1. Make private key readable by www-data (grants www-data read access to the file):
   pc@elf:~/.ssh$ setfacl -m u:www-data:r id_rsa-vmchecker
  1. Edit vmchecker config file:
   pc@elf:~/vmchecker-storer$ emacs config

You'll find a sample config file in the repository.

  1. Add tests to vmchecker-storer/tests/<asg-name>.zip files. <asg-name> is the name of each assignment as specified in the config file.

  2. Add new class to global list:

   $ ssh root@elf.cs.pub.ro
   elf:~# emacs /etc/vmchecker/config.list
   Add this to the file:
   PC:/home/pc/vmchecker-storer/config

Class Folder

The class folder is generally stored in /home/class_name/vmchecker-storer (where class_name is the name of the class: so, or so2 or programare) and will look something like this:

/home/class_name/vmchecker-storer
|--tests
    |--asg1_tests.zip
|--build.sh
|--run.sh
|--config
|--auth_file.json
|--vmchecker.db

auth_file.json is a plain text file that contains the user name and password for the users that are not in the ldap server.

vmchecker.db is an SQLite3 database that contains the paths to the assignment archives that will be graded.

Adding a New Assignment

In order to add a new assignment the user must:

  • Add the assignment in the config file.
  • Upload the assignment tests.
  • Upload the run script.
  • Upload the build script

Assignment Repository

Each class specific folder (vmchecker-storer/) resides on the storer system contains a repository where assignments are saved. The repository follows a hierarchical structure consisting, on each level, of:

  • 1st level: assignment name
  • 2nd level: student name that submitted assignment for the 1st level
  • 3rd level: submission id of submission by student for the given assignment
  • 4th level: three items
    • archive.zip: a compressed archive of the submission
    • archive/: a folder representing the uncompressed submission
    • results/: a folder that stores assignment test results and grading

The results/ folder is created after the homework is sent back from the tester machine after being tested/checker. It stores the entries below:

  • submission-config: contains assignment details.
  • grade.vmr: contains the grades given by the teacher
  • run-stdout.vmr: contains the results of the automatic tests.
  • build-stdout.vmr: contains the results of the build.

Uploading an Assignment File

In order to upload an assignment a user must log in the WebUI, browse for the assignment archive and click submit.

On the click of the submit button a new directory is created in the course name with the user name and the assignment archive, if it does not exist.

Resubmitting an Assignment from CLI

It may happen that due to an error on the tester system side, an assignment needs to be resubmitted; to preserver timestamp and other. You may use the submit.py script for this, while located in the class folder. A sample run is:

./bin/submit.py 1-minishell-linux Alexandru\ Moșoi path_to_some_archive.zip
Add a custom footer
Add a custom sidebar
Clone this wiki locally