xnat-aws

Deploy XNAT on AWS using Terraform and Ansible.

Terraform is used to create the infrastructure on AWS and Ansible is then used to configure the instances for XNAT deployment.

Below is an overview of the infrastructure that will be created. See the notes on provisioning the infrastructure for a more detailed description.

Requirements

• An AWS account
• AWS CLI v2
• AWS credentials stored locally using aws configure (or aws configure sso)
• Terraform >= 0.15
• Ansible >= 2.15.0 - we recommend installing Ansible in a virtual environment, using either Conda or Virtualenv
• Other Python dependencies - these should be installed in the same virtual environment as Ansible. These dependencies are required if you would like to create a project and upload sample data to it.

Quick start

Once you have installed and set up the requirements, there are three steps to deploying XNAT on AWS:

1. Generate credentials. From the xnat-aws/credentials directory, type:

   terraform init
   terraform apply

   This will will create and SSH key and various passwords that will be used for accessing the AWS servers and configuring XNAT.

2. Set the necessary Terraform variables. From the xnat-aws/provision directory:

   First copy the terraform.tfvars.example file to terraform.tfvars:

   cp terraform.tfvars.example terraform.tfvars

   Then edit the terraform.tfvars file to change any variables as necessary. In particular, you may want to change the EC2 instance types.

3. Create the AWS instances. From the xnat-aws/provision directory, type:

   terraform init
   terraform apply

   This will create the infrastructure on AWS.

   Note: after the terraform apply steps have completed, you may need to wait 1-2 minutes before running the configuration. This is to allow time for the AWS instances to finish starting up.

4. Install XNAT. From the xnat-aws/configure directory, first type:

   python -m pip install -r requirements.txt

   This will install Ansible and other Python dependencies. Then, from the same directory, type:

   ./install_xnat.sh

   This will run several Ansible commands to install and configure XNAT.

   See Logging into the web server for notes on how to log into XNAT once it has been deployed.

5. [Optional] Create a sample project and upload data. From the xnat-aws/configure directory, type:

   ./setup_xnat_project.sh

   This will create a project on your XNAT server and upload data to it.

   The sample data can be used for running a workshop on implementing reproducible medical image analysis pipelines with XNAT.

Important!

Don't forget to destroy your infrastructure when you're finished! Otherwise you may end up with a large AWS bill at the end of the month.

To destroy the infrastructure, go to the xnat-aws/provision directory and type:

terraform destroy

AWS cost estimate

It is estimated the AWS resources will cost approximately $270 USD per month (assuming the workshop is run for a whole month) or about $68 USD per week.

This is assuming usage of 50 hours per week (10 hours per day times 5 days per week) with the following resources:

• web server: t3.large
• Container Service server: m4.xlarge
• database server: db.t3.medium
• up to 20 GB EFS storage
• AppStream instances for up to 10 users

The largest cost here are the AppStream instances, which cost about $124 USD per month. Without using AppStream, the cost is reduced to about $146 USD per month or about $37 USD per week.

Note, it is recommended that you set a monthly budget to avoid unexpected large bills.

Testing

To test the infrastructure setup with multiple XNAT users, go to the xnat-aws/configure directory and type:

./run_tests.sh <number of users>

This will create the specified number of users on the XNAT server, each with their own project and some example data. This can be useful to test whether the infrastructure can handle multiple users running various tasks simultaneously.

See Testing with multiple users for more details.

Acknowledgement

This work was funded by Health and Bioscience IDEAS, a project supported by the UKRI Innovation Scholars: Data Science Training in Health and Bioscience (MR/V03863X/1).