xenon-adaptors-cloud

Cloud related adaptors for xenon library.

Implemented adaptors:

• s3, Xenon filesystem adaptor for Amazon S3 blob store
• azureblob, Xenon filesystem adaptor for Azure blob store
• awsbatch, Xenon scheduler adaptor for Amazon Batch service

See adaptor documentation for what each adaptor offers and requires.

Usage

The library can be added as a Gradle dependency to your own project with

repositories {
    // ... others
    jcenter()
}
dependencies {
    // ... others
    implementation group: 'nl.esciencecenter.xenon.adaptors', name: 'xenon-adaptors-cloud', version: '3.0.2'
}

Testing

Testing of Xenon adaptor is described in the Xenon testing document.

Additional integration test requirements:

• moto_server, a stand-alone mocked AWS infrastructure server. The moto_server executable should be in PATH or location set in MOTO_SERVER environment variable.

To run live test on AWS S3, first create a bucket containing the test fixtures listed in create_symlinks script and then run live test command:

./gradlew liveTest -Dxenon.filesystem=s3 \
 -Dxenon.filesystem.location=https://s3.<region>.amazonaws.com/<bucket> \
 -Dxenon.username=<access key> -Dxenon.password=<secret key> \
 -Dxenon.filesystem.basedir=/ -Dxenon.filesystem.expected.workdir=/

To run the live tests on Microsoft Azure, first create a container containing the test fixtures mentioned above and run the following test command:

./gradlew liveTest -Dxenon.filesystem=azure \
 -Dxenon.filesystem.location=https://<user>.blob.core.windows.net/<container> \
 -Dxenon.username=<access key> -Dxenon.password=<secret key> \
 -Dxenon.filesystem.basedir=/ -Dxenon.filesystem.expected.workdir=/

New release

Chapter is for xenon developers.

The major version should be the same as the used xenon library.

1. If new adaptor is added,
   1. Add adaptor to gradle/adaptor.gradle:adaptorDocumentation and src/main/resources/META-INF/services/nl.esciencecenter.xenon.adaptors.*Adaptor
   2. Update adaptor doc by running ./gradlew publishSite
2. Bump version in README.md, build.gradle and CITATION.cff
3. Update CHANGELOG.md
4. Commit and push changes
5. Publish to bintray with BINTRAY_USER=*** BINTRAY_KEY=**** ./gradlew bintrayUpload
6. Create GitHub release, for title use the version and for description use first chapter of README as intro and changelog section belonging to the version.
7. In Zenodo entry, add Xenon doi as is referenced by this upload.
8. Announce release so users of library like xenon-cli can use new version.