- Azure subscription. This can be a free trial subscription, MSDN, or the one you use for other work.
- Azure Python SDK and azure-mgmt-batchai, if you like to run recipes using Python Jupyter notebook. See How to install Azure SDK.
- Azure CLI 2.0, if you like to run recipes using Azure CLI - See Install Azure CLI 2.0 for instructions.
- Azure Storage Account in East US (required for all recipes). See How to create Azure storage accounts
To start, please Clone or download this repo
Please follow Azure CLI 2.0 Batch AI specific documentation to install and configure Azure CLI 2.0 for using with Batch AI.
We provide setup wizard scripts of setup.ps1 for Windows users and setup.sh for Linux users. Please execute the script from you cloned recipe directory <your clone root>/BatchAI/recipes.
Note that Azure CLI 2.0 is required for the setup scripts. Please follow documentation to install and configure Azure CLI 2.0. For Linux Users, jq package is also required to process json data.
Alternative to use setup wizard scripts. You can also configure Batch AI manually. Please follow Preparation.md for instructions.
Use the following links for a quick navigation:
For your convenience, we provide a collection of helper functions in BatchAI/utilities used for each recipes:
- Read parameters from configuration file
- Create python client object (BatchAIManagementClient) to access Azure Batch AI service
- Create/Update resource group
- Download file with given shared access signature (SAS)
- Print Job/Cluster status
- File Streaming
Since all recipes utlize APIs from other Azure products (e.g, Azure storage, credentials), it is also required to install the full package of Azure Python SDK:
pip install azureInstall Batch AI management client using the following command:
pip install azure-mgmt-batchai --upgradePlease install Jupyter Notebook from https://jupyter.org/ or run
python -m pip install jupyter- Route into the root your cloned recipe directory
cd <your clone root>/BatchAI/recipes- Launch the Jupyter Notebook by
jupyter notebook- In the prompted brower brower, navigate into the recipe of interest, and start the *.ipynb file.
Please follow Azure CLI 2.0 Batch AI specific documentation to install and configure Azure CLI 2.0 for using with Batch AI.
During Cluster and File Server creation you will need to specify a name and authentication method for administrator account which will be created on each compute node (you can use this account to ssh to the node).
You can provide a password and/or ssh public key as authentication method via --password (-p) and --ssh-public-key (-k) parameters.
GNU/Linux users (including Cloud Shell users) can generate authentication key for ssh using ssh-keygen command.
Note, GNU/Linux part of recipes expects you to have a public ssh key at ~/.ssh/id_rsa.pub, if you prefer to use different ssh key, please update -k parameter value.
Training data used in recipes is compressed in zip archives and requires unzip utility to be installed on the host, please install it using your distribution package manager.
Cloud Shell has unzip already installed.
Each recipe contains cli-instructions.md file which describes input data, cluster and job configuration and provides instructions for cluster and job creation.
If you have any problems or questions, you can reach the Batch AI team at AzureBatchAITrainingPreview@service.microsoft.com or you can create an issue on GitHub.
We also welcome your contributions of additional sample notebooks, scripts, or other examples of working with Batch AI.