Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure connection strings for /vsiaz/ have additional valid constructions #6870

Closed
drwelby opened this issue Dec 7, 2022 · 1 comment
Closed
Assignees

Comments

@drwelby
Copy link
Contributor

drwelby commented Dec 7, 2022

Azure connection strings are parsed in ParseStorageConnectionString:

/* ParseStorageConnectionString() */

which looks for the Azure account name and access key within the connection string.

These connection strings also can have an alternate construction for using SAS tokens which takes the format:

"BlobEndpoint=https://my_test.blob.core.windows.net;SharedAccessSignature=sp=rl&st=2022-12-06T20:41:17Z&se=2022-12-07T04:41:17Z&spr=https&sv=2021-06-08&sr=c&sig=xxxxxxxx"

This string could be parsed to obtain the /vsiaz/ configuration variables that map to AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_SAS_TOKEN environment variables. A Python proof of concept is:

from azure.core.utils import parse_connection_string
from urllib.parse import urlparse

conn_string = 'BlobEndpoint=https://my_test.blob.core.windows.net;SharedAccessSignature=sp=rl&st=2022-12-06T20:41:17Z&se=2022-12-07T04:41:17Z&spr=https&sv=2021-06-08&sr=c&sig=xxxxxxxx'

# SAS token is the SharedAccessSignature portion

parsed = parse_connection_string(conn_string)
sas_token = parsed['sharedaccesssignature']

# Account name starts the blob URL: https://ACCOUNT_NAME.blob.core.windows.net
 
blob_url = urlparse(parsed["blobendpoint"]).netloc
account_name = blob_url.split(".")[0]

os.environ['AZURE_STORAGE_ACCOUNT'] = account_name
os.environ['AZURE_STORAGE_SAS_TOKEN'] = sas_token

While workarounds like the above are not difficult, the current situation is that a valid token will work for other software packages that use AZURE_STORAGE_CONNECTION_STRING envvars to read files in Azure such as adlfs but then cause errors when using GDAL with the same string to access files in the same location.

@rouault rouault self-assigned this Dec 9, 2022
@drwelby
Copy link
Contributor Author

drwelby commented Dec 9, 2022

Thanks Even for jumping on this so quickly!!

rouault added a commit that referenced this issue Dec 10, 2022
/vsiaz/: accept Azure connection string with only BlobEndpoint and SharedAccessSignature (fixes #6870)
rouault added a commit that referenced this issue Dec 10, 2022
rouault added a commit that referenced this issue Dec 10, 2022
[Backport release/3.6] /vsiaz/: accept Azure connection string with only BlobEndpoint and SharedAccessSignature (fixes #6870)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants