Support azure-cli credentials for local development using azure-based components

[onionhammer]

Describe the feature

It would be a huge time saver for developers to be able to use azure-cli based credentials to authenticate azure-based dapr components; this would allow developers to having to create service principles and managing secrets or certificates just to run their components locally.

The analog here for .NET developers would be use of the DefaultAzureCredential, which automatically checks several sources on your machine for azure credentials (including VS Code, CLI, token cache, and managed identity endpoints).

Release Note

ADD Support for Azure-CLI credentials with azure-based dapr components

• Depends on Update all other Azure components to use the common auth layer (and support Azure AD auth) #1103

[berndverst]

I have a local prototype for this and tried to get it into the release but it won't be possible. This requires azidentity 1.3.0-beta2. We will not accept beta dependencies for such core libraries and are currently waiting for the 1.3.0 release.

https://github.com/Azure/azure-sdk-for-go/blob/e085950674aad851a010f82381472acc5e7fbace/sdk/azidentity/CHANGELOG.md#130-beta2-2023-01-10

As a result this will not make it into 1.10.

[onionhammer]

Looks promising! we'll just have to wait and suffer :P

[berndverst]

The DefaultAzureCredential likely doesn't work here by the way because we need to also support user assigned managed identities as well as other Azure cloud endpoints (like on premises, sovereign clouds etc)

I added the CLI Credential as a fallback option after the Managed Identity auth times out. This is essentially what the DefaultAzureCredential does also.

[onionhammer]

The DefaultAzureCredential likely doesn't work here by the way because we need to also support user assigned managed identities as well as other Azure cloud endpoints (like on premises, sovereign clouds etc)

I added the CLI Credential as a fallback option after the Managed Identity auth times out. This is essentially what the DefaultAzureCredential does also.

DefaultAzureCredential does support user assigned managed identities; you just have to pass it the client id

[berndverst]

The DefaultAzureCredential likely doesn't work here by the way because we need to also support user assigned managed identities as well as other Azure cloud endpoints (like on premises, sovereign clouds etc)

I added the CLI Credential as a fallback option after the Managed Identity auth times out. This is essentially what the DefaultAzureCredential does also.

DefaultAzureCredential does support user assigned managed identities; you just have to pass it the client id

It uses a default 1 second timeout for attempting managed identity however which isn't good enough.

It also doesn't seem to have a way to specify the various Azure environments (e.g. China cloud)

Hence we need to use a custom chained token credential anyway.

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

[onionhammer]

bumping

[berndverst]

bumping

Once dapr/dapr#5991 is merged you can build Dapr yourself or wait for the next nightly build (or Dapr 1.11) to use the CLI Credential. Documentation won't be ready until 1.11... but essentially this works the same way Managed Identity authentication with system assigned identity works. You will only have to specify things like the Storage account name, Keyvault name etc etc

Give it a try and let us know @onionhammer

[onionhammer]

bumping

Once dapr/dapr#5991 is merged you can build Dapr yourself or wait for the next nightly build (or Dapr 1.11) to use the CLI Credential. Documentation won't be ready until 1.11... but essentially this works the same way Managed Identity authentication with system assigned identity works. You will only have to specify things like the Storage account name, Keyvault name etc etc

Give it a try and let us know @onionhammer

I will give it a shot next week

[berndverst]

@onionhammer I did some testing - it works great, but I need to point out some caveats.

By default the Azure CLI only has management scopes. It is not authorized for any data plane operations.

Also, CLI authentication is a bit slow, so you need to increase the default init timeout from 5s to something else.

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: statestore-aad
spec:
  initTimeout: 30s
  type: state.azure.blobstorage
  metadata:
    - name: accountName
      value: "mystorageaccount"
    - name: containerName
      value: "somecontainerthatdoesnotexist"

You will see that the storage container is successfully created. However any attempts to store data will not succeed as that is a data plane operation.

You must either grant the necessary permissions to your CLI's credentials, or log the CLI into a different credential. For example I used a service principal which I knew had the correct permissions via az login -u $CLIENTID -p $CLIENTSECRET --tenant $TENANTID.

While signed into the correct credential using the CLI the above component yaml worked for me without issues and all data plane operations worked too.

Please note that this is the default behavior of the Azure CLI, DefaultAzureCredential / AzureCLICredential.

[onionhammer]

Have you given yourself roles to access data? Usually something like blob data contributor role for access to blob data, is that what you're referring to by "by default"?

I use azure cli credentials a lot in our codebase for local development, just not with dapr

[berndverst]

Have you given yourself roles to access data? Usually something like blob data contributor role for access to blob data, is that what you're referring to by "by default"?

I use azure cli credentials a lot in our codebase for local development, just not with dapr

Yes I created a new service principal and then gave that service principal blob data contributor permission. Then I logged into the CLI using that service principal instead.

The default "az login" obtains a service principal with only management permissions. I used "az login --service-principal". You could of course also figure out what service principal the CLI is already logged in as and grant those roles after the fact.

[onionhammer]

Hmm.. not sure if I follow. I'll see if I can find time tomorrow to play with it to try to see what you mean.

How I generally work is just assign whatever role I need in the resource group to my AAD user account (in the IAM pane), and az cli uses that credential.

[berndverst]

I am under the impression that the Azure CLI creates a separate service principal where the only authorized scopes are management operations when you run "az login".

If your approach works - great.

Note that the CLI actually has some additional code to query and then use other credentials. That's however not available when using any of the Azure SDKs in Python, Go etc

So maybe what you are observing is the convenience functionality baked into the CLI?

Play around tomorrow - all I can say is that the behavior we implemented looks correct to me and working as intended. Maybe you have some best practices for managing your CLI permissions I don't know.

[berndverst]

@onionhammer you are absolutely right. It seems that az login properly impersonates my current user and permissions. By default my user does not have those data plane scopes. I used the Azure Portal to grant myself Azure Blob Data Contributor for my storage account, and then ran az login and everything worked perfectly.

I'm glad to have been wrong on this!

FYI @ItalyPaleAle

[berndverst]

So I guess the only caveat to remember is to set initTimeout in the spec section of each of your components for which you want to use CLI auth. 5 seconds seems to be just too slow (and I'm on Gigabit Fiber with 2ms latency...)

[cwistedt]

So i have upgraded to dapr cli 1.11. Works great. However, when i run two or more apps locally with dapr that uses MI to get secrets from a secrets store i get this error:

It happens when i start them at the same time. Looks like the one that is slower to start breaks the faster one on startup. And then i see the error above. I have not observed this error when i start them one by one and wait and make sure the first app starts correctly. And then i start the second one.

[ItalyPaleAle]

Possibly related to #2916

[onionhammer]

I have yet to experience any issues running locally with az cli, I have not configured any timeouts

[Yeg-A]

PSA: Make sure to upgrade to the latest version of Dapr CLI