Expired Access token

[ingthu]

Hi!

Just wonder if it is a way to extend the access token. Now it is 1 hour but that is not enough when deploying large datasets.

BR
/I

[mthierba]

Could you provide some further context for your scenario? Which access is expiring? The deployment token or one used for your external datasources? Why is the deployment taking so long? Is it because you have a long-running refresh? Are there any logs/error messages you could share?

[ingthu]

The dataset I need to deploy is large (6GB), refresh window is used, last 2 months based on a business date. When I have made changes in the dataset I would like to do a full refresh of the partitions/dataset. A full refresh takes approx 1.5 hrs. But the deployment token is only valid for approx. 1 hour. After that the deployment fails. Is there any workaround for this?

This is part of deployment manifest:

Error message:
Starting dataset refresh (Full) ...
Using default refresh type: Full
Refreshing model (Full)...
An unhandled exception occurred.
System.AggregateException: One or more errors occurred. ---> Microsoft.AnalysisServices.ConnectionException: MWC token validation failed: Microsoft.IdentityModel.Tokens.SecurityTokenExpiredException at Microsoft.IdentityModel.Tokens.Validators.ValidateLifetime(Nullable1 notBefore, Nullable1 expires, SecurityToken securityToken, TokenValidationParameters validationParameters)
at Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler.ValidateTokenPayload(JsonWebToken jsonWebToken, TokenValidationParameters validationParameters, BaseConfiguration configuration)
at Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler.ValidateJWS(String token, TokenValidationParameters validationParameters, BaseConfiguration configuration)

Technical Details:
RootActivityId: 76943baa-321d-4623-bf6d-91cefcb62ab0
Date (UTC): 6/12/2023 8:19:00 PM
at Microsoft.AnalysisServices.HttpStream.HttpChannelController.GetResponseMessageImpl(HttpStream owner, Object context, Task1 pendingResponse) at Microsoft.AnalysisServices.HttpStream.HttpChannelController.CompleteHttpClientBasedOperation(HttpStream owner, Object context, Task1 pendingResponse)
at Microsoft.AnalysisServices.HttpStream.HttpXmlaOperation.HttpClientOperationManager.GetResponse(HttpXmlaOperation operation)
at Microsoft.AnalysisServices.HttpStream.PaasInfraController.PaasInfraXmlaOperation.EnsureCanRead()
at Microsoft.AnalysisServices.HttpStream.HttpXmlaOperation.GetResponseDataType()
at Microsoft.AnalysisServices.HttpStream.GetResponseDataType()
at Microsoft.AnalysisServices.CompressedStream.GetResponseDataType()
at Microsoft.AnalysisServices.XmlaClient.EndRequest(Boolean useBinaryXml)
at Microsoft.AnalysisServices.XmlaClient.SendExecuteAndReadResponse(Boolean skipResults, Boolean throwIfError, Boolean useBinaryXml)
at Microsoft.AnalysisServices.Core.AnalysisServicesClient.RollbackTransaction()
at Microsoft.AnalysisServices.Core.Server.RollbackTransactionInternal()
at Microsoft.AnalysisServices.Tabular.Server.RollbackTransaction()
at Microsoft.AnalysisServices.Tabular.Model.SaveChangesImpl(SaveFlags flags, Int32 maxParallelism)
at PbiTools.Deployments.XmlaRefreshManager.RunRefresh() in X:\pbi-tools\BRANCHES\fody\src\PBI-Tools\Deployments\XmlaRefreshManager.cs:line 94
at PbiTools.Deployments.DeploymentManager.d__32.MoveNext() in X:\pbi-tools\BRANCHES\fody\src\PBI-Tools\Deployments\DeploymentManager.Dataset.cs:line 452
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at PbiTools.Deployments.DeploymentManager.d__30.MoveNext() in X:\pbi-tools\BRANCHES\fody\src\PBI-Tools\Deployments\DeploymentManager.cs:line 97
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at PbiTools.Cli.CmdLineActions.Deploy(String folder, String label, String environment, String basePath, Boolean whatIf) in X:\pbi-tools\BRANCHES\fody\src\PBI-Tools\Cli\Deploy.cs:line 39
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at PowerArgs.ArgAction.Invoke()
at PbiTools.Program.Main(String[] args) in X:\pbi-tools\BRANCHES\fody\src\PBI-Tools\Program.cs:line 110
---> (Inner Exception #0) Microsoft.AnalysisServices.ConnectionException: MWC token validation failed: Microsoft.IdentityModel.Tokens.SecurityTokenExpiredException at Microsoft.IdentityModel.Tokens.Validators.ValidateLifetime(Nullable1 notBefore, Nullable1 expires, SecurityToken securityToken, TokenValidationParameters validationParameters)
at Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler.ValidateTokenPayload(JsonWebToken jsonWebToken, TokenValidationParameters validationParameters, BaseConfiguration configuration)
at Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler.ValidateJWS(String token, TokenValidationParameters validationParameters, BaseConfiguration configuration)

Technical Details:
RootActivityId: 76943baa-321d-4623-bf6d-91cefcb62ab0
Date (UTC): 6/12/2023 8:19:00 PM
at Microsoft.AnalysisServices.HttpStream.HttpChannelController.GetResponseMessageImpl(HttpStream owner, Object context, Task1 pendingResponse) at Microsoft.AnalysisServices.HttpStream.HttpChannelController.CompleteHttpClientBasedOperation(HttpStream owner, Object context, Task1 pendingResponse)
at Microsoft.AnalysisServices.HttpStream.HttpXmlaOperation.HttpClientOperationManager.GetResponse(HttpXmlaOperation operation)
at Microsoft.AnalysisServices.HttpStream.PaasInfraController.PaasInfraXmlaOperation.EnsureCanRead()
at Microsoft.AnalysisServices.HttpStream.HttpXmlaOperation.GetResponseDataType()
at Microsoft.AnalysisServices.HttpStream.GetResponseDataType()
at Microsoft.AnalysisServices.CompressedStream.GetResponseDataType()
at Microsoft.AnalysisServices.XmlaClient.EndRequest(Boolean useBinaryXml)
at Microsoft.AnalysisServices.XmlaClient.SendExecuteAndReadResponse(Boolean skipResults, Boolean throwIfError, Boolean useBinaryXml)
at Microsoft.AnalysisServices.Core.AnalysisServicesClient.RollbackTransaction()
at Microsoft.AnalysisServices.Core.Server.RollbackTransactionInternal()
at Microsoft.AnalysisServices.Tabular.Server.RollbackTransaction()
at Microsoft.AnalysisServices.Tabular.Model.SaveChangesImpl(SaveFlags flags, Int32 maxParallelism)
at PbiTools.Deployments.XmlaRefreshManager.RunRefresh() in X:\pbi-tools\BRANCHES\fody\src\PBI-Tools\Deployments\XmlaRefreshManager.cs:line 94
at PbiTools.Deployments.DeploymentManager.d__32.MoveNext() in X:\pbi-tools\BRANCHES\fody\src\PBI-Tools\Deployments\DeploymentManager.Dataset.cs:line 452
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at PbiTools.Deployments.DeploymentManager.d__30.MoveNext() in X:\pbi-tools\BRANCHES\fody\src\PBI-Tools\Deployments\DeploymentManager.cs:line 97<---

[mthierba]

The 3600 second token lifetime is set and enforced by AAD, that's a hard limit. Key here is to re-structure your operation so that it doesn't result in a very long-running process.

This can be done by refreshing individual partitions in batches. Alternatively, the async refresh REST API could be used. The latter, however, is not currently supported by pbi-tools, I'm afraid. It is on the backlog: #172

[ingthu]

Thanks for your reply, I will use TMSL refresh command for full refreshes.