azure deployment - integers being passed as string

[vpopescu]

SUMMARY

I am trying to deploy an cluster via azure_rm_deployment and an inline template. I am getting an error when trying to send an integer property, it appears it's being serialized as a string.

The value in question is count, below. I have a string value of "1" and I'm trying to send it as an integer:

...
    count: "{{ agent_count | int}}"
...

I found this filter in another bug report, and it appered that it would fix my needs. However my JSON still comes out as a string:

...
   "count": "1",
...

If I however replace the line with:

...
    count: 1
...

Then It gets generated correctly. What am I doing wrong?

ISSUE TYPE

• Bug Report

COMPONENT NAME

azure_rm_deployment

ANSIBLE VERSION

ansible 2.7.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/val/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0]

CONFIGURATION

OS / ENVIRONMENT

Ubuntu / WSL

STEPS TO REPRODUCE

...

1. Create any playbook that contains an inline azure deployment template and has an integer field.
2. Define the field as a string, and try to pass it via

...

...

- set_fact:
    agent_count: "{{ lookup('env', 'CONFIGURED_AGENT_COUNT) }}"

...

- name: Create AKS cluter via Azure template
  azure_rm_deployment:
    state: present
    resource_group_name: "{{ rg }}"
    template:
      $schema: "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#managedCluster"
      contentVersion: "1.0.0.0"
      resources:
        - type: "Microsoft.ContainerService/managedClusters"
          apiVersion: '2018-03-31'
          location: "{{ location }} }"
          name: "{{ cn }}"
          properties:
            dnsPrefix: "{{ prefix }} "
            agentPoolProfiles:
              - name: "{{ anp }} "
                count: "{{ agent_count | int}}"
#                count: 1  (this works)
                vmSize:  "{{ vm ) }}"
            linuxProfile:
              adminUsername: "{{ luid }} "
              ssh:
                publicKeys:
                - keyData: "{{ lkey }} "
            servicePrincipalProfile:
              clientId: "{{ appId }}"
              secret:  "{{ password }}"
            addonProfiles: {}
            enableRBAC: true

EXPECTED RESULTS

The playbook should execute.

ACTUAL RESULTS

he full traceback is:
WARNING: The below traceback may *not* be related to the actual failure.
  File "/tmp/ansible_azure_rm_deployment_payload_TScRh7/__main__.py", line 515, in deploy_template
    deployment_result = self.get_poller_result(result)
  File "/tmp/ansible_azure_rm_deployment_payload_TScRh7/ansible_azure_rm_deployment_payload.zip/ansible/module_utils/azure_rm_common.py", line 484, in get_poller_result
    poller.wait(timeout=delay)
  File "/usr/local/lib/python2.7/dist-packages/msrestazure/azure_operation.py", line 511, in wait
    raise self._exception

fatal: [localhost]: FAILED! => {
    "changed": false,
    "failed_deployment_operations": [
        {
            "id": "/subscriptions/0f2f4954-26bf-4aed-bc0c-24ffc80d1c18/resourceGroups/agnxRG/providers/Microsoft.Resources/deployments/ansible-arm/operations/C435A2563172D838",
            "operation_id": "C435A2563172D838",
            "provisioning_state": "Failed",
            "status_code": "BadRequest",
            "status_message": {
                "code": "UnmarshalError",
                "message": "UnmarshalEntity encountered error: json: cannot unmarshal string into Go struct field AgentPoolProfile.count of type int."
            },
...,
    "invocation": {
        "module_args": {
            "ad_user": null,
            "adfs_authority_url": null,
            "api_profile": "latest",
            "append_tags": true,
            "auth_source": null,
            "cert_validation_mode": null,
            "client_id": null,
            "cloud_environment": "AzureCloud",
            "deployment_mode": "incremental",
            "deployment_name": "ansible-arm",
            "location": "westus",
            "parameters": null,
            "parameters_link": null,
            "password": null,
            "profile": null,
            "resource_group_name": "reactor42Test",
            "secret": null,
            "state": "present",
            "subscription_id": null,
            "tags": null,
            "template": {
                "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#managedCluster",
                "contentVersion": "1.0.0.0",
                "resources": [
                    {
...
                            "addonProfiles": {},
                            "agentPoolProfiles": [
                                {
                                    "count": "1",
                                    "name": "sampletestagnx",
                                    "vmSize": "Standard_DS2_v2"
                                }
                            ],
...
}

[ansibot]

Hi @vpopescu, thank you for submitting this issue!

click here for bot help

[ansibot]

Files identified in the description:

• lib/ansible/modules/cloud/azure/azure_rm_deployment.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibot]

cc @haroldwongms @nitzmahone @trstringer @xscript @yuwzho @zikalino
click here for bot help

[ansibot]

@vpopescu, just so you are aware we have a dedicated Working Group for azure.
You can find other people interested in this in #ansible-azure on Freenode IRC
For more information about communities, meetings and agendas see https://github.com/ansible/community

click here for bot help

[zikalino]

@vpopescu this is actually generic Ansible issue, not relayed to azure.
when you do templating "{{ … }}" everything is treated as a string.

so when you do this: "{{ agent_count | int}}"
it's first converted into int internally, and then it gets converted back intro string because of everything in quotes is treated as a string.

there is already a discussion on this.....

#36818
#35536

seems it needs to be revisited....

[vpopescu]

@zikalino Thanks, I thought my googling returned some solutions for other modules where the "int" filter actually worked, I guess i was wrong. But this is a fundamental issue then, if you can't use integer values in ansible, since many systems validate results against a schema of some sort. What should I do with the bug, how do I make it a generic one?

Doesn't help that the other discussions are all closed if they were not resolved. :(

[zikalino]

I think there was another, more generic discussion about this, but couldn't find it yet.
actually I have noticed that you want to use deployment because aks module doesn't provide enough functionality. you could also try to use rest api directly, I can create a sample (under another issue)

[vpopescu]

Thanks. Wouldn't the example in the other post have the same issue when i try to set the count from an external variable, and it would convert it to a string, thereby making the REST API choke?

[zikalino]

actually, probably yes, I think REST API should be more forgiving when passing integer as string.... but needs to be checked.

[zikalino]

also it should be possible to pass JSON.
having said that it could perhaps be a workaround for deployment templates...

[kaelscion]

If you don't mind, I'd like to take a look at this but am new to open source. Is there any way I can assign myself to it per chance?

[zikalino]

@kaelscion unfortunately it's possible to assign only members.... I can assign myself, and then help you merge if you create pr :-)

[zikalino]

seems like, when just using single numeric variable, say "a", "{{ a }}" will be actually a number.
but any more complex expression will result in a string......

[kaelscion]

Hmm. Once I dig in a bit more I'll be able to answer this for myself I'm sure (cloning the version in the bug report right now) but just to save some time, is there templating support built in? Like Jinja or Django's template system or the like? I'm guessing an Azure deployment has web template support at some area of the stack. Thank you so much btw for answering my questions. If it gets converted to an int via the Python code, then back to a string by them template engine, that gives me a place to start at least.

[vpopescu]

#9362
#32738

Last one talks about jinja2_native config setting. Have not yet been able to get it to work.

[kaelscion]

Fair enough. I work with Jinja2 templating alot in my Flask development projects so I'm pretty familiar with it. I'll take a look at the differences between web and native and how they hand int input. We'll get this tracked down. I'll be taking a look this weekend and see what I can find out 😁😁

[Fred-sun]

@kaelscion Thanks for your contribution, Could you update the tracking result？ Thanks!

[kaelscion]

I haven't made a pull request as of yet. I'm having a bit of trouble getting things set up properly on Azure to test honestly. I'm usually an AWS and GCloud guys. I use the Jinja2 template all the time, hence my interest in this particular bug, but I cannot seem to get the environment set up correctly to test :O. I will update as soon as I have submitted changes. Apologies for delay!

[Fred-sun]

@kaelscion Thanks for your update. Once you submitted changes, I will push maintainer to review and merge. Thanks!

[Fred-sun]

kindly ping

[kaelscion]

Unfortunately, you may want to remove me from this bug. Azure is much different than what I am used to and, while I am sure I can work within the context of the bug once all is deployed to Azure, I simply cannot seem to get a working version of Ansible up and running on the Azure platform. I apologize for wasting folk's time as I would really like to work on this, but I am just too green in Azure to get a playbook up and running, let alone to the point of reproducing the issue. I'm sure I could figure it out eventually, but I simply do not have enough time to play with it among my other commitments to do this in a timely manner. Unless there is a tutorial or something that the Ansible team uses to deploy to Azure that could be linked here, I should probably just let you guys move on.

[vpopescu]

@kaelscion just curious, why do you need it running in Azure? You can run the playbook on your local computer and manage Azure that way.

[kaelscion]

My word. Well, that's "bug vision" for ya.. I got so consumed with the fact that the issue took place in management of Azure that I absolutely missed the fact that I could run it locally. Wow. Okay I'll get back at it thank you so much for letting me know!

[Fred-sun]

@zikalino how about the issue going ahead? Could you help take a look? Thanks!

[ansibot]

cc @devigned @lmazuel @obsoleted
click here for bot help

[Fred-sun]

kindly ping

[Fred-sun]

@zikalino

[Fred-sun]

@zikalino

[Fred-sun]

@zikalino

[Fred-sun]

@zikalino

[Fred-sun]

kindly ping

[Fred-sun]

@zikalino Do you have any update about this issue?

[yuwzho]

resolved_by_pr #32738

You can simply set the environment variable or modify the ansible.cfg to enable native jinjia2 syntax.

reference:
#32738 (comment)
#32738 (comment)