int is not a number

[amather]

ISSUE TYPE

Bug Report

COMPONENT NAME

core

ANSIBLE VERSION

ansible 2.0.1.0
ansible 2.1.0 (devel 9aa91cb6a3) last updated 2016/04/01 125706 (GMT +200)

CONFIGURATION

OS / ENVIRONMENT

Testet on OS X only.

SUMMARY

Applying the int YAML builtin filter does not make the variable permanently a number. I've to apply it every time I use the variable (e.g. when comparing to another number). The number nature gets also lost when the filter is applied within set_fact.

STEPS TO REPRODUCE

  vars:
    # 15G virtual size
    output: |
      {
        "virtual-size": 16106127360,
        "filename": "customer-dev-root.qcow2",
        "cluster-size": 65536,
        "format": "qcow2",
        "actual-size": 197120,
        "format-specific": {
            "type": "qcow2",
            "data": {
                "compat": "1.1",
                "lazy-refcounts": false
            }
        },
       "dirty-flag": false
      }
  tasks:
    - set_fact:
        a: 15
        b_obj: "{{output | from_json}}"
    - set_fact:
        # Will be 15
        b: "{{ (b_obj['virtual-size'] | int) // 1024 // 1024 // 1024 }}"
    - set_fact:
        is_smaller: "{{ a < b }}"
        c: "{{ (b|int) }}"
    - debug: msg="{{b}} (b) is a number -- {{b is number}}"
    - debug: msg="{{b}} (direct conversion) is a number -- {{(b|int) is number}}"
    - debug: msg="{{b}} (c) is a number -- {{c is number}}"
    - debug: msg="{{a}} is_smaller than {{b}} (b) -- {{is_smaller}}"
    - debug: msg="{{a}} is_smaller than {{c}} (c) -- {{a < c}}"
    - debug: msg="{{a}} is_smaller than {{b}} (direct conversion) -- {{ a < (b|int)}}"

EXPECTED RESULTS

"msg": "15 (b) is a number -- True"
"msg": "15 (direct conversion) is a number -- True"
"msg": "15 (c) is a number -- True"
"msg": "15 is_smaller than 15 (b) -- False"
"msg": "15 is_smaller than 15 (c) -- False"
"msg": "15 is_smaller than 15 (direct conversion) -- False"

ACTUAL RESULTS

"msg": "15 (b) is a number -- False"
"msg": "15 (direct conversion) is a number -- True"
"msg": "15 (c) is a number -- False"
"msg": "15 is_smaller than 15 (b) -- True"
"msg": "15 is_smaller than 15 (c) -- True"
"msg": "15 is_smaller than 15 (direct conversion) -- False"

[dagwieers]

If you print out the tasks, you get:

TASK: [set_fact ] ************************************************************* 
ok: [localhost] => {"ansible_facts": {"a": 15, "b_obj": {"actual-size": 197120, "cluster-size": 65536, "dirty-flag": false, "filename": "customer-dev-root.qcow2", "format": "qcow2", "format-specific": {"data": {"compat": "1.1", "lazy-refcounts": false}, "type": "qcow2"}, "virtual-size": 16106127360}}}

TASK: [set_fact ] ************************************************************* 
ok: [localhost] => {"ansible_facts": {"b": "15"}}

TASK: [set_fact ] ************************************************************* 
ok: [localhost] => {"ansible_facts": {"c": "15", "is_smaller": "True"}}

And as you can see: b and c are considered strings. This behaviour is identical to v1.9.5 so it doesn't seem to be a regression. It doesn't feel right, as I don't see how else you could create proper integers this way.

[bcoca]

• int is a jinja2 filter, not YAML
• use the filters on 'consumption' to assure type as they can be 'stringified' through templating because jinja2 defaults to emitting strings.

[dagwieers]

@bcoca So it means one cannot create proper integers or other types, simply because Jinja2 is evaluated after YAML. (and we have to make YAML strings so it doesn't interprete '{' as the start of a YAML/JSON dict).

Using Jinja2 filters 'on consumption' feels more like a workaround, although there's probably no real solution to be had, right ?

[bcoca]

@dagwieers I'm open to suggestions to make it otherwise

[dagwieers]

@bcoca If the YAML parser would understand {{ }} is not a dictionary, we would be halfway already. But then we still need to make a difference between actual strings and jinja templates. And what's worse, it would still be considered a string once templated if there's stuff appended after the template, so it is unclear to me how this would finally be interpreted. It would almost mean having YAML skip it during parsing, but have it re-parse the templated stuff at consumption (to determine the type). Overly complex and prone to ambiguity and errors.

[bcoca]

With vars, YAML is out of the picture by the time anything is templated, it is only reintroduced if using it for assignment in a task option and at that point it is getting just jinja2 output which is by default a string.

[antoineco]

@bcoca I'm a bit puzzled, why does

  set_fact:
    enable_server: '{{ "True" | bool }}'

work in that case ? How is it any different ? (see referenced issue above)

[bcoca]

@dagwieers even if yaml recognized {{ }} it still would not matter as this is not a YAML issue but a jinja2 issue. If YAML actually saw it as anything other than a string, it would error out when templating as an int would not work as a type for jinaj2 to template. The tempalting happens AFTER YAML processes the vars, not before, so it is a mute point what YAML understands or not.

@antoineco jinja2 processes booleans as such when it identifies them, with "15" it can be either string or int, jinja2 defaults to treating it as a string, that is how they are different.

[simplesteph]

Is there a workaround for this issue? I need some variables to be cast to integer types in set_fact, or anytime after?

[awr]

Depending on your context, you may be able to work around the issue by passing something which contains the item.

I ran into this with azure_rm_securitygroup, which accepts rules as a list of dicts. One property on the dict (priority) needed to be an int, which I couldn't pass in due to this issue. If I instead passed the entire dict, then the property on the dict did manage to stay as an int.

Not ideal, but may help.

Before:

  azure_rm_securitygroup:
    resource_group: "{{resource_group}}"
    location: "{{location}}"
    name: "{{network_security_name}}"
    purge_rules: no
    rules: 
      - name: "{{rule_name}}"
        protocol: "{{protocol}}"
        destination_port_range: "{{port}}"
        access: Allow
        priority: "{{priority|int}}" # <-- this resolved as a string
        direction: Inbound    

After:

  azure_rm_securitygroup:
    resource_group: "{{resource_group}}"
    location: "{{location}}"
    name: "{{network_security_name}}"
    purge_rules: no
    rules: 
      - "{{rule|combine({'direction':'Inbound', 'access':'Allow'})}}"

[alikins]

(Mentioning #17992 here to keep track of related issues)

[berney]

It would be nice to be able to set the type of variables, so that we don't need to be constantly casting when consuming. The way Ansible is right now using it feels unnatural and clunky, it feels like we are using workarounds rather than something elegant.

It would be nice if we had the option to do something like: -

- command: some command
   register: some_registered_result
- set_fact:
    my_var:
       value: (some_registered_result.stdout | from_yaml)['some line']
       type: int
- command: some other command
   when: my_var != 1

Note: value: doesn't have Jinja2 interpolation escaping (e.g. no{{ expr }}), it's assumed to be Jinja just like how changed_when: is assumed to be Jinja.

Rather than what we have to do now which is: -

- command: some command
   register: some_registered_result
- set_fact:
    my_var: "{{ (some_registered_result.stdout | from_yaml)['some line'] }}"
- command: some other command
   when: "{{ (my_var | int) != 1 }}"

If we can set the type in set_fact, we can set it once, and then when we use it many times we do not have to keep cast it many times.

Even better would be to get the type from Jinja2 programmatically from the AST or similar so that the following actually worked: -

- command: some command
   register: some_registered_result
- set_fact:
    my_var: "{{ (some_registered_result.stdout | from_yaml)['some line'] | int }}"
- command: some other command
   when: my_var != 1

[dagwieers]

@berney You can set the type in YAML, see https://en.wikipedia.org/wiki/YAML

year: !!int 2017
length: !!float 123
mode: !!str 644
check: !!bool Yes

And you can already do:

- command: some command
   register: some_registered_result
- set_fact:
    my_var: '{{ (some_registered_result.stdout | from_yaml)["some line"] }}'
- command: some other command
   when: my_var|int != 1

[berney]

@dagwieers That's true and useful, but does not work with Jinja2 expressions AFAICT from experimenting.

---
- set_fact:
    my_var: !!int "{{ (some_registered_result.stdout | from_yaml)['some line'] }}"

Gets ERROR! Unexpected Exception: invalid literal for int() with base 10: "{{ (some_registered_result.stdout | from_yaml)['some line'] }}".

From what I understand based on what @bcoca said above YAML processing is done before Jinja2 templates, and the experiment seems to concur with this.

[diego-treitos]

I am having also this issue. Sadly not even this works:

- set_fact:
      my_int_var: 1*{{ integer_variable }}

Even in this case my_int_var is still evaluated as a string.

[dagwieers]

@diego-treitos That will never work. Read above if you want to understand why.

[diego-treitos]

@dagwieers I know that Yaml would get that as a string but for some reason I exepcted ansible to parse that as an integer after the jinja2 evaluation, which I realize now that it makes no sense. I guess that in my desperation I just tried anything that came to my mind. Thank you anyway.

Having something to specifiy the type like what @berney suggested would be great.

[yajo]

Then, to fix this problem, Ansible should allow and parse YAML application-specific local tags that force a type cast after the value has been jinja2-evaluated.

[dagwieers]

@yajo Feel free to create a proposal to detail how that would work at: https://github.com/ansible/proposals

[yajo]

Done: ansible/proposals#72

[dagwieers]

There is a light at the end of the tunnel. We made a change to Jinja2 so we don't see all variable types changed into strings. See: pallets/jinja#708

[sivel]

This has been resolved by #32738

$ ANSIBLE_JINJA2_NATIVE=true ansible-playbook -v 15249.yml | grep msg
    "msg": "15 (b) is a number -- True"
    "msg": "15 (direct conversion) is a number -- True"
    "msg": "15 (c) is a number -- True"
    "msg": "15 is_smaller than 15 (b) -- False"
    "msg": "15 is_smaller than 15 (c) -- False"
    "msg": "15 is_smaller than 15 (direct conversion) -- False"

If you have further questions please stop by IRC or the mailing list:

• IRC: #ansible on irc.freenode.net
• mailing list: https://groups.google.com/forum/#!forum/ansible-project