Complex argument parsing returns arrays or objects as strings in ansible 2.1.0

[yourcelf]

ISSUE TYPE

• Bug Report

COMPONENT NAME

core

ANSIBLE VERSION

ansible 2.1.0.0
  config file = /home/user/path/ansible.cfg
  configured module search path = ['some/path/library']

(I've altered the paths referenced here; but both are custom).

CONFIGURATION

My ansible.cfg specifies a custom library path and a custom roles path.

OS / ENVIRONMENT

Ubuntu 14.04.4

SUMMARY

In Ansible 2.1.0, it seems that complex arguments are not getting properly parsed into array or object types, and are presented to the module as strings instead.

STEPS TO REPRODUCE

Here is a minimal module that demonstrates the problem. All the module does is check that the argument mylist is a list type, and fails of it is not, indicating the type in the failure message:

## library/argcheck.py
#!/usr/bin/python

def main():
    module = AnsibleModule(argument_spec={'mylist': {'required': True}})
    mylist = module.params['mylist']
    if not isinstance(mylist, (list, tuple)):
        module.fail_json(msg="expected array; found %s: %s" % (type(mylist), repr(mylist)))
    module.exit_json(changed=False)

from ansible.module_utils.basic import *
if __name__ == "__main__":
    main()

with this playbook:

 - name: Check argument parsing
   argcheck:
     mylist:
       - one
       - two
       - three

Running this playbook fails with the following error:

FAILED! => {"changed": false, "failed": true, "msg": "expected array; found <type 'str'>: \"['one', 'two', 'three']\""}

EXPECTED RESULTS

I expected that retrieving an argument within a module with module.params['varname'] returns the parsed structured type, be that a list, dictionary, or what have you. This is how it worked in ansible v2.0.

ACTUAL RESULTS

Retrieving an argument via module.params['varname'] is returning a string, even when varname is a structured object or list in the yaml file.

Are we now expected to do further parsing/eval'ing of argument contents in order to get structured output, or is there some other way to retrieve the structured params?

[yourcelf]

Ah, it seems that it is now necessary to specify a type in argument_spec, e.g.:

    module = AnsibleModule(argument_spec={'mylist': {'required': True, 'type': 'list'}})

Would love to see some docs on this, as it's a change in behavior.

[nordjc]

I also ran into this with a custom module. The undocumented workaround specified by @yourcelf does not help if you actually want a complex arg that might be any of several types. As a module author, if you were relying on the yaml parser to produce nested dictionaries, you now have to figure out a safe way to eval the stringified representation of what once was a python object.
I will be handling this by adding an additional argument that indicates to my module when I need to parse a string into a complex object vs using the string directly.

[drybjed]

@nordjc When you do this, can you provide an example solution to this issue so that other module developers could use it? Thanks!

[abadger]

@nordjc Couple things to note (Already talked to @drybjed on IRC). I'll start with the one that should help you first:

• type="raw" will store the value as given by yaml and then passed to the module as json directly into the parameter. So you can get that in without it being transformed in other ways.
• The caveat and one of the reasons we changed this: For modules that are being used by other people, types are only applied if the user uses yaml dict syntax. If key=value syntax is used instead, then the module will still get strings. Here's an example:

- copy:
    src: /etc/passwd
    dest: /var/tmp/p
    mode: 420
- copy: src=/etc/passwd dest=/var/tmp/p mode=420

The first example is in yaml dict format. When ansible parses the playbook, the yaml library will parse each dictionary entry and hand them back to Ansible in a structured format. The mode will be an integer number 420 in base 10. So the module will be handed the base 10 integer 420. base 10 420 is octal 0644, a common file permission.

The second example is in key=value format. In this case, yaml will hand Ansible a single string as the value of copy. Ansible will then parse the string into key=value pairs. In this case, all of the values are strings after this parsing. Ansible passes these strings across the wire to the modules. So the copy module receives mode="420". The module then assumes the mode is octal 0420 and sets it accordingly.

So if a module really wants to be non-buggy and uses type=raw it has to be able to handle both cases: when it is given the parameter via yaml dicts it will be in a real type. When given via key=value it will be a string htat has to be parsed further.

[hackndoes]

I have the same issue with the ec2_elb_lb module that have an expected numeric parameter in the stickiness object but is valued as string by ansible thus not functioning correctly.
in the ec2_elb_lb module argument_spec does specify stickiness type as dict

[dagwieers]

(Mentioning #17992 here to keep track of related issues)

There is a light at the end of the tunnel. We made a change to Jinja2 so we don't see all variable types changed into strings. See: pallets/jinja#708

[sivel]

This is expected behavior as we did purposefully change the default of a missing type to default to str.

As this issue report is as old as it is, I don't think a porting guide entry is necessary at this time.

If you have further questions please stop by IRC or the mailing list:

• IRC: #ansible on irc.freenode.net
• mailing list: https://groups.google.com/forum/#!forum/ansible-project