ansible-lint reports key-duplicates in EXAMPLE doc of plugins

[guidograzioli]

Summary

ansible-lint 2.21.0 reports duplicate-keys inside EXAMPLE documentation of plugins

Issue Type

• Bug Report

OS / ENVIRONMENT

ansible-lint --version
ansible-lint 6.21.0 using ansible-core:2.15.5 ansible-compat:4.1.10 ruamel-yaml:0.17.35 ruamel-yaml-clib:0.2.8

• ansible installation method: pip
• ansible-lint installation method: pip

STEPS TO REPRODUCE

Run ansible-lint against a plugin with the following code:

EXAMPLES = """
# Filename must end with kubevirt.[yml|yaml]

# Authenticate with token, and return all VirtualMachineInstances for all accessible namespaces
plugin: kubevirt.core.kubevirt
connections:
- host: https://192.168.64.4:8443
  api_key: xxxxxxxxxxxxxxxx
  validate_certs: false

# Use default config (~/.kube/config) file and active context, and return VirtualMachineInstances
# from namespace testing with interfaces connected to network bridge-network
plugin: kubevirt.core.kubevirt
connections:
- namespaces:
  - testing
  network_name: bridge-network

Desired Behavior

Should not report an en error

Actual Behavior

Reports an error: yaml[key-duplicates]: Duplication of key "plugin" in mapping

https://github.com/kubevirt/kubevirt.core/actions/runs/6570606188/job/17848302926#step:6:93

[ssbarnea]

That is a genuine error as the entire EXAMPLES is no longer a valid YAML file. Still, I do not know what is the correct way to include more than one example...

[cidrblock]

From the docs:

"The DOCUMENTATION block must be valid YAML."

and for the examples:

"Here you show users how your module works with real-world examples in multi-line plain-text YAML format. The best examples are ready for the user to copy and paste into a playbook."

This is the way I think about the examples, which is why linting them is so important, they should be cut-and-paste ready for the user.

WRT the example above, that's a tricky one :) Examples are generally assumed to playbook/tasks in which case they are a list and we don't have to worry about duplicate keys.

But for 2 examples of an inventory plugin, each of which is a dictionary it makes sense why it was done like it was.

In this case, the error could be ignored or maybe it would pass with a yaml doc seperator between them, which is probably more accurate because these are 2 examples of the same file.

(I haven't tested the doc seperator)

[ssbarnea]

@Qalthos I think that we might want to modify the current implementation to allow it to use multi-documents. What we need to do is to split the multi-document into multiple file before linting the example, that is because ansible itself does not allow loading of multi-document files.

That is also opening few other questions regarding how can we guess/infer the nature of the example (file kind). I guess we could ask our users to add magic comments? The alternative is to only support tasks and playbooks and determine it based on what we find (dictionary or list).

[guidograzioli]

Is there a way to disable the rule for DOCUMENTATION and EXAMPLE only? duplicated-key is a very important rule but for those two strings, I'd disable it and forget forever

[ssbarnea]

I think that excluding the plugin file should work.

[tremble]

For anyone else hitting this, if you use the document separator (---) between each of the examples, then with ansible-lint 6.22.1 duplicate key errors only fire if there's a duplicate key within the example:

eg

EXAMPLES = r""" 
---

# Example using groups to assign the running hosts to a group based on vpc_id
plugin: amazon.aws.aws_ec2
profile: aws_profile
# Populate inventory with instances in these regions
regions:
  - us-east-2
filters:
  # All instances with their state as `running`
  instance-state-name: running
keyed_groups:
  - prefix: tag
    key: tags
compose:
  ansible_host: public_dns_name
groups:
  libvpc: vpc_id == 'vpc-####'

---

# Define prefix and suffix for host variables coming from AWS.
plugin: amazon.aws.aws_ec2
regions:
  - us-east-1
hostvars_prefix: 'aws_'
hostvars_suffix: '_ec2'
"""

unfortunately ansible-test's sanity checks then complain

ERROR: plugins/inventory/aws_ec2.py:157:1: unparsable-with-libyaml: expected a single document in the stream - but found another document

[bcoca]

o longer a valid YAML file. Still, I do not know what is the correct way to include more than one example...

it is not required to be YAML, EXAMPLES is treated as text by ansible-doc.

[atocko]

It is still an issue for the EXAMPLES of dynamic inventory plugins. Those cant have - name blocks.
False positives on multiple examples:

# Example 1
---
plugin: "kind.of.inventory"
param_one: value1

# Example 2
plugin: "kind.of.inventory"
param_one: value2

Error: yaml[key-duplicates]: Duplication of key "plugin" in mapping

Interesting, how to avoid it...

[valkiriaaquatica]

The yaml[key-duplicates]: Duplication of key "plugin" in mapping error was solved as @tremble said using --- between different plugin examples.

But then using ansible-test sanity received:
ERROR: Found 1 yamllint issue(s) which need to be resolved:
ERROR: plugins/inventory/inventory.py:73:1: unparsable-with-libyaml: expected a single document in the stream - but found another document

[tremble]

ERROR: plugins/inventory/inventory.py:73:1: unparsable-with-libyaml: expected a single document in the stream - but found another document

This is a problem with ansible-test. It's been fixed in Ansible Core 2.17 (currently in release candidate stage).