ecs_service_info lists all services in all clusters when no cluster attribute is given #2058

b0tting · 2024-03-18T13:55:27Z

SUMMARY

Added a function to internally query clusters to allow listing of all services without requiring a cluster name in the task.

The current version of ecs_service_info requires a cluster name. There is no ansible task option to list cluster names requiring us to use a command task. By internally querying the clusters we can list all services without needing to know a cluster name. When a cluster ARN (or a list of cluster ARNs) is given only the services for those clusters will be included in the response

Fixes #2056

ISSUE TYPE

Feature Pull Request

COMPONENT NAME

ecs_service_info

ADDITIONAL INFORMATION

As suggested by @markuman in #2056 (suggestion 1)

Task response example without having to add a cluster:

..Ansible task:

- community.aws.ecs_service_info:
    details:false

..response:

        "services": [
            "arn:aws:ecs:eu-west-1:0123456789:service/MyCluster1/SomeService1",
            "arn:aws:ecs:eu-west-1:0123456789:service/MyCluster2/SomeService2"
         ]

softwarefactory-project-zuul · 2024-03-18T14:29:27Z

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/020bd78ec50e4477a847f03ba78d60dc

❌ ansible-galaxy-importer FAILURE in 5m 02s (non-voting)
✔️ build-ansible-collection SUCCESS in 15m 21s
✔️ ansible-test-splitter SUCCESS in 6m 27s
❌ integration-community.aws-1 FAILURE in 16m 12s
Skipped 21 jobs

github-actions · 2024-03-18T15:07:56Z

Docs Build 📝

Thank you for contribution!✨

The docsite for this PR is available for download as an artifact from this run:
https://github.com/ansible-collections/community.aws/actions/runs/8347015912

You can compare to the docs for the main branch here:
https://ansible-collections.github.io/community.aws/branch/main

File changes:

M collections/community/aws/ecs_service_info_module.html

Click to see the diff comparison.

NOTE: only file modifications are shown here. New and deleted files are excluded.
See the file list and check the published docs to see those files.

diff --git a/home/runner/work/community.aws/community.aws/docsbuild/base/collections/community/aws/ecs_service_info_module.html b/home/runner/work/community.aws/community.aws/docsbuild/head/collections/community/aws/ecs_service_info_module.html
index c8095dd..be7018a 100644
--- a/home/runner/work/community.aws/community.aws/docsbuild/base/collections/community/aws/ecs_service_info_module.html
+++ b/home/runner/work/community.aws/community.aws/docsbuild/head/collections/community/aws/ecs_service_info_module.html
@@ -215,9 +215,9 @@ see <a class="reference internal" href="#ansible-collections-community-aws-ecs-s
 </tr>
 <tr class="row-odd"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-cluster"></div><p class="ansible-option-title" id="ansible-collections-community-aws-ecs-service-info-module-parameter-cluster"><strong>cluster</strong></p>
-<a class="ansibleOptionLink" href="#parameter-cluster" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">string</span></p>
+<a class="ansibleOptionLink" href="#parameter-cluster" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">list</span> / <span class="ansible-option-elements">elements=string</span></p>
 </div></td>
-<td><div class="ansible-option-cell"><p>The cluster ARNS in which to list the services.</p>
+<td><div class="ansible-option-cell"><p>The cluster ARNS in which to list the services. If not provided, all clusters are listed.</p>
 </div></td>
 </tr>
 <tr class="row-even"><td><div class="ansible-option-cell">
@@ -321,7 +321,7 @@ see <a class="reference internal" href="#ansible-collections-community-aws-ecs-s
 <a class="ansibleOptionLink" href="#parameter-service" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-aliases">aliases: name</span></p>
 <p class="ansible-option-type-line"><span class="ansible-option-type">list</span> / <span class="ansible-option-elements">elements=string</span></p>
 </div></td>
-<td><div class="ansible-option-cell"><p>One or more services to get details for</p>
+<td><div class="ansible-option-cell"><p>One or more services to get details for. If not provided, all services are listed.</p>
 </div></td>
 </tr>
 <tr class="row-even"><td><div class="ansible-option-cell">
@@ -379,9 +379,15 @@ see <a class="reference internal" href="#ansible-collections-community-aws-ecs-s
 <span class="w">    </span><span class="nt">details</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">true</span>
 <span class="w">  </span><span class="nt">register</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">output</span>
 
-<span class="c1"># Basic listing example</span>
+<span class="c1"># Basic listing example for all services in all clusters</span>
 <span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">community.aws.ecs_service_info</span><span class="p">:</span>
-<span class="w">    </span><span class="nt">cluster</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">test-cluster</span>
+<span class="w">    </span><span class="nt">details</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">true</span>
+<span class="w">  </span><span class="nt">register</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">output</span>
+<span class="c1"># Basic listing example for the list of services in two specific clusters</span>
+<span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">community.aws.ecs_service_info</span><span class="p">:</span>
+<span class="w">    </span><span class="nt">cluster</span><span class="p">:</span>
+<span class="w">      </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">test-cluster</span>
+<span class="w">      </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">prod-cluster</span>
 <span class="w">  </span><span class="nt">register</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">output</span>
 </pre></div>
 </div>

softwarefactory-project-zuul · 2024-03-18T15:48:09Z

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/e0fff5c22756439baaa020ef6c87d803

❌ ansible-galaxy-importer FAILURE in 5m 20s (non-voting)
✔️ build-ansible-collection SUCCESS in 15m 26s
✔️ ansible-test-splitter SUCCESS in 6m 29s
❌ integration-community.aws-1 FAILURE in 16m 03s
Skipped 21 jobs

softwarefactory-project-zuul · 2024-03-18T17:06:27Z

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/f9ae4ef3136e4e2ab45df678900fef65

✔️ ansible-galaxy-importer SUCCESS in 5m 31s (non-voting)
✔️ build-ansible-collection SUCCESS in 15m 39s
✔️ ansible-test-splitter SUCCESS in 5m 55s
❌ integration-community.aws-1 FAILURE in 15m 14s
Skipped 21 jobs

…rs, returning a non-existent service in a cluster result does not make sense.

softwarefactory-project-zuul · 2024-03-18T23:26:35Z

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/978b498143ed4707ab87647209afa213

❌ ansible-galaxy-importer FAILURE in 5m 09s (non-voting)
✔️ build-ansible-collection SUCCESS in 14m 47s
✔️ ansible-test-splitter SUCCESS in 5m 29s
❌ integration-community.aws-1 FAILURE in 16m 18s
Skipped 21 jobs

softwarefactory-project-zuul · 2024-03-19T00:30:21Z

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/f852ddfe0ecc49889913ce9135795d91

✔️ ansible-galaxy-importer SUCCESS in 4m 54s (non-voting)
✔️ build-ansible-collection SUCCESS in 15m 59s
✔️ ansible-test-splitter SUCCESS in 6m 01s
✔️ integration-community.aws-1 SUCCESS in 23m 02s
Skipped 21 jobs

plugins/modules/ecs_service_info.py

Co-authored-by: Markus Bergholz <git@osuv.de>

softwarefactory-project-zuul · 2024-03-19T17:26:50Z

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/42e971ed48cc43409b48aebe44bdfc00

❌ ansible-galaxy-importer FAILURE in 5m 27s (non-voting)
✔️ build-ansible-collection SUCCESS in 15m 25s
✔️ ansible-test-splitter SUCCESS in 7m 18s
✔️ integration-community.aws-1 SUCCESS in 16m 55s
Skipped 21 jobs

markuman · 2024-03-20T18:56:07Z

recheck

softwarefactory-project-zuul · 2024-03-20T19:33:51Z

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/02d3bf84d6b64f58ad115c458cb74193

✔️ ansible-galaxy-importer SUCCESS in 5m 01s (non-voting)
✔️ build-ansible-collection SUCCESS in 15m 21s
✔️ ansible-test-splitter SUCCESS in 5m 57s
✔️ integration-community.aws-1 SUCCESS in 17m 33s
Skipped 21 jobs

markuman · 2024-03-21T05:10:41Z

hm, the failing CI jobs are not related to this PR

ERROR: Found 2 yamllint issue(s) which need to be resolved:
ERROR: tests/sanity/ignore-2.17.txt:1:1: ansible-test: Ignoring 'unparsable-with-libyaml' on 'plugins/connection/aws_ssm.py' is unnecessary
FATAL: The 1 sanity test(s) listed below (out of 24) failed. See error output above for details.
ERROR: tests/sanity/ignore-2.17.txt:2:1: ansible-test: Ignoring 'unparsable-with-libyaml' on 'plugins/inventory/aws_mq.py' is unnecessary

plugins/modules/ecs_service_info.py

markuman · 2024-03-21T05:14:09Z

plugins/modules/ecs_service_info.py

+        else:
+            clusters = self.list_clusters()
+
+        cluster_services = {}


Suggested change

cluster_services = {}

cluster_services = {}

Service name per cluster are unique. What will happen when more than one cluster contains the same service name?
Will the last service name overwrite the first service name?

I think there is an issue when the details flag is not added or set to false. This would return just the list of services with no attributes. Considering the scenario of 2 clusters, with the same HelloWorld service.

- community.aws.ecs_service_info: details: false

..would get you a list of 2 service ARNs

- community.aws.ecs_service_info: service: arn:aws:ecs:eu-central-1:123456789012:service/my-cluster/HelloWorld

..would get you a single ARN

But giving service names (not service ARNs) with details: false in the current ecs_service_info returns you just the existing services. So in this scenario:

- community.aws.ecs_service_info: details: false service: HelloWorld

..would in the current module fail, or force you to add a cluster and return just [HelloWorld]. But what should be the response if you have the given HelloWorld scenario? [HelloWorld] is ambiguous right? But adding cluster info or returning a cluster:service dict would be a change that changes the module response compared to the current version.

I am not an experienced contributer. Do you have a suggestion how to proceed here?

Ah maybe I also misinterpreted the return value

cluster_services[cluster] = ...

So it's already grouped by cluster.

returning a cluster:service dict would be a change that changes the module response compared to the current version.

I think this is the only solution. That would also mean that it is a breaking change and the changes will release with 8.0.0 and not in 7.2.0

Okay. I need some guidance in what would be good form for the output.

So (with details: false) the current output:

services: - hello_world-1 - hello_world-2

If I were to slide in the clusters I could add them as an extra level below services:

services: my-cluster: - hello_world-1 - hello_world-2 my-other-cluster: - hello_world-1 - hello_world-2

Or should this be more explicit (and renaming the root item) in the lines of:

result: - cluster: my-cluster services: - hello_world-1 - hello_world-2 - cluster: my-other-cluster services: - hello_world-1 - hello_world-2

any opinions @tremble @alinabuzachis?

Ok, current return value for details: false is

services: - hello_world-1 - hello_world-2

using details: true results in

services: - clusterArn: ...blue serviceName: hello_world-1 .... - clusterArn: ...red serviceName: hello_world-1 ...

I guess we should keep this structure.
And once cluster name is omit, details should be automatically set to true

- name: test register: out ecs_service_info: cluster: "{{ omit }}" details: true

This won't break any backwards compatiblity and won't return any ambiguous results in case a service name exist in more than one cluster.

What do you think @b0tting

tremble · 2024-03-23T08:23:51Z

hm, the failing CI jobs are not related to this PR

Fixed by #2059

Short version, there was a bug in the sanity tests, fix was merged a while back but it too time before milestone was updated.

softwarefactory-project-zuul · 2024-04-01T23:25:28Z

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/b86cd2a0e24f42b399301b59543e60f4

❌ ansible-galaxy-importer FAILURE in 4m 47s (non-voting)
✔️ build-ansible-collection SUCCESS in 14m 48s
✔️ ansible-test-splitter SUCCESS in 6m 46s
✔️ integration-community.aws-1 SUCCESS in 18m 12s
Skipped 21 jobs

markotting added 2 commits March 18, 2024 12:47

Add cluster(less) query to ecs_service_info

4de8ebd

Added test to validate commits with no cluster

eb16fcf

b0tting changed the title ~~[WIP] ecs_service_info lists all services in all clusters when no expliciet cluster is given~~ [WIP] ecs_service_info lists all services in all clusters when no cluster attribute is given Mar 18, 2024

b0tting changed the title ~~[WIP] ecs_service_info lists all services in all clusters when no cluster attribute is given~~ ecs_service_info lists all services in all clusters when no cluster attribute is given Mar 18, 2024

b0tting changed the title ~~ecs_service_info lists all services in all clusters when no cluster attribute is given~~ [WIP] ecs_service_info lists all services in all clusters when no cluster attribute is given Mar 18, 2024

Fixed not being able to deal with service names

d381118

b0tting changed the title ~~[WIP] ecs_service_info lists all services in all clusters when no cluster attribute is given~~ ecs_service_info lists all services in all clusters when no cluster attribute is given Mar 18, 2024

Fixed using the wrong service name in tests

c1028cf

Modified integration test. Given that we can now pass multiple cluste…

0d1ebee

…rs, returning a non-existent service in a cluster result does not make sense.

Last fix to integration tests

253a097

markuman reviewed Mar 19, 2024

View reviewed changes

plugins/modules/ecs_service_info.py Outdated Show resolved Hide resolved

markuman reviewed Mar 19, 2024

View reviewed changes

plugins/modules/ecs_service_info.py Outdated Show resolved Hide resolved

markuman reviewed Mar 19, 2024

View reviewed changes

plugins/modules/ecs_service_info.py Outdated Show resolved Hide resolved

b0tting and others added 3 commits March 19, 2024 17:52

Update plugins/modules/ecs_service_info.py

eed692d

Co-authored-by: Markus Bergholz <git@osuv.de>

Update plugins/modules/ecs_service_info.py

f63e625

Co-authored-by: Markus Bergholz <git@osuv.de>

Update plugins/modules/ecs_service_info.py

049e728

Co-authored-by: Markus Bergholz <git@osuv.de>

markuman reviewed Mar 21, 2024

View reviewed changes

plugins/modules/ecs_service_info.py Outdated Show resolved Hide resolved

markuman reviewed Mar 21, 2024

View reviewed changes

Removed explicit error when no clusters or services exist

456012c

Merge branch 'main' of github.com:b0tting/community.aws into main

2dc19a3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ecs_service_info lists all services in all clusters when no cluster attribute is given #2058

ecs_service_info lists all services in all clusters when no cluster attribute is given #2058

b0tting commented Mar 18, 2024 •

edited

softwarefactory-project-zuul bot commented Mar 18, 2024

github-actions bot commented Mar 18, 2024 •

edited

softwarefactory-project-zuul bot commented Mar 18, 2024

softwarefactory-project-zuul bot commented Mar 18, 2024

softwarefactory-project-zuul bot commented Mar 18, 2024

softwarefactory-project-zuul bot commented Mar 19, 2024

softwarefactory-project-zuul bot commented Mar 19, 2024

markuman commented Mar 20, 2024

softwarefactory-project-zuul bot commented Mar 20, 2024

markuman commented Mar 21, 2024

markuman Mar 21, 2024

b0tting Mar 21, 2024

markuman Mar 23, 2024

b0tting Apr 1, 2024 •

edited

markuman Apr 2, 2024

markuman Apr 10, 2024

tremble commented Mar 23, 2024

softwarefactory-project-zuul bot commented Apr 1, 2024

ecs_service_info lists all services in all clusters when no cluster attribute is given #2058

Are you sure you want to change the base?

ecs_service_info lists all services in all clusters when no cluster attribute is given #2058

Conversation

b0tting commented Mar 18, 2024 • edited

SUMMARY

ISSUE TYPE

COMPONENT NAME

ADDITIONAL INFORMATION

softwarefactory-project-zuul bot commented Mar 18, 2024

github-actions bot commented Mar 18, 2024 • edited

Docs Build 📝

softwarefactory-project-zuul bot commented Mar 18, 2024

softwarefactory-project-zuul bot commented Mar 18, 2024

softwarefactory-project-zuul bot commented Mar 18, 2024

softwarefactory-project-zuul bot commented Mar 19, 2024

softwarefactory-project-zuul bot commented Mar 19, 2024

markuman commented Mar 20, 2024

softwarefactory-project-zuul bot commented Mar 20, 2024

markuman commented Mar 21, 2024

markuman Mar 21, 2024

Choose a reason for hiding this comment

b0tting Mar 21, 2024

Choose a reason for hiding this comment

markuman Mar 23, 2024

Choose a reason for hiding this comment

b0tting Apr 1, 2024 • edited

Choose a reason for hiding this comment

markuman Apr 2, 2024

Choose a reason for hiding this comment

markuman Apr 10, 2024

Choose a reason for hiding this comment

tremble commented Mar 23, 2024

softwarefactory-project-zuul bot commented Apr 1, 2024

b0tting commented Mar 18, 2024 •

edited

github-actions bot commented Mar 18, 2024 •

edited

b0tting Apr 1, 2024 •

edited