Docs [2.10] Backportapalooza 6 (#71129)

* Misc typo fixes (#71089) (cherry picked from commit 504ef60) * Add some documentation for the format of meta/runtime.yml (#71035) * Document the format of meta/runtime.yml * Document multiple Ansible versions Clarify difference between deprecation and tombstone fields * add note (cherry picked from commit a9eb8b0) * add note to uninstall older versions of ansible for pip (#71023) * add note to uninstall older versions of ansible for pip * combine with the other PR (cherry picked from commit 72d3d44) * VMware: Inventory scenario guide for hostnames (#71055) Added a scenario guide for ``hostnames`` parameter for vmware_vm_inventory. Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com> (cherry picked from commit 0055673) * Document string tests a bit more (#71049) - Explain how `regex` differs from `match` and `search`. - Document `multiline` and `ignorecase`. Signed-off-by: Rick Elrod <rick@elrod.me> (cherry picked from commit 701c638) * docs: Add a note about package requirements for fact gathering (#70796) Fixes: #26148 Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com> (cherry picked from commit a6725d6) * added note about fakeroot (#71018) see #70895 (cherry picked from commit 11a31e9) * Update documentation of httpapi's handle_httperror method for clarity (#70991) (cherry picked from commit a0523e5) * DOCS: add 2.10 collections roadmap (#70975) * draft of 2.10 collections roadmap * incorporates feedback from felixfontein * gundalow and samccann feedback, fix link Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com> (cherry picked from commit 9879da8) * updates changelog types; some updates for easier translation (#71027) Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com> Co-authored-by: Felix Fontein <felix@fontein.de> (cherry picked from commit 4f4436c) * Document common return values with examples (#71046) * adding return value examples * shift to console code blocks * cleaning up whitespace and shortening invocation example * reordering diff section (cherry picked from commit 864573a) * Update intro_getting_started.rst (#71039) Added two additional learning resources in the See also: section- forgot closing backticks (cherry picked from commit 9850915) * Guide users to use ansible-runner (#71063) Update the docs to guide users to use `ansible-runner` instead of using Python API directly. In many use cases, executing Ansible playbooks are sufficient. In those use cases, `ansible-runner` is easier and much stable to use comparing with Python API, but there is no mention of it. (cherry picked from commit 0c855dc) * Porting guides for ansible-base 2.10 and ansible 2.10 (#70891) * Fix changelog link title. * Rename Ansible 2.10 and 2.11 porting guides to Ansible-base porting guides. * Add stub for automatically generated 2.10 porting guide. * Move things that should not be in the ansible-base porting guide to the ansible porting guide. * Apply changes to base porting guides. * Add remark that ansible-base is mainly for developers. * Ansible Base -> Ansible-base * Fix link in base porting guide. * Add generated porting guide. * Use same header signs as antsibull-changelog's RST builder. * Update generated porting guide. (cherry picked from commit 61b36c6) * Update network platform guides with FQCN (#70699) * fqcn all the docs things! (cherry picked from commit 54bee71) * Document how to upgrade to ansible with pip (#70768) Fixes #70348 (cherry picked from commit 5019335) * document how to migrate between collections (#70243) * document how to migrate between collections * Apply suggestions from code review Co-authored-by: John R Barker <john@johnrbarker.com> Co-authored-by: Felix Fontein <felix@fontein.de> (cherry picked from commit 58145df) * remove github link from plugins (#70951) (cherry picked from commit e28b20d) * Add latest rc from ansible-base (#70974) * Add latest rc from ansible-base (cherry picked from commit d62dffa) * Document to_json will convert to ASCII strings by default (#70954) ... as reported in issue #68702 (cherry picked from commit 8c48366) * Update the porting guide for ansible-2.10.0a8 (#71141) (cherry picked from commit 0a9638c) Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com> Co-authored-by: Sloane Hertel <shertel@redhat.com> Co-authored-by: Rick Elrod <rick@elrod.me> Co-authored-by: Brian Coca <bcoca@users.noreply.github.com> Co-authored-by: Nathaniel Case <ncase@redhat.com> Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com> Co-authored-by: Terciero <terciero@users.noreply.github.com> Co-authored-by: Brendon O'Sullivan <49501251+bjosullivan@users.noreply.github.com> Co-authored-by: EthanHur <ethan0311@gmail.com> Co-authored-by: Felix Fontein <felix@fontein.de> Co-authored-by: Baptiste Mille-Mathias <baptiste.millemathias@gmail.com> Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
ansible · Aug 7, 2020 · b9a406f · b9a406f
1 parent 4b03d89
commit b9a406f
Show file tree

Hide file tree

Showing 42 changed files with 1,052 additions and 305 deletions.
diff --git a/docs/docsite/_themes/sphinx_rtd_theme/breadcrumbs.html b/docs/docsite/_themes/sphinx_rtd_theme/breadcrumbs.html
@@ -44,10 +44,14 @@
               <a href="{{ meta['github_url'] }}" class="fa fa-github"> {{ _('Edit on GitHub') }}</a>
             {% else %}
               <!-- Ansible-specific additions for modules etc -->
-                {% if pagename.endswith('_module') %}
-                  <a href="https://{{ github_host|default("github.com") }}/{{ github_user }}/{{ github_repo }}/{{ theme_vcs_pageview_mode|default("blob") }}/{{ github_module_version }}{{ meta.get('source', '') }}?description=%23%23%23%23%23%20SUMMARY%0A%3C!---%20Your%20description%20here%20--%3E%0A%0A%0A%23%23%23%23%23%20ISSUE%20TYPE%0A-%20Docs%20Pull%20Request%0A%0A%2Blabel:%20docsite_pr" class="fa fa-github"> {{ _('Edit on GitHub') }}</a>
-                {% elif pagename.startswith('plugins') and meta.get('source', None) %}
-                  <a href="https://{{ github_host|default("github.com") }}/{{ github_user }}/{{ github_repo }}/{{ theme_vcs_pageview_mode|default("blob") }}/{{ github_root_dir }}/{{ pagename }}.py?description=%23%23%23%23%23%20SUMMARY%0A%3C!---%20Your%20description%20here%20--%3E%0A%0A%0A%23%23%23%23%23%20ISSUE%20TYPE%0A-%20Docs%20Pull%20Request%0A%0A%2Blabel:%20docsite_pr" class="fa fa-github"> {{ _('Edit on GitHub') }}</a>
+                {% if (pagename.endswith('_module')) or (pagename.endswith('_become'))
+                  or (pagename.endswith('_cache')) or (pagename.endswith('_callback'))
+                  or (pagename.endswith('_connection')) or (pagename.endswith('_inventory'))
+                  or (pagename.endswith('_lookup')) or (pagename.endswith('_shell'))
+                  or (pagename.endswith('_strategy')) or (pagename.endswith('_vars'))
+                 %}
+                <!--  <a href="https://{{ github_host|default("github.com") }}/{{ github_user }}/{{ github_repo }}/{{ theme_vcs_pageview_mode|default("blob") }}/{{ github_module_version }}{{ meta.get('source', '') }}?description=%23%23%23%23%23%20SUMMARY%0A%3C!---%20Your%20description%20here%20--%3E%0A%0A%0A%23%23%23%23%23%20ISSUE%20TYPE%0A-%20Docs%20Pull%20Request%0A%0A%2Blabel:%20docsite_pr" class="fa fa-github"> {{ _('Edit on GitHub') }}</a> -->
+                  <br>
                 {% elif pagename.startswith('cli') and meta.get('source', None) %}
                   <a href="https://{{ github_host|default("github.com") }}/{{ github_user }}/{{ github_repo }}/{{ theme_vcs_pageview_mode|default("blob") }}/{{ github_cli_version }}{{ meta.get('source', '') }}?description=%23%23%23%23%23%20SUMMARY%0A%3C!---%20Your%20description%20here%20--%3E%0A%0A%0A%23%23%23%23%23%20ISSUE%20TYPE%0A-%20Docs%20Pull%20Request%0A%0A%2Blabel:%20docsite_pr" class="fa fa-github"> {{ _('Edit on GitHub') }}</a>
                 {% elif (not 'list_of' in pagename) and (not 'category' in pagename) %}

diff --git a/docs/docsite/rst/community/development_process.rst b/docs/docsite/rst/community/development_process.rst
@@ -43,7 +43,7 @@ Here's an overview of the PR lifecycle:
 Automated PR review: ansibullbot
 --------------------------------
 
-Because Ansible receives many pull requests, and because we love automating things, we've automated several steps of the process of reviewing and merging pull requests with a tool called Ansibullbot, or Ansibot for short.
+Because Ansible receives many pull requests, and because we love automating things, we have automated several steps of the process of reviewing and merging pull requests with a tool called Ansibullbot, or Ansibot for short.
 
 `Ansibullbot <https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md>`_ serves many functions:
 
@@ -139,11 +139,7 @@ We don't merge every PR. Here are some tips for making your PR useful, attractiv
 Changelogs
 ----------
 
-Changelogs help users and developers keep up with changes to Ansible.
-Ansible builds a changelog for each release from fragments.
-You **must** add a changelog fragment to any PR that changes functionality or fixes a bug in ansible-base.
-You don't have to add a changelog fragment for PRs that add new
-modules and plugins, because our tooling does that for you automatically.
+Changelogs help users and developers keep up with changes to Ansible. Ansible builds a changelog for each release from fragments. You **must** add a changelog fragment to any PR that changes functionality or fixes a bug in ansible-base. You do not have to add a changelog fragment for PRs that add new modules and plugins, because our tooling does that for you automatically.
 
 We build short summary changelogs for minor releases as well as for major releases. If you backport a bugfix, include a changelog fragment with the backport PR.
 
@@ -152,44 +148,37 @@ We build short summary changelogs for minor releases as well as for major releas
 Creating a changelog fragment
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-A basic changelog fragment is a ``.yaml`` file placed in the
-``changelogs/fragments/`` directory.  Each file contains a yaml dict with
-keys like ``bugfixes`` or ``major_changes`` followed by a list of
-changelog entries of bugfixes or features.  Each changelog entry is
-rst embedded inside of the yaml file which means that certain
-constructs would need to be escaped so they can be interpreted by rst
-and not by yaml (or escaped for both yaml and rst if that's your
-desire).  Each PR **must** use a new fragment file rather than adding to
-an existing one, so we can trace the change back to the PR that introduced it.
+A basic changelog fragment is a ``.yaml`` file placed in the ``changelogs/fragments/`` directory.  Each file contains a yaml dict with keys like ``bugfixes`` or ``major_changes`` followed by a list of changelog entries of bugfixes or features.  Each changelog entry is rst embedded inside of the yaml file which means that certain constructs would need to be escaped so they can be interpreted by rst and not by yaml (or escaped for both yaml and rst if you prefer).  Each PR **must** use a new fragment file rather than adding to an existing one, so we can trace the change back to the PR that introduced it.
 
-To create a changelog entry, create a new file with a unique name in the ``changelogs/fragments/`` directory of corresponding repository.
-The file name should include the PR number and a description of the change.
-It must end with the file extension ``.yaml``. For example: ``40696-user-backup-shadow-file.yaml``
+To create a changelog entry, create a new file with a unique name in the ``changelogs/fragments/`` directory of the corresponding repository. The file name should include the PR number and a description of the change. It must end with the file extension ``.yaml``. For example: ``40696-user-backup-shadow-file.yaml``
 
-A single changelog fragment may contain multiple sections but most will only contain one section.
-The toplevel keys (bugfixes, major_changes, and so on) are defined in the
-`config file <https://github.com/ansible/ansible/blob/devel/changelogs/config.yaml>`_ for our release note tool. Here are the valid sections and a description of each:
+A single changelog fragment may contain multiple sections but most will only contain one section. The toplevel keys (bugfixes, major_changes, and so on) are defined in the `config file <https://github.com/ansible/ansible/blob/devel/changelogs/config.yaml>`_ for our `release note tool <https://github.com/ansible-community/antsibull-changelog/blob/main/docs/changelogs.rst>`_. Here are the valid sections and a description of each:
+
+**breaking_changes**
+  Changes that break existing playbooks or roles. This includes any change to existing behavior that forces users to update tasks. Displayed in both the changelogs and the :ref:`Porting Guides <porting_guides>`.
 
 **major_changes**
-  Major changes to Ansible itself. Generally does not include module or plugin changes.
+  Major changes to Ansible itself. Generally does not include module or plugin changes. Displayed in both the changelogs and the :ref:`Porting Guides <porting_guides>`.
 
 **minor_changes**
   Minor changes to Ansible, modules, or plugins. This includes new features, new parameters added to modules, or behavior changes to existing parameters.
 
 **deprecated_features**
-  Features that have been deprecated and are scheduled for removal in a future release.
+  Features that have been deprecated and are scheduled for removal in a future release. Displayed in both the changelogs and the :ref:`Porting Guides <porting_guides>`.
 
 **removed_features**
-  Features that were previously deprecated and are now removed.
+  Features that were previously deprecated and are now removed. Displayed in both the changelogs and the :ref:`Porting Guides <porting_guides>`.
+
+**security_fixes**
+  Fixes that address CVEs or resolve security concerns. Include links to CVE information.
 
 **bugfixes**
   Fixes that resolve issues.
 
 **known_issues**
   Known issues that are currently not fixed or will not be fixed.
 
-Each changelog entry must contain a link to its issue between parentheses at the end.
-If there is no corresponding issue, the entry must contain a link to the PR itself.
+Each changelog entry must contain a link to its issue between parentheses at the end. If there is no corresponding issue, the entry must contain a link to the PR itself.
 
 Most changelog entries will be ``bugfixes`` or ``minor_changes``. When writing a changelog entry that pertains to a particular module, start the entry with ``- [module name] -`` and the following sentence with a lowercase letter.
 
@@ -222,9 +211,7 @@ Once you've written the changelog fragment for your PR, commit the file and incl
 Backporting merged PRs
 ======================
 
-All Ansible PRs must be merged to the ``devel`` branch first.
-After a pull request has been accepted and merged to the ``devel`` branch, the following instructions will help you create a
-pull request to backport the change to a previous stable branch.
+All Ansible PRs must be merged to the ``devel`` branch first. After a pull request has been accepted and merged to the ``devel`` branch, the following instructions will help you create a pull request to backport the change to a previous stable branch.
 
 We do **not** backport features.
 

diff --git a/docs/docsite/rst/dev_guide/developing_api.rst b/docs/docsite/rst/dev_guide/developing_api.rst
@@ -6,7 +6,7 @@ Python API
 
 .. contents:: Topics
 
-.. note:: This API is intended for internal Ansible use. Ansible may make changes to this API at any time that could break backward compatibility with older versions of the API. Because of this, external use is not supported by Ansible.
+.. note:: This API is intended for internal Ansible use. Ansible may make changes to this API at any time that could break backward compatibility with older versions of the API. Because of this, external use is not supported by Ansible. If you want to use Python API only for executing playbooks or modules, consider `ansible-runner <https://ansible-runner.readthedocs.io/en/latest/>`_ first.
 
 There are several ways to use Ansible from an API perspective.   You can use
 the Ansible Python API to control nodes, you can extend Ansible to respond to various Python events, you can

diff --git a/docs/docsite/rst/dev_guide/developing_collections.rst b/docs/docsite/rst/dev_guide/developing_collections.rst
@@ -25,6 +25,8 @@ Collections follow a simple data structure. None of the directories are required
     collection/
     ├── docs/
     ├── galaxy.yml
+    ├── meta/
+    │   └── runtime.yml
     ├── plugins/
     │   ├── modules/
     │   │   └── module1.py
@@ -194,6 +196,64 @@ command completion, or environment variables are presented throughout the
 Ansible Collection Testing because the act of installing the stable release of
 Ansible containing `ansible-test` is expected to setup those things for you.
 
+.. _meta_runtime_yml:
+
+meta directory
+--------------
+
+A collection can store some additional metadata in a ``runtime.yml`` file in the collection's ``meta`` directory. The ``runtime.yml`` file supports the top level keys:
+
+- *requires_ansible*:
+
+  The version of Ansible required to use the collection. Multiple versions can be separated with a comma.
+
+  .. code:: yaml
+
+     requires_ansible: ">=2.10,<2.11"
+
+  .. note:: although the version is a `PEP440 Version Specifier <https://www.python.org/dev/peps/pep-0440/#version-specifiers>`_ under the hood, Ansible deviates from PEP440 behavior by truncating prerelease segments from the Ansible version. This means that Ansible 2.11.0b1 is compatible with something that ``requires_ansible: ">=2.11"``.
+
+- *plugin_routing*:
+
+  Content in a collection that Ansible needs to load from another location or that has been deprecated/removed.
+  The top level keys of ``plugin_routing`` are types of plugins, with individual plugin names as subkeys.
+  To define a new location for a plugin, set the ``redirect`` field to another name.
+  To deprecate a plugin, use the ``deprecation`` field to provide a custom warning message and the removal date or version. If the plugin has been renamed or moved to a new location, the ``redirect`` field should also be provided. If a plugin is being removed entirely, ``tombstone`` can be used for the fatal error message and removal date or version.
+
+  .. code:: yaml
+
+     plugin_routing:
+       inventory:
+         kubevirt:
+           redirect: community.general.kubevirt
+         my_inventory:
+           tombstone:
+             removal_version: "2.0.0"
+             warning_text: my_inventory has been removed. Please use other_inventory instead.
+       modules:
+         my_module:
+           deprecation:
+             removal_date: "2021-11-30"
+             warning_text: my_module will be removed in a future release of this collection. Use another.collection.new_module instead.
+           redirect: another.collection.new_module
+         podman_image:
+           redirect: containers.podman.podman_image
+       module_utils:
+         ec2:
+           redirect: amazon.aws.ec2
+         util_dir.subdir.my_util:
+           redirect: namespace.name.my_util
+
+- *import_redirection*
+
+  A mapping of names for Python import statements and their redirected locations.
+
+  .. code:: yaml
+
+     import_redirection:
+       ansible.module_utils.old_utility:
+         redirect: ansible_collections.collection_name.plugins.module_utils.new_location
+
 
 .. _creating_collections_skeleton:
 
@@ -469,24 +529,70 @@ Collection versions use `Semantic Versioning <https://semver.org/>`_ for version
 
 .. _migrate_to_collection:
 
-Migrating Ansible content to a collection
-=========================================
+Migrating Ansible content to a different collection
+====================================================
+
+To migrate content from one collection to another, you need to create three PRs as follows:
+
+#. Create a PR against the old collection to remove the content.
+#. Create a PR against the new collection to add the files removed in step 1.
+#. Update the ``ansible/ansible:devel`` branch entries for all files moved.
+
+
+Removing the content from the old collection
+----------------------------------------------
+
+Create a PR against the old collection repo to remove the modules, module_utils, plugins, and docs_fragments related to this migration:
+
+#. If you are removing an action plugin, remove the corresponding module that contains the documentation.
+#. If you are removing a module, remove any corresponding action plugin that should stay with it.
+#. Remove any entries about removed plugins from ``meta/runtime.yml``. Ensure they are added into the new repo.
+#. Remove sanity ignore lines from ``tests/sanity/ignore\*.txt``
+#. Remove associated integration tests from ``tests/integrations/targets/`` and unit tests from ``tests/units/plugins/``.
+#. if you are removing from content from ``community.general`` or ``community.network``, remove entries from ``.github/BOTMETA.yml``.
+#. Carefully review ``meta/runtime.yml`` for any entries you may need to remove or update, in particular deprecated entries.
+#. Update ``meta/runtime.yml`` to contain redirects for EVERY PLUGIN, pointing to the new collection name.
+#. If possible, do not yet add deprecation warnings to the new ``meta/runtime.yml`` entries, but only for a later major release. So the order should be:
+    1. Remove content, add redirects in 3.0.0;
+    2. Deprecate redirects in 4.0.0;
+    3. Set removal version to 5.0.0 or later.
 
-You can experiment with migrating existing modules into a collection using the `content_collector tool <https://github.com/ansible/content_collector>`_. The ``content_collector`` is a playbook that helps you migrate content from an Ansible distribution into a collection.
 
 .. warning::
 
-	This tool is in active development and is provided only for experimentation and feedback at this point.
+	Maintainers for the old collection have to make sure that the PR is merged in a way that it does not break user experience and semantic versioning:
+
+	#. A new version containing the merged PR must not be released before the collection the content has been moved to has been released again, with that content contained in it. Otherwise the redirects cannot work and users relying on that content will experience breakage.
+	#. Once 1.0.0 of the collection from which the content has been removed has been released, such PRs can only be merged for a new **major** version (i.e. 2.0.0, 3.0.0, etc.).
+
+
+Adding the content to the new collection
+-----------------------------------------
+
+Create a PR in the new collection to:
+
+#. Add ALL the files removed in first PR (from the old collection).
+#. If it is an action plugin, include the corresponding module with documentation.
+#. If it is a module, check if it has a corresponding action plugin that should move with it.
+#. Check ``meta/ `` for relevant updates to ``action_groups.yml`` and ``runtime.yml`` if they exist.
+#. Carefully check the moved ``tests/integration`` and ``tests/units`` and update for FQCN.
+#. Review ``tests/sanity/ignore-\*.txt`` entries.
+#. Update ``meta/runtime.yml``.
+
+Updating ``ansible/ansible:devel`` branch entries for all files moved
+----------------------------------------------------------------------
+
+Create a third PR on ``ansible/ansible`` repository to:
 
-See the `content_collector README <https://github.com/ansible/content_collector>`_ for full details and usage guidelines.
+#. Update ``lib/ansible/config/ansible_builtin_runtime.yml`` (the redirect entry).
+#. Update ``.github/BOTMETA.yml`` (the migrated_to entry)
 
 BOTMETA.yml
 -----------
 
 The `BOTMETA.yml <https://github.com/ansible/ansible/blob/devel/.github/BOTMETA.yml>`_ in the ansible/ansible GitHub repository is the source of truth for:
 
 * ansibullbot
-* the docs build for collections-based modules
 
 Ansibulbot will know how to redirect existing issues and PRs to the new repo.
 The build process for docs.ansible.com will know where to find the module docs.