Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ubuntu/devel #1921

Merged
merged 36 commits into from Dec 16, 2022
Merged

Ubuntu/devel #1921

merged 36 commits into from Dec 16, 2022

Conversation

blackboxsw
Copy link
Collaborator

@blackboxsw blackboxsw commented Dec 15, 2022

new-upstream-snapshot using canonical/uss-tableflip#103 and manual merge conflict resolution to fix daily build recipes for lunar

Proposed Commit Message

do not squash merge

Additional Context

Steps to generate this PR

git clone -b new-new-upstream-snapshot git@github.com:TheRealFalcon/uss-tableflip.git
# add uss-tableflip/scripts to your $PATH
cd /tmp
git clone -b ubuntu/devel git@github.com:canonical/cloud-init.git -o upstream
cd cloud-init
new_upstream_snapshot.py # expect failure
git mergetool --tool=vimdiff   # fix the merge conflict,  :wq,  :xa
git add doc-requirements.txt 
git merge --continue
new_upstream_snapshot.py -p merge
# PROFIT

Test Steps

Checklist:

  • My code follows the process laid out in the documentation
  • I have updated or added any unit tests accordingly
  • I have updated or added any documentation accordingly

s-makin and others added 30 commits November 22, 2022 13:34
- index.rst: changed contributing link to :ref:
- found CONTRIBUTING.rst had rtd.io links still in it
- added labels to several topic pages to fix that issue
- changed all of those to :ref: links
Since 3.12 is the development release, failures of it
shouldn't fail the overall build.
Handles any situation where `status.json` does not yet contain
datasource information, by gracefully fulfilling a 
`datasource=None` in `StatusDetails`.

LP: #1997559
Bump the version in cloudinit/version.py to 22.4.2 and
update ChangeLog.
Currently, FreeBSD ignores IPv6 addresses. This PR adds support for static IPv6 addresses
Remove managed key in network config on LXD preseed configs
as it is not a valid key in LXD > v4.
As previously discussed, the Analyze page contains the content we want
to present under the "Performance" topic and should be uprated.

I have changed the title of the "Analyze" page to "Performance" so
that it shows correctly in the LHS menu. The anchors are the same,
and I have not changed the file name so as to avoid having to update
any links.

I took the opportunity to make some minor tidying edits to the text.
Hopefully these will add clarity and not change the intended meaning.
Currently, Alibaba cloud provides a security hardening mode for its
metadata server, which is alike IMDSv2, and we should support it.

Detailed information be found here:
https://www.alibabacloud.com/help/en/elastic-compute-service/latest/view-instance-metadata#concept-dwj-y1x-wgb
All components of cloudinit need to run, and in a specific order.
If cloudinit is to be enabled, it should only rely on one variable.
This change better encodes that, than canonical#161

Sponsored by: FreeBSD Foundation
This commit adds Rocky Linux, AlmaLinux, CloudLinux, EuroLinux, Miracle
Linux, and Virtuozzo to the is_rhel list. Recent downstream patch from
Red Hat causes issues with RHEL derivatives with the cloud.cfg template,
which leads to derivatives having to make small changes to bring back
expected functionality.
overriding __init__() means we need to call super().__init__()

Sponsored By: FreeBSD Foundation
…#1888)

* Add support for setting uid when creating users on FreeBSD

* Test if uid in config is used when creating user in FreeBSD

* Merge two branches in if statement.

Unit test uid in pw call for create user on freebsd.
Guide the user through setting up a virtual machine.
Introduce commonly used terminology and architecture.
Include a debugging page and example script to guide
the audience through pitfalls.
Networkd supports multiple [Route] sections within the same file.
Currently all [Route] section tags are squashed into one and if there
is a default gateway it means defining a device route is not possible
as the target is set to the default gateway.

This patch adds support for multiple [Route] sections allowing us to
support device routes. This is done by tracking each route in the route
list individually and ensuring the key-value pairs are maintained within
their respective [Route] section. This both maintains backwards
compatibility with previous behavior and allows the specification of
routes with no destination IP, causing the destination to be added with
a device target.
If utf-8 characters are used in the report, it will fail to encode:

azure.py[ERROR]: exception while reporting ready: 'latin-1' codec can't
encode characters in position 392-397:
Body ('乱写一些单词') is not valid Latin-1.
Use body.encode('utf-8') if you want to send it encoded in UTF-8.

Explicitly encode document as utf-8.

Signed-off-by: Chris Patterson cpatterson@microsoft.com
Add autoinstall pointer to subiquity docs to FAQ.

Remove cc_ubuntu_autoinstall from docs module definitions
since it doesn't give users actionable information.
On netplan systems, network v2 is passed directly though and written
to /etc/netplan/50-cloud-init.yaml without validation. Current
netplan configuration provides the ability to configure sensitive
information such as `wifi:access-points:password`.

Limit permissions for /etc/network/50-cloud-init.yaml as read-only
for root (600). Since configuration or modification or netplan config
needs to be performed by an admin user this permission restriction
aligns with netplan tooling.

Set root read-only only always and not just 'if' sensitive material
exists within custom config because it will add confusion to have
two expected modes for this file based on external conditions.
Avoid breakage on newer Azure SDK versions
Bump commit to c9db5bfc
net: clarify network function name and docstring
To make retaining original behavior in stable downstreams easier,
provide a feature flag NETPLAN_CONFIG_ROOT_READ_ONLY so
/etc/netplan/50-cloud-init.yaml config can remain unchanged as
world-readable.

Set this flag False to ensure world-readable 50-cloud-init.yaml.

Add tests.integration_tests.util.get_feature_flag to extract
feature values from cloudinit.features on test system.

Co-authored-by: James Falcon <james.falcon@canonical.com>
Use the sphinx-copybutton extension.
Separate commands from output for ease of use.
Currently tests attempt to delete /run/dhclient.pid and
/run/dhclient.leases from the host. This leads to host-dependent test
failure. Fix it.
Add coverage for:
- atomic_helper.write_file()
- util.human2bytes()
Add test and support for parsing IEC prefix format.
aciba90 and others added 3 commits December 15, 2022 17:13
NoCloud meta-data seedfrom (or kernel commandline seedfrom) URL can
now provide variable expansion for system-specific DMI values as seen
in /sys/class/dmi/id on Linux or kenv on FreeBSD platforms.

Variable names of the format __dmi.SOME_VAR__ will be replaced when
determining the URL from which NoCloud datasource GETs its user-data
and meta-data.

This allows for a common templated seedfrom URL which can be reused
for mass deployments, but can allow for unique URLs based on classes
of DMI system characteristics such as chassis serial, product name,
UUID etc.

LP: #1994980
@holmanb
Copy link
Member

holmanb commented Dec 15, 2022

Compared against the old script:

git fetch upstream
git checkout ubuntu/devel
git reset --hard upstream/ubuntu/devel
new-upstream-snapshot
<resolve> 
git merge --continue

the remaining diff is changelog-related, so I think our git bits are in order

diff --git a/debian/changelog b/debian/changelog
index 25d4fd919..1c32173b1 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,11 +1,3 @@
-cloud-init (22.4.2-0ubuntu2) UNRELEASED; urgency=medium
-
-  * Upstream snapshot based on upstream/main at 040090a3.
-    - Bugs fixed in this snapshot: (LP: #1994980, #1998655, #1999042)
-      (LP: #1997559)
-
- -- Chad Smith <chad.smith@canonical.com>  Thu, 15 Dec 2022 14:52:03 -0700
-
 cloud-init (22.4.2-0ubuntu1) lunar; urgency=medium
   
 * Upstream snapshot based on upstream commit 893df0d6.

@blackboxsw
Copy link
Collaborator Author

blackboxsw commented Dec 15, 2022

Something in that diff isn't right,
with the old tooling, we'd fall into the refresh-fix shell, you'd need to fix, then git add doc-requirements.txt; git merge --continue; exit 0. Then you'd continue to setting up the d/changelog with a bulk of individual commits listed in d/changelog.

Here's the diff I get with old tool vs new tool (and abbreviated d/changelog)

diff --git a/debian/changelog b/debian/changelog
index 25d4fd91..fe15ce5c 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,10 +1,48 @@
-cloud-init (22.4.2-0ubuntu2) UNRELEASED; urgency=medium
-
-  * Upstream snapshot based on upstream/main at 040090a3.
-    - Bugs fixed in this snapshot: (LP: #1994980, #1998655, #1999042)
-      (LP: #1997559)
-
- -- Chad Smith <chad.smith@canonical.com>  Thu, 15 Dec 2022 14:52:03 -0700
+cloud-init (22.4-49-g040090a3-0ubuntu1) lunar; urgency=medium
+
+  * New upstream snapshot.
+    + nocloud: add support for dmi variable expansion for seedfrom URL
+    + tools: read-version drop extra call to git describe --long
+    + doc: improve cc_write_files doc (#1916)
+    + read-version: When insufficient tags, use
+    cloudinit.version.get_version
+    + mounts: document weird prefix in schema (#1913)
+    + add utility function test cases (#1910) [sxt1001]
+    + test: mock file deletion in dhcp tests (#1911)
+    + Ensure network ready before cloud-init service runs on RHEL (#1893)
+    + docs: add copy button to code blocks (#1890) [s-makin]
+    + netplan: define features.NETPLAN_CONFIG_ROOT_READ_ONLY flag
+    + azure: fix support for systems without az command installed (#1908)
+    + Networking Clarification (#1892)
+    + Fix the distro.osfamily output problem in the openEuler system.
+      (#1895) [sxt1001]
+    + pycloudlib: bump commit dropping azure api smoke test
+    + * net: netplan config root read-only as wifi config can contain creds
+    + autoinstall: clarify docs for users
+    + sources/azure: encode health report as utf-8 (#1897) [Chris Patterson]
+    + Add back gateway4/6 deprecation to docs (#1898)
+    + networkd: Add support for multiple [Route] sections (#1868)
+      [Nigel Kukard]
+    + doc: add qemu tutorial (#1863)
+    + lint: fix tip-flake8 and tip-mypy (#1896)
+    + Add support for setting uid when creating users on FreeBSD (#1888)
+      [einsibjarni]
+    + Fix exception in BSD networking code-path (#1894) [Mina Galić]
+    + Append derivatives to is_rhel list in cloud.cfg.tmpl (#1887)
+      [Louis Abel]
+    + FreeBSD init: use cloudinit_enable as only rcvar (#1875) [Mina Galić]
+    + feat: add support aliyun metadata security harden mode (#1865)
+      [Manasseh Zhou]
+    + docs: uprate analyze to performance page [s-makin]
+    + test: fix lxd preseed managed network config (#1881)
+    + Add support for static IPv6 addresses for FreeBSD (#1839)
+      [einsibjarni]
+    + Release 22.4.2 (#1878)
+    + status: handle ds not defined in status.json (#1876)
+    + Make 3.12 failures not fail the build (#1873)
+    + Docs: adding relative links [s-makin]
+
+ -- Chad Smith <chad.smith@canonical.com>  Thu, 15 Dec 2022 15:43:08 -0700
 
 cloud-init (22.4.2-0ubuntu1) lunar; urgency=medium

@holmanb
Copy link
Member

holmanb commented Dec 15, 2022

Reproducing your steps now with new-new:

  diff --git a/debian/changelog b/debian/changelog
  index 210289536..da9fa88ba 100644
  --- a/debian/changelog
  +++ b/debian/changelog
  @@ -4,7 +4,7 @@ cloud-init (22.4.2-0ubuntu2) lunar; urgency=medium
       - Bugs fixed in this snapshot: (LP: #1994980, #1998655, #1999042)
         (LP: #1997559)
   
  - -- Chad Smith <chad.smith@canonical.com>  Thu, 15 Dec 2022 16:11:37 -0700
  + -- Brett Holman <brett.holman@canonical.com>  Thu, 15 Dec 2022 16:21:29 -0700
   
   cloud-init (22.4.2-0ubuntu1) lunar; urgency=medium
   

@holmanb
Copy link
Member

holmanb commented Dec 15, 2022

+1 Also reproducing this now with the old tooling.

Copy link
Contributor

@aciba90 aciba90 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Lunar sbuild succeeded with no lintian errors.

cloud-init (22.4.2-0ubuntu2) lunar; urgency=medium

* Upstream snapshot based on upstream/main at 040090a3.
- Bugs fixed in this snapshot: (LP: #1994980, #1998655, #1999042)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this line wrapping expected? I would expect something like:

    - Bugs fixed in this snapshot: (LP: #1994980, #1998655, #1999042,
    #1997559)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this 'formatting' is intentional because the changes file which is generated by dpkg-buildpackage has a limited regex for matching Launchpad-Bugs-Fixed: that cannot parse multi-line LP: #123,\n#456. So a build with this mult-line suggestion would redact 1997559 from the Launchpad-Bugs-Fixed: in the sources file.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this existing d/changelog we get the following in ../out/cloud-init_22.4.2-0ubuntu2_source.changes:
Launchpad-Bugs-Fixed: 1994980 1997559 1998655 1999042

with your amended suggestion:
Launchpad-Bugs-Fixed: 1994980 1998655 1999042

@blackboxsw blackboxsw merged commit c13bf73 into canonical:ubuntu/devel Dec 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet