ytt converts floating point numbers to integers

[maxwell-k]

What steps did you take:

ytt appears to convert input floats to integers, for example, running the command below:

  echo 1.0 | ytt --file=-

I am passing ytt a single floating point number - 1.0 - encoded as YAML.

What happened:

The output from ytt is an integer:

1

What did you expect:

I expected the output to be a float:

1.0

Anything else you would like to add:

I understand that integer and floating point data types differ in the YAML spec. This is a simplified example from input to an application. That application behaves differently if its YAML input contains an integer or a float. In the actual use case these numbers are embedded inside more complex structures.

Environment:

Version:

% ytt --version
 ytt version 0.45.0

OS Fedora Linux 26 on AMD 64.

/etc/os-release

NAME="Fedora Linux"
VERSION="36 (Workstation Edition)"
ID=fedora
VERSION_ID=36
VERSION_CODENAME=""
PLATFORM_ID="platform:f36"
PRETTY_NAME="Fedora Linux 36 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:36"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f36/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=36
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=36
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
SUPPORT_END=2023-05-16
VARIANT="Workstation Edition"
VARIANT_ID=workstation

---

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

[github-actions]

This issue is being marked as stale due to a long period of inactivity and will be closed in 5 days if there is no response.

[100mik]

There seems to have been prior discussion on this as well on this slack thread. And is actually related to how the yaml.v2 library we use to marshal our YAML treats floating points, a user had previously created an issue related to this here.

The key thoughts that would be worth capturing here as well would be:

1. "In 1.2.2 yaml, floating regex seems to have changed again making .X optional", which should mean that the YAML value can be loaded as a float even if its missing .X
2. "So if an application is behaving in a different way when someone enters "42" or "42.0", that sounds like a bug worth fixing in the application."
   • Is the consumer of ytts output unable to differentiate between these two values in your case?

I ran a few cases real quick through ytt, to see how the language behaves if we drop the .X and ytt seems to be able to realise when a float-like representation is required.

Input:

a: #@ 1.0
b: #@ 1.1
c: #@ 1.0 + 2
d: #@ 1.0 + 1.1
e: #@ 2.0 * 2.1

Output:

a: 1
b: 1.1
c: 3
d: 2.1
e: 4.2

[100mik]

I switched up the tags as we are gathering more information here.
What would be useful to understand is, if this is blocking you from using ytt for your needs? This would help us prioritise this concern!

Furthermore, it is worth adding that we are thinking about moving from yaml.v2 => yaml.v3 for our marshalling needs as we would like to support emitting comments (#63 ). And judging from this comment. It should also ensure that original formatting is retained while marshalling 🚀

[maxwell-k]

Thank you for looking at this. I understand that you are relying on the underlying library. Moving yaml.v2 => yaml.v3 sounds like an interesting solution.

What would be useful to understand is, if this is blocking you from using ytt for your needs? This would help us prioritise this concern!

This issue is currently blocking our use of ytt.

To explain a few more details of the use case: we're trying to use ytt overlays to change parameters before we run a vessel simulation. ytt is perfect for succinctly describing the edits (or you could call them patches or overlays) that we need to apply before each run.

The problem is that as well as applying the overlay as expected, ytt unexpectedly converts certain floats to integers. This unexpected change causes a part of our application to error.

Reading the links that you provided we could perhaps tag the input as a workaround. It seems that ytt also converts certain floats that are explicitly tagged to be floats to integers:

Input

a: !!float 1.0
b: !!float 1
c: !!float 1.1

Output

a: 1
b: 1
c: 1.1

[vmunishwar]

@maxwell-k Thanks for the details on your use case. As mentioned above, we are soon planning on moving to yaml.v3 in certain scenarios and that would allow preserving the intermediate representation in the original format.
Meanwhile, looks like the explicit tagging with !!float would work on most cases except for floats like 1.0, 2.0, 3.0 where it gets converted to integer.

Here are few examples I ran through the ytt playground.

We would be happy to discuss any further questions if you have. Thanks for reaching out!

[maxwell-k]

Thanks @vmunishwar ; it is d and h that block our use of ytt.

If h resulted in 1.0 we could workaround the unpredictable result from d. Ideally neither d nor h would result in an integer data type.

I guess we'll wait for the move to yaml.v3; thank you.