Performance and inaccuracies of function fixedToFloat()

[luc-vanrompaey]

The docstring of function "fixedToFloat()" (in file 'Lib/fontTools/misc/fixedTools.py') warns about its poor performance, saying:
"This is pretty slow compared to a simple division. Use sporadically."

In addition, it will not always return an accurate result. For instance, when its value argument is set to 66191 (representing version 1.01), it will return 1.00999 instead; for an argument value of 137624 (i.e., version 2.10), it will return 2.09998; and there are more examples. (I actually found these example just by looking at the mscorefonts.)

I coded a version of the function that should solve both of these problems, and is much shorter than the original. It will return the correct floating-point value for any 16:16-bit fixed-point value with up to four decimal digits in its fractional part (more than four decimal digits won't fit in the 16 bits that are reserved for it anyway).

It does (what should be) the equivalent of a round(value/scale*10000), but in integer, instead of floating-point, arithmetic; then, it divides that result by 10000 (doing a floating-point division), to effectively move the minor version into the fractional part.

This is what I came up with:

def fixedToFloat(value, precisionBits):
	scale = 1 << precisionBits
	return (((value * 100000) // scale + 5) // 10) / 10000

Does this make sense?

[justvanrossum]

It makes sense except for the hard-coded number of digits. For example, your function doesn't do the right thing for 2.14 fixed numbers.

[luc-vanrompaey]

You mean, the number of decimal digits should be a function of the "precisionBits" argument value, right?
Now, the docstring says that "precisionBits is only supported up to 16", so I guess that greater values need not be considered. Also, at least four bits are required, because fewer bits cannot hold even a single decimal digit.
So, if I can make the function work correctly for "precisiobBits" values in the range from 4 through 16, then it will work right?

[luc-vanrompaey]

OK - So, I now get that the binary fixed-point data type is being used primarily for values that relate to the font metrics. Such values should not get rounded or truncated, because any change in such values, no matter how subtle, will affect the visual appearance of the fonts.
That the font version is stored as a binary fixed-point number is just accidental. We, humans, typically view version numbers as decimal values. The binary fixed-point representation of such values is inexact and must really be rounded in order to arrive at a value that is meaningful to us mere mortals. It most likely appears useful only when converted to string.
I think that sums up the situation. I was confused about the use of the fixed-point binary data type because I encountered them in the context of the font version values, where it's not a particularly natural format.

Anyway, I played around a little more with my version of the fixedToFloat() function, and came up with the following:

def fixedToFloat(value, precisionBits):
   scale   = 1 << precisionBits
   digits  = (precisionBits * 3) // 10
   scale10 = 10 ** digits
   return (((value * (scale10 * 10)) // scale + 5) // 10) / scale10

It calculates the number of decimal digits as the number of bits multiplied by 0.3. That is not universally correct (the exact multiplier should be log(2), which is close to 0.30103), but it works for any bit count up to something like 80.

I tested this version of the function for all bit counts from 4 to 19, and it calculated all values correctly. I couldn't go any higher than 19 bits, since my verification code was an awk script that constructed the expected strings with 5 significant digits, while 20 bits would require 6 decimal digits.

[anthrotype]

fixedToFloat (like floatToFixed) doesn't do any rounding, and is not slow

fonttools/Lib/fontTools/misc/fixedTools.py

Line 60 in c552a77

return value / (1 << precisionBits)

Those are used when decompiling/compiling from/to binary font. We decided to not round any more when converting fixedToFloat on decompile (or when parsing from XML), see #737 for more info.

I guess you meant fixedToStr (andfloatToFixedToStr ), which returns the shortest string representation of the decimal float that best approximates a fixed-point number with given precisionBits, using the least number of decimal digits. Those are used only when saving a TTFont to xml to avoid unnecessarily long floats in the TTX dump. They are relatively slower because they manipulate strings, but I believe it's acceptable.

[luc-vanrompaey]

Guess, then, that the function must have been rewritten since the version that I was looking at.
Version 3.21.2, as distributed with Ubuntu 18.04, looks like the following:

def fixedToFloat(value, precisionBits):
	"""Converts a fixed-point number to a float, choosing the float
	that has the shortest decimal reprentation.  Eg. to convert a
	fixed number in a 2.14 format, use precisionBits=14.  This is
	pretty slow compared to a simple division.  Use sporadically.

	precisionBits is only supported up to 16.
	"""
	if not value: return 0.0

	scale = 1 << precisionBits
	value /= scale
	eps = .5 / scale
	lo = value - eps
	hi = value + eps
	# If the range of valid choices spans an integer, return the integer.
	if int(lo) != int(hi):
		return float(round(value))
	fmt = "%.8f"
	lo = fmt % lo
	hi = fmt % hi
	assert len(lo) == len(hi) and lo != hi
	for i in range(len(lo)):
		if lo[i] != hi[i]:
			break
	period = lo.find('.')
	assert period < i
	fmt = "%%.%df" % (i - period)
	value = fmt % value
	return float(value)

Oh, and BTW: Thanks for the pointer to the discussion about why the rounding is no longer being done.