How does FontTools decode byte-strings in name table?

[http403]

I'm trying to get the font name in the name table, but some aren't encoded with any UTF standard despite the unicode flag is set to true and platformID is 1. However, saveXML have decoded them correctly. I wonder how. I tried dive into the code but I can't get a sense of it.

[behdad]

fonttools/Lib/fontTools/ttLib/tables/_n_a_m_e.py

Lines 456 to 505 in feb4e75

	def toUnicode(self, errors='strict'):
	"""
	If self.string is a Unicode string, return it; otherwise try decoding the
	bytes in self.string to a Unicode string using the encoding of this
	entry as returned by self.getEncoding(); Note that self.getEncoding()
	returns 'ascii' if the encoding is unknown to the library.
	Certain heuristics are performed to recover data from bytes that are
	ill-formed in the chosen encoding, or that otherwise look misencoded
	(mostly around bad UTF-16BE encoded bytes, or bytes that look like UTF-16BE
	but marked otherwise). If the bytes are ill-formed and the heuristics fail,
	the error is handled according to the errors parameter to this function, which is
	passed to the underlying decode() function; by default it throws a
	UnicodeDecodeError exception.
	Note: The mentioned heuristics mean that roundtripping a font to XML and back
	to binary might recover some misencoded data whereas just loading the font
	and saving it back will not change them.
	"""
	def isascii(b):
	return (b >= 0x20 and b <= 0x7E) or b in [0x09, 0x0A, 0x0D]
	encoding = self.getEncoding()
	string = self.string
	if isinstance(string, bytes) and encoding == 'utf_16_be' and len(string) % 2 == 1:
	# Recover badly encoded UTF-16 strings that have an odd number of bytes:
	# - If the last byte is zero, drop it. Otherwise,
	# - If all the odd bytes are zero and all the even bytes are ASCII,
	# prepend one zero byte. Otherwise,
	# - If first byte is zero and all other bytes are ASCII, insert zero
	# bytes between consecutive ASCII bytes.
	#
	# (Yes, I've seen all of these in the wild... sigh)
	if byteord(string[-1]) == 0:
	string = string[:-1]
	elif all(byteord(b) == 0 if i % 2 else isascii(byteord(b)) for i,b in enumerate(string)):
	string = b'\0' + string
	elif byteord(string[0]) == 0 and all(isascii(byteord(b)) for b in string[1:]):
	string = bytesjoin(b'\0'+bytechr(byteord(b)) for b in string[1:])
	string = tostr(string, encoding=encoding, errors=errors)
	# If decoded strings still looks like UTF-16BE, it suggests a double-encoding.
	# Fix it up.
	if all(ord(c) == 0 if i % 2 == 0 else isascii(ord(c)) for i,c in enumerate(string)):
	# If string claims to be Mac encoding, but looks like UTF-16BE with ASCII text,
	# narrow it down.
	string = ''.join(c for c in string[1::2])
	return string

[http403]

Thank you