Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VARC] Variable Composites table #3395

Merged
merged 114 commits into from May 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
ef6903e
[VARC] Start
behdad Dec 14, 2023
c3175b2
[transform] Add __eq__ / __ne__ to DecomposedTransform
behdad Dec 14, 2023
aad01a9
[VARC] Towards XML
behdad Dec 15, 2023
1a1e9e1
[VARC] Use one varIndexBase only
behdad Dec 15, 2023
6f4feff
[varStore] Add storeMastersMany / storeDeltasMany
behdad Dec 15, 2023
a800513
[varStore] Cache individual items in store*Many()
behdad Dec 15, 2023
b7ab1d8
[varStore] Fix overflow logic
behdad Dec 15, 2023
0e9eff8
Add MultiVarStore
behdad Dec 15, 2023
02c6a94
[VARC] Use TupleVariation value encoding
behdad Dec 15, 2023
6bcab78
[varStore] Fix caching when setModel() is called repeatedly
behdad Dec 15, 2023
5fe9da4
[MultiVarStore] Fix caching
behdad Dec 15, 2023
b4d3fc5
[TupleVariation] Support 32bit encoding in delta-encoding
behdad Dec 15, 2023
c78ba01
[VarCompositeGlyph] Use two varIdxes per component
behdad Dec 15, 2023
ec78b57
[MultiVarStore] Fix up XML read/write
behdad Dec 16, 2023
bcd5e4c
Rip out glyf1 VarComposites
behdad Feb 6, 2024
e9551c4
Remove some more remnants of VarComposites in ttGlyphSet
behdad Dec 16, 2023
a30ebf0
Revert "Remove some more remnants of VarComposites in ttGlyphSet"
behdad Dec 16, 2023
0e52857
[VARC] Start drawing VARC glyphs
behdad Dec 16, 2023
8cce745
[TupleVariation] Assert message
behdad Dec 16, 2023
68277fc
[TupleVariation] Fix 32bit reading / writing
behdad Dec 16, 2023
4b6c574
[VARC] More towards drawing
behdad Dec 16, 2023
c69fd12
[VARC] Move code around
behdad Dec 16, 2023
bbad70e
[VARC] bool(Vector) is useless... :(
behdad Dec 16, 2023
950d39b
[VARC] Finish drawing!
behdad Dec 16, 2023
e22953e
Black
behdad Dec 16, 2023
c50a0f6
[VARC] Rename a type
behdad Dec 16, 2023
c952237
[otConverters] Make _LazyList generic
behdad Dec 16, 2023
5faf139
[CFF2Index] Minor massage
behdad Dec 16, 2023
450c8f1
[CFF2Index] Make loading lazy
behdad Dec 16, 2023
feb6820
[_LazyList] Make much faster
behdad Dec 16, 2023
7de1306
[CFF2Index] Make even faster
behdad Dec 16, 2023
61916c1
[gvar] Speed up loading a bit
behdad Dec 17, 2023
7471ac6
[gvar] Speed up loading by not reading all offsets
behdad Dec 17, 2023
3806fd2
Move a couple of functions outline
behdad Dec 17, 2023
3ff2ee6
Move lazy datastructures to misc.lazyTools
behdad Dec 17, 2023
cae76d5
[glyf] Load using LazyDict
behdad Dec 17, 2023
0a79939
Revert "[glyf] Load using LazyDict"
behdad Dec 17, 2023
4db90f5
[loca] Minor speedup
behdad Dec 17, 2023
44a32f8
Minor refactor
behdad Dec 17, 2023
42d6b6b
[svgPen] Write two digits after decimal by default
behdad Dec 17, 2023
1514158
[subset] Support VARC
behdad Dec 17, 2023
f37f2e4
[subset] Close over MATH before GSUB
behdad Dec 17, 2023
5c27846
[subset/VARC] Subset MultiVarStore
behdad Dec 17, 2023
27e5182
[VARC/instancer] Implement
behdad May 23, 2024
272f736
[VARC] Fix instanciating component
behdad Dec 17, 2023
38d190a
Black
behdad Dec 17, 2023
57dd288
[VARC/instancer] Comment
behdad Dec 17, 2023
3c60c0e
[VARC/subset] Fix closure
behdad Dec 17, 2023
ebd877b
[VARC/test] Start adding
behdad Dec 17, 2023
2ca627d
[VARC/instancer] Fix
behdad Dec 17, 2023
22e02cc
[VARC/test] Adjust a test
behdad Dec 17, 2023
28385ec
[VARC/test] Adjust test
behdad Dec 17, 2023
ce3e260
[VARC/test] Update the rest of the test expectations
behdad Dec 17, 2023
2852055
[VARC] Allow drawing same-name glyph
behdad Dec 17, 2023
2056577
[VARC/scaleUpem] Start
behdad Dec 17, 2023
f24808c
Black
behdad Dec 17, 2023
febbb34
[CFF2Index] Avoid infinite loop visitor by hiding symbol
behdad Dec 17, 2023
0f0148e
[VARC/scaleUpem] Implement
behdad Dec 18, 2023
2b09b0d
Black
behdad Dec 18, 2023
45f7f4f
Try fixing Python < 3.12
behdad Dec 18, 2023
93fe240
[scaleUpem] Comment
behdad Dec 18, 2023
232d9cf
[VARC/scaleUpem] Remove early return
behdad Dec 18, 2023
a958c68
[VARC] Simplify TupleValues
behdad Dec 18, 2023
c91984e
[VARC] Use sparse-regions in MultiVarStore
behdad Dec 18, 2023
42a5fbd
[VARC] Redesign table
behdad Dec 19, 2023
9cc3689
[VARC] Towards drawing new design
behdad Dec 19, 2023
f73d6f2
[VARC] Fixups
behdad Dec 20, 2023
8ea9765
[VARC] Fix copilot mistake in decompile
behdad Dec 20, 2023
2229607
Black
behdad Dec 20, 2023
43e054b
[VARC] Minor
behdad Dec 20, 2023
822351f
[VARC] Minor rename
behdad Dec 20, 2023
76d293e
[VARC] Simplify reading
behdad Dec 20, 2023
b772f1d
Reuse a variable
behdad Dec 20, 2023
f7337b5
[VARC] Make HAVE_VARIATIONS flag automatic
behdad Dec 20, 2023
d07d960
Fix RESERVED
behdad Dec 20, 2023
cd15139
[VARC] Minor
behdad Dec 20, 2023
263d4d9
Handle scaleY
behdad Dec 20, 2023
d37b394
[VARC] Encode indices as 1,2,3,4 bytes long
behdad Dec 20, 2023
58bb96b
[VARC] Use a DeltaSetIndexMap
behdad Dec 20, 2023
6af1d5c
Fix RESERVED
behdad Dec 20, 2023
bfb8490
[VARC] Apparently __iadd__ is optional
behdad Dec 20, 2023
668a40d
[VARC] Pivot on design again
behdad Dec 20, 2023
cfc66a3
[VARC] Simplify VarCompositeRecord
behdad Dec 20, 2023
735859f
[VARC] Implement XML read/write
behdad Dec 20, 2023
a7ca67a
[VARC] Update subsetting
behdad Dec 20, 2023
902b2a1
[VARC] Fix scaleUpem
behdad Dec 20, 2023
e88e47f
[VARC] Take a stab at instancing
behdad Dec 20, 2023
bc82985
[VARC] Speed up subsetting
behdad Dec 20, 2023
c3dfe10
[VARC] Use TupleVariations tuple encoding for axisValues
behdad Dec 21, 2023
6a09096
[VARC] Use variable-length encoding for VarIdx'es
behdad Dec 21, 2023
297e0bd
[VARC] Use var-int encoding for AxisIndicesIndex
behdad Dec 21, 2023
b1142b6
[VARC] Adjust to latest flags change
behdad Dec 25, 2023
9f66edb
[varLib.models] Add validate=False to normalizeLocation
behdad Jan 2, 2024
d6482c9
[VARC] Rename VarInt32 to uint32var
behdad Jan 3, 2024
87ddb24
[otBase] Add comment based on review feedback
behdad Jan 16, 2024
b3a0a21
[transform] Remove redundant methods
behdad Jan 16, 2024
e3ba7a7
[lazyTools] Simplify based on review feedback
behdad Jan 16, 2024
5ad4045
[VarComponent] Nicer XML output
behdad Jan 18, 2024
037bbe1
[VARC] Load axisValues as float
behdad Jan 18, 2024
5b2df6b
[VARC] Compute flags from XML transform components
behdad Jan 20, 2024
7d6df04
[VARC] Minor, match spec better
behdad Jan 22, 2024
40584ad
[VARC] Adjust XML output based on review feedback
behdad Jan 22, 2024
1acc80e
[VARC] Deepcopy the component
behdad Jan 26, 2024
3bada5d
Fix otConverters lazy reader
behdad Jan 26, 2024
a1641d9
Newer black
behdad Feb 6, 2024
88828e0
[CFF2IndexOf] Fix data_base
behdad Mar 16, 2024
c155632
[varc] Skip reserved records
behdad Mar 20, 2024
74f870f
[varc] Add ConditionSets
behdad Mar 20, 2024
d53c08a
[varc] Conditionals tested!
behdad Mar 21, 2024
9cb73da
[varc] Add a conditional VarComponent test
behdad Mar 22, 2024
973dc5c
[varc] Use Condition instead of ConditionSet
behdad Apr 22, 2024
77add05
Don't emit addVarComponent() if the component references the parent g…
justvanrossum Apr 23, 2024
973072b
[Condition] Implement ConditionValue
behdad Apr 23, 2024
753197e
[varc] Use multiVarStore instead of GDEF varStore
behdad Apr 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 1 addition & 5 deletions Lib/fontTools/fontBuilder.py
Expand Up @@ -656,11 +656,7 @@ def setupGlyf(self, glyphs, calcGlyphBounds=True, validateGlyphFormat=True):

if validateGlyphFormat and self.font["head"].glyphDataFormat == 0:
for name, g in glyphs.items():
if g.isVarComposite():
raise ValueError(
f"Glyph {name!r} is a variable composite, but glyphDataFormat=0"
)
elif g.numberOfContours > 0 and any(f & flagCubic for f in g.flags):
if g.numberOfContours > 0 and any(f & flagCubic for f in g.flags):
raise ValueError(
f"Glyph {name!r} has cubic Bezier outlines, but glyphDataFormat=0; "
"either convert to quadratics with cu2qu or set glyphDataFormat=1."
Expand Down
2 changes: 1 addition & 1 deletion Lib/fontTools/merge/tables.py
Expand Up @@ -225,7 +225,7 @@ def merge(self, m, tables):
g.removeHinting()
# Expand composite glyphs to load their
# composite glyph names.
if g.isComposite() or g.isVarComposite():
if g.isComposite():
g.expand(table)
return DefaultTable.merge(self, m, tables)

Expand Down
12 changes: 12 additions & 0 deletions Lib/fontTools/misc/iterTools.py
@@ -0,0 +1,12 @@
from itertools import *

# Python 3.12:
if "batched" not in globals():
# https://docs.python.org/3/library/itertools.html#itertools.batched
def batched(iterable, n):
# batched('ABCDEFG', 3) --> ABC DEF G
if n < 1:
raise ValueError("n must be at least one")

Check warning on line 9 in Lib/fontTools/misc/iterTools.py

View check run for this annotation

Codecov / codecov/patch

Lib/fontTools/misc/iterTools.py#L9

Added line #L9 was not covered by tests
it = iter(iterable)
while batch := tuple(islice(it, n)):
yield batch
42 changes: 42 additions & 0 deletions Lib/fontTools/misc/lazyTools.py
@@ -0,0 +1,42 @@
from collections import UserDict, UserList

__all__ = ["LazyDict", "LazyList"]


class LazyDict(UserDict):
def __init__(self, data):
super().__init__()
self.data = data

def __getitem__(self, k):
v = self.data[k]
if callable(v):
v = v(k)
self.data[k] = v
return v


class LazyList(UserList):
def __getitem__(self, k):
if isinstance(k, slice):
indices = range(*k.indices(len(self)))
return [self[i] for i in indices]
v = self.data[k]
if callable(v):
v = v(k)
self.data[k] = v
return v

def __add__(self, other):
if isinstance(other, LazyList):
other = list(other)
elif isinstance(other, list):
pass
else:
return NotImplemented
return list(self) + other

def __radd__(self, other):
if not isinstance(other, list):
return NotImplemented
return other + list(self)
13 changes: 13 additions & 0 deletions Lib/fontTools/misc/transform.py
Expand Up @@ -422,6 +422,19 @@
tCenterX: float = 0
tCenterY: float = 0

def __bool__(self):
return (

Check warning on line 426 in Lib/fontTools/misc/transform.py

View check run for this annotation

Codecov / codecov/patch

Lib/fontTools/misc/transform.py#L426

Added line #L426 was not covered by tests
self.translateX != 0
or self.translateY != 0
or self.rotation != 0
or self.scaleX != 1
or self.scaleY != 1
or self.skewX != 0
or self.skewY != 0
or self.tCenterX != 0
or self.tCenterY != 0
)

@classmethod
def fromTransform(self, transform):
# Adapted from an answer on
Expand Down
10 changes: 8 additions & 2 deletions Lib/fontTools/pens/svgPathPen.py
Expand Up @@ -2,7 +2,7 @@
from fontTools.pens.basePen import BasePen


def pointToString(pt, ntos=str):
def pointToString(pt, ntos):
return " ".join(ntos(i) for i in pt)


Expand Down Expand Up @@ -37,7 +37,13 @@ class SVGPathPen(BasePen):
print(tpen.getCommands())
"""

def __init__(self, glyphSet, ntos: Callable[[float], str] = str):
def __init__(
self,
glyphSet,
ntos: Callable[[float], str] = (
lambda x: ("%.2f" % x) if x != int(x) else str(int(x))
),
):
BasePen.__init__(self, glyphSet)
self._commands = []
self._lastCommand = None
Expand Down
122 changes: 109 additions & 13 deletions Lib/fontTools/subset/__init__.py
Expand Up @@ -14,7 +14,7 @@
from fontTools.subset.util import _add_method, _uniq_sort
from fontTools.subset.cff import *
from fontTools.subset.svg import *
from fontTools.varLib import varStore # for subset_varidxes
from fontTools.varLib import varStore, multiVarStore # For monkey-patching
from fontTools.ttLib.tables._n_a_m_e import NameRecordVisitor
import sys
import struct
Expand Down Expand Up @@ -2630,6 +2630,88 @@
s.glyphs.update(variants)


@_add_method(ttLib.getTableClass("VARC"))
def subset_glyphs(self, s):
indices = self.table.Coverage.subset(s.glyphs)
self.table.VarCompositeGlyphs.VarCompositeGlyph = _list_subset(
self.table.VarCompositeGlyphs.VarCompositeGlyph, indices
)
return bool(self.table.VarCompositeGlyphs.VarCompositeGlyph)


@_add_method(ttLib.getTableClass("VARC"))
def closure_glyphs(self, s):
if self.table.VarCompositeGlyphs is None:
return

Check warning on line 2645 in Lib/fontTools/subset/__init__.py

View check run for this annotation

Codecov / codecov/patch

Lib/fontTools/subset/__init__.py#L2645

Added line #L2645 was not covered by tests

glyphMap = {glyphName: i for i, glyphName in enumerate(self.table.Coverage.glyphs)}
glyphRecords = self.table.VarCompositeGlyphs.VarCompositeGlyph

glyphs = s.glyphs
covered = set()
new = set(glyphs)
while new:
oldNew = new
new = set()
for glyphName in oldNew:
if glyphName in covered:
continue

Check warning on line 2658 in Lib/fontTools/subset/__init__.py

View check run for this annotation

Codecov / codecov/patch

Lib/fontTools/subset/__init__.py#L2658

Added line #L2658 was not covered by tests
idx = glyphMap.get(glyphName)
if idx is None:
continue
glyph = glyphRecords[idx]
for comp in glyph.components:
name = comp.glyphName
glyphs.add(name)
if name not in covered:
new.add(name)


@_add_method(ttLib.getTableClass("VARC"))
def prune_post_subset(self, font, options):
table = self.table

store = table.MultiVarStore
if store is not None:
usedVarIdxes = set()
table.collect_varidxes(usedVarIdxes)
varidx_map = store.subset_varidxes(usedVarIdxes)
table.remap_varidxes(varidx_map)

axisIndicesList = table.AxisIndicesList.Item
if axisIndicesList is not None:
usedIndices = set()
for glyph in table.VarCompositeGlyphs.VarCompositeGlyph:
for comp in glyph.components:
if comp.axisIndicesIndex is not None:
usedIndices.add(comp.axisIndicesIndex)
usedIndices = sorted(usedIndices)
table.AxisIndicesList.Item = _list_subset(axisIndicesList, usedIndices)
mapping = {old: new for new, old in enumerate(usedIndices)}
for glyph in table.VarCompositeGlyphs.VarCompositeGlyph:
for comp in glyph.components:
if comp.axisIndicesIndex is not None:
comp.axisIndicesIndex = mapping[comp.axisIndicesIndex]

conditionList = table.ConditionList
if conditionList is not None:
conditionTables = conditionList.ConditionTable
usedIndices = set()

Check warning on line 2699 in Lib/fontTools/subset/__init__.py

View check run for this annotation

Codecov / codecov/patch

Lib/fontTools/subset/__init__.py#L2698-L2699

Added lines #L2698 - L2699 were not covered by tests
for glyph in table.VarCompositeGlyphs.VarCompositeGlyph:
for comp in glyph.components:
if comp.conditionIndex is not None:
usedIndices.add(comp.conditionIndex)
usedIndices = sorted(usedIndices)
conditionList.ConditionTable = _list_subset(conditionTables, usedIndices)

Check warning on line 2705 in Lib/fontTools/subset/__init__.py

View check run for this annotation

Codecov / codecov/patch

Lib/fontTools/subset/__init__.py#L2703-L2705

Added lines #L2703 - L2705 were not covered by tests
mapping = {old: new for new, old in enumerate(usedIndices)}
for glyph in table.VarCompositeGlyphs.VarCompositeGlyph:
for comp in glyph.components:
if comp.conditionIndex is not None:
comp.conditionIndex = mapping[comp.conditionIndex]

Check warning on line 2710 in Lib/fontTools/subset/__init__.py

View check run for this annotation

Codecov / codecov/patch

Lib/fontTools/subset/__init__.py#L2710

Added line #L2710 was not covered by tests

return True


@_add_method(ttLib.getTableClass("MATH"))
def closure_glyphs(self, s):
if self.table.MathVariants:
Expand Down Expand Up @@ -3298,33 +3380,33 @@
self.glyphs.add(font.getGlyphName(i))
log.info("Added first four glyphs to subset")

if self.options.layout_closure and "GSUB" in font:
with timer("close glyph list over 'GSUB'"):
if "MATH" in font:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the swapping of GSUB and MATH closure order should arguably go to a separate PR.. unless is really required for the changes at hand

with timer("close glyph list over 'MATH'"):
log.info(
"Closing glyph list over 'GSUB': %d glyphs before", len(self.glyphs)
"Closing glyph list over 'MATH': %d glyphs before", len(self.glyphs)
)
log.glyphs(self.glyphs, font=font)
font["GSUB"].closure_glyphs(self)
font["MATH"].closure_glyphs(self)
self.glyphs.intersection_update(realGlyphs)
log.info(
"Closed glyph list over 'GSUB': %d glyphs after", len(self.glyphs)
"Closed glyph list over 'MATH': %d glyphs after", len(self.glyphs)
)
log.glyphs(self.glyphs, font=font)
self.glyphs_gsubed = frozenset(self.glyphs)
self.glyphs_mathed = frozenset(self.glyphs)

if "MATH" in font:
with timer("close glyph list over 'MATH'"):
if self.options.layout_closure and "GSUB" in font:
with timer("close glyph list over 'GSUB'"):
log.info(
"Closing glyph list over 'MATH': %d glyphs before", len(self.glyphs)
"Closing glyph list over 'GSUB': %d glyphs before", len(self.glyphs)
)
log.glyphs(self.glyphs, font=font)
font["MATH"].closure_glyphs(self)
font["GSUB"].closure_glyphs(self)
self.glyphs.intersection_update(realGlyphs)
log.info(
"Closed glyph list over 'MATH': %d glyphs after", len(self.glyphs)
"Closed glyph list over 'GSUB': %d glyphs after", len(self.glyphs)
)
log.glyphs(self.glyphs, font=font)
self.glyphs_mathed = frozenset(self.glyphs)
self.glyphs_gsubed = frozenset(self.glyphs)

for table in ("COLR", "bsln"):
if table in font:
Expand All @@ -3345,6 +3427,20 @@
log.glyphs(self.glyphs, font=font)
setattr(self, f"glyphs_{table.lower()}ed", frozenset(self.glyphs))

if "VARC" in font:
with timer("close glyph list over 'VARC'"):
log.info(
"Closing glyph list over 'VARC': %d glyphs before", len(self.glyphs)
)
log.glyphs(self.glyphs, font=font)
font["VARC"].closure_glyphs(self)
self.glyphs.intersection_update(realGlyphs)
log.info(
"Closed glyph list over 'VARC': %d glyphs after", len(self.glyphs)
)
log.glyphs(self.glyphs, font=font)
self.glyphs_glyfed = frozenset(self.glyphs)

if "glyf" in font:
with timer("close glyph list over 'glyf'"):
log.info(
Expand Down