You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The introduction of the is_circular_reference function call within the for loop of stages/resolve_locals.lua since version v0.26 seems to significantly impact performance as observed in this commit: a24332e
The long story
Greetings, I've been utilizing luacheck v0.25 via luarocks in a private project for an extended period. Within this project, there exists a sizable Lua file (>100K lines) primarily comprising data tables (~95%) interspersed with logic code (~5K lines). Under version v0.25, linting this file takes approximately 10 seconds, which is acceptable given its substantial size.
However, subsequent to updating to the latest release, luacheck v1.1.2, the linting process for this file consumes over 10 minutes. Further investigation revealed that this performance regression commenced with v0.26.
Upon analyzing the differences between v0.25 and v0.26, I attempted to revert the alterations near the aforementioned for loop. Consequently, the linting time returned to normal, averaging around ~10 seconds.
How to reproduce
Given the private nature of the project, sharing the actual file is not feasible. Instead, I will outline the file structure and provide a minimal test file capable of reproducing the issue.
File Structure:
-- Standard module requirelocalabc=require"abc"...-- Approximately 200 local function definitions used by the data table belowlocalfunctionhandlerA(...)
...end-- Over 5000 data templates formatted as { [id] = <template> }localDATA= {
[id] = {
-- Approximately 20 attribute fields for each data templateattr1=1,
attr2="2",
attr3=true,
...-- Not all templates have `on_use`; roughly 50% of templates include iton_use=function (self)
-- This section references the local functions defined abovereturnhandlerA(...)
end,
},
...
}
returnDATA
Observations
Removing the entire local DATA = {...} portion, leaving only the local function definitions (~5K lines), results in a lint time of ~1 second for both luacheck versions.
Conversely, removing the entire local function definitions, leaving only the local DATA = {...} (~95K lines), yields a lint time of ~8 seconds for both luacheck versions.
Combining both parts causes luacheck v0.26 to take 10+ minutes.
Additionally, removing all on_use functions from the data templates, such that none of them have upvalues pointing above, leads to a lint time of ~1 minute for luacheck v0.26.
Minimal Reproducing File
I have crafted a gen_test.lua script to generate a test.lua file in the aforementioned format:
localNUM_FUNCS=50-- control num handlers to be generatedlocalNUM_DATAS=200-- control data templates to be generatedlocalNUM_ATTRS=10-- control attributes count of each data templatelocalf=io.open("test.lua", "w")
-- gen handlersfori=1, NUM_FUNCSdof:write(("local function handler%03d(...) return ... end\n"):format(i))
end-- gen datasf:write("local DATA = {\n")
forid=1, NUM_DATASdof:write((" [%d] = {\n"):format(id+10000))
forattr=1, NUM_ATTRSdof:write((" attr%02d = 0,\n"):format(attr))
endf:write((" on_use = function (...) return handler%03d(...) end,\n"):format(
NUM_FUNCS>0andid%NUM_FUNCS+1or0
))
f:write(" },\n")
endf:write("}\n")
f:write("return DATA\n")
On my MacBook Pro 2016, v0.25 takes less than 1 second while v0.26 takes around 30 seconds.
Thoughts
I acknowledge that this file structure may be unconventional, but refactoring is challenging within the project. I aim to lint this file given the presence of logic code in the upper portion and the inline on_use functions in the data templates. Are there alternative suggestions on how to work around or disable the newly added circular reference check? Could the is_circular_reference() function be optimized, perhaps by caching the result of a checked node?
Your assistance in addressing this performance concern would be greatly appreciated.
The text was updated successfully, but these errors were encountered:
Contributions welcome. I've never seen a speed regression on this scale, but obviously we'd be happy to see it fixed. I probably won't have time to work on it for a while, but perhaps you or anybody else that comes along can come up with a solution. I'll be happy to facilitate contributions and getting them released.
Two approaches I can think of to look into:
If the answer to some question is just being recalculated from the same inputs over and over, it might work to just add memoization to the function.
If that isn't possible perhaps we can add a lint id and gate the check behind it such that it can be ignored for this particular case while still getting the benefit of other linting.
If the answer to some question is just being recalculated from the same inputs over and over, it might work to just add memoization to the function.
It looks like a solid approach! After reviewing the code for contains_call, it seems that caching the result into node.is_contains_call before the function returns could be a viable solution. To elaborate, the function would return the value of node.is_contains_call at the beginning if it is not nil.
I've just tested the above changes on latest version of luacheck locally and confirmed that it now has the same performance as v0.25. I'll soon open a PR on it. 🎉
TL;DR
The introduction of the
is_circular_reference
function call within the for loop ofstages/resolve_locals.lua
since versionv0.26
seems to significantly impact performance as observed in this commit: a24332eThe long story
Greetings, I've been utilizing luacheck
v0.25
vialuarocks
in a private project for an extended period. Within this project, there exists a sizable Lua file (>100K lines) primarily comprising data tables (~95%) interspersed with logic code (~5K lines). Under versionv0.25
, linting this file takes approximately 10 seconds, which is acceptable given its substantial size.However, subsequent to updating to the latest release, luacheck
v1.1.2
, the linting process for this file consumes over 10 minutes. Further investigation revealed that this performance regression commenced withv0.26
.Upon analyzing the differences between v0.25 and v0.26, I attempted to revert the alterations near the aforementioned for loop. Consequently, the linting time returned to normal, averaging around ~10 seconds.
How to reproduce
Given the private nature of the project, sharing the actual file is not feasible. Instead, I will outline the file structure and provide a minimal test file capable of reproducing the issue.
File Structure:
Observations
local DATA = {...}
portion, leaving only the local function definitions (~5K lines), results in a lint time of ~1 second for both luacheck versions.local function
definitions, leaving only thelocal DATA = {...}
(~95K lines), yields a lint time of ~8 seconds for both luacheck versions.v0.26
to take 10+ minutes.on_use
functions from the data templates, such that none of them have upvalues pointing above, leads to a lint time of ~1 minute for luacheckv0.26
.Minimal Reproducing File
I have crafted a
gen_test.lua
script to generate atest.lua
file in the aforementioned format:On my MacBook Pro 2016,
v0.25
takes less than 1 second whilev0.26
takes around 30 seconds.Thoughts
I acknowledge that this file structure may be unconventional, but refactoring is challenging within the project. I aim to lint this file given the presence of logic code in the upper portion and the inline
on_use
functions in the data templates. Are there alternative suggestions on how to work around or disable the newly added circular reference check? Could theis_circular_reference()
function be optimized, perhaps by caching the result of a checked node?Your assistance in addressing this performance concern would be greatly appreciated.
The text was updated successfully, but these errors were encountered: