Performance hit while using the KeeTrayTOTP column on large databases

[Swoy]

So, a long story short:

I've had problems with what I thought was @navossoc's YAFD plugins fault. However, it seems to be KeeTrayTOTP that is the culprit;

I received the following error code in Windows 10 (latest build), running KeePass 2.40 and KeeTrayTOTP 0.9.5.0 combined with HIBPOffline plugin and YAFD:

Application: KeePass.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.NullReferenceException
   at KeePass.UI.UIUtil.SetAlternatingBgColors(System.Windows.Forms.ListView, System.Drawing.Color, Boolean)
   at KeePass.Forms.MainForm.UpdateEntryList(KeePassLib.PwGroup, Boolean)
   at KeePass.Forms.MainForm.UpdateUI(Boolean, KeePass.UI.PwDocument, Boolean, KeePassLib.PwGroup, Boolean, KeePassLib.PwGroup, Boolean, System.Windows.Forms.Control)
   at KeePass.Forms.MainForm.UpdateUI(Boolean, KeePass.UI.PwDocument, Boolean, KeePassLib.PwGroup, Boolean, KeePassLib.PwGroup, Boolean)
   at HIBPOfflineCheck.HIBPOfflineColumnProv.UpdateStatus()
   at HIBPOfflineCheck.HIBPOfflineColumnProv.PwdTouchedHandler(System.Object, KeePassLib.ObjectTouchedEventArgs)
   at KeePassLib.PwEntry.Touch(Boolean, Boolean)
   at YetAnotherFaviconDownloader.FaviconDialog+<>c__DisplayClass6.<BgWorker_DoWork>b__0(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()

As you can see, it breaks during list repaint. This happens when using the column TOTP _(Using either open view or asteriks) with any number of entries present. What is more, is that if used with a large number of entries such as this test Database.zip (password to open db is 123 (there is no password on the zip file itself.)) You can clearly see the effect of the seconds counter working the list every second.

• I think a urgent fix would be to allow this to update only once every time the TOTP changes - as opposed to every second, or simply disable the column feature all together.
• Another, maybe more sustainable solution would be to somehow check if there are other plugins updating the list, and pause this action until their job is complete on the list.
• Yet one more; if no TOTP is present in the current list, I see no reason to perform this update at all, and as it stands now, it will continue to check or try to update empty fields.

I've noticed some sluggishness on my main db (around 300-350 entries spanned ~10 groups as well, so it is definitively a performance hit.

Note: Using the column feature of KeeTrayTOTP with other plugins that require list repaint can potentially corrupt data during save, and any change is definitively lost when KeePass crash. So do not try this on your mainDB.

Let me know if you need anything else.

[robinvanpoppel]

Hi, thanks for reporting in such detail. I tried the steps this morning using your test database, but I could not yet get it reproduced.

I had YAFD already, and installed the HIBP Offline plugin just yet (without a password file though).

• What actions did you take after opening the database to cause this to happen?
• You mention _This happens when using the column TOTP (Using either open view or asteriks) with any number of entries present.
• The stack trace indicates a null reference exception in SetAlternatingBgColors. Do you suspect that is also caused by this plugin, or is it just that it is slowing down / hanging?

[Swoy]

Update: navossoc updated the issue in his repo where he says that the HIBP-module mainly caused the crashes when it tries to update each entry with new data in it's own cell. So this issue is mostly because of the slugishness. I apologize for that.

Hello again, if you try and add a few more TOTP's in the Database file? And then try to scroll? I mean, it could be that my computer is the culprit? I'm on a Dell 9570 ( i7 / with 1050 Ti Q-max ). Maybe there is something I can do on my side. But I swear, once I enable the TOTP column, the fans start to spin immediately:

And once I disable it again, it quickly goes down to normal:

Steps I take ( updated ):

• Open the DB
• Enable TOTP column
• Scroll quickly in the root folder where all entries are.
• I observe that the list is lagging each second and the computer spins up the fan due to high CPU load.
• I disable the TOTP column.
• And I observe that the list is butter smooth and the computer spins down the fan due to low CPU load.

For this I am only using the TOTP plugin enabled.

[chiwou]

Hi, had the same issues, it has something to do with the fields you display in KeePass, try reducing it to a minimum.
Title | Username | Url
And try again

[robinvanpoppel]

I've been investigating this one. When the column is enabled a timer will try and update the otp code every second (KeeTrayTOTPExt.OnTimerTick) . There are some checks whether to do a repaint , and although this part could be improved a little bit, the real drag is requesting a repaint. The plugin is calling PluginHost.MainWindow.RefreshEntriesList(); which is really heavy and also causes all other plugins to provide new entry data. Preferably we could ask Keepass to repaint / refresh only the password entries for which we know there are totp settings.

For now I don't have a definite solution. I'll have to take a look at the source code of Keepass if such a method exists. We could touch the password entry, which might trigger a repaint, but that would also prompt you to save the database when exiting so I'd rather not do that. Otherwise we might have to reach out to the Keepass author.