Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GORM fix up + no more digest #3136

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

GORM fix up + no more digest #3136

wants to merge 2 commits into from

Conversation

0sewa0
Copy link
Contributor

@0sewa0 0sewa0 commented May 7, 2024

Description

Initial change:
We no longer need to use the digest to pull the image in case of CodeModulesImage, so now we just use the ref we get from the user, and use that string (base64 encoded) as the folder name.

To make the change actually work:
I had to update the way we do things all around the new GORM implementation, so that we are utilising the data in the new database schema in a way that "make more sense".

  • This includes changes to the GC, now it actually runs when there is more likely some stuff to collect (example: when a dynakube is deleted)
  • It actually cleans things up (⚠️ this is very important)
    • The OSMounts have to be cleaned up, otherwise the TenantConfigs can't be cleaned up
      • So we don't fill the database with bunch of deleted TenantConfigs (this is more of a cosmetic problem)
    • The CodeModule foreign-key in an AppMount has to be set to Null in case the AppMount is deleted, so the CodeModule can be deleted if its not used. (Deleted AppMounts no longer use the related CodeModule)
      • If a soft deleted CodeModule is not cleaned-up soon ™️ then you will have a problem with the UNIQUE constraint if the same version would be downloaded.

⚠️ ⚠️ ⚠️ ⚠️ ⚠️ REALLY IMPORTANT NOTES ⚠️ ⚠️ ⚠️ ⚠️ ⚠️

  • If you have previously installed the Operator that was already using the GORM implementation, you WILL face errors
    • The reason for this is that the current way we use the database is very messy, and things don't get cleaned up (just soft deleted)
    • If the soft-deleted things are not removed, there may be problems due to the UNIQUE constraint. (mainly for the CodeModules)
    • OSMounts were not updated properly, so they will point to a VolumeMeta that is actually not correct anymore

⚠️ ⚠️ ⚠️ ⚠️ ⚠️

How can this be tested?

Play around with the csi-driver, check that the CodeModulesImage works, check that there are no database errors in the logs.
Check if the garbage-collection works, make sure its not too aggressive.

Sidenote: I still want to add more unit-tests, especially to the "Cleaner" parts, but its ready to be reviewed

@0sewa0 0sewa0 force-pushed the feature/csi-no-digest branch 2 times, most recently from 0c32c95 to b9960e7 Compare May 23, 2024 12:08
@0sewa0 0sewa0 changed the title POC GORM fix up + no more digest GORM fix up + no more digest May 23, 2024
@0sewa0 0sewa0 added the core Changes to core functionality of the Operator label May 23, 2024
@codecov-commenter
Copy link

codecov-commenter commented May 23, 2024

Codecov Report

Attention: Patch coverage is 47.95918% with 102 lines in your changes are missing coverage. Please review.

Project coverage is 58.55%. Comparing base (a3a6eb2) to head (7f8f3be).
Report is 4 commits behind head on main.

Current head 7f8f3be differs from pull request most recent head 3cd19ad

Please upload reports for the commit 3cd19ad to get more accurate results.

Files Patch % Lines
pkg/controllers/csi/metadata/cleaner.go 0.00% 30 Missing ⚠️
...kg/controllers/csi/driver/volumes/app/publisher.go 71.73% 7 Missing and 6 partials ⚠️
pkg/controllers/csi/gc/reconciler.go 25.00% 9 Missing and 3 partials ⚠️
pkg/controllers/csi/provisioner/controller.go 62.50% 7 Missing and 5 partials ⚠️
...g/controllers/csi/driver/volumes/host/publisher.go 40.00% 7 Missing and 2 partials ⚠️
pkg/controllers/csi/gc/binaries.go 40.00% 4 Missing and 2 partials ⚠️
pkg/controllers/csi/metadata/sqlite_gorm_client.go 14.28% 4 Missing and 2 partials ⚠️
pkg/controllers/csi/provisioner/install.go 78.57% 4 Missing and 2 partials ⚠️
pkg/controllers/csi/driver/server.go 0.00% 5 Missing ⚠️
pkg/injection/codemodule/installer/image/unpack.go 0.00% 3 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3136      +/-   ##
==========================================
- Coverage   58.72%   58.55%   -0.18%     
==========================================
  Files         326      324       -2     
  Lines       18117    18018      -99     
==========================================
- Hits        10640    10550      -90     
+ Misses       6298     6283      -15     
- Partials     1179     1185       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Changes to core functionality of the Operator
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants