Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rubocop cache broken in CI #8629

Closed
dvandersluis opened this issue Sep 1, 2020 · 0 comments · Fixed by #8633
Closed

Rubocop cache broken in CI #8629

dvandersluis opened this issue Sep 1, 2020 · 0 comments · Fixed by #8633

Comments

@dvandersluis
Copy link
Member

In #8304, the way that the rubocop_checksum was calculated changed, to use File.mtime for each file. Unfortunately, this has broken using the rubocop cache in CI, because when the code is checked out for CI, the mtime timestamp for each file is the checkout time (as far as I can find, git is not able to set mtime on checkout).

That means that every single CI run as of this change now gets a different rubocop_checksum value, and the cache is never going to be useful in CI.

I reran the same build multiple times (caching the rubocop cache dir in between) and you can see this in action (again note that there were no code or library changes between runs):

$ ls tmp/cache/rubocop_cache -la
total 24
drwxr-xr-x 6 circleci circleci 4096 Sep  1 16:48 .
drwxr-xr-x 3 circleci circleci 4096 Sep  1 16:48 ..
drwxr-xr-x 3 circleci circleci 4096 Sep  1 16:32 2f5fad92f0c09057f669462766ac298d1d8b1c07
drwxr-xr-x 3 circleci circleci 4096 Sep  1 16:48 7c2374f31998085cc8f08a38ca5518e494557194
drwxr-xr-x 3 circleci circleci 4096 Sep  1 16:36 b42c65b831c7b6892fcfdbf6955b8922a3ec5943
drwxr-xr-x 3 circleci circleci 4096 Sep  1 16:43 cc07ecda0c297a96eab49706225c962fbfb9d5ed

I have confirmed this in CircleCI. I imagine it's also the case in Travis but I don't have a project to check with. This is impactful for us because the time to run rubocop in CI has ballooned from 3-4s with the cache to about a minute without it.


Expected behavior

Rubocop cache should be able to be persisted between CI runs.

Actual behavior

Each CI build gets a new value for rubocop_checksum, causing the cache to not be used.

RuboCop version

Include the output of rubocop -V or bundle exec rubocop -V if using Bundler. Here's an example:

$ bundle exec rubocop -V
0.89.1 (using Parser 2.7.1.4, rubocop-ast 0.3.0, running on ruby 2.6.2 x86_64-linux)
dvandersluis added a commit to dvandersluis/rubocop that referenced this issue Sep 3, 2020
…for each file rather than using mtime.

File.mtime is faster, but inconsistent in CI, because in each CI build the mtime changes, which means that the rubocop cache implicitly cannot be used as-is. It's obviously slower to calculate a hash for each file, but this is still more performant (both in memory consumption and iterations per second) than the original.
bbatsov pushed a commit that referenced this issue Sep 4, 2020
…h file rather than using mtime.

File.mtime is faster, but inconsistent in CI, because in each CI build the mtime changes, which means that the rubocop cache implicitly cannot be used as-is. It's obviously slower to calculate a hash for each file, but this is still more performant (both in memory consumption and iterations per second) than the original.
dvandersluis added a commit to dvandersluis/rubocop that referenced this issue Sep 10, 2020
koic added a commit that referenced this issue Sep 10, 2020
Put changelog item for #8629 in correct version
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant