{precommit} can't be used when a {renv} is already present in repo #342

smingerson · 2021-12-09T12:40:18Z

Similar to #316, I'm having trouble. I recently updated from 0.1.3 to 0.2.0, and I can't get anything working again. I followed the updating instructions which failed, and then tried to remove everything and try with a fresh installation. It doesn't seems to have something to do with the fact I use renv -- I tried deactivating it and still have the same issues.

I'm on Windows 10 with R 4.1.1.

style-files..............................................................Failed
- hook id: style-files
- exit code: 1

Warning: Setting LC_COLLATE=en_US.UTF-8 failed
Warning: Setting LC_CTYPE=en_US.UTF-8 failed
Warning: Setting LC_MONETARY=en_US.UTF-8 failed
Warning: Setting LC_TIME=en_US.UTF-8 failed
Error in packageVersion("precommit") : 
  there is no package called 'precommit'
Execution halted
spell-check..............................................................Failed
- hook id: spell-check
- exit code: 1

Warning: Setting LC_COLLATE=en_US.UTF-8 failed
Warning: Setting LC_CTYPE=en_US.UTF-8 failed
Warning: Setting LC_MONETARY=en_US.UTF-8 failed
Warning: Setting LC_TIME=en_US.UTF-8 failed
Error in loadNamespace(x) : there is no package called 'docopt'
Calls: loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
Execution halted
lintr....................................................................Failed
- hook id: lintr
- exit code: 1

Warning: Setting LC_COLLATE=en_US.UTF-8 failed
Warning: Setting LC_CTYPE=en_US.UTF-8 failed
Warning: Setting LC_MONETARY=en_US.UTF-8 failed
Warning: Setting LC_TIME=en_US.UTF-8 failed
Error in loadNamespace(x) : there is no package called 'docopt'
Calls: loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
Execution halted

readme-rmd-rendered..................................(no files to check)Skippedparsable-R...............................................................Failed
- hook id: parsable-R
- exit code: 1

Warning: Setting LC_COLLATE=en_US.UTF-8 failed
Warning: Setting LC_CTYPE=en_US.UTF-8 failed
Warning: Setting LC_MONETARY=en_US.UTF-8 failed
Warning: Setting LC_TIME=en_US.UTF-8 failed
Error in loadNamespace(x) : there is no package called 'knitr'
Calls: lapply ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
Execution halted
no-browser-statement.....................................................PassedCheck for added large files..............................................PassedFix End of Files.........................................................Passed
Don't commit common R artifacts......................(no files to check)Skipped
[INFO] Restored changes from C:\Users\Sean.Ingerson\.cache\pre-commit\patch1638996953-4064.

The text was updated successfully, but these errors were encountered:

smingerson · 2021-12-09T12:48:59Z

I fixed this by finding the automatically created pre-commit R project and manually installing packages for both renv and renv-default folders.

lorenzwalthert · 2021-12-09T13:55:59Z

Thanks for letting me know @smingerson.

tried to remove everything and try with a fresh installation

Did this involve the clean up suggested in #316, namely

delete the file .git/hooks/pre-commit.
run $ pre-commit clean and
try again with precommit::use_precommit() and then commit

?

Were you using {renv} in this project (just to make sure we are on the same page: the {renv} used by {precommit} is not located in your repository directory.

I fixed this by finding the automatically created pre-commit R project and manually installing packages for both renv and renv-default folders.

Ok, this seem like a hack -.- sorry for the bad experience.

smingerson · 2021-12-09T15:15:22Z

Did this involve the clean up suggested in #316, namely
delete the file .git/hooks/pre-commit.
run $ pre-commit clean and
try again with precommit::use_precommit() and then commit
?

Yes, it did.

Were you using {renv} in this project (just to make sure we are on the same page: the {renv} used by {precommit} is not located in your repository directory.

Correct. My own project used renv, and I was deactivating/reactivating renv within my own project. At the time, I didn't understand that precommit was building its own instance somewhere.

Ok, this seem like a hack -.- sorry for the bad experience.

It most certainly is a hack! No worries on the bad experience, precommit is more than worth it!

lorenzwalthert · 2021-12-09T15:22:23Z

Correct. My own project used renv, and I was deactivating/reactivating renv within my own project. At the time, I didn't understand that precommit was building its own instance somewhere.

Is this a public repo? We have not yet explored compatibility of {precommit} with {renv} in a {renv} project. Maybe that's causing the problems.

smingerson · 2021-12-09T15:45:02Z

Work project so I cannot share. I will spend some time this weekend to see if I can reproduce it in a public repo.

lorenzwalthert · 2021-12-09T15:48:16Z

Thanks a lot. Would be great to see a minimal reproducible example of this. Also note that I created release v0.2.0.9001, so if you autoupdate() from a v0.1.3 in a repo, you will have a new hook environment and your workaround won't work there (because it creates a new renv or renv-default folder).

smingerson · 2021-12-11T12:32:43Z

I tried to reproduce on my personal computer, and ended up with some very different errors. I suspect the difference being going from no precommit to 0.2.0, rather than 0.1.3 to 0.2.0, and not having had reticulate installed anywhere prior. Progress so far:

install.packages("precommit")
install.packages("reticulate")
reticulate::install_miniconda()
precommit::install_precommit()
precommit::use_precommit()
#----Try to commit -------------------------------------------------------------
# received many lines of errors, resulting from
#> ImportError: DLL load failed while importing _sqlite3: The specified module could not be found.
# Added r-miniconda/library/bin to system PATH
#----Try to commit -------------------------------------------------------------
#> An unexpected error has occurred: CalledProcessError: command: ('C:\\Program Files\\R\\R-4.0.2\\bin\\x64\\Rscript.EXE', '--vanilla', '-e', '            prefix_dir <- \'C:\\\\Users\\\\Sean\\\\.cache\\\\pre-commit\\\\repo1vl0kwya\'\n            options(\n                repos = c(CRAN = "https://cran.rstudio.com"),\n                renv.consent = TRUE\n            )\n            source("renv/activate.R")\n            renv::restore()\n            activate_statement <- paste0(\n              \'suppressWarnings({\',\n              \'old <- setwd("\', getwd(), \'"); \',\n              \'source("renv/activate.R"); \',\n              \'setwd(old); \',\n              \'renv::load("\', getwd(), \'");})\'\n            )\n            writeLines(activate_statement, \'activate.R\')\n            is_package <- tryCatch(\n              {\n                  path_desc <- file.path(prefix_dir, \'DESCRIPTION\')\n                  suppressWarnings(desc <- read.dcf(path_desc))\n                  "Package" %in% colnames(desc)\n              },\n              error = function(...) FALSE\n            )\n            if (is_package) {\n                renv::install(prefix_dir)\n            }\n            ')
#> return code: 1
#> expected return code: 0
#> stdout: (none)
#> stderr:
#>   During startup - Warning message:
#>   Setting LC_CTYPE=en_US.UTF-8 failed 
#> # Bootstrapping renv 0.14.0-37 ------------
#> * Downloading renv 0.14.0-37 from GitHub ... OK
#> * Installing renv 0.14.0-37 ... Done!
#>   * Successfully installed and loaded renv 0.14.0-37.
#> Error in renv_json_quote(aliased_path(path)) : 
#>   could not find function "renv_json_quote"
#> Calls: source ... eval -> <Anonymous> -> renv_load_switch -> warningf
#> Traceback (most recent calls last):
#>   17: source("renv/activate.R")
#> 16: withVisible(eval(ei, envir))
#> 15: eval(ei, envir)
#> 14: eval(ei, envir)
#> 13: local(...)
#> 12: eval.parent(substitute(eval(quote(expr), envir)))
#> 11: eval(expr, p)
#> 10: eval(expr, p)
#> 9: eval(quote(...), new.env())
#> 8: eval(quote(...), new.env())
#> 7: renv::load()
#> 6: renv_load_switch(project)
#> 5: warningf(fmt, renv_path_pretty(project))
#> 4: warning(sprintf(fmt, ...), call. = call., immediate. = immediate.)
#> 3: sprintf(fmt, ...)
#> 2: base::sprintf(fmt, ...)
#> 1: renv_path_pretty(project)
#> Execution halted
#> 
#> Check the log at C:\Users\Sean\.cache\pre-commit\pre-commit.log# 
# 
# Somehow, precommit is using my local project renv, which has version 0.11.0
# Upgrade renv to 0.14.0
#-----Try to commit-------------------------------------------------------------
#> [INFO] Installing environment for https://github.com/lorenzwalthert/precommit.
#> [INFO] Once installed this environment will be reused.
#> [INFO] This may take a few minutes...
# Actually does take a few minutes, then 
#> An unexpected error has occurred: CalledProcessError: command: ('C:\\Users\\Sean\\AppData\\Local\\r-miniconda\\envs\\r-precommit\\python.exe', '-mvirtualenv', 'C:\\Users\\Sean\\.cache\\pre-commit\\repo1f2hx72b\\py_env-python3.9')
#> return code: 1
#> expected return code: 0
#> stdout: (none)
#> stderr:
#>   Traceback (most recent call last):
#>   File "C:\Users\Sean\AppData\Local\r-miniconda\envs\r-precommit\lib\runpy.py", line 188, in _run_module_as_main
#> mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
#> File "C:\Users\Sean\AppData\Local\r-miniconda\envs\r-precommit\lib\runpy.py", line 147, in _get_module_details
#> return _get_module_details(pkg_main_name, error)
#> File "C:\Users\Sean\AppData\Local\r-miniconda\envs\r-precommit\lib\runpy.py", line 111, in _get_module_details
#> __import__(pkg_name)
#> File "C:\Users\Sean\AppData\Local\r-miniconda\envs\r-precommit\lib\site-packages\virtualenv\__init__.py", line 3, in <module>
#>   from .run import cli_run, session_via_cli
#> File "C:\Users\Sean\AppData\Local\r-miniconda\envs\r-precommit\lib\site-packages\virtualenv\run\__init__.py", line 11, in <module>
#>   from ..seed.wheels.periodic_update import manual_upgrade
#> File "C:\Users\Sean\AppData\Local\r-miniconda\envs\r-precommit\lib\site-packages\virtualenv\seed\wheels\__init__.py", line 3, in <module>
#>   from .acquire import get_wheel, pip_wheel_env_run
#> File "C:\Users\Sean\AppData\Local\r-miniconda\envs\r-precommit\lib\site-packages\virtualenv\seed\wheels\acquire.py", line 12, in <module>
#>   from .bundle import from_bundle
#> File "C:\Users\Sean\AppData\Local\r-miniconda\envs\r-precommit\lib\site-packages\virtualenv\seed\wheels\bundle.py", line 4, in <module>
#>   from .periodic_update import periodic_update
#> File "C:\Users\Sean\AppData\Local\r-miniconda\envs\r-precommit\lib\site-packages\virtualenv\seed\wheels\periodic_update.py", line 10, in <module>
#>   import ssl
#> File "C:\Users\Sean\AppData\Local\r-miniconda\envs\r-precommit\lib\ssl.py", line 98, in <module>
#>   import _ssl             # if we can't import it, let the error propagate
#> ImportError: DLL load failed while importing _ssl: The specified module could not be found.
#> 
#> Check the log at C:\Users\Sean\.cache\pre-commit\pre-commit.log

lorenzwalthert · 2021-12-11T12:36:20Z

oh dear this does not look great indeed. Thanks for your time. I'll also try with a local {renv}.

lorenzwalthert · 2021-12-11T12:43:18Z

Repo: https://github.com/lorenzwalthert/precommit.and.project.renv

smingerson · 2021-12-11T12:54:03Z

I've made some progress. I think a lot of the errors weren't really precommit related, but due to python/reticulate pain. I did the below, and now its looking more like my original problem

# Deleted pre-commit python env, re-installed ----------------------------------
#> [INFO] Locking pre-commit directory
#> style-files..............................................................Failed
#> - hook id: style-files
#> - exit code: 1
#> 
#> During startup - Warning messages:
#>   1: Setting LC_COLLATE=en_US.UTF-8 failed 
#> 2: Setting LC_CTYPE=en_US.UTF-8 failed 
#> 3: Setting LC_MONETARY=en_US.UTF-8 failed 
#> 4: Setting LC_TIME=en_US.UTF-8 failed 
#> Error in packageVersion("precommit") : 
#>   there is no package called 'precommit'
#> Execution halted
#> 
#> spell-check..........................................(no files to check)Skippedlintr....................................................................Failed
#> - hook id: lintr
#> - exit code: 1
#> 
#> During startup - Warning messages:
#>   1: Setting LC_COLLATE=en_US.UTF-8 failed 
#> 2: Setting LC_CTYPE=en_US.UTF-8 failed 
#> 3: Setting LC_MONETARY=en_US.UTF-8 failed 
#> 4: Setting LC_TIME=en_US.UTF-8 failed 
#> Error in loadNamespace(x) : there is no package called 'docopt'
#> Calls: loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
#> Execution halted
#> 
#> readme-rmd-rendered..................................(no files to check)Skippedparsable-R...............................................................Passedno-browser-statement.....................................................PassedCheck for added large files..............................................PassedFix End of Files.........................................................Passed
#> Don't commit common R artifacts......................(no files to check)Skipped

Back to something I'm familiar with! I noticed in the pre-commit python project, there is now a py_env-python3.9 folder -- there was nothing like that previously. Perhaps the miniconda install not being on path caused problems?

Both the precommit project renv and renv-default libraries are empty. Note however, that my local project DOES have all these dependencies, so it is using the correct renv.

lorenzwalthert · 2021-12-11T13:05:03Z

Both the precommit project renv and renv-default libraries are empty

I noticed that too in my repo.

lorenzwalthert · 2021-12-11T17:40:48Z

Setup:

lastest pre-commit (v2.16.0).
latest RStudio (v2021.09.1). This version changed some low-level terminal related things that could be related to our problem.
lorenzwalthert/precommit@344 (release candidate).
global installation of {renv} that is not equal to our hooks, i.e. not 0.14.0-37.

The problem seems RStudio, as the following workflow does error in a system terminal opened with RStudio, but not with a terminal like iTerm.

Workflow with system terminal in RStudio (fails)

open a repo with {renv}, e.g. https://github.com/lorenzwalthert/precommit.and.project.renv. initialize renv with renv::activate() and turn it off again with renv::deactivate().
$ pre-commit clean to clear cache.
$ pre-commit install --install-hooks to install the hooks into the cache.
$ git commit -m ... to commit.
gives missing packages

style-files..............................................................Failed
- hook id: style-files
- exit code: 1

Error in packageVersion("precommit") : 
  there is no package called ‘precommit’
Execution halted

spell-check..........................................(no files to check)Skipped
lintr....................................................................Failed
- hook id: lintr
- exit code: 1

Error in loadNamespace(x) : there is no package called ‘docopt’
Calls: loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
Execution halted

Workflow with system terminal outside RStudio (succeeds)

The same as the above workflow, run outside RStudio

Workflow with R in RStudio

A slightly different workflow in R alone:

initialize repo with precommit::use_precommit(install_hooks = TRUE) with options(precommit.block_install_hooks = FALSE) (the defaults).
$ git commit ...
gives no error.
However, once we run $ pre-commit autoupdate (or precommit::autoupdate()), a new hook environment will only be initialized at commit, which - if done via the terminal in RStudio (not R prompt, system terminal) - leads to the same outcome as the first workflow.

What is unknown

if this happens only when {renv} is configured. Initial experiment suggest yes. Can others confirm?

How to go forward

It would be helpful to narrow down the issue further, i.e. instead of pre-commit install --install-hooks that calls pre_commit.languages.r.install_environment() -> calls {renv}, we could open interactive R from the RStudio system terminal and run the {renv} installation code directly to see if we can reproduce the problem as well that way. Update: It seems as the renv can't be loaded properly when RStudio -> system teriminal -> R is used. We get:

> source('renv/activate.R')
Warning message:
could not load renv from project "~/.cache/pre-commit/repoop2cc2mv/renv-default"; reloading previously-loaded renv 
>

Which means the project renv is loaded instead of the {precommit} renv. This should be reproducible for any two renvs.

Probably file an issue with {renv} or RStudio.
The workaround is:
- $ pre-commit clean, then
- $ pre-commit install --install-hooks in a directory that uses pre-commit, from outside RStudio.
- commit as usual, from RStudio or outside RStudio.
This workaround needs to be applied on every autoupdate as well.
This questions the usefulness of language: r.
Maybe we can detect if renv.lock is present in repo and warn the user, as the problem seems limited and most users are package developers who don't use {renv}.

Note that R should not be called from R without path, which might be a probem. The terminal in RStudio has for example $R_HOME set, which iTerm does not have. I hence adapted the upstream pre-commit framework to use full path to Rscript instead of just Rscript with lorenzwalthert/pre-commit@f09dfd8. But this did not help to overcome the problem when ran from Rstudio system terminal.
Maybe pre-commit could add an option --install-hooks to pre-commit autoupdate.

lorenzwalthert · 2021-12-25T10:56:37Z

@smingerson I think one workaround for you is to use the dev version of {renv} from GitHub now. Also requires dev version of {precommit}. Do you want to give it a try? Kevin updated it due to rstudio/renv#900.

lorenzwalthert · 2022-01-10T21:23:35Z

Closed with release v0.2.2.9009.

franperic · 2022-07-20T12:03:06Z

Hey
I ran into the same issues.
Specifically, we want to introduce precommit hooks to our R shiny project. We are using renv for the project. So I ran into the same problems described above.

I am using precommit v0.3.2 and still I experienced the issues from above.

The workaround worked for me. Thank you!
However, rolling out precommit hooks for all team members seems to be quite cumbersome.

Is there an update regarding this issue?

lorenzwalthert · 2022-07-21T08:17:19Z

What exactly do you mean by workaround?
Are you also using up to date versions of {renv}, pre-commit and the pre-commit hook version (in .pre-commit-config.yaml)?

franperic · 2022-07-21T08:46:49Z

What exactly do you mean by workaround?

Deleting .git/hooks/precommit, then
$ pre-commit clean, then
$ pre-commit install --install-hooks in a directory that uses pre-commit, from outside RStudio.
commit as usual, from RStudio or outside RStudio.

Are you also using up to date versions of {renv}, pre-commit and the pre-commit hook version (in .pre-commit-config.yaml)?

precommit v0.3.2
pre-commit-hooks v4.3.0
renv 0.15.1
In the process of the precommit setup I updated to renv 0.15.5

But I updated renv after using the workaround. So I guess this might have been the problem on my side.
Once I clean up the project code I will test the precommit setup with my colleagues.

I think for now my comment from before can be ignored.

lorenzwalthert · 2022-07-21T08:58:34Z

precommit v0.3.2
pre-commit-hooks v4.3.0
renv 0.15.1

This looks good. But you also have a pre-commit executable, which needs to be reasonably recent ideally, i.e. after pre-commit/pre-commit#2170, that is, at least v2.17.0. What does pre-commit --version and precommit::version_precommit() give you?

franperic · 2022-07-21T09:25:42Z

$ pre-commit --version
pre-commit 2.20.0

I think updating the used renv package would have solved my problems.
I will have to setup my colleagues environment to use precommits as well.
So I can check then and will update whether we encountered any issues.

lorenzwalthert · 2022-07-21T11:04:49Z

Ok great. Also note that if you have many projects and you don’t want to use the defaults config file (.pre-commit-config.yaml), you can store a template config file on a file system or on the web and set the R option precommit.config_source In your .Rprofile to always use this config file instead of the default one. See config_source for use_precommit().

lorenzwalthert · 2022-07-21T11:47:56Z

If an URL, it must be a raw file at the target, like this:
https://github.com/lorenzwalthert/precommit/raw/main/.pre-commit-config.yaml

you could contribute that example to the docs if you wanted under the examples of use_config().

lorenzwalthert changed the title ~~commit failure due to missing packages~~ {precommit} can't be used when a {renv} is already present in repo Dec 11, 2021

lorenzwalthert added Complexity: High Priority: Critical Type: Bug Status: Unassigned labels Dec 11, 2021

lorenzwalthert mentioned this issue Dec 12, 2021

Instruct users to downgrade to v0.1.3.9014 if renv.lock is present #346

Closed

lorenzwalthert mentioned this issue Jan 6, 2022

Release Scheduling #365

Closed

5 tasks

lorenzwalthert closed this as completed Jan 10, 2022

danielinteractive mentioned this issue Jun 28, 2022

precommit fails to set up environment after upgrade to R 4.2.1 #424

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

{precommit} can't be used when a {renv} is already present in repo #342

{precommit} can't be used when a {renv} is already present in repo #342

smingerson commented Dec 9, 2021

smingerson commented Dec 9, 2021 •

edited

lorenzwalthert commented Dec 9, 2021 •

edited

smingerson commented Dec 9, 2021

lorenzwalthert commented Dec 9, 2021

smingerson commented Dec 9, 2021

lorenzwalthert commented Dec 9, 2021

smingerson commented Dec 11, 2021 •

edited

lorenzwalthert commented Dec 11, 2021

lorenzwalthert commented Dec 11, 2021

smingerson commented Dec 11, 2021

lorenzwalthert commented Dec 11, 2021

lorenzwalthert commented Dec 11, 2021 •

edited

lorenzwalthert commented Dec 25, 2021 •

edited

lorenzwalthert commented Jan 10, 2022

franperic commented Jul 20, 2022

lorenzwalthert commented Jul 21, 2022 •

edited

franperic commented Jul 21, 2022

lorenzwalthert commented Jul 21, 2022

franperic commented Jul 21, 2022

lorenzwalthert commented Jul 21, 2022 •

edited

lorenzwalthert commented Jul 21, 2022 •

edited

{precommit} can't be used when a {renv} is already present in repo #342

{precommit} can't be used when a {renv} is already present in repo #342

Comments

smingerson commented Dec 9, 2021

smingerson commented Dec 9, 2021 • edited

lorenzwalthert commented Dec 9, 2021 • edited

smingerson commented Dec 9, 2021

lorenzwalthert commented Dec 9, 2021

smingerson commented Dec 9, 2021

lorenzwalthert commented Dec 9, 2021

smingerson commented Dec 11, 2021 • edited

lorenzwalthert commented Dec 11, 2021

lorenzwalthert commented Dec 11, 2021

smingerson commented Dec 11, 2021

lorenzwalthert commented Dec 11, 2021

lorenzwalthert commented Dec 11, 2021 • edited

Workflow with system terminal in RStudio (fails)

Workflow with system terminal outside RStudio (succeeds)

Workflow with R in RStudio

What is unknown

How to go forward

lorenzwalthert commented Dec 25, 2021 • edited

lorenzwalthert commented Jan 10, 2022

franperic commented Jul 20, 2022

lorenzwalthert commented Jul 21, 2022 • edited

franperic commented Jul 21, 2022

lorenzwalthert commented Jul 21, 2022

franperic commented Jul 21, 2022

lorenzwalthert commented Jul 21, 2022 • edited

lorenzwalthert commented Jul 21, 2022 • edited

smingerson commented Dec 9, 2021 •

edited

lorenzwalthert commented Dec 9, 2021 •

edited

smingerson commented Dec 11, 2021 •

edited

lorenzwalthert commented Dec 11, 2021 •

edited

lorenzwalthert commented Dec 25, 2021 •

edited

lorenzwalthert commented Jul 21, 2022 •

edited

lorenzwalthert commented Jul 21, 2022 •

edited

lorenzwalthert commented Jul 21, 2022 •

edited