Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dockerize R package build #5182

Merged
merged 4 commits into from Dec 21, 2021
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/master.yml
Expand Up @@ -137,6 +137,10 @@ jobs:
Rscript -e 'covr::codecov()' || :
env:
COVR_RUNNING: true
- name: Test package build
working-directory: mlflow/R/mlflow
run: |
./build-package.sh
Comment on lines +140 to +143
Copy link
Member Author

@harupy harupy Dec 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step takes about 5 minutes, which I think is acceptable:

https://github.com/mlflow/mlflow/runs/4577625774?check_suite_focus=true#step:16:1

- name: Show 00check.log on failure
if: ${{ failure() }}
run: |
Expand Down
4 changes: 4 additions & 0 deletions mlflow/R/mlflow/.Rbuildignore
Expand Up @@ -19,3 +19,7 @@ Reference_Manual_mlflow.md
^logs/
^depends\.Rds
^R-version
^\.utils\.R$
^\.build-package\.R$
^build-package\.sh$
^Dockerfile\.build$
16 changes: 16 additions & 0 deletions mlflow/R/mlflow/.build-package.R
@@ -0,0 +1,16 @@
source(".utils.R")

# Increase the timeout length for `utils::download.file` because the default value (60 seconds)
# could be too short to download large packages such as h2o.
options(timeout=300)
# Install dependencies required for the submission check.
devtools::install_deps(".", dependencies = TRUE)
# Bundle up the package into a .tar.gz file. This file will be submitted to CRAN.
package_path <- devtools::build(".")
# Run the submission check against the built package.
devtools::check_built(
package_path,
remote = should_enable_cran_incoming_checks(),
error_on = "note",
args = c("--no-tests", "--as-cran"),
)
1 change: 1 addition & 0 deletions mlflow/R/mlflow/.gitignore
Expand Up @@ -19,3 +19,4 @@ Reference_Manual_mlflow.md
*.Rcheck/
depends.Rds
R-version
*.tar.gz
25 changes: 8 additions & 17 deletions mlflow/R/mlflow/.run-tests.R
@@ -1,24 +1,15 @@
source("../.utils.R")

parent_dir <- dir("../", full.names = TRUE)
package <- parent_dir[grepl("mlflow_", parent_dir)]

library(reticulate)
use_condaenv(mlflow:::mlflow_conda_env_name())

# Disable CRAN incoming feasibility check within a week after the latest release because it fails.
#
# Relevant code:
# https://github.com/wch/r-source/blob/4561aea946a75425ddcc8869cdb129ed5e27af97/src/library/tools/R/QC.R#L8005-L8008
install.packages(c("xml2", "rvest"))
library(xml2)
library(rvest)

url <- "https://cran.r-project.org/web/packages/mlflow/index.html"
html <- read_html(url)
xpath <- '//td[text()="Published:"]/following-sibling::td[1]/text()'
published_date <- as.Date(html_text(html_nodes(html, xpath=xpath)))
today <- Sys.Date()
days_since_last_release <- difftime(today, published_date, units="days")
remote <- as.numeric(days_since_last_release) > 7

devtools::check_built(path = package, remote = remote, error_on = "note", args = "--no-tests")
devtools::check_built(
path = package,
remote = should_enable_cran_incoming_checks(),
error_on = "note",
args = "--no-tests"
)
source("testthat.R")
18 changes: 18 additions & 0 deletions mlflow/R/mlflow/.utils.R
@@ -0,0 +1,18 @@

# This script defines utility functions only used during development.

should_enable_cran_incoming_checks <- function() {
# The CRAN incoming feasibility check performs a package recency check (this is undocumented)
# that examines the number of days since the last release and raises a NOTE if it's < 7.
# Relevant code:
# https://github.com/wch/r-source/blob/4561aea946a75425ddcc8869cdb129ed5e27af97/src/library/tools/R/QC.R#L8005-L8008
# This check needs to be disabled for a week after releasing a new version.
desc_url <- url("https://cran.r-project.org/web/packages/mlflow/DESCRIPTION")
field <- "Date/Publication"
desc <- read.dcf(desc_url, fields = c(field))
close(desc_url)
publication_date <- as.Date(unname(desc[1, field]))
today <- Sys.Date()
days_since_last_release <- as.numeric(difftime(today, publication_date, units="days"))
days_since_last_release > 7
}
5 changes: 5 additions & 0 deletions mlflow/R/mlflow/Dockerfile.build
@@ -0,0 +1,5 @@
FROM rocker/r-ver:4.1.2
Copy link
Member Author

@harupy harupy Dec 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rocker/r-ver is maintained by R maintainers:
https://hub.docker.com/r/rocker/r-ver


RUN apt-get update -y
RUN apt-get install pandoc -y
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

devtools::check_built needs pandoc to analyze README.

RUN Rscript -e 'install.packages("devtools", dependencies = TRUE)'
5 changes: 5 additions & 0 deletions mlflow/R/mlflow/build-package.sh
@@ -0,0 +1,5 @@
#!/usr/bin/env bash
set -ex

docker build -f Dockerfile.build -t r-build-package .
docker run --rm --workdir /app -v $(pwd):/app r-build-package Rscript -e 'source(".build-package.R", echo = TRUE)'