Build R documentation using Docker #5188

harupy · 2021-12-21T11:05:37Z

What changes are proposed in this pull request?

Build the R documentation using docker to make it easier to run and test + simplify the CircleCI workflow.

How is this patch tested?

Confirm the R documentation can be built without errors.
Run our internal Jenkins job for documentation build against this PR's branch and confirm it succeeds.

Does this PR change the documentation?

No. You can skip the rest of this section.
Yes. Make sure the changed pages / sections render correctly by following the steps below.

Check the status of the ci/circleci: build_doc check. If it's successful, proceed to the
next step, otherwise fix it.
Click Details on the right to open the job page of CircleCI.
Click the Artifacts tab.
Click docs/build/html/index.html.
Find the changed pages / sections and make sure they render correctly.

Release Notes

Is this a user-facing change?

No. You can skip the rest of this section.
Yes. Give a description of this change to be included in the release notes for MLflow users.

(Details in 1-2 sentences. You can just refer to another PR with a description if this PR is part of a larger change.)

What component(s), interfaces, languages, and integrations does this PR affect?

Components

Interface

area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
area/windows: Windows support

Language

language/r: R APIs and clients
language/java: Java APIs and clients
language/new: Proposals for new client languages

Integrations

integrations/azure: Azure and Azure ML integrations
integrations/sagemaker: SageMaker integrations
integrations/databricks: Databricks integrations

How should the PR be classified in the release notes? Choose one:

rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
rn/feature - A new user-facing feature worth mentioning in the release notes
rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
rn/documentation - A user-facing documentation change worth mentioning in the release notes

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

harupy · 2021-12-22T08:10:32Z

mlflow/R/mlflow/Dockerfile.dev

+FROM rocker/r-ver:4.1.2
+
+WORKDIR /mlflow/mlflow/R/mlflow
+RUN apt-get update -y
+RUN apt-get install git wget libxml2-dev libgit2-dev -y
+RUN TEMP_DEB=$(mktemp) && \
+    wget --directory-prefix $TEMP_DEB https://github.com/jgm/pandoc/releases/download/2.16.2/pandoc-2.16.2-1-amd64.deb && \
+    dpkg --install $(find $TEMP_DEB -name '*.deb') && \
+    rm -rf $TEMP_DEB
+COPY DESCRIPTION .
+COPY .install-deps.R .
+RUN Rscript -e 'source(".install-deps.R", echo = TRUE)'


Renamed Docker.build to Docker.dev and add packages required to build the docs.

harupy · 2021-12-22T08:13:40Z

docs/source/R-api.rst

+-----------+---------------------------------------------------------+
+| Argument  | Description                                             |
+===========+=========================================================+
+| ``model`` | The loaded MLflow model flavor.                         |
+-----------+---------------------------------------------------------+
+| ``data``  | A data frame to perform scoring.                        |
+-----------+---------------------------------------------------------+
+| ``...``   | Optional additional arguments passed to underlying      |
+|           | predict methods.                                        |
+-----------+---------------------------------------------------------+


I updated the version of pandoc to 2.16.2 from 2.7.1, which added some changes in this file.

harupy · 2021-12-22T08:14:59Z

.circleci/config.yml

-      - run:
-          name: Install dev tools
-          environment:
-            DEBIAN_FRONTEND: noninteractive
-          command: |
-            apt-get update --yes
-            apt-get install sudo git wget curl jq software-properties-common apt-transport-https --yes
-
-      - run:
-          name: Install Java
-          command: |
-            sudo apt-get install default-jdk --yes
-            java -version
-
-      - run:
-          name: Install R
-          command: |
-            # How To Install R on Ubuntu 20.04:
-            # https://www.digitalocean.com/community/tutorials/how-to-install-r-on-ubuntu-20-04
-
-            sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9
-            sudo add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu focal-cran40/'
-            sudo apt update -y
-            sudo apt install -y r-base libssl-dev libxml2-dev libcurl4-openssl-dev
-            R --version
-
-      - run:
-          name: Install pandoc
-          command: |
-            # Install a recent version of pandoc
-            TEMP_DEB="$(mktemp)"
-            wget -O "$TEMP_DEB" 'https://github.com/jgm/pandoc/releases/download/2.7.2/pandoc-2.7.2-1-amd64.deb'
-            sudo dpkg -i "$TEMP_DEB"
-            rm -f "$TEMP_DEB"
-
-      - run:
-          name: Dump R dependencies
-          working_directory: mlflow/R/mlflow
-          command: |
-            Rscript .dump-r-dependencies.R
-
-      - restore_cache:
-          keys:
-            - r-cache-{{ checksum "mlflow/R/mlflow/R-version" }}
-


Removed these steps because the docker image contains all required tools.

harupy · 2021-12-22T08:17:32Z

.circleci/config.yml

-    docker:
-      - image: ubuntu:20.04
+    machine:
+      image: ubuntu-2004:202111-01


To use docker volume mounting, we need to use the machine executor:
https://support.circleci.com/hc/en-us/articles/360007324514-How-can-I-use-Docker-volume-mounting-on-CircleCI-

WeichenXu123 · 2021-12-22T10:08:09Z

mlflow/R/mlflow/.build-doc.R

+# Note that this commit is equivalent to commit 6b48255 of Rd2md master
+# (https://github.com/quantsch/Rd2md/tree/6b4825579a2df8a22898316d93729384f92a756b)
+# with a single extra commit to fix rendering of \link tags between methods in R documentation.
+devtools::install_git("https://github.com/smurching/Rd2md", ref = "mlflow-patches")


Shall we use commit hash instead of branch which may be changed ?

Good point, but we usually don't update this branch.

WeichenXu123 · 2021-12-22T10:15:01Z

mlflow/R/mlflow/Dockerfile.dev

+RUN apt-get install git wget libxml2-dev libgit2-dev -y
+RUN TEMP_DEB=$(mktemp) && \
+    wget --directory-prefix $TEMP_DEB https://github.com/jgm/pandoc/releases/download/2.16.2/pandoc-2.16.2-1-amd64.deb && \
+    dpkg --install $(find $TEMP_DEB -name '*.deb') && \


Can we install pandoc by RUN apt-get install pandoc -y ?

The version installed by pandoc is 2.5 which is too old and contains a bug.

I inserted a comment.

WeichenXu123

LGTM

github-actions bot added the rn/none List under Small Changes in Changelogs. label Dec 21, 2021

harupy added 2 commits December 22, 2021 13:54

rename dockerfile

c40663f

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

build R doc using docker

f240662

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

harupy force-pushed the install-r-dependencies branch from e03ad6d to f240662 Compare December 22, 2021 04:55

harupy added 3 commits December 22, 2021 14:35

update .Rbuildignore

c40993d

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

fix volume target

eebd9d6

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

use working_directory

2ec6db1

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

harupy commented Dec 22, 2021

View reviewed changes

WeichenXu123 reviewed Dec 22, 2021

View reviewed changes

WeichenXu123 approved these changes Dec 22, 2021

View reviewed changes

add comment

26734cd

harupy merged commit 4deb9b7 into mlflow:master Dec 22, 2021

harupy deleted the install-r-dependencies branch December 22, 2021 12:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build R documentation using Docker #5188

Build R documentation using Docker #5188

harupy commented Dec 21, 2021 •

edited

harupy Dec 22, 2021

harupy Dec 22, 2021

harupy Dec 22, 2021

harupy Dec 22, 2021

WeichenXu123 Dec 22, 2021

harupy Dec 22, 2021

WeichenXu123 Dec 22, 2021

harupy Dec 22, 2021

harupy Dec 22, 2021

WeichenXu123 left a comment

Build R documentation using Docker #5188

Build R documentation using Docker #5188

Conversation

harupy commented Dec 21, 2021 • edited

What changes are proposed in this pull request?

How is this patch tested?

Does this PR change the documentation?

Release Notes

Is this a user-facing change?

What component(s), interfaces, languages, and integrations does this PR affect?

How should the PR be classified in the release notes? Choose one:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WeichenXu123 left a comment

Choose a reason for hiding this comment

harupy commented Dec 21, 2021 •

edited