Enable Past CI #17919

ydshieh · 2022-06-28T09:13:06Z

What does this PR do?

Enable Past CI

HuggingFaceDocBuilderDev · 2022-06-28T09:23:27Z

The documentation is not available anymore as the PR was closed or merged.

ydshieh · 2022-07-01T09:30:15Z

docker/transformers-past-gpu/Dockerfile

@@ -0,0 +1,51 @@
+ARG BASE_DOCKER_IMAGE="nvidia/cuda:11.2.2-cudnn8-devel-ubuntu20.04"


I will remove some commented blocks before merge - the installations which won't be included.

ydshieh · 2022-07-01T09:30:34Z

docker/transformers-past-gpu/Dockerfile

+### RUN python3 -m pip install --no-cache-dir git+https://github.com/facebookresearch/detectron2.git pytesseract https://github.com/kpu/kenlm/archive/master.zip
+RUN python3 -m pip install -U "itsdangerous<2.1.0"
+
+### RUN python3 -m pip install --no-cache-dir git+https://github.com/huggingface/accelerate@main#egg=accelerate


no accelerate

ydshieh · 2022-07-01T09:32:34Z

docker/transformers-past-gpu/Dockerfile

+### RUN python3 -m pip install --no-cache-dir torch-scatter -f https://data.pyg.org/whl/torch-$(python3 -c "from torch import version; print(version.__version__.split('+')[0])")+$CUDA.html
+### RUN python3 -m pip install --no-cache-dir intel_extension_for_pytorch==$INTEL_TORCH_EXT+cpu -f https://software.intel.com/ipex-whl-stable
+
+### RUN python3 -m pip install --no-cache-dir git+https://github.com/facebookresearch/detectron2.git pytesseract https://github.com/kpu/kenlm/archive/master.zip


not to include those 3rd party libs

ydshieh · 2022-07-01T09:32:50Z

docker/transformers-past-gpu/Dockerfile

+### RUN python3 -m pip install --no-cache-dir torch-scatter -f https://data.pyg.org/whl/torch-$(python3 -c "from torch import version; print(version.__version__.split('+')[0])")+$CUDA.html
+### RUN python3 -m pip install --no-cache-dir intel_extension_for_pytorch==$INTEL_TORCH_EXT+cpu -f https://software.intel.com/ipex-whl-stable
+
+### RUN python3 -m pip install --no-cache-dir git+https://github.com/facebookresearch/detectron2.git pytesseract https://github.com/kpu/kenlm/archive/master.zip


not to include those 3rd party libs

ydshieh · 2022-07-01T09:35:59Z

utils/past_ci_versions.py

@@ -0,0 +1,116 @@
+import argparse


This file is to make the installation commands easier to read in Docker files.
This should be used for other Docker files once this PR is merged.

For example, the following block hurts my head a bit already 😄

transformers/docker/transformers-pytorch-gpu/Dockerfile

Lines 19 to 21 in 49cd736

RUN [ ${#PYTORCH} -gt 0 ] && VERSION='torch=='$PYTORCH'.*' || VERSION='torch'; python3 -m pip install --no-cache-dir -U $VERSION --extra-index-url https://download.pytorch.org/whl/cu113

RUN [ ${#TORCH_VISION} -gt 0 ] && VERSION='torchvision=='TORCH_VISION'.*' || VERSION='torchvision'; python3 -m pip install --no-cache-dir -U $VERSION --extra-index-url https://download.pytorch.org/whl/cu113

RUN [ ${#TORCH_AUDIO} -gt 0 ] && VERSION='torchaudio=='TORCH_AUDIO'.*' || VERSION='torchaudio'; python3 -m pip install --no-cache-dir -U $VERSION --extra-index-url https://download.pytorch.org/whl/cu113

ydshieh · 2022-07-01T09:39:55Z

utils/past_ci_versions.py

+            "torch": "1.10.2",
+            "torchvision": "0.11.3",
+            "torchaudio": "0.10.2",
+            "python": 3.9,


This is the highest python version for that PyTorch version. Not really important

LysandreJik

This is great, super impressive! Let's merge it when you're happy with it and monitor failures. Please open an issue with the first reports when they're here so that we may discuss there which versions we'd like to keep or not.

LysandreJik · 2022-07-04T08:14:54Z

utils/past_ci_versions.py

+        # need python 3.7
+        # "1.3": {
+        #     "torch": "1.3.1",
+        #     "torchvision": "0.4.2",
+        #     "torchaudio": None,
+        #     "python": 3.7,
+        #     "cuda": "cu100",
+        #     "docker-base": "10.0-cudnn7-devel-ubuntu18.04",
+        #     "install": "python3 -m pip install --no-cache-dir -U torch==1.3.1 torchvision==0.4.2 torchaudio==0.4.0 --extra-index-url https://download.pytorch.org/whl/cu100",
+        # },


Let's remove it then

LysandreJik · 2022-07-04T08:15:12Z

utils/past_ci_versions.py

+        #     "install": "python3 -m pip install --no-cache-dir -U torch==1.3.1 torchvision==0.4.2 torchaudio==0.4.0 --extra-index-url https://download.pytorch.org/whl/cu100",
+        # },
+    },
+    # Should we include `tensorflow_probability`?


Let's do without it as a first step, let's start simple

LysandreJik · 2022-07-04T08:15:35Z

utils/past_ci_versions.py

+        # "2.3": {
+        #     "tensorflow": "2.3.4",
+        #     "install": "python3 -m pip install --no-cache-dir -U tensorflow==2.3.4",
+        # },


This one isn't available either in the python version ?

TF 2.3 requires CUDA 10.1 / cudnn 7.6, for which I don't find a good docker image on nvidia/.

(There is one for Ubuntu 18.04, but I remembered I have some trouble with it. I can re-run docker build to see what goes wrong)

Hi @LysandreJik . I remember what's wrong 😄

The best image I could find is nvidia/cuda:10.1-cudnn7-devel-ubuntu18.04, which has Python 3.6. During build, I got errors like

ERROR: No matching distribution found for accelerate>=0.10.0; extra == "dev"

Also, we have python": "python>=3.7.0 in setup.py.

As we discussed shortly before, I could try to build an image from scratch with python >= 3.7 with the necessary cuda/cudnn. I didn't do that though but just tried to launch the Past-CI with those versions easier to deal with.

I will merge this PR first, and if we would like to give a stab on TF 2.3, I can make it.

Let's merge it like this and ask the TF crew if they think 2.3 should be supported before putting too much efforts into it :)

ydshieh · 2022-07-18T13:02:37Z

merge it ... and monitor failures.

I read your comment too quickly. So far the Past CI will be triggered only on pushing to run-past-ci* branches. I ran it ~ June 20 however, and I opened #18181 today.

I think we can launch past CI monthly or even bimonthly. Please let me know if you have different opinion, @LysandreJik. Thanks.

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

ydshieh marked this pull request as draft June 28, 2022 10:04

Enable Past CI

aee2dbd

ydshieh marked this pull request as ready for review July 1, 2022 09:25

ydshieh force-pushed the past-ci branch from f51f1f2 to aee2dbd Compare July 1, 2022 09:25

ydshieh commented Jul 1, 2022

View reviewed changes

fix style

750d07d

ydshieh commented Jul 1, 2022

View reviewed changes

ydshieh requested a review from LysandreJik July 1, 2022 09:41

LysandreJik approved these changes Jul 4, 2022

View reviewed changes

ydshieh added 5 commits July 5, 2022 09:18

Add TF 2.3 in the list

9ed2c26

clean-up

00c6c71

try to install torch-scatter

4ca80d7

Disable torch-scatter installation

c630a3a

fix style

db0a986

ydshieh merged commit f681437 into main Jul 5, 2022

ydshieh deleted the past-ci branch July 5, 2022 16:08

viclzhu pushed a commit to viclzhu/transformers that referenced this pull request Jul 18, 2022

Enable Past CI (huggingface#17919)

236d170

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable Past CI #17919

Enable Past CI #17919

ydshieh commented Jun 28, 2022

HuggingFaceDocBuilderDev commented Jun 28, 2022 •

edited

ydshieh Jul 1, 2022

ydshieh Jul 1, 2022

ydshieh Jul 1, 2022

ydshieh Jul 1, 2022

ydshieh Jul 1, 2022

ydshieh Jul 1, 2022

LysandreJik left a comment

LysandreJik Jul 4, 2022

LysandreJik Jul 4, 2022

LysandreJik Jul 4, 2022

ydshieh Jul 4, 2022

ydshieh Jul 5, 2022 •

edited

LysandreJik Jul 5, 2022

ydshieh commented Jul 18, 2022 •

edited

		@@ -0,0 +1,51 @@
		ARG BASE_DOCKER_IMAGE="nvidia/cuda:11.2.2-cudnn8-devel-ubuntu20.04"

	RUN [ ${#PYTORCH} -gt 0 ] && VERSION='torch=='$PYTORCH'.*' \|\| VERSION='torch'; python3 -m pip install --no-cache-dir -U $VERSION --extra-index-url https://download.pytorch.org/whl/cu113
	RUN [ ${#TORCH_VISION} -gt 0 ] && VERSION='torchvision=='TORCH_VISION'.*' \|\| VERSION='torchvision'; python3 -m pip install --no-cache-dir -U $VERSION --extra-index-url https://download.pytorch.org/whl/cu113
	RUN [ ${#TORCH_AUDIO} -gt 0 ] && VERSION='torchaudio=='TORCH_AUDIO'.*' \|\| VERSION='torchaudio'; python3 -m pip install --no-cache-dir -U $VERSION --extra-index-url https://download.pytorch.org/whl/cu113

Enable Past CI #17919

Enable Past CI #17919

Conversation

ydshieh commented Jun 28, 2022

What does this PR do?

HuggingFaceDocBuilderDev commented Jun 28, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LysandreJik left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ydshieh Jul 5, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ydshieh commented Jul 18, 2022 • edited

HuggingFaceDocBuilderDev commented Jun 28, 2022 •

edited

ydshieh Jul 5, 2022 •

edited

ydshieh commented Jul 18, 2022 •

edited