Add git-lfs Large File Storage support #40
Conversation
|
Can this cause any build reproducibility issues between different distros? i.e. Would the behavior be the same between Fedora 29 and Ubuntu 18.04, but not between Fedora 29 and Ubuntu 16.04. I ask because that was one useful aspect, being able to check for common behavior across distros. |
moto-timo
force-pushed
the
git-lfs
branch
2 times, most recently
from
dee3586
to
9a535ed
Compare
|
@rewitt1 I completely agree about your concern, so the most recent updates to the pull request address debian-9, ubuntu-16.04 and opensuse-15.0,15.1 |
| # wget -O - https://packagecloud.io/github/git-lfs/gpgkey | gpg --with-fingerprint | ||
| RUN apt-get update && apt-get install -y apt-transport-https && \ | ||
| echo "deb https://packagecloud.io/github/git-lfs/ubuntu/ xenial main" >> /etc/apt/sources.list && \ | ||
| apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 6D398DBD30DD78941E2C4797FE2A5F8BDC282033 && \ |
There was a problem hiding this comment.
Is there a fingerprint for the key you can use and verify, since the key will be downloaded every time the image is built?
The manpage for apt-key even says
With adv --recv-key you can e.g. download key from
keyservers directly into the trusted set of keys. Note that there are no checks
performed, so it is easy to completely undermine the apt-secure(8) infrastructure if
used without care.
Does the same question apply in the zypper case?
There was a problem hiding this comment.
@rewitt1 --recv-keys 6D398DBD30DD78941E2C4797FE2A5F8BDC282033 is the PGP fingerprint. It seemed better to hard code a verifiable fingerprint than to trust running a shell script or carrying the gpg key as an artifact. This also applies for zypper because OpenSUSE has no packaging for git-lfs, at least not in the 15.1 release. As soon as we feel comfortable dropping Ubuntu 16.04 and OpenSUSE 15.2 is released we can get rid of this packagecloud workaround.
"Git Large File Storage (LFS) replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server." https://git-lfs.github.com/ This is useful when including proprietary pre-built binaries and also when adding virtualization guests, such as Windows as a Guest or Android as a Guest. Since http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=a2d6792f3a531b557bffaec4387d8e0e4eb3dab0 git-lfs is a non-fatal HOST_TOOLS addition, meaning it will be linked if present. debian-9 only supports it with stretch-backports, patch to follow opensuse-15.0,-15.1 do not have git-lfs except in package cloud, patch to follow ubuntu-16.04 does not have git-lfs except in package cloud, patch to follow Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Add stretch-backports to /etc/apt/sources.list and use it to install git-lfs. Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Add "github_git-lfs" rpm repo excerpted from install script: https://packagecloud.io/github/git-lfs/install#bash-rpm (e.g. https://packagecloud.io/install/repositories/github/git-lfs/config_file.repo?os=opensuse&dist=15.0&source=script) Automatically add gpg keys with: zypper --gpg-auto-import-keys refresh github_git-lfs Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Install gpg key from wget https://packagecloud.io/github/git-lfs/gpgkey Install apt-transport-https as packagecloud is https only Add packagecloud apt repo, as excerpted from install script: https://packagecloud.io/github/git-lfs/install#bash-deb (e.g. https://packagecloud.io/install/repositories/github/git-lfs/config_file.list?os=ubuntu&dist=xenial&source=script) Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
|
Hi guys, any chance to get this merged? Don't know how busy you are and if I can take over. |
I can merge it if it gets updated. But right now, I don't have time to update it myself. I'll circle back in a few weeks when I have more time, and if it hasn't been updated, then I'll try to help out with updating the pull request. |
|
Internally we went a different route to add "repo" and "git-lfs" via a derivative container model (which is in the process of being open sourced). The issue with "git-lfs" specifically was that it wasn't directly installable as a package for all the items in the supported Distro/Version matrix. Ergo, I have no motivation to rebase the PR at this time. Happy to have you take it over @dagmcr if you have a driving need and can bring it up to date. It is possible the Distro/Version package manager issue is no longer a problem (for "git-lfs" specifically). |
Although ubuntu-16.04 is in Extended Security Maintenance until 2026, it is holding us back from adding git-lfs (see crops#40) Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Although ubuntu-16.04 is in Extended Security Maintenance until 2026, it is holding us back from adding git-lfs (see crops#40) Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Although ubuntu-16.04 is in Extended Security Maintenance until 2026, it is holding us back from adding git-lfs (see crops#40) Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Although ubuntu-16.04 is in Extended Security Maintenance until 2026, it is holding us back from adding git-lfs (see crops#40) Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Although ubuntu-16.04 is in Extended Security Maintenance until 2026, it is holding us back from adding git-lfs (see crops#40) Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Although ubuntu-16.04 is in Extended Security Maintenance until 2026, it is holding us back from adding git-lfs (see #40) Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Ubuntu "Jammy Jellyfish" 22.04 has been released, with long-term support until at least 2027 (maintenance) and 2032 (extended security maintenance). https://ubuntu.com/about/release-cycle https://ubuntu.com/blog/ubuntu-22-04-lts-released Although ubuntu-16.04 is in Extended Security Maintenance until 2026, it is holding us back from adding git-lfs (see crops/yocto-dockerfiles#40). Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Ubuntu "Jammy Jellyfish" 22.04 has been released, with long-term support until at least 2027 (maintenance) and 2032 (extended security maintenance). https://ubuntu.com/about/release-cycle https://ubuntu.com/blog/ubuntu-22-04-lts-released Although ubuntu-16.04 is in Extended Security Maintenance until 2026, it is holding us back from adding git-lfs (see crops/yocto-dockerfiles#40). Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Ubuntu "Jammy Jellyfish" 22.04 has been released, with long-term support until at least 2027 (maintenance) and 2032 (extended security maintenance). https://ubuntu.com/about/release-cycle https://ubuntu.com/blog/ubuntu-22-04-lts-released Although ubuntu-16.04 is in Extended Security Maintenance until 2026, it is holding us back from adding git-lfs (see crops/yocto-dockerfiles#40). Signed-off-by: Tim Orling <tim.orling@konsulko.com>
"Git Large File Storage (LFS) replaces large files such as audio samples,
videos, datasets, and graphics with text pointers inside Git, while storing
the file contents on a remote server."
https://git-lfs.github.com/
This is useful when including proprietary pre-built binaries and also
when adding virtualization guests, such as Windows as a Guest or
Android as a Guest.
Since http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=a2d6792f3a531b557bffaec4387d8e0e4eb3dab0
git-lfs is a non-fatal HOST_TOOLS addition, meaning it will be linked if present.
debian-9 only supports it with stretch-backports, so skip
opensuse-15.0,-15.1 do not have git-lfs except in community packages, so skip
ubuntu-16.04 does not have git-lfs except in package cloud, so skip
If someone really wants support in those distro versions, patches are welcome.
Signed-off-by: Tim Orling timothy.t.orling@linux.intel.com