Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Prevent dependencies issue #142

Merged
merged 7 commits into from May 18, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
27 changes: 27 additions & 0 deletions .github/workflows/ci.yml
Expand Up @@ -22,6 +22,10 @@ jobs:
task: format
os: ubuntu-latest
rust: stable
- make:
task: check-dependencies
os: ubuntu-latest
rust: stable
- make:
task: lint
os: ubuntu-latest
Expand Down Expand Up @@ -76,6 +80,15 @@ jobs:
steps:
- uses: actions/checkout@v2

- name: Check versions on crates.io
id: dep-versions
run: |
# load helper script
source .github/workflows/scripts/crates.io-script.bash
# get latest version from crates.io
echo "::set-output name=cargo-deny_version::$(get_crate_latest_version cargo-deny)"
# add other dependencies versions here

- name: Install Rust ${{ matrix.rust }}
if: ${{ ! matrix.make.rust-free }}
uses: actions-rs/toolchain@v1
Expand All @@ -98,6 +111,20 @@ jobs:
${{ env.CACHE_RESET_KEY }}-${{ runner.os }}-${{ matrix.rust }}-cargo-${{ matrix.make.task }}-
${{ env.CACHE_RESET_KEY }}-${{ runner.os }}-${{ matrix.rust }}-cargo-

- name: Cache install cargo-deny
id: cache-cargo-deny
uses: actions/cache@v2
continue-on-error: false
with:
path: |
~/.cargo/bin/cargo-deny
key: ${{env.CACHE_RESET_KEY}}-${{ runner.os }}-${{ matrix.rust }}-cargo-deny-${{ steps.dep-versions.outputs.cargo-deny_version }}

- name: Install cargo-deny
if: ${{ steps.cache-cargo-deny.outputs.cache-hit == false && matrix.rust == 'stable'}}
run: |
cargo install cargo-deny

- name: Install cargo-make
env:
BASE_URL: https://github.com/sagiegurari/cargo-make/releases/download
Expand Down
13 changes: 13 additions & 0 deletions .github/workflows/scripts/crates.io-script.bash
@@ -0,0 +1,13 @@

# function: get_crate_latest_version
# arg#1: crate name
# returns: latest version number on crates.io
#
# this calls crates.io REST api via curl, that listing all versions of specified crate.
# `jq` query filter for not yanked and extract version number.
# sort semver and takes last one.
function get_crate_latest_version() {
curl -L https://crates.io/api/v1/crates/$1/versions | jq -r '.versions[] | select( .yanked == false) | .num' | sort --version-sort | tail -n 1
}

export -f get_crate_latest_version
8 changes: 8 additions & 0 deletions Makefile.toml
Expand Up @@ -28,11 +28,19 @@ script = ['''
RUSTFLAGS='-D warnings' cargo clippy --workspace --all-targets --all-features
''']

[tasks.check-dependencies]
condition = { rust_version = { min = "1.56.1" } } # cargo-deny 0.11.4 : supports rust 1.56.1 later
script = ['''
#!/usr/bin/env bash -eux
cargo deny check
''']

[tasks.build]
script = ['''
#!/usr/bin/env bash -eux
RUSTFLAGS='-D warnings' cargo build --workspace --all-targets --all-features
''']
dependencies=["check-dependencies"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems too severe.

build should run even with bad dependencies.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build process executes the crate build scripts and procedural macros that came in from the dependencies.

For this reason, we need to verify that the dependencies are safe before building, which is represented by dependencies.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build process executes the crate build scripts and procedural macros

Yes. They can be malicious.

But I don't think putting this dependency saves developers (they might use cargo build).

I think it's OK to run malicious codes in CI's containers at build time.

IMO, running check-dependencies as a required job on pull-request should be enough.
(Maybe better to run it also on creating release artifacts)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I don't think putting this dependency saves developers (they might use cargo build).

The problem is certainly not solved this way.

I'm looking for a solution, but I haven't found a solution at the moment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ref: #148


[tasks.test]
script = ['''
Expand Down
206 changes: 206 additions & 0 deletions deny.toml
@@ -0,0 +1,206 @@
# This template contains all of the possible sections and their default values

# Note that all fields that take a lint level have these possible values:
# * deny - An error will be produced and the check will fail
# * warn - A warning will be produced, but the check will not fail
# * allow - No warning or error will be produced, though in some cases a note
# will be

# The values provided in this template are the default values that will be used
# when any section or field is not specified in your own configuration

# If 1 or more target triples (and optionally, target_features) are specified,
# only the specified targets will be checked when running `cargo deny check`.
# This means, if a particular package is only ever used as a target specific
# dependency, such as, for example, the `nix` crate only being used via the
# `target_family = "unix"` configuration, that only having windows targets in
# this list would mean the nix crate, as well as any of its exclusive
# dependencies not shared by any other crates, would be ignored, as the target
# list here is effectively saying which targets you are building for.
targets = [
# The triple can be any string, but only the target triples built in to
# rustc (as of 1.40) can be checked against actual config expressions
#{ triple = "x86_64-unknown-linux-musl" },
# You can also specify which target_features you promise are enabled for a
# particular target. target_features are currently not validated against
# the actual valid features supported by the target architecture.
#{ triple = "wasm32-unknown-unknown", features = ["atomics"] },
]

# This section is considered when running `cargo deny check advisories`
# More documentation for the advisories section can be found here:
# https://embarkstudios.github.io/cargo-deny/checks/advisories/cfg.html
[advisories]
# The path where the advisory database is cloned/fetched into
db-path = "~/.cargo/advisory-db"
# The url(s) of the advisory databases to use
db-urls = ["https://github.com/rustsec/advisory-db"]
# The lint level for security vulnerabilities
vulnerability = "deny"
# The lint level for unmaintained crates
unmaintained = "warn"
# The lint level for crates that have been yanked from their source registry
yanked = "warn"
# The lint level for crates with security notices. Note that as of
# 2019-12-17 there are no security notice advisories in
# https://github.com/rustsec/advisory-db
notice = "warn"
# A list of advisory IDs to ignore. Note that ignored advisories will still
# output a note when they are encountered.
ignore = [
#"RUSTSEC-0000-0000",
]
# Threshold for security vulnerabilities, any vulnerability with a CVSS score
# lower than the range specified will be ignored. Note that ignored advisories
# will still output a note when they are encountered.
# * None - CVSS Score 0.0
# * Low - CVSS Score 0.1 - 3.9
# * Medium - CVSS Score 4.0 - 6.9
# * High - CVSS Score 7.0 - 8.9
# * Critical - CVSS Score 9.0 - 10.0
#severity-threshold =

# This section is considered when running `cargo deny check licenses`
# More documentation for the licenses section can be found here:
# https://embarkstudios.github.io/cargo-deny/checks/licenses/cfg.html
[licenses]
# The lint level for crates which do not have a detectable license
unlicensed = "deny"
# List of explicitly allowed licenses
# See https://spdx.org/licenses/ for list of possible licenses
# [possible values: any SPDX 3.11 short identifier (+ optional exception)].
allow = [
"MIT",
"Apache-2.0",
"BSD-3-Clause",
laysakura marked this conversation as resolved.
Show resolved Hide resolved
#"Apache-2.0 WITH LLVM-exception",
]
# List of explicitly disallowed licenses
# See https://spdx.org/licenses/ for list of possible licenses
# [possible values: any SPDX 3.11 short identifier (+ optional exception)].
deny = [
#"Nokia",
]
# Lint level for licenses considered copyleft
copyleft = "warn"
# Blanket approval or denial for OSI-approved or FSF Free/Libre licenses
# * both - The license will be approved if it is both OSI-approved *AND* FSF
# * either - The license will be approved if it is either OSI-approved *OR* FSF
# * osi-only - The license will be approved if is OSI-approved *AND NOT* FSF
# * fsf-only - The license will be approved if is FSF *AND NOT* OSI-approved
# * neither - This predicate is ignored and the default lint level is used
allow-osi-fsf-free = "neither"
# Lint level used when no other predicates are matched
# 1. License isn't in the allow or deny lists
# 2. License isn't copyleft
# 3. License isn't OSI/FSF, or allow-osi-fsf-free = "neither"
default = "deny"
# The confidence threshold for detecting a license from license text.
# The higher the value, the more closely the license text must be to the
# canonical license text of a valid SPDX license file.
# [possible values: any between 0.0 and 1.0].
confidence-threshold = 0.8
# Allow 1 or more licenses on a per-crate basis, so that particular licenses
# aren't accepted for every possible crate as with the normal allow list
exceptions = [
# Each entry is the crate and version constraint, and its specific allow
# list
#{ allow = ["Zlib"], name = "adler32", version = "*" },
]

# Some crates don't have (easily) machine readable licensing information,
# adding a clarification entry for it allows you to manually specify the
# licensing information
#[[licenses.clarify]]
# The name of the crate the clarification applies to
#name = "ring"
# The optional version constraint for the crate
#version = "*"
# The SPDX expression for the license requirements of the crate
#expression = "MIT AND ISC AND OpenSSL"
# One or more files in the crate's source used as the "source of truth" for
# the license expression. If the contents match, the clarification will be used
# when running the license check, otherwise the clarification will be ignored
# and the crate will be checked normally, which may produce warnings or errors
# depending on the rest of your configuration
#license-files = [
# Each entry is a crate relative path, and the (opaque) hash of its contents
#{ path = "LICENSE", hash = 0xbd0eed23 }
#]

[licenses.private]
# If true, ignores workspace crates that aren't published, or are only
# published to private registries.
# To see how to mark a crate as unpublished (to the official registry),
# visit https://doc.rust-lang.org/cargo/reference/manifest.html#the-publish-field.
ignore = false
# One or more private registries that you might publish crates to, if a crate
# is only published to private registries, and ignore is true, the crate will
# not have its license(s) checked
registries = [
#"https://sekretz.com/registry
]

# This section is considered when running `cargo deny check bans`.
# More documentation about the 'bans' section can be found here:
# https://embarkstudios.github.io/cargo-deny/checks/bans/cfg.html
[bans]
# Lint level for when multiple versions of the same crate are detected
multiple-versions = "warn"
# Lint level for when a crate version requirement is `*`
wildcards = "allow"
# The graph highlighting used when creating dotgraphs for crates
# with multiple versions
# * lowest-version - The path to the lowest versioned duplicate is highlighted
# * simplest-path - The path to the version with the fewest edges is highlighted
# * all - Both lowest-version and simplest-path are used
highlight = "all"
# List of crates that are allowed. Use with care!
allow = [
#{ name = "ansi_term", version = "=0.11.0" },
]
# List of crates to deny
deny = [
# Each entry the name of a crate and a version range. If version is
# not specified, all versions will be matched.
#{ name = "ansi_term", version = "=0.11.0" },
#
# Wrapper crates can optionally be specified to allow the crate when it
# is a direct dependency of the otherwise banned crate
#{ name = "ansi_term", version = "=0.11.0", wrappers = [] },
]
# Certain crates/versions that will be skipped when doing duplicate detection.
skip = [
#{ name = "ansi_term", version = "=0.11.0" },
]
# Similarly to `skip` allows you to skip certain crates during duplicate
# detection. Unlike skip, it also includes the entire tree of transitive
# dependencies starting at the specified crate, up to a certain depth, which is
# by default infinite
skip-tree = [
#{ name = "ansi_term", version = "=0.11.0", depth = 20 },
]

# This section is considered when running `cargo deny check sources`.
# More documentation about the 'sources' section can be found here:
# https://embarkstudios.github.io/cargo-deny/checks/sources/cfg.html
[sources]
# Lint level for what to happen when a crate from a crate registry that is not
# in the allow list is encountered
unknown-registry = "warn"
# Lint level for what to happen when a crate from a git repository that is not
# in the allow list is encountered
unknown-git = "warn"
# List of URLs for allowed crate registries. Defaults to the crates.io index
# if not specified. If it is specified but empty, no registries are allowed.
allow-registry = ["https://github.com/rust-lang/crates.io-index"]
# List of URLs for allowed Git repositories
allow-git = ["https://github.com/laysakura/simple-server"]

[sources.allow-org]
# 1 or more github.com organizations to allow git sources for
# github = [""]
# 1 or more gitlab.com organizations to allow git sources for
# gitlab = [""]
# 1 or more bitbucket.org organizations to allow git sources for
# bitbucket = [""]