Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Safety 2.2.0 patch #407

Merged
merged 20 commits into from Sep 19, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
efcb321
Eat stderr messages from git commands.
tarmack Jul 26, 2022
e010baa
Remove unused import of safety package in utils.
tarmack Jul 26, 2022
7895e10
Merge pull request #399 from tarmack/develop
yeisonvargasf Aug 22, 2022
3b43e4e
Merge pull request #404 from pyupio/master
yeisonvargasf Sep 14, 2022
69af896
Starting version 2.2.0.dev
yeisonvargasf Sep 14, 2022
341507b
Adding basic github action workflow
yeisonvargasf Sep 14, 2022
718bb7b
Test action only in main branches
yeisonvargasf Sep 14, 2022
51ad4ad
Building image only on master branch
yeisonvargasf Sep 14, 2022
6735a3e
Merge pull request #405 from pyupio/feature/remove-appveyor-travis
yeisonvargasf Sep 14, 2022
815e812
Fixes on WIN binaries generation
yeisonvargasf Sep 14, 2022
de5ba12
Uploading binaries depending on the right os
yeisonvargasf Sep 14, 2022
3a7f560
Upgrading dparse because a ReDoS security issue - PVE-2022-50571 - GH…
yeisonvargasf Sep 14, 2022
1aea37e
Using dparse to read requirements and fixes for custom integrations
yeisonvargasf Sep 14, 2022
d91950b
Handling unpinned dep and make a warn about that
yeisonvargasf Sep 15, 2022
77f8a7a
Merge pull request #406 from pyupio/feature/use-dparse-to-parsing-dep…
yeisonvargasf Sep 15, 2022
c95eeca
Adding more logging
yeisonvargasf Sep 15, 2022
af2776f
Debug log level on binaries testing
yeisonvargasf Sep 15, 2022
5b953cf
More logging...
yeisonvargasf Sep 15, 2022
6423099
Including the dep file in the logging
yeisonvargasf Sep 15, 2022
003eb3a
Using the latest dparse version
yeisonvargasf Sep 19, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/build.yml
@@ -1,6 +1,8 @@
name: Safety Action Build And Publish

on: [push]
on:
push:
branches: [master]

env:
DOCKER_BUILDKIT: 1
Expand Down
117 changes: 117 additions & 0 deletions .github/workflows/main.yml
@@ -0,0 +1,117 @@
name: Python package

on: [ push ]

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [ "3.6", "3.7", "3.8", "3.9", "3.10" ]
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Display Python version
run: python -c "import sys; print(sys.version)"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r test_requirements.txt
- name: Test with pytest
run: |
pytest -rP tests/ --cov=dparse/ --cov-report=xml --cov-report=html

build-binaries:
needs: test
runs-on: ${{ matrix.os }}

if: contains(fromJson('["refs/heads/master", "refs/heads/develop", "refs/heads/binaries-fixes"]'), github.ref) || startsWith(github.ref, 'refs/tags')

strategy:
matrix:
os: ['windows-latest', 'ubuntu-latest', 'macos-latest']
env:
BINARY_OS: '${{ matrix.os }}'
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Install Dependencies
run: python binaries.py install
- name: Test Safety
run: python binaries.py test
- name: Producing Binaries
run: python binaries.py dist
- uses: actions/upload-artifact@v3
if: ${{ matrix.os == 'windows-latest' }}
with:
name: safety-win-i686.exe
path: dist/safety-win-i686.exe
if-no-files-found: error
- uses: actions/upload-artifact@v3
if: ${{ matrix.os == 'windows-latest' }}
with:
name: safety-win-x86_64.exe
path: dist/safety-win-x86_64.exe
if-no-files-found: error
- uses: actions/upload-artifact@v3
if: ${{ matrix.os == 'ubuntu-latest' }}
with:
name: safety-linux-i686
path: dist/safety-linux-i686
if-no-files-found: error
- uses: actions/upload-artifact@v3
if: ${{ matrix.os == 'ubuntu-latest' }}
with:
name: safety-linux-x86_64
path: dist/safety-linux-x86_64
if-no-files-found: error
- uses: actions/upload-artifact@v3
if: ${{ matrix.os == 'macos-latest' }}
with:
name: safety-macos-x86_64
path: dist/safety-macos-x86_64
if-no-files-found: error


deploy-pypi:
needs: build-binaries
runs-on: ubuntu-latest

if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')

steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build
- name: Build package
run: python -m build
- name: Publish package
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
user: __token__
password: ${{ secrets.SAFETY_PYPI_API_TOKEN }}

create-gh-release:
needs: deploy-pypi
runs-on: ubuntu-latest
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
permissions:
contents: write
steps:
- uses: actions/checkout@v2
- uses: ncipollo/release-action@v1
with:
artifacts: "dist/safety-win-i686.exe,dist/safety-win-x86_64.exe,dist/safety-linux-i686,dist/safety-linux-x86_64,dist/safety-macos-x86_64"
token: ${{ secrets.SAFETY_GITHUB_TOKEN }}
4 changes: 3 additions & 1 deletion .github/workflows/test-insecure.yml
Expand Up @@ -2,7 +2,9 @@
######## set on the action step, and a further step to ensure the previous step failed (and actually fail if it _didn't_)
name: Safety Action Insecure Tests

on: [push]
on:
push:
branches: [master, develop]

jobs:
##### Auto mode tests
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/test-secure.yml
Expand Up @@ -3,7 +3,9 @@
######## fail, the pinned version might need to be updated.
name: Safety Action Secure Tests

on: [push]
on:
push:
branches: [master, develop]

jobs:
##### Auto mode tests
Expand Down
44 changes: 0 additions & 44 deletions .travis.yml

This file was deleted.

2 changes: 2 additions & 0 deletions CHANGELOG.md
Expand Up @@ -5,6 +5,8 @@ All notable changes to this project will be documented in this file.
The format is partly based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) and [PEP 440](https://peps.python.org/pep-0440/)

## [Unreleased] 2.2.0.dev

## [2.1.1] - 2022-07-18
- Fix crash when running on systems without git present (Thanks @andyjones)

Expand Down
66 changes: 0 additions & 66 deletions appveyor.yml

This file was deleted.

49 changes: 27 additions & 22 deletions appveyor.py → binaries.py
@@ -1,7 +1,7 @@
"""AppVeyor Build
"""Github Action Build

This file is used to build and distribute the safety binary on appveyor. Take
a look at the corresponding appveyor.yml as well.
This file is used to build and distribute the safety binary on Github actions.
Take a look at the corresponding main.yml as well.

"""
import os
Expand All @@ -18,35 +18,38 @@ class environment:

def __init__(self):
os_mapping = {
"Visual Studio 2019": self.WIN,
"Ubuntu": self.LINUX,
"macOS": self.MACOS
"windows-latest": self.WIN,
"ubuntu-latest": self.LINUX,
"macos-latest": self.MACOS
}
self.os = os_mapping[os.getenv("APPVEYOR_BUILD_WORKER_IMAGE")]
self.os = os_mapping[os.getenv("BINARY_OS")]

@property
def python(self):
for arch, python in self.PYTHON_BINARIES[self.os].items():
yield arch, python

WIN_BASE_PATH = "C:\\hostedtoolcache\\windows\\Python\\3.10.7"

PYTHON_BINARIES = {
WIN: {
64: "C:\\Python38-x64\\python.exe",
32: "C:\\Python38\\python.exe",
64: f"{WIN_BASE_PATH}\\x64\\python.exe", # setup-python default
32: f"{WIN_BASE_PATH}\\x86\\python.exe"
},

# Order is important. If the 32 bit release gets built first,
# you'll run into permission problems due to docker clobbering
# up the current working directory.
LINUX: OrderedDict([
(64, "python"),
(32, f"docker run -t -v {os.getcwd()}:/app 32-bit-linux python3"),
(64, "python3"),
(32,
f"docker run --platform linux/386 -t "
f"-v {os.getcwd()}:/app 32-bit-linux "
f"python3"),
]),

MACOS: {
# Trying to use Python 3 compatible with PyInstaller according
# https://www.appveyor.com/docs/macos-images-software/#python
64: "~/venv3.8/bin/python",
64: "python3",
}
}

Expand All @@ -59,7 +62,10 @@ def run(self, command):
try:
print(f"RUNNING: {command}")
print("-" * 80)
subprocess.run(command, shell=True, check=True)
result = subprocess.run(command, shell=True, check=True,
stdout=subprocess.PIPE)
if result:
print(result.stdout.decode('utf-8').strip())
except subprocess.CalledProcessError as e:
print(f"ERROR calling '{command}'")
print("-" * 20)
Expand All @@ -74,13 +80,12 @@ def install(self):
# - build the 32 bit binary for linux on docker
# - create dist/ path to circumvent permission errors
if self.os == self.LINUX:
self.run("docker build -t 32-bit-linux -f Dockerfilei386 .")
self.run("docker build --platform linux/386 "
"-t 32-bit-linux -f Dockerfilei386 .")

for arch, python in self.python:
self.run(f"{python} -m pip install setuptools")
self.run(f"{python} -m pip install pyinstaller")
self.run(f"{python} -m pip install pytest")
self.run(f"{python} -m pip install -e .")
self.run(f"{python} -m pip install -r test_requirements.txt")

def dist(self):
"""Runs Pyinstaller producing a binary for every platform arch."""
Expand All @@ -92,7 +97,7 @@ def dist(self):
f" --distpath {build_path}")

# There seems to be no way to tell pyinstaller the binary name.
# This leads to problems with appveyors artifact collector because
# This leads to problems with artifact collector because
# every binary is named the same.
#
# Move them around so they can be picked up correctly
Expand All @@ -113,13 +118,13 @@ def test(self):
Runs tests for every available arch on the current platform.
"""
for arch, python in self.python:
self.run(f"{python} -m pytest")
self.run(f"{python} -m pytest --log-level=DEBUG")


if __name__ == "__main__":

if len(sys.argv) <= 1 or sys.argv[1] not in ['install', 'test', 'dist']:
print("usage: appveyor.py [install|test|dist]")
print("usage: binaries.py [install|test|dist]")
sys.exit(-1)

env = environment()
Expand Down
2 changes: 1 addition & 1 deletion safety/VERSION
@@ -1 +1 @@
2.1.1
2.2.0.dev