Add Tidelift alignment action and badge #5763

aclark4life · 2021-10-14T17:50:38Z

README.md

hugovk · 2021-10-14T18:50:02Z

Wonder why this failed:

Running alignment. Uploading files. (This may take a few minutes to complete)
Error: invalid authentication. are you using the correct API key? (ptm.tl/docs-apikeys).

https://github.com/python-pillow/Pillow/pull/5763/checks?check_run_id=3897935345

When the first one passed:

Running alignment. Uploading files. (This may take a few minutes to complete)

Revision: 5c69dc7c9a1b12eae14e6ab4823f73da6a5911e7 (Check alignment status by running 'tidelift status 5c69dc7c9a1b12eae14e6ab4823f73da6a5911e7')
+ echo 'Waiting for the alignment to start'
Waiting for the alignment to start
+ sleep 15
+ ./tidelift status --wait 5c69dc7c9a1b12eae14e6ab4823f73da6a5911e7
Waiting for alignment to finish

Alignment Details URL: tidelift.com/scans/842c3013-bd63-4667-8582-8b583c87f43e
Revision: 5c69dc7c9a1b12eae14e6ab4823f73da6a5911e7 - Status: success

https://github.com/python-pillow/Pillow/runs/3897866038?check_suite_focus=true

hugovk · 2021-10-14T19:21:41Z

If the workflow requires secrets to run, we'll need a guard to prevent it running and failing on forks. For example:

 jobs:
   build:
+    if: github.repository_owner == 'python-pillow'
     name: Run Tidelift to ensure approved open source packages are in use

radarhere · 2021-10-19T06:12:03Z

I've pushed a commit so that there's now a badge at https://pillow--5763.org.readthedocs.build/en/5763/

radarhere · 2021-11-21T01:59:02Z

Hi. What's the reason for this being 'Do Not Merge'? Is it just because it's failing, I think due to missing secrets?

I also see https://github.com/python-pillow/Pillow/runs/4124628947#step:4:7 ?

⚠️ Warning: this Github Action is deprecated. Please use https://github.com/tidelift/alignment-action instead.

hugovk · 2021-11-21T10:55:17Z

Yes, that's the main reason: there's no point merging this and then it then fails every build.

I also have questions about what the action is actually for: #5762 (comment)

aclark4life · 2021-11-21T13:10:38Z

Right. Let me get back to this now and provide an update by EOM.

aclark4life · 2021-12-01T19:03:02Z

OK added @hugovk suggestion and double checked TIDELIFT_API_KEY, if it still fails I'll check API key again but I think it's set correctly. Thanks @hugovk @radarhere !

aclark4life · 2021-12-01T19:17:01Z

@JeffStern Is ae4d5d9 correct for org and proj? Thx

jeffstade · 2021-12-01T20:46:21Z

That looks good to me. Are you still seeing issues?

hugovk · 2021-12-01T21:27:18Z

Thanks!

API token looks good now, but fails like this. Should we use https://github.com/tidelift/alignment-action instead?

⚠️ Warning: this Github Action is deprecated. Please use tidelift/alignment-action instead.
Downloading latest CLI
+ chmod +x ./tidelift
+ echo 'Current Tidelift CLI version'
Current Tidelift CLI version
+ ./tidelift version
+ echo 'Uploading manifests for alignment'
+ ./tidelift alignment save --revision f26e370ebbb874b3afb1aa6206c5404bf0f08a70 --branch merge --directory /github/workspace
1.2.5-build-4330
Uploading manifests for alignment
Running alignment. Uploading files. (This may take a few minutes to complete)

Revision: f26e370ebbb874b3afb1aa6206c5404bf0f08a70 (Check alignment status by running 'tidelift status f26e370ebbb874b3afb1aa6206c5404bf0f08a70')
+ echo 'Waiting for the alignment to start'
+ sleep 15
Waiting for the alignment to start
+ ./tidelift status --wait f26e370ebbb874b3afb1aa6206c5404bf0f08a70
Waiting for alignment to finish
Error: blocking issues.

Alignment Details URL: tidelift.com/scans/12340224-5aa7-40ef-b05a-9878e540039e
Revision: f26e370ebbb874b3afb1aa6206c5404bf0f08a70 - Status: failure


Blocking issues found. See Alignment Details URL for more detail.

https://github.com/python-pillow/Pillow/runs/4386180151?check_suite_focus=true

.github/workflows/tidelift.yml

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>

aclark4life · 2021-12-02T00:21:41Z

@hugovk @radarhere @JeffStern OK here's where it gets potentially "wacky". I hate to add more files to the root of the Pillow repository, but in this case the two Pipenv files I've added may satisfy the Tidelift CI 🤞 (https://support.tidelift.com/hc/en-us/articles/4406286692756-How-to-generate-lockfiles-from-manifests) What's particularly "wacky" for me is I've never, ever, EVER used Pipenv before now. At a glance, it looks like something some people use and others maybe don't use … I certainly don't see Pipfiles in every Python package repository. So, if the CI passes and no one minds the intrusion of two more files in the root, I say we merge for experiment's sake. And if anyone has any relevant Pipenv experience share-away! Thanks all 🙏

aclark4life · 2021-12-02T00:34:53Z

Progress …

aclark4life · 2021-12-02T00:49:39Z

More progress …

aclark4life · 2021-12-02T11:02:03Z

@JeffStern OK looks like we're just waiting for babel and sphinx to get approved ^^^ thanks!

hugovk · 2021-12-02T13:17:32Z

pipenv is used for pinning dependencies (and sub-dependencies) to get reproducible builds, usually used for developing applications rather than libraries like Pillow.

(We use requirements.txt for test/docs/release so there is an argument to be made for pinning (and a counter-argument for having to manage/update those pins).)

If possible, it would be nice to avoid pipenv, because it means we need to duplicate the content of requirements.txt into Pipfile and then run a command to regenerate Pipfile.lock, but if it's the only way, so be it. (In general, if another library had runtime dependencies they probably should not pin.)

Can we have them listing the same things? That is, add packaging and remove docutils from Pipfile?

Also, is it possible to give access to the rest of the Pillow team to see the scans, so we can see why they fail?

At https://tidelift.com/scans/e4fc2d1a-f253-4831-9ff6-bebe624a75f1 I see:

Unable to find scan e4fc2d1a-f253-4831-9ff6-bebe624a75f1

Revision: 3d0c10ca6463e7ef35ae4a4a9a8b51f4989df519 (Check alignment status by running 'tidelift status 3d0c10ca6463e7ef35ae4a4a9a8b51f4989df519')
+ echo 'Waiting for the alignment to start'
+ sleep 15
Waiting for the alignment to start
+ ./tidelift status --wait 3d0c10ca6463e7ef35ae4a4a9a8b51f4989df519
Waiting for alignment to finish
Error: blocking issues.

Alignment Details URL: tidelift.com/scans/e4fc2d1a-f253-4831-9ff6-bebe624a75f1
Revision: 3d0c10ca6463e7ef35ae4a4a9a8b51f4989df519 - Status: failure

https://github.com/python-pillow/Pillow/runs/4393978672?check_suite_focus=true

aclark4life · 2021-12-02T17:13:48Z

@hugovk I'll let @JeffStern address adding Pillow team to Tidelift dashboard, but for now let's just assume that failures are caused by babel and sphinx not being in the catalog. I'm not even sure where babel comes from, I ran pipdeptree on requirements.txt and I don't see it:


╰─[:)] % for i in `cat requirements.txt | grep -v ^# `
do
echo -- $i --
pipdeptree -p $i
done
-- black --
black==19.3b0
  - appdirs [required: Any, installed: 1.4.4]
  - attrs [required: >=18.1.0, installed: 21.2.0]
  - click [required: >=6.5, installed: 8.0.3]
  - toml [required: >=0.9.4, installed: 0.10.2]
-- check-manifest --

-- coverage --

-- defusedxml --

-- docutils==0.16 --

-- markdown2 --

-- olefile --

-- packaging --

-- pyroma --

-- pytest --

-- pytest-cov --

-- pytest-timeout --

-- sphinx>=2.4 --

-- sphinx-copybutton --

-- sphinx-issues --

-- sphinx-removed-in --

-- sphinx-rtd-theme --

-- sphinxext-opengraph --

I'm tempted to remove all the packages and re-add them one-by-one.

hugovk · 2021-12-03T10:59:07Z

That command only worked for Black.

Doing a reverse search, babel is from Sphinx:

$ pipdeptree -p babel --reverse
babel==2.9.1
  - Sphinx==4.3.1 [requires: babel>=1.3]
    - sphinx-copybutton==0.4.0 [requires: sphinx>=1.8]
    - sphinx-issues==1.2.0 [requires: sphinx]
    - sphinx-removed-in==0.2.1 [requires: Sphinx]
    - sphinx-rtd-theme==1.0.0 [requires: sphinx>=1.6]
    - sphinxext-opengraph==0.5.0 [requires: sphinx>=2.0]

auvipy · 2021-12-12T14:36:00Z

.github/workflows/tidelift.yml

+        env:
+          TIDELIFT_API_KEY: ${{ secrets.TIDELIFT_API_KEY }}
+          TIDELIFT_ORGANIZATION: team/aclark4life
+          TIDELIFT_PROJECT: pillow


TIDELIFT_PROJECT should be name of exact package repo name right? or anything tidelift specific

i got it by trial n error. now only figure out tidelift API key authenticate failure

auvipy · 2021-12-12T14:44:49Z

Pipfile

@@ -0,0 +1,22 @@
+[[source]]


is this file mandatory for tidelift?

@auvipy If you want to provide a lock file, then either Pipfile.lock or poetry.lock according to https://support.tidelift.com/hc/en-us/articles/4406293161492-Compatible-languages-and-package-files

OK I am familiar with both, but not so with tidelift CLI :D I was able to make tidelift authenticate work though

but should I put key in a secret env variable in the local tidelift project?

auvipy · 2021-12-12T14:46:30Z

Progress …

I can see security advised PyPI catalog, did you created any custom catalog? is it possible to share that sample to the slack chat?

aclark4life · 2021-12-20T19:31:24Z

Success (finally)!

hugovk · 2021-12-21T15:03:40Z

Almost there! To pass CI linting, MANIFEST.in needs updating to include the new files:

lint run-test: commands[1] | check-manifest
lists of files in version control and sdist do not match!
missing from sdist:
  Pipfile
  Pipfile.lock
suggested MANIFEST.in rules:
  include *.lock
  include Pipfile

https://support.tidelift.com/hc/en-us/articles/4406293161492-Compatible-languages-and-package-files says that while "Tidelift can generate the most complete bill of materials if we have both files", it seems to suggest that requirements.txt is fine on its own without a lockfile.

But I'm fine with including it for now just to get it merged, and we can look into removing it later.

aclark4life · 2021-12-21T16:16:56Z

Almost there! To pass CI linting, MANIFEST.in needs updating to include the new files:

lint run-test: commands[1] | check-manifest
lists of files in version control and sdist do not match!
missing from sdist:
  Pipfile
  Pipfile.lock
suggested MANIFEST.in rules:
  include *.lock
  include Pipfile

Done, thanks!

https://support.tidelift.com/hc/en-us/articles/4406293161492-Compatible-languages-and-package-files says that while "Tidelift can generate the most complete bill of materials if we have both files", it seems to suggest that requirements.txt is fine on its own without a lockfile.

But I'm fine with including it for now just to get it merged, and we can look into removing it later.

It says that, but the UI indicates otherwise, and the issue of (or confusion about) whether or not locking is appropriate in any given context remains (for me at least). I added the lock file to make Tidelift UI warning go away … but yeah, we can decide later if we want to remove it. Particularly if @JeffStern makes it so we optionally don't have to look at that warning in the Tidelift UI (e.g. by adding a [✔️ ] No lockfiles in this project option.) 😄

With lockfile: 🎉

Without lockfile: 😡

auvipy · 2021-12-21T16:30:57Z

my alignments were working without lockfile, or may be false positive?

aclark4life · 2021-12-21T16:32:35Z

@auvipy Right, alignment without lock file is "real". Just complaining about the UI while we're on this thread. 😄

aclark4life · 2021-12-25T12:22:18Z

Thanks all! 🎄

Not sure if we still care about this? cf. #5762 #5763

aclark4life mentioned this pull request Oct 14, 2021

Align development requirements with Tidelift's Security-advised PyPI catalog #5762

Closed

hugovk reviewed Oct 14, 2021

View reviewed changes

README.md Outdated Show resolved Hide resolved

hugovk reviewed Oct 14, 2021

View reviewed changes

README.md Outdated Show resolved Hide resolved

hugovk mentioned this pull request Oct 14, 2021

Delete tidelift.yml #5765

Merged

aclark4life and others added 4 commits October 14, 2021 22:57

Add tidelift alignment badge

c8822a6

Create tidelift.yml

76b5760

Don't run for forks: missing secrets would fail

836f740

Fix badge HTML

3c92b34

hugovk force-pushed the new-badge branch from 9029af9 to 3c92b34 Compare October 14, 2021 20:04

hugovk changed the title ~~Add tidelift alignment badge~~ Add Tidelift alignment action and badge Oct 14, 2021

hugovk and others added 2 commits October 16, 2021 15:22

Merge branch 'main' into new-badge

3c8128d

Added Tidelift Align badge to docs

06ab032

hugovk marked this pull request as draft October 19, 2021 06:40

hugovk added the Do Not Merge label Oct 28, 2021

Add hugovk suggestion

91fb7fc

aclark4life marked this pull request as ready for review December 1, 2021 19:02

aclark4life removed the Do Not Merge label Dec 1, 2021

aclark4life added 2 commits December 1, 2021 14:13

Remove org and proj because deprecated, I think

c373ac1

Not deprecated, re-add org and proj

ae4d5d9

hugovk reviewed Dec 1, 2021

View reviewed changes

.github/workflows/tidelift.yml Outdated Show resolved Hide resolved

aclark4life and others added 2 commits December 1, 2021 17:05

Update .github/workflows/tidelift.yml

d23afb3

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>

Add pipenv files for tidelift cli

1d82b6b

Add packaging remove docutils

360b169

aclark4life added 2 commits December 3, 2021 07:26

Remove sphinx for testing tidelift alignment

789fcb8

Update lock file

ca8fc48

auvipy reviewed Dec 12, 2021

View reviewed changes

Add lock file

7a42610

aclark4life requested a review from hugovk December 20, 2021 19:30

Lint fix

085b05d

hugovk approved these changes Dec 24, 2021

View reviewed changes

aclark4life merged commit 9d0703a into main Dec 25, 2021

aclark4life deleted the new-badge branch December 25, 2021 12:21

aclark4life added a commit that referenced this pull request Nov 14, 2022

Remove Tidelift alignment action and badge

8a3ba65

Not sure if we still care about this? cf. #5762 #5763

aclark4life mentioned this pull request Nov 14, 2022

Remove Tidelift alignment action and badge #6739

Merged

hugovk mentioned this pull request Dec 8, 2022

Remove unnecessary Pipfile #6790

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Tidelift alignment action and badge #5763

Add Tidelift alignment action and badge #5763

aclark4life commented Oct 14, 2021

hugovk commented Oct 14, 2021

hugovk commented Oct 14, 2021

radarhere commented Oct 19, 2021

radarhere commented Nov 21, 2021

hugovk commented Nov 21, 2021

aclark4life commented Nov 21, 2021

aclark4life commented Dec 1, 2021

aclark4life commented Dec 1, 2021

jeffstade commented Dec 1, 2021

hugovk commented Dec 1, 2021

aclark4life commented Dec 2, 2021 •

edited

aclark4life commented Dec 2, 2021

aclark4life commented Dec 2, 2021

aclark4life commented Dec 2, 2021

hugovk commented Dec 2, 2021

aclark4life commented Dec 2, 2021

hugovk commented Dec 3, 2021

auvipy Dec 12, 2021

auvipy Dec 14, 2021

auvipy Dec 12, 2021

aclark4life Dec 20, 2021

auvipy Dec 21, 2021

auvipy Dec 21, 2021

auvipy commented Dec 12, 2021

aclark4life commented Dec 20, 2021

hugovk commented Dec 21, 2021

aclark4life commented Dec 21, 2021

auvipy commented Dec 21, 2021

aclark4life commented Dec 21, 2021

aclark4life commented Dec 25, 2021

Add Tidelift alignment action and badge #5763

Add Tidelift alignment action and badge #5763

Conversation

aclark4life commented Oct 14, 2021

hugovk commented Oct 14, 2021

hugovk commented Oct 14, 2021

radarhere commented Oct 19, 2021

radarhere commented Nov 21, 2021

hugovk commented Nov 21, 2021

aclark4life commented Nov 21, 2021

aclark4life commented Dec 1, 2021

aclark4life commented Dec 1, 2021

jeffstade commented Dec 1, 2021

hugovk commented Dec 1, 2021

aclark4life commented Dec 2, 2021 • edited

aclark4life commented Dec 2, 2021

aclark4life commented Dec 2, 2021

aclark4life commented Dec 2, 2021

hugovk commented Dec 2, 2021

aclark4life commented Dec 2, 2021

hugovk commented Dec 3, 2021

auvipy Dec 12, 2021

Choose a reason for hiding this comment

auvipy Dec 14, 2021

Choose a reason for hiding this comment

auvipy Dec 12, 2021

Choose a reason for hiding this comment

aclark4life Dec 20, 2021

Choose a reason for hiding this comment

auvipy Dec 21, 2021

Choose a reason for hiding this comment

auvipy Dec 21, 2021

Choose a reason for hiding this comment

auvipy commented Dec 12, 2021

aclark4life commented Dec 20, 2021

hugovk commented Dec 21, 2021

aclark4life commented Dec 21, 2021

auvipy commented Dec 21, 2021

aclark4life commented Dec 21, 2021

aclark4life commented Dec 25, 2021

aclark4life commented Dec 2, 2021 •

edited