Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use 🛻 pickup from api #2310

Merged
merged 6 commits into from Mar 22, 2023
Merged

feat: use 🛻 pickup from api #2310

merged 6 commits into from Mar 22, 2023

Conversation

olizilla
Copy link
Collaborator

@olizilla olizilla commented Feb 9, 2023

pickup pulls dags into E-IPFS, so this PR updates the api to reflect that.

  • Update the pinning service add and update api routes to record ElasticIpfs as the service that the pin is queued on.
  • Update GET /pins/:req-id to check for a status change from pickup and update out DB, so we can inform users early when a pin request is pinned or failed.

Cron changes are in #2339

Fixes #2309

TODO

  • add PICKUP_BASIC_AUTH_TOKEN as a secret to the repo for cron jobs.
  • add PICKUP_BASIC_AUTH_TOKEN as worker secret for dev, staging, and prod
  • deploy
  • remove CLUSTER env vars

License: MIT
Signed-off-by: Oli Evans oli@protocol.ai

pickup pulls dags into E-IPFS, so this PR updates the api and cron jobs to reflect that in our db.

- Update the pinning service add and update api routes to record `ElasticIpfs` as the service that the pin is queued on.
- Update pins and pins-failed cron jobs to check pin status in pickup instead of cluster

Fixes #2309

TODO

- [ ] add `PICKUP_BASIC_AUTH_TOKEN` as a secret to the repo for cron jobs.
- [ ] update `CLUSTER-*` api env vars.

License: MIT
Signed-off-by: Oli Evans <oli@protocol.ai>
@cloudflare-pages
Copy link

cloudflare-pages bot commented Feb 9, 2023

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: efa19ee
Status: ✅  Deploy successful!
Preview URL: https://cc296ca7.nft-storage-1at.pages.dev
Branch Preview URL: https://use-pickup.nft-storage-1at.pages.dev

View logs

@olizilla
Copy link
Collaborator Author

This is blocked on pickup validator improvements web3-storage/pickup#97

Comment on lines +54 to +59
pins: [
{
status: 'PinQueued',
service: 'ElasticIpfs', // via pickup
},
],
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do this to override the default pin info that createUpload would set if we didn't provide this. New pins via pickup are provided via E-IPFS.

License: MIT
Signed-off-by: Oli Evans <oli@protocol.ai>
@olizilla olizilla changed the title feat: use 🛻 pickup to fetch pinning service pins feat: use 🛻 pickup from api Mar 7, 2023
cron changes now in #2339

License: MIT
Signed-off-by: Oli Evans <oli@protocol.ai>
so we can set them in advance of deploy, and have the new code make use of them when it goes live.

License: MIT
Signed-off-by: Oli Evans <oli@protocol.ai>
License: MIT
Signed-off-by: Oli Evans <oli@protocol.ai>
@olizilla
Copy link
Collaborator Author

Raised an issue for the dev worker deploy fail #2349

Copy link
Member

@vasco-santos vasco-santos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ✨

@olizilla olizilla merged commit e327bb3 into main Mar 22, 2023
@olizilla olizilla deleted the use-pickup branch March 22, 2023 14:11
olizilla pushed a commit that referenced this pull request Mar 22, 2023
🤖 I have created a release *beep* *boop*
---


##
[4.3.0](api-v4.2.1...api-v4.3.0)
(2023-03-22)


### Features

* use 🛻 pickup from api
([#2310](#2310))
([e327bb3](e327bb3))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use 🛻 pickup to handle pinning service requests
3 participants