Skip to content
This repository has been archived by the owner on Apr 18, 2024. It is now read-only.

Saturn L2 V0 #22

Merged
merged 13 commits into from Jul 21, 2022
Merged

Saturn L2 V0 #22

merged 13 commits into from Jul 21, 2022

Conversation

aarshkshah1992
Copy link
Contributor

@aarshkshah1992 aarshkshah1992 commented Jun 8, 2022

No description provided.

Copy link

@willscott willscott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should @bajtos also be reviewing these for integration with station?


# Define a job to be invoked later in a workflow.
# See: https://circleci.com/docs/2.0/configuration-reference/#jobs
orbs:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you hook this up to the standard PL git management setup rather than hand-maintaining this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@willscott What is this "standard PL git management setup" that thou refer to ?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aarshkshah1992
Copy link
Contributor Author

@willscott Don't wanna bother @bajitos with reviewing Golang code and that too one that uses so much of the PL Stack.

I'll have a separate discussion/doc/long running interaction with him for integrating with Station.

@willscott
Copy link

him or the L1 devs will know what sort of responses they want to be getting from the l2 interaction so may be better reviewers than me

@bajtos
Copy link
Contributor

bajtos commented Jun 8, 2022

@aarshkshah1992 I am happy to learn more about Go :) I quickly skimmed through your code and have one question - how is this new code and the new HTTP server integrating with the existing main binary? Do we want two HTTP servers listening on different ports (one for web, another for Saturn)?

@willscott
Copy link

My understanding is that the golang code here will be a libp2p node, and will make connections outbound to L1's.
The HTTP server here gets requests funneled through that libp2p connection - the actual network request is a long lived L2->L1 to handle cases where L2's are behind NATs

)

var (
defaultURL = "https://ipfs.io/api/v0/dag/export"
Copy link

@lidel lidel Jun 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: go-ipfs 0.13+ allows you to replace legacy RPC API at /api/v0/dag/export with implementation-agnostic Gateway request GET https://ipfs.io/ipfs/{cid}?format=car (or GET https://ipfs.io/ipfs/{cid} with Accept: application/vnd.ipld.car).

This is already supported by ipfs.io:

curl -H "Accept: application/vnd.ipld.car" "https://ipfs.io/ipfs/bafybeidd2gyhagleh47qeg77xqndy2qy3yzn4vkxmk775bg2t5lpuy7pcu" > webui.car

Switching from RPC to Gateway enables you to swap gateway endpoint and its implementation in the future + benefit from planned selector support.

gwAPI := carstore.NewGatewayAPI(svc.URL)
lg := logs.NewSaturnLogger()
cfg := carstore.Config{}
cs, err := carstore.New(temp, gwAPI, cfg, lg)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When running inside Filecoin Station, we need Saturn L2 node to store all cache files in the directory provided by the Station. (Typically, this directory is OS-specific. See #36 and ipfs/ipfs-desktop#1656 for more details.)

I am proposing to introduce a new env var, e.g. CACHE_DIR or SATURN_CACHEDIR:

  • Filecoin Station sets CACHE_DIR env var, similarly to how it sets FIL_WALLET_ADDRESS
  • Saturn L2 Node uses this value as the root dir for all caches

To make it easier to run saturn-l2 on its own, we can introduce a sensible default when CACHE_DIR is not provided. I don't think t.TempDir() is a good one - IMHO, we should preserve cached data across node restarts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey I have introduced an ENV VAR called ROOT_DIR. This will be the root directory for the L2 Node where it will persist all it's state and cached data.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't added a sensible default yet but we can add it down the line if need be. Should be easy.

@bajtos
Copy link
Contributor

bajtos commented Jul 20, 2022

@aarshkshah1992 in my limited knowledge of Go, I believe it's a common convention (a best practice?) to put CLI main files into cmd/{name} folder, see e.g. https://github.com/cli/cli/tree/trunk/cmd and https://github.com/ipfs/kubo/tree/master/cmd.

In our case, I think we should move /main directory to /cmd/saturn-l2.

WDYT?

As a nice side effect, go build will call the output saturn-l2 out of the box, there will be no need to pass an explicit -o arg.

I guess such a change is out of scope of your PR, so the question is whether you want to do it before or after your PR lands?

@aarshkshah1992
Copy link
Contributor Author

@bajtos -> Yes good suggestion, I have moved all binaries to cmd/.

@aarshkshah1992 aarshkshah1992 merged commit d8f754e into main Jul 21, 2022
@aarshkshah1992 aarshkshah1992 deleted the feat/v0-car-traversal branch July 21, 2022 08:20
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants