RFC: Decentralized Package Registry

[vovacodes]

TLDR: Like npm, but built with an economically sustainable open-source ecosystem in mind.

Goals

• Decentralized permissionless JS package registry
• Financially sustainable ecosystem that supports maintainers and service providers
• Community governance
• Standard ESM modules
• Compatibility with npm cli

Competitive Advantages (vs npm)

• Decentralization and central censorship resistance
• Package registry that rewards authors, thus nurturing sustainable open-source ecosystem
• All packages are optimized for browser consumption:
  • bundled into a file-per-entrypoint
  • dependencies are using bare module specifiers
  • can be cached permanently due to their immutability

Governance and Economics

When publishing, tiny Solana and Arweave transaction costs (below 1$) are paid by the author. This ensures spam-free usage. In situations when the cost of the transaction is too high for the author, the project can ask for sponsorship on its homepage, this will encourage and promote the culture of supporting opensource projects financially.

Registry issues a token that's being paid out to package authors on the basis of their package usage. The usage is estimated by the number of hits of the package URL. We can count the hits by running our own Vartex Gateway and proxying the requests through it. Can we do that in a decentralized way?
The token is tradable on Solana platform and gives the owner voting rights in the governance process.

There is no initial tokens allocation, the tokes are minted only as rewards to the package authors. This should serve as testament of good intentions of the protocol - we aim for creating sustainable JS open-source ecosystem, not personal wealth of the protocol authors.
That said, some token pool should be created for the foundation to sponsor the development of the protocol through grants, the exact details of how to do that in a fair way need to be researched, perhaps following the example of other protocols recognized by the community as the ones having high integrity.

CDN

How would a sustainable decentralized CDN that serves the packages look like? Perhaps monetizing through the paying for access (authorization though cookie). Guy Bedford must've investigated this option already, we should discuss if paying with the registry tokens can be a viable solution.

Challenges

Giving token value

If we use tokens as reward for the ecosystem participants, we need to make sure the token has some actual value besides just speculative power.
Participation in the governance process, while being somewhat attractive, is not enough for a regular person to justfy paying for the token.
An obvious idea here is to make the token a payment instrument for some services and functionality the registry ecosystem provides. Examples can be:

• Private (encrypted) packages;
• Access to the CDN;

Moderation

The first question that arises when it comes to building a decentralized registry is how to moderate the content. The solution is community governance. The token holders can vote for removing some packages/versions from the registry. Despite files still being present on Arweave, references to them will be deleted from the latest state of the registry. Also there will not gonna be accounts (Solana term) in the ledger corresponding to the deleted version/package, those will be de-funded and pruned by the network.

Encouraging token holders to participate in governance

We need to think about how we could create incentives for the token holders to actively participate in governance/voting. Some ideas:

• Reward these actions with more tokens.
• Allow staking tokens, assigning their voting power to volunteer moderators/governors.

Ideas can probably be borrowed from other blockchain protocols that have success stories in the area of governance.

Technical Details

Package is a set if files accompanied by a package.json. In this regard, package structure is no different from one of a typical npm package.
All package files are persisted on Arweave. All "registry" data is stored on Solana blockchain, that includes:

• Mapping between existing package versions and their corresponding arweave hashes
• Packages ownership data: package scope or individual scope/name to one or many Solana public keys

Plan of Attack

• Write a simple script that uploads a file to Arweave and persists package metadata on Solana
• TODO

[jollytoad]

There is prior art for this, I've seen a few ideas like this pass by over the years, I'll add more links when I find them...

• https://nest.land/ (Deno, Arweave)

[vovacodes]

Yeah, nest.land is certainly an interesting effort in this area. Unfortunately, it doesn't look like they managed to make it attractive enough for the ecosystem participants to actively participate. This is where the main challenge is going to be: build a sustainable economic model.
I don't believe it's possible to compete with npm by just offering a decentralized clone with an option of participating in governance.

[guybedford]

Thanks for starting this discussion!

In terms of payment I would strongly recommend keeping publishing free and allowing paid / corporate / private packages to subsidize the free ones.

Thinking about how payment / private transactions work with a provider model would be interesting. I think it can work using standard crypto techniques though such as nested encryption (provider - registry communication is encrypted with their mutual encryption, provider - user communication is encrypted with their mutual encryption in turn, which can permit a nested key for the user to do the final decryption if you didn't even want the provider to have access to private code...).

Would be super interesting to explore code encryption in-browser on top of es-module-shims as well, and very much doable.

In terms of the action plan, the problem I think would be a great one to focus on first and foremost would be how to ensure developres can be compensated for their open source work, by allowing them to charge companies for high usage in companies. Solving that problem in the simplest way first and foremost, over trying to get all the crypto stuff completely perfect to start.

It's a big thing, and mostly marketing at the end of the day. A cross-collaboration of builder projects is exactly the sort of group that is needed to launch such a thing. Exciting space indeed!

Perhaps we can create a home for further discussion? Did you want to start a Discord or other space further to invite participation? I'd also recommend compiling the hated "whitepaper" or something similar that really drives home the details of the concept.

The other thing is just to charge for initial package name registration or package transfers, to avoid spam.

[vovacodes]

Thanks for the feedback!
If we are going to use one of the existing blockchains for storing registry information, I don't think we can avoid transaction fees for publishing, somebody just has to pay them. What we can do though is to ask, say Solana Foundation, for a grant that would finance the publishing costs, at least for the early adopters, so we can show the economics work, after that participation in the token economy should be enough of an incentive for the authors to pay the transaction fees. Besides, one of the goals of this project, how I see it, is to make it explicit for both providers and consumers that work and service always costs money, it's just a question who pays for it. Introducing payments, even tiny ones, early on supports the transparency in this area.

I agree that we should start with a whitepaper. Success or failure of this project, indeed, depends upon our ability to come up with an attractive and sustainable economic model. I will start working on that, and of course happy to collaborate.

[guybedford]

I don't think we can avoid transaction fees for publishing

It actually depends on whether you need the version feed to be absolutely immutable or not. If the version feed is cryptographically immutable yes it needs to be on the blockchain as a transaction per version.

But if the version feed is just treated as a signed data feed then it can just be file-based signed by the package author signature chain and hosted anywhere.

Such an architecture would also more directly permit changes / unpublishing than having the version feed direclty on the blockchain as well.

So it's that sort of crypto question of how hard to make the cyptographic constraints. If a core requirement of a registry is the ability to get a fixed lock to guarantee a given execution, then the question is is it ok for locks to just be final hashes? In such a case hidden version immutability may not be so bad, when convention would for the most part avoid hitting those mutation cases in the 99.9% usage.

In turn that moves transaction costs to only registering a package name, and not per version.

I agree that we should start with a whitepaper.

Maybe whitepaper is the wrong term even - it's really just a write-up / description / post of the approach, where the process of fleshing that out collaboratively could start to form a coalition around such a launch.

Success or failure of this project, indeed, depends upon our ability to come up with an attractive and sustainable economic model.

I definitely agree that is a huge goal here. I would also argue cross-organizational collaboration, marketing and distribution are just as important!