sourcecred v0.11.2

Summary

Critical bug patch to solve discord server ids passing the quintillion mark

Contributions

3c76945 Discord message IDs have passed the quintillion mark and this has led to an infinite loop in the codebase due to a string comparison on numeric IDs. This PR proposes a simple fix of casting the IDs to numbers before doing the comparison. (#3361)

sourcecred v0.11.1

Summary

Critical bug patch to solve [Error: ENOENT: no such file or directory, open '/home/runner/work/org/repo/output/graphs/sourcecred/discourse/graph'] during the graph step of the Generate Cred Action.

Contributions

24f8be3 Bugfix: explicit directory creation for plugin outputs (#3348)

sourcecred v0.11.0

Note: critical bug involving directory creation patched in 0.11.1

Summary

Wowee! As our core developers move on to other opportunities, this will be the last 1st-party update for the foreseeable future. But it is an EXCITING one. And it should be a safe, zero-effort upgrade (except for a small breaking change in our developer library).

CredEquate: an experimental alternative core algorithm

CredEquate is a new scoring algorithm based on simple algebra. It can be used instead of, or in parallel with, our existing pagerank-based algorithm CredRank. As of this release we consider CredEquate to be sufficiently production-ready. So far, it supports implementations in the Discord and Github plugins, and it comes with a broad set of advantages compared to CredRank. To learn more about the comparison, and to learn how to try it out, go to https://sourcecred.io/docs/guides/core-algorithm

Configuration Editor GUI

Setting up a SourceCred instance has long been a very manual and technical process that requires local git managment, terminal commands, and hand-editing JSON. In this version, a new Configuration Editor GUI makes a major step towards reducing the barriers to using SourceCred.

Now, you can edit the sourcecred.json file (which is especially important when using CredEquate) in a local-hosted admin portal by navigating to the new menu option "Edit Plugins." There, your sourcecred.json file will be loaded into a webform, where you can edit it and then click "Save" to save the changes back to disk. For more information on how to configure the CredEquate plugins section of this page, go to https://sourcecred.io/docs/guides/core-algorithm

Additionally, we have created an experimental no-clone, no-terminal way of using this GUI with a static remote-hosted site. This experience involves inputting a GitHub API key so that the page can read and write to GitHub instead of to a local disk. The API key is only stored in your browser's memory, and only used to make client-to-GitHub API calls. There are potential security risks with this method, but the benefits are worth exploring. To try it out, go to https://cred.sourcecred.io/#/instance-config in Google Chrome (make sure you trust your Chrome extensions).

Our dream is that one day, this config editing experience will be available for all of the JSON configuration files used by a SourceCred instance. We hope that others will be inspired to build this dream out using the foundation that we have laid here.

CSV Grain Integration and Payout Address storage

FINALLY, a feature that makes it easy to distribute tokens using SourceCred! While this is not a direct on-chain integration, it is a HUGE improvement to treasury workflows. The basic outline is:

1. An integration currency can be configured in grain.json
2. Participant payout addresses can be stored in the SourceCred ledger.
3. The Grain distribution CLI / Action can be configured to output CSV files compatible with either disperse.app or Gnosis Safe's CSV Airdrop app.

For detailed instructions, go to https://sourcecred.io/docs/guides/csv-grain

Load Discord data starting at a configured date

Discord has always been a major scalability limit. Some Discord servers are just too big to reasonable load and process all their historical data. Now, you can add an attribute to the config/plugins/sourcecred/discord/config.json file to set a beginning date, and the plugin will only load data after that date. For example, to only load Discord data after December 1st, 2021, add "beginningDate": "2021/12/1"

[devs] [Breaking Change] Reversed the GithubStorage breaking change from 0.10.1/0.10.2

In 0.10.1/0.10.2 we renamed the github writing class from GithubStorage to WritableGithubStorage, which was a breaking change for 3rd-party code that used GithubStorage. In this version, we have renamed the export back to GithubStorage. We apologize to developers who have already updated to 0.10.2 for the churn, but we are glad to say that developers who hadn't updated yet will not experience a breaking change as they move to 0.11.0.

[devs] Polymorphic WriteInstance class

Our library now exports the new WriteInstance module, which allows for easy reading and writing of SourceCred inputs and outputs. It's constructor accepts an object that implements our WritableDataStorage interface (get and set methods). The module also includes a factory for creating a GitHub-compatible WriteInstance like so: sc.instance.writeInstance.getGithubWriteInstance(myApiKey, myRepo, myBranch)

Contributions

cb45917 Create a GUI config editor for RawInstanceConfig (#3343)
0506e48 Addng WriteInstance into LocalInstance inheritance hierarchy (#3325)
6f66bce Github action for generating docs (#3346)
6a4c808 CredRank: add timestamp to MarkovProcessGraph nodes (#3345)
37144a0 take and pass args to storage in persist (#3339)
3939ea0 fix backward compatibility for gh storage (#3340)
67af254 Discord: fetch backwards and add a beginningDate config (#3329)
404b11c Change the Go CLI to run contributions, and use hybrid instead of credrank (#3337)
ac5d05d CredEquate: create MVP GitHub plugin (#3338)
68b9fbd Helper method for finding CredEquate contributions by key/subkey (#3330)
e8950b3 Dependency cred support for CredEquate (#3328)
b5298c0 Get CSV grain integration production-ready (#3320)
86189db Add hybrid CLI that calls both credrank and credequate (#3322)
77dde28 Add static method for merging CredGrainViews (#3323)
7e4518d Explorer Home: Make Grain column name match configuration (#3318)
8bafdf9 Support for Gnosis in Csv Grain Integration (#3321)
2f5e6ef CredEquate support in Load CLI (#3319)
5e8e86e Nav & Sidebar Adjustments (#3266)

New Contributors

• @black32167 made their first contribution in #3325

Full Changelog: sourcecred-v0.10.3...sourcecred-v0.11.0

sourcecred v0.10.3

Summary

File name length bugfix for Windows

Windows users (and some other environments) have not been able to import or set up sourcecred properly due to some autogenerated assets that had file names exceeding the max for those systems. This has been fixed.

Alpha release of experimental core algorithm CredEquate

We will document this more after some initial internal testing. If you want to try it out for yourself in the meantime, see the test plan of #3299, and if you want to read more about what this is and why we're building it, see the original design spec.

Contributions

966c48f CredEquate: Fix circular dependencies (#3312)
0fe70ab Instructional sharness snapshot error message (#3309)
505d5da CredEquate: expose core code in library (#3310)
d77bf0e Mergeable CredEquate Design Doc for recordkeeping (#3304)
f735638 Change the grain API to take a credGrainview instead of a credGraph (#3303)
8b98711 CredEquate: create credequate CLI/API (#3299)
e6c6147 CredEquate: add Discord plugin and Contributions CLI/API (#3295)
94b9e2a CredEquate core algorithm, experimental alternative to CredRank (#3273)
809f6fc Ignore contributions from huxbot (#3289)
b8110a1 Author Pre-Audit Merkle Distribution Contract (#3148)
b5709e6 Feature: Disable balance accounting in the ledger (#3192)
8651733 combo bugfix: spacer option renamed to space (#3280)
47af83f Convert snapshot file name serialization to hashes (#3279)

sourcecred v0.10.2

Patches

This patch only concern 3rd-party developers or users of 3rd-party tools. This changes our API's sc.ledger.storage.GithubStorage export to a newer class of the same name. Because the interface is unchanged for users of the LedgerManager, scripts using that class will work after changing GithubStorage to WritableGithubStorage. If you use the GithubStorage class directly without a LedgerManager, then you should note that read and write have been changed to get and set and take different parameters.

Contributions

fff0d2a Change GithubStorage in API to the newer version (#3277)
9217fc1 v0.10.1 docs generation (#3276)

sourcecred v0.10.1

Breaking changes

If you are not a 3rd-party developer or user of 3rd-party tools for managing your ledger, there are no breaking changes that concern you and this is a stable build.

This version changes the GithubStorage input required by the LedgerManager class in our JS library. Unfortunately, we also forgot to switch the GithubStorage provided in the API to the new one. So if you use the LedgerManager class, this is not a viable build. The correct WritableGithubStorage class is provided in 0.10.2, however.

Summary

Export the CredGraph to Neo4j

Neo4j is a graph database with its own query language and visual explorer. We now support exporting the SourceCred graph to Neo4j after generating it. Running yarn sourcecred analysis -n will generate CSV files in a output/neo4j folder and log pre-built neo4j-admin commands that can be used with Neo4j's admin CLI to import the CSV files into a Neo4j databse. The fastest way to get started is by downloading Neo4j Desktop and Java 11 and setting up local Neo4j db.

Note, you can also add yarn sourcecred analysis -n to your instance's GitHub Action configuration to automatically generate these files in your gh-pages branch.

UI improvements

• The Grain Accounts page now has pagination, filtering, and sorting by earned, balance, and active status.
• The Cred Explorer page now has a column showing % of total Cred.

[Dev] Computer-generated API documentation

We have primitive API docs! We'll be working on making sure they're all accurate and providing entry points to know where to start. In the mean time, enjoy our progress so far: https://api.sourcecred.io/sourcecred/

[Dev] Simpler API usage when creating a ReadInstance object

If you've been using our getNetworkReadInstance library functionality to pull SourceCred data from an instance, there is now an improved getRawGithubReadInstance interface that does the same thing without requiring you to find the raw.githubusercontent base URL. Just pass in the github org name, repo name, and branch name like this:

instance = sc.instance.readInstance.getRawGithubReadInstance("sourcecred","cred","gh-pages")

Don't know what this means but interested? You can find info about our ReadOnlyInstance interface here: https://api.sourcecred.io/sourcecred/#readonlyinstance
https://observablehq.com/@sourcecred/api-tutorial

Contributions

2380c7b Add Needs Triage label to feature request template (#3274)
3c80039 Support Pagination and Filter-by-accont for Grains Accounts (#3267)
0e03eb1 Analysis API: Neo4j csv files (#3258)
71387ef Util: improve performance of batch (#3261)
a61df47 Add support for sorting by Active accounts. (#3263)
9f356fa Fix sorting in accounts preview (#3262)
24cf73a Util: add batchArray and batchIterator methods (#3259)
5459e0c Better error message for dependency id missing in ledger (#3255)
362b72e GitHub storage conforms data storage (#3209)
5f88abe Set up GH Pages config for api docs (#3252)
a40646a Add computer-generated API documentation (#3251)
2c91304 Add RawGithubReadInstance wrapper for improved UX (#3250)
584d45f Using Node as default jest's test environment & Utilizing jest's docblock (#3242)
dfbade6 ADD Postable Origin Storage (#3235)
7f66575 Add Payout Address Text Box for Identities (#3233)
b95f7fb Utilize an Opaque type to key on currencies in the Ledger (#3231)
eb2600c Fix colSpan so row takes up full width of table (#3238)
261e48d ADD Cred per contributor for each period. (#3236)

sourcecred v0.10.0

Breaking Changes

The below Grain Config format change is breaking for many instances. Fortunately, we included an updater. To use:

1. Check out the dependabot branch locally (or manually update the sourcecred version in your instance's package.json)
2. Run yarn && yarn sourcecred update
3. Commit and push changes, including changes to the file data/updater.txt

Summary

[Affects Cred] Discord skips zero-weighted channels

Before, the only way to fully exclude channels from the load and graph steps was to wrestle the bot's permissions. Now, channels with a weight of 0 (explicitly or using defaultWeight) will not be loaded. This should improve performance significantly for instances with zero-weighted channels. This may affect cred scores slightly.

[Affects Cred] Discord Graph Simplification

Now the Discord plugin excludes messages that don't have any positively weighted reactions attached. This will improve performance, and may also affect cred scores slightly.

Grain Config format change

Months ago, we introduced the option to configure your grain distributions using an array format in the attribute allocationPolicies. In this release, we are deprecating the old format in favor of the new. See the Breaking Changes note above to easily reformat your existing grain.json

Discord Channel Category Weights

Now, you can set a weight for a discord channel category, which will be used if a channel's weight is not explicitly set. To do so, simply add the category's ID into the same channelWeightConfig.weights array that you use to configure channel weights.

Recent Policy Exclusions

Many communities have salaried employees, which can complicate their use of SourceCred since they don't want people double-dipping. Now, you can pay salaried employees directly through SourceCred by adding a SPECIAL distribution for that employee and adding them to the new exclusions attribute on the RECENT distribution. This allows SourceCred's BALANCED policy to do smarter accounting and even pay salaried employees if their contributions exceed their salary. This also improves transparency in the community.

Example grain.json

{
  "maxSimultaneousDistributions": 100,
  "allocationPolicies": [
    {
      "policyType": "SPECIAL",
      "budget": 100,
      "memo": "Ongoing salary for Garnet's full-time community organizing contributions",
      "recipient": "6wyPOVWLmZ2b35eCmOrz1w"
    },
    {
      "policyType": "RECENT",
      "budget": 100,
      "discount": 0.16,
      "exclusions": ["6wyPOVWLmZ2b35eCmOrz1w"]
    },
    {
      "policyType": "BALANCED",
      "budget": 100,
      "numIntervalsLookback": 0
    }
  ]
}

File Disk Friendly exports in our node library

For our 3rd-party developers out there, our library now has some extra goodies included when importing into a node environment.

• sc.api.instance.LocalInstance("path/to/instance/root") streamlines reading and writing instance configurations and outputs on local disk.
• The Plugin class implementations for each plugin are included, meaning you can now programmatically call our graph api with ease.

Contributions

1c31ba1 Bugfix: use discord user id if the name is all special characters (#3223)
b2bfd16 Support Sorting in Current Balance and Earned Column. (#3227)
b472cc9 Deprecate old GrainConfig attributes + add updater (#3210)
852294f API: create extended library for node (#3214)
e905f33 Discord: Simplify default graph + simplifyGraph config (#3203)
00a2f59 Add Spacing to stringify output. (#3202)
29b4c69 Fix color property (#3211)
4f8aba2 Add raw GrainConfig parsers (#3206)
bcdb968 Combo: Add fmap as a parser class method (#3207)
55adb29 Recent Policy: add exclusions list of identities (#3204)
4cc0484 Initial Cleanup of UI color palette (#3197)
90c7efc Discord: Channel Category weight to use inheritance instead of multiply (#3200)
04abbb8 - now accounts update when update name (#3198)
46706be Discord: Skip channels with a weight of zero (#3199)
0ea3d8a Discord: add support for Channel Category weights (#3195)
8ecc779 support delimiter for Discord's guildid (#3194)
435f6c3 Implement Grain Integration functionality in the CLI (#3085)

sourcecred v0.9.3

Summary

This is primarily a patch for the build bugs that broke 0.9.2, though it does also contain some minor UX improvements on our Ledger History and Transfer Grain UI pages.

There were a bunch of awesome new features added in 0.9.2, read about those here.

sourcecred v0.9.2

Summary

⚠️ ⚠️ Warning⚠️ ⚠️

This release has bugs in the build. To use the awesome features from this release, please upgrade to 0.9.3.

---

Support for adding text descriptors to certain Discord and Discourse config attributes

Now, you can add text descriptors to certain attributes by adding // and any description you want afterwards at the end of the attribute key. Example:
"477551557723029514//general channel": 2,
The attributes that support this are:

• Discourse category ids
• Discord role ids
• Discord channel ids

ExternalPlugin: flexible/powerful 3rd-party plugins

The ExternalPlugin is used for loading a graph and optionally a declaration and
identityProposals from either:

1. the plugin config folder on disk
   • To use this method, simply place the files into the
     config/plugins/external/X folder.
2. a base url that statically serves the files
   • To use this method, simply serve the files statically with cross-origin
     enabled, and add a config.json file in the
     instance's config/plugins/external/X folder with form:
     { "graphUrl": "https://www.myhost.com/path/to/graph", "declarationUrl": "https://www.myhost.com/path/to/declaration", "identityProposalsUrl": "https://www.myhost.com/path/to/proposals" }

Supported files for either method are:

1. graph.json/graph (required) - works whether or not it is
   compressed using our library
2. declaration.json (optional) - if omitted, a default declaration with
   minimal node/edge types is used, but also graphs don't have to adhere to the
   declaration if they don't desire to be configured using our
   Weight Configuration UI.
3. identityProposals.json (optional) - if omitted, no identities are proposed

The External plugin can be used multiple times, because it simply uses the
PluginId pattern "external/X" where X can be any name an instance admin chooses.

We are working on this tutorial for developers to work with the ExternalPlugin. It should be done by the end of August 2021.
https://observablehq.com/@sourcecred/external-plugin-tutorial

Option to write JSON outputs without compression

If you want to get the non-compressed JSON of your credGraph, credGrainView, or other compressed output files, you can simply run:

yarn sourcecred graph --no-zip
yarn sourcecred credrank --no-zip

Contributions

77e6898 Add --no-zip flag to CLI, deprecate gzip suffix (#3162)
41d6044 Add ConstructorPlugin for ExternalPlugin testing (#3154)
5270b58 CredGrainView: Add advice to common error message (#3164)
14d7d07 Add Cache version suffix in circleCI (#3159)
82378e5 Use combo parser instead of compat for PluginDeclaration (#3153)
4d468ac Bsodenkamp allocation identities refactor (#3115)
c124d7b Revert "UI: Add Filtering to to Ledger View (#3122)" (#3147)
d4df10d Create External Plugin that reads from disk or url (#3114)
ea05c00 Add support for a text descriptor in number strings in config files (#3145)
c347f5a Set up Grain Integration support in Config API (#3079)
56aa079 UI: Add Filtering to to Ledger View (#3122)
3d3eb21 Implement CSV Grain Integration Plugin (#3078)
9f11c25 Update the root clean cmd to remove all node_modules (#3124)
57ecb9c Update Grain Integration to Accept file output (#3077)
1b723b1 UI: Fixed react leak on identities page (#3104)
216e1c7 converted immediate policy to CredGrainView (#3102)

sourcecred v0.9.1

Summary

BREAKING CHANGE: Balanced Policy now can be filtered to last X weeks

If using the allocationPolicies attribute in the GrainConfig, numIntervalsLookback must now be added to BALANCED policies to filter the cred-per-grain that the policy takes into account to the last X weeks. To look at all-time, set numIntervalsLookback to 0. Example:

{
  allocationPolicies: [
    {
      policyType: "BALANCED",
      budget: 5000,
      numIntervalsLookback: 0
    }
  ],
  maxSimultaneousDistributions: 100
}

Latency performance for UI

Now the credrank CLI outputs a smaller aggregated data structure called CredGrainView, which the UI now loads instead of the much larger CredGraph. This results in a much faster UI load time.

Discord message retention after Author leaves server

The behavior of the Discord plugin has changed to include messages in the graph even after their author leaves or is kicked from the server. The author will no longer be included in the cred graph, but their messages will be, so anyone that their messages mentioned will keep that cred. This will likely affect Cred scores upon upgrading.

[Experimental] Discord Confidence Dampener configuration

A new optional confidenceDampener number attribute can be used in the Discord reactionWeightConfig when applyAveraging is enabled to add "0 votes" to the averaging logic. This means that as more people react to a message, the total cred logarithmically approaches the role-weighted average. This calculates out along the lines of:

Confidence dampener = 1
1 people adding 1 emoji => 1/2 => 0.5
2 people adding 1 emoji => 2/3 => 0.67
3 people adding 1 emoji => 3/4 => 0.75
4 people adding 1 emoji => 4/5 => 0.80
5 people adding 1 emoji => 5/6 => 0.83

Confidence dampener = 2
1 people adding 1 emoji => 1/3 => 0.33
2 people adding 1 emoji => 2/4 => 0.5
3 people adding 1 emoji => 3/5 => 0.6
4 people adding 1 emoji => 4/6 => 0.67
5 people adding 1 emoji => 5/7 => 0.71

Contributions

7cfd3b8 UI: Add identity id tooltip to Grain Accounts page (#3099)
f472480 Bsodenkamp balanced policy lookback 2678 (#3093)
60ffcfb Discord: add confidence dampener config to emoji weighting (#3098)
a28fc77 Add default labels to issue templates (#3095)
6509852 Monorepo fix: put github templates in root (#3094)
fc06d08 changed block under conditional check for author existencenow message node always added (#3092)
461d60b Add dev flag to LoginButton
3f7967f Add web3 login button in AppBar
0722e53 Setup Web3Context with web3modal and ethers.js
f44ae63 localinstance extends readinstance (code dedup) (#3088)
ec13926 Cosmetic updates to ledger events (#3084)
d3fdf74 Cache CredGrainView for improved UI load times (#3086)
6fc431c Better error for wrong repository name (#3083)