Core Services: 3.17 Tracking issue

[tsenart]

Status

This tracking issue is being worked on.

Availability

Period is from 21st May to 20th of June. Please write the days you won't be working and the number of working days for the period.

• @keegancsmith
• @tsenart: 10d
• @unknwon: 22d

Workload

@bobheadxi: 0.50d

• src config commands fail #10419 0.5d 🐛

@keegancsmith

• errors seen in indexed-search on bigdata cluster with 250k repos #11200
• Index multiple (non-master) branches #6728 10d 👩

@ryanslade: 5.50d

• Download raw file fails on large repos #10466 1d 🐛
• Reduce usage of raw string for external service kind #10296 1d 🧶
• Validate unique rate limit configuration #10026 0.5d
• Use internal rate limiters for all source requests #9953 2d
• RFC 132: Use rate limit data when scheduling changeset syncs #9105 1d

@tsenart

• No Git file history for binary files #10999 🐛
• gitlab-org/gitlab ResolveRev errors with 504 Gateway Timeout on sourcegraph.com #10960 🐛
• repo-updater: Use per HTTP request timeouts #11105 👩
• repo-updater: Change Postgres isolation level to default #10957
• Add golangci-lint GitHub Action #10858 🧶

@unknwon: 16.00d

• Background permissions syncing should not rely on ServiceID to match authz provider #11023 4d 🐛
• Background permission fetching causes repos from 2nd code host to be inaccessible #11318 1d 🐛
• Detect GitHub 404 errors during background permissions syncing #11007 0.5d
• Update GraphQL mutation to support multi-level permissions #10952 0.5d 🐛
• Ping investigation: Instances are offline #10598 1d
• License expired notice is not showing up for users #10363 1d 🐛
• Ensure that we are tracking Go test coverage correctly #10356 2d 🛠️
• Docs: How background permissions syncing works #10190 1d
• RFC 142: Migrate backend e2e test with unit tests #10067 4d 🛠️
• authz: better heuristic of NextSyncAt by PermsSyncer #9098 1d
• PoC: src-expose push based #10045 2d 🕵️

@uwedeportivo

• repo-updater: make handleStatusMessages log entry shorter and more useful #11203

Legend

• 👩 Customer issue
• 🐛 Bug
• 🧶 Technical debt
• 🛠️ Roadmap
• 🕵️ Spike
• 🔒 Security issue
• Pull Request

[unknwon]

Last week

• Some polish work to be ready for making background permissions syncing as the default method.
• Integrated rate limit into background permissions syncing. Waiting for review authz: make PermsSyncer respect rate limit #10950.
• Fixed license expired notice is not showing to users. Waiting for review license: show expired notice to users #10951.

W20 Plan [2020-05-25]

• RFC 142: Migrate backend e2e test with unit tests.
• Turn on background permissions syncing by default.
• PoC: src-expose push based.
• Docs: How background permissions syncing works.

[ryanslade]

Last Week

Focused on campaign related work, specifically around webhook syncing.

This week

Will continue working on campaign work by finishing of webhook sync changes and then move onto work on src-cli

[unknwon]

Last week

• Made our GraphQL APIs support multi-level permissions.
• Made background permissions syncing as the default.
• Added docs and links to background permissions syncing in UI.
• Finished PoC of cloning Git repo over WebSocket.

W21 Plan [2020-06-01]

• Finish migrating/adding unit tests derived from backend e2e tests, also help restructure the code of Add notifications when sourcegraph.com licenses are close to expiring… #10362 and add tests.
• Detect GitHub 404 errors during background permissions syncing.
• Ensure that we are tracking Go test coverage correctly.
• Remove dead code in the usage stats and db package.

[ryanslade]

Last Week

• Added a progress bar to src-cli as well as a few minor cleanups
• Removed Bitbucket webhook syncing

This week

• Finish off src-cli issues
• Fix infinite retry bug: Repo-updater can fall into infinite retry loop #10798
• Finally get around to finishing off some dangling rate limiter issues

[unknwon]

Last week

• Finished migrating applicable backend e2e tests to unit tests.
• Finished detecting GitHub 404 errors during background permissions syncing.
• Removed dead code in the usage stats and db package.
• Investigated -coverpkg option to increase our Go test coverage but it took 16m (vs 2m) to finish and only increased 3%. test: use -coverpkg=./... to increase coverage #11332

W22 Plan [2020-06-08]

• Fix background permission fetching causes repos from 2nd code host to be inaccessible.
• Fix background permissions syncing should not rely on ServiceID to match authz provider.
• Start migrating backend e2e tests to integration tests.
• Start drafting RFC 178: Sourcegraph Cloud authorization model

[ryanslade]

Last Week

• Fixed a few campaign bugs
• Made a bunch of changes to src-cli
• Sped up the raw archive file endpoint. It no longer needs to download an entire archive, just the requested file
• Debt task to remove uses of raw external service kinds and type strings

This week

• Campaign OOM bug
• Rate limiter tasks

[tsenart]

Start drafting RFC 178: Sourcegraph Cloud authorization model

@unknwon: Bear in mind that I asked @sqs to help with this since he did a great job thinking about this for campaigns recently. I'd wait for Quinn's initial proposal, and you'd take it from there. This, however, was in the context of RFC 162: Devolve the site admin role. Did you have something slightly different in mind?

[unknwon]

Bear in mind that I asked @sqs to help with this since he did a great job thinking about this for campaigns recently. I'd wait for Quinn's initial proposal, and you'd take it from there. This, however, was in the context of RFC 162: Devolve the site admin role. Did you have something slightly different in mind?

@tsenart: Thanks for the reminder! It would be more like my brain-dump of some rough ideas I've been keeping in my TODO (more technical, and site-admin is only one aspect for example), I can get busy enough with migrating e2e tests this week so no problem limit my initial effort on this RFC and wait for @sqs.

[unknwon]

Last week

• Fixed background permission fetching causes repos from 2nd code host to be inaccessible.
• Fixed background permissions syncing should not rely on ServiceID to match authz provider.
• Finished e2e test prototype, started migrating browser-based e2e tests.

W23 Plan [2020-06-15]

• Finish Spike: Use Postgres intarray to store permissions object_ids.
• RFC 142: Migrate backend e2e test with integration tests.

[ryanslade]

Last week

• Clean up raw strings used for external service "kind" and "type"
• Add code to estimate rate limit "cost" for GitHub
• Move internal rate limiting into code host clients
• Simplify code after the above change

W23 Plan

• Not working on Tuesday
• Focusing on a few performance related issues: Repository language query not returning correct results in a timely fashion #11466,
  src actions exec can cause frontend to be OOMKilled #10540, repository status page with "cloning" filter is extremely slow #11029
• Planning for next iteration

[uwedeportivo]

Dear all,

This is your release captain speaking. 🚂🚂🚂

Branch cut for the 3.17 release is scheduled for tomorrow.

Is this issue / PR going to make it in time? Please change the milestone accordingly.
When in doubt, reach out!

Thank you