Element Capture

[eladalon1983]

Hej TAG!

I'm requesting a TAG review of Element Capture.

A combination of pre-existing mechanisms (getDisplayMedia, Region Capture) already allows Web applications to capture a portion of the current tab as video MediaStreamTrack, robustly cropping away irrelevant pixels. Such videos can than be transmitted remotely; removing pixels not intended for sharing helps the sharing user's privacy, and prevents distraction by the receiving users. It also helps conserve compute and network resources.

Our new API, Element Capture, takes this a step further, allowing Web applications to remove unwanted occlusions. For example, if a private message notification appears over the shared region, it is possible to avoid capturing that message, which also avoids transmitting it remotely, and therefore helps uphold privacy guarantees implicitly made to the user, who had only intended to share the target-region, and not whatever happened to be drawn over it.

• Explainer: https://github.com/screen-share/element-capture/blob/main/README.md
• User research: N/A
• Security and Privacy self-review: https://github.com/screen-share/element-capture/blob/main/questionnaire.md
• GitHub repo: https://github.com/screen-share/element-capture/
• Primary contacts (and their relationship to the specification):
  • Elad Alon, @eladalon1983, Google
  • Mark Foltz, @mfoltzgoogle, Google
  • Jordan Bayles, @baylesj, Google
• Organization/project driving the design: Google Chrome and Google Meet
• External status/issue trackers: https://chromestatus.com/feature/5198989277790208

Further details:

• I have reviewed the TAG's Web Platform Design Principles
• The group where the incubation/design work on this is being done (or is intended to be done in the future): Screen Capture CG
• The group where standardization of this work is intended to be done ("unknown" if not known): Screen Capture CG and/or WebRTC WG
• Existing major pieces of multi-stakeholder review or discussion of this design: TPAC 2023 SCCG / WebRTC WG joint meeting
• Major unresolved issues with or opposition to this design: There are disagreements between Google and Mozilla on privacy concerns.
• This work is being funded by: Google

You should also know that...

Strong positive Web developer feedback for this feature was expressed on screen-share/element-capture#3 and during Screen Capture CG meetings.