mandatory property missing from MediaTrackConstraints

[jameshfisher]

Preflight Checklist

• I have read the Contributing Guidelines for this project.
• I agree to follow the Code of Conduct that this project adheres to.
• I have searched the issue tracker for an issue that matches the one I want to file, without success.

Issue Details

• Electron Version: v11.1.0
• Operating System: macOS 11
• Last Known Working Electron version: -

Expected Behavior

TypeScript compiles with no type errors.

Actual Behavior

error TS2322: Type '{ mandatory: { chromeMediaSource: string; chromeMediaSourceId: string; }; }' is not assignable to type 'boolean | MediaTrackConstraints | undefined'.
  Object literal may only specify known properties, and 'mandatory' does not exist in type 'MediaTrackConstraints'.

To Reproduce

In a TypeScript project, use the desktopCapturer API as described in its docs, e.g. just use the example code on that page, or:

const screenCaptureStream = await navigator.mediaDevices.getUserMedia({
  audio: false,
  video: {
    mandatory: {
      chromeMediaSource: 'desktop',
      chromeMediaSourceId: source.id,
      // etc
    }
  }
});

Additional Information

It looks like this API is part of Chrome rather than part of Electron, and are parsed somewhere here in media_constraints_impl.cc. It also looks like this mandatory section is part of some old/deprecated getUserMedia API. It would be great if we could use desktopCapturer API with the modern getUserMedia API - maybe we can, but the docs don't say how.

It would also be great to know what other constraints we can pass in this mandatory section. For example, can we set the frame rate for desktop capture?

[codebytere]

@jameshfisher pretty sure this is just a docs update - you should be able to use the new syntax just fine. What this error is saying (correctly) is that mandatory is no longer an accepted property, and you should use syntax specified here.

[jameshfisher]

Hey, thanks @codebytere - however when I try using these properties using the modern getUserMedia API, they're ignored: I get the default camera video stream instead of a screen capture. Here's what I've tried:

// This works as expected - gets a screen capture
screenCaptureStream = await navigator.mediaDevices.getUserMedia({
  audio: false,
  video: {
    mandatory: {
      chromeMediaSource: 'desktop',
      chromeMediaSourceId: source.id,
    }
  }
});

// But this just gets the default camera stream instead of the screen -
// same behavior as if these params were not present
const screenCaptureStream = await navigator.mediaDevices.getUserMedia({
  audio: false,
  video: {
    chromeMediaSource: 'desktop',
    chromeMediaSourceId: source.id,
  }
});

// Similarly, using `ConstrainDOMStringParameters` long form -
// same behavior, gets the default camera stream instead of the screen
const screenCaptureStream = await navigator.mediaDevices.getUserMedia({
  audio: false,
  video: {
    chromeMediaSource: { exact: 'desktop' },
    chromeMediaSourceId: { exact: source.id },
  }
});

So it seems like this API really is stuck in the dark ages of getUserMedia. It would be lovely if it were updated to use the modern API, but I think that's a separate problem to the lack of type definitions for the current API.

(Edit: above testing is on Electron 11, on macOS 11.)

[codebytere]

@jameshfisher hmm ok i'll check into it and see what i come up with :)

[adampah]

Has there been any resolution that allows using the getUserMedia function with a specific sourceId? I found the following two threads which contain a similar issue: [1] and [2] but no further luck. As far as I can tell, every time I try to use getUserMedia, the only video stream that I can retrieve is one of the entire screen. Is there a still way to retrieve specific windows like mentioned at the start of this thread?

[Icehunter]

Any update on this at all?

[nornagon]

Seems relevant to #30702

[ch1ny]

It seems this bug has not been fixed yet.
I'm not sure would this helped, but I found a weird way to implement this feature.

I write my React source code in TypeScript and write a preload script in JavaScript.
And here is my code:

// preload.js
const { contextBridge, ipcRenderer } = require('electron');

contextBridge.exposeInMainWorld('captureDesktop', function () {
	return new Promise((resolve) => {
		ipcRenderer.invoke('DESKTOP_CAPTURE').then((source) => {
			navigator.mediaDevices
				.getUserMedia({
					audio: {
						mandatory: {
							chromeMediaSource: 'desktop',
							chromeMediaSourceId: source.id,
						},
					},
					video: {
						mandatory: {
							chromeMediaSource: 'desktop',
							chromeMediaSourceId: source.id,
						},
					},
				})
				.then((stream) => {
					console.log(stream);
					const video = document.createElement('video');
					video.srcObject = stream;
					resolve(video);
					// If we resolved MediaStream directly, we could only get an empty object.
				});
		});
	});
});

// TypeScript
(window as any).captureDesktop().then((videoDOM: HTMLVideoElement) => {
	console.log(videoDOM.srcObject); // MediaStream
})

I guess this is not the best way to implement this feature, but it does work.

[Goldiserv]

Based on this https://stackoverflow.com/questions/37613981/how-to-use-a-typescript-cast-with-jsx-tsx
Seems like TS follows the latest WebRTC specification. Each browser can have differences in API as WebRTC specification is not an official standard yet.
Hence TS code needs to cast to any, which worked for me:
const stream = await (navigator.mediaDevices as any).getUserMedia(constraints)

[github-actions]

This issue has been automatically marked as stale. If this issue is still affecting you, please leave any comment (for example, "bump"), and we'll keep it open. If you have any new additional information—in particular, if this is still reproducible in the latest version of Electron or in the beta—please include it with your comment!

[github-actions]

This issue has been closed due to inactivity, and will not be monitored. If this is a bug and you can reproduce this issue on a supported version of Electron please open a new issue and include instructions for reproducing the issue.

[Jeshvenkatesh]

Hi @codebytere Any update on this issue ?

[neobyteUK]

Would also like some updates on this as have the same questions as OP, can we get this ticket reopened?

[aziz-boudi4]

Any update on this issue ?