Support non-US keyboard layouts

[jasongrout]

There are a number of issues with using JupyterLab with non-US keyboards. To summarize, we have two keyboard shortcut systems:

• Lumino (phosphor) keyboard shortcut system, used throughout JupyterLab. This shortcut system allows for non-US keyboards, but we need to have relevant keyboard mappings. We've talked about writing some UX to make it easier to make a keymap (basically capturing the key character and usb code and associating them, perhaps combining maps from different browsers)
• codemirror keyboard shortcut system for text editors and code cells. Should we expose this in some way that integrates with the Lumino shortcut system? Or should we expose this as a separate mechanism?

Designing a UX for capturing new keyboard mappings would be a great new extension, for example. The actual keyboard mapping data can be contributed to Lumino.

Related issues

#4778

[vidartf]

Would you mind expanding a little on why a keymap is needed / what problems they would solve?

[calvin-sykes]

Hi - just to point out a related issue is here: #4778

Forgive the unsolicited comment, but it does seem somewhat bizarre that a web app need concern itself with the mapping between physical keys and characters. Naively I would've assumed that this would remain the OS' and/or browser's job.

[cstenkamp]

Would you mind expanding a little on why a keymap is needed / what problems they would solve?

Many Non-US-Keyboards don't have certain buttons, like the "/". So on these keyboards, it is impossible to for example use the "comment code block"-functionality, which is really annoying.

[jasongrout]

Naively I would've assumed that this would remain the OS' and/or browser's job.

How we wish that were true! When we were designing the Lumino keyboard system a few years ago, the browsers were woefully inconsistent about how they applied these settings, and indeed still are last I checked, so we had to go with a lower common denominator of keycodes rather than relying on the browser giving us accurate key character information. Unfortunately, this means that we need to explicitly give the mapping between keycodes and actual silkscreened characters.

[jasongrout]

Many Non-US-Keyboards don't have certain buttons, like the "/". So on these keyboards, it is impossible to for example use the "comment code block"-functionality, which is really annoying.

FYI, those specific keys are set here:

jupyterlab/packages/codemirror/src/factory.ts

Lines 24 to 25 in 4fe4dcf

	'Cmd-/': 'toggleComment',
	'Ctrl-/': 'toggleComment'

jupyterlab/packages/codemirror/src/factory.ts

Lines 34 to 35 in 4fe4dcf

	'Cmd-/': 'toggleComment',
	'Ctrl-/': 'toggleComment',

I think it makes sense to have those specific bindings configurable via settings or JLab's shortcut system, if someone wants to submit a PR.

[vidartf]

Would you mind expanding a little on why a keymap is needed / what problems they would solve?

Many Non-US-Keyboards don't have certain buttons, like the "/". So on these keyboards, it is impossible to for example use the "comment code block"-functionality, which is really annoying.

I agree that this is a problem, but I'm saying that I'm not sure if I see how a keymap will solve it. Are we planning to special case the most popular maps, or is there some general way to solve the problem given a keymap?

[vidartf]

For context, I use a Norwegian keyboard layout, but use Window's Alt+Shift shortcut to toggle to a US layout for coding so that the øæå¨' keys next to Enter map to ;'[]\ instead of having to use Alt Gr + 8 / 9 for square brackets. I'm not sure if I want Lab to interfere in this process (i.e. certain symbols can only be generated in certain layout modes, so something interfering can render those inaccessible to me).

[jasongrout]

The current plan in Lumino is that it will maintain a database of keymaps and the user will be able to select the appropriate one to use manually. Perhaps there could be some autoguessing logic to aid finding the right keymap like is popular in various OS installations (like press the key to the right of Shift, etc.).

Here is the current keymap Lumino has: https://github.com/jupyterlab/lumino/blob/master/packages/keyboard/src/index.ts#L210

[jasongrout]

When we were designing the Lumino keyboard system a few years ago, the browsers were woefully inconsistent about how they applied these settings, and indeed still are last I checked, so we had to go with a lower common denominator of keycodes rather than relying on the browser giving us accurate key character information. Unfortunately, this means that we need to explicitly give the mapping between keycodes and actual silkscreened characters.

I just wrote up more explicit details about this in jupyterlab/lumino#74

[JasonWeill]

@jasongrout noted that Chrome can now provide info about the user's keyboard layout to web applications. Other browsers do not yet support this. To cover all users, provide an option for the user to manually set their keyboard layout.

[jasongrout]

@jasongrout noted that Chrome can now provide info about the user's keyboard layout to web applications.

See jupyterlab/lumino#271

[krassowski]

A minimal improvement that we could make would be ensuring that if a default shortcut conflicts with a combo for a key that is used on non-US keyboard, then typing a key resulting from the combo takes a priority over the shortcut. For example Alt + [ on an Italian Mac should still produce [ (#15744).

This is we could distinguish between combos which fire beforeinput and those which do not. Here is a PoC:

<input id="the-input" placeholder="Enter some text" />
<p id="values"></p>

const input = document.getElementById("the-input");
const log = document.getElementById("values");

input.addEventListener("keydown", maybeShortcut);
document.addEventListener("beforeinput", onbeforeinput);

let lastOnBeforeInput = null;

function onbeforeinput(e) {
  lastOnBeforeInput = e;
}

function maybeShortcut(e) {
  setTimeout(() => {
    const delta = lastOnBeforeInput.timeStamp - e.timeStamp;
    const data = JSON.stringify({
      code: e.code,
      shiftKey: e.shiftKey,
      metaKey: e.metaKey,
      altKey: e.altKey,
      tdelta: delta
    });
    if (lastOnBeforeInput && e.target === lastOnBeforeInput.target && Math.abs(delta) < 10) {
      log.textContent += `\ninput: ${data}`;
    } else {
      log.textContent += `\nshortcut: ${data}`;
    }
  }, 0)
}

it unfortunately relies on correct timestamps and these might be obfuscated in Firefox, especially in some privacy modes - this needs checking.

Edit: works fine on Firefox, but is not reliable when pressing keys quickly as the setTimeout trick moving it to the back of execution queue does not guarantee that beforeinput (dispatched by the browser after keydown has completed) was already executed.

Edit2: increasing timeout from 0 to 10ms and increasing threshold to 20ms does make it robust enough in practice. The downside is that waiting with the decision on shortcut activation reduces responsiveness. It would be best if we could test for whether a given key combo would invoke beforeinput but synthetic events are hard.

[krassowski]

Here is a cleaner implementation which I believe is good enough - I did not find any false positive for a shortcut in Firefox nor Chrome:

const input = document.getElementById("the-input");
const log = document.getElementById("values");

input.addEventListener("keydown", maybeShortcut);

async function maybeShortcut(e) {
  const causesInputPromise = Promise.race([
    new Promise(resolve => {
      e.target.addEventListener("beforeinput", () => {
        resolve(true)
      }, {once: true});
    }),
    new Promise(resolve => {
      setTimeout(() => resolve(false), 10)
    })
  ]);
  causesInputPromise.then((value) => {
    const modifiers = [];
    if (e.altKey) {
      modifiers.push('Alt')
    }
    if (e.shiftKey) {
      modifiers.push('Shift')
    }
    if (e.metaKey) {
      modifiers.push('Meta')
    }
    const data = JSON.stringify({
      code: e.code,
      modifiers: modifiers
    });
    if (value) {
      log.textContent = `input: ${data}\n` + log.textContent;
    } else {
      log.textContent = `shortcut: ${data}\n` + log.textContent;
    }
  });
}

<input id="the-input" placeholder="Enter some text" />
<pre id="values"></pre>

[JasonWeill]

As discussed at the JupyterLab meeting today, we could potentially pop up a notification on first launch if we detect that the user's browser locale (which we should be able to detect for all modern browsers) is not en_US. We could direct the user to the settings editor. Having palettes of known-good keyboard shortcuts for particular keyboards might be useful.