Electron upgrade 11.4.7

[sadick254]

TODO

• Update the CI with the proper node and npm versions
• Update electron version
• Update electron dependencies (electron-chromedriver and electron-mksnapshot) to match the electron version
• Get the app to bootstrap on the new electron version
• Once the build job passes make sure the artifacts are runnable/installable
• Fix any failing tests and make sure the test jobs are passing on the CI
• Test the artifacts against the regression checklist below
• Check performance issue after upgrade on Linux (Check Slow UI and busy GPU thread with Electron 6 on Linux (with Intel CPU and no GPU?) #20924 and the conversation below)

BLOCKERS AND ISSUES

• Failing github package tests
• Project-wide find and replace not working.
• macOS: Key binding with composition characters are ignored - alt-l inserting the - char

---

Regression Checklist

General testing

• Review issues tagged with electron to see if any of them are fixed

• Verify that you can collaborate with Teletype

  • Host using Atom's current stable release can collaborate with guest using Atom with the new version of Electron
  • Guest using Atom's current stable release can collaborate with host using Atom with the new version of Electron
  • Host and guest can collaborate when both are using Atom with the new version of Electron
  Steps

  1. Open a file 1. Share a portal 1. Have a guest join the portal 1. When guest edits the text in the file, verify that the host sees those changes

Check for regressions experienced in previous upgrades

Text Input/Keybindings

See how to setup keyboard layouts.

• IME not working (1.19.0-beta2: Cursor stucks at the first letter when using macOS Chinese IME #14911)

  • macOS
  • Windows 10
  • Ubuntu
  Repro steps

  1. Install an IME layout, for example Pinyin simplified
  2. Open Atom and use the layout you installed as the keyboard layout
  3. Example, if you picked Pinyin simplified type in zhongwenshuru

  Expected: in Pinyin simplified 中文输入 is expected

  

  Issues we've seen: Only shows latin first character and not every character typed.

  

• macOS: Key binding with composition characters are ignored (key-mappings with alt don't suppress Mac OS composition characters anymore (atom-1.19) #15189)

  Repro steps

  1. Open your keymap.cson file and add the following:

     'atom-text-editor':
       # use hjkl; vim-like when pressing alt (word by word, paragraph by paragraph)
       # or alt (letter by letter)
       'alt-h': 'core:move-left'
       'alt-j': 'core:move-down'
       'alt-k': 'core:move-up'
       'alt-l': 'core:move-right'

  2. Save keymap.cson and reload atom.
  3. Try the new keymappings under ABC - Extended keyboard layout

  Expected: the keybindings work

  Issues we've seen: it types out the modified keys and ignores the key mapping

• macOS: Composition characters mess up insertion point (Composition characters mess up insertion point #15344)

  Repro steps

  1. Open keymap.cson and add the following:

     'atom-text-editor':
       'alt-n': 'core:move-down'
       'alt-o': 'core:move-down'

  2. Have the following in your buffer:

     Foo
     Bar
     

  3. Have the cursor at the beginning of file(before F character) Type alt-n, then arrow right, then insert a character. Observe that the character is inserted at the beginning of the line (where the alt-n left you).
  4. Repeat steps using alt-o instead of alt-n.

  Expected: Both alt-o and alt-n behave the same

  Issues we've seen:
  

• Ubuntu: Keystrokes involving ctrl resolve to the default layout instead of the active layout (Keystrokes involving ctrl on Linux resolve to the default layout instead of the active layout #13170)

  Repro steps

  This should be tested on Linux with gnome

  1. Install a Linux ditro with gnome, for example Ubuntu
  2. Install French AZERTY and English US QWERTY layouts
  3. Change the default layout to be French AZERTY
  4. Open Atom and switch to English US QWERTY layout
  5. Press ctrl-w

  Expected: core:close to dispatch (Or the keybinding-resolver to resolve to ctrl-w). Should be resolving to the keyboard layout that is chosen and not the OS default layout.

  Issues we've seen: core:undo dispatched because it was resolved as ctrl-z because AZERTY has Z where W is on QWERTY.

• Other keyboard layouts on new Electron version

  • macOS: ABC - Extended
  • macOS: Slovak
  • macOS: U.S. International
  • macOS: Squirrel (https://github.com/rime/squirrel)
  • macOS: Full-Width Alphanumeric (Google) - Japanese layout (https://www.google.co.jp/ime/)
  • Windows: U.S.
  • Linux: Pinyin Simplified

UI

• tree-view drag image (Display the correct drag image on Electron >= 1.14 tree-view#1054)

  • macOS
  • Windows 10
  • Ubuntu
  Repro steps

  1. Open a project in Atom with multiple files and a few folders
  2. Drag a file into a folder

  Expected:
  

  Issues we've seen:

  

• drag-and-drop indicator (Fix missing drop indicator on Electron >= 1.14 tree-view#1055, Fix missing drop indicator on Electron >= 1.4 tabs#426, Fix mistakenly shown docks drop indicator on Electron >= 1.4 tabs#437)

  • macOS
  • Windows 10
  • Ubuntu
  Repro steps

  1. Open Atom with a few folder projects on tree-view (you can do this on terminal in a directory that has existing folder and having a space between each folder. Example: atom folderA folderB folderC)
  2. Drag and drop all of the folders to reorder them on Atom

  Expected:
  The placeholder indicates where the folder/project is being dropped to.

  

  Issues we've seen:
  No placeholder shows up after drag and dropping

  

• Large file rendering (Bad rendering after ~1 million lines #16591)

  • macOS
  • Windows 10
  • Ubuntu
  Repro steps

  1. Open a file that has over one million lines
  2. Scroll down to the bottom

  Expected: To not regress the number of lines that can be rendered. Rendering to be correct for lines past a certain point. Atom 1.25 can render around 800k to 1 million lines correctly.

  Issues we've seen: Increased number of lines rendered but bad rendering past a certain point

  

• Loss of subpixel AA when the cursor is at the end of long lines (Loss of subpixel AA on soft-wrapped line #16889, Don't break subpixel AA when cursor is at the end of longest line #16595)

  • macOS
  • Windows 10
  • Ubuntu
  Repro steps

  1. Open text-editor.js in Atom
  2. Copy the first 21 lines to a new untitled window
  3. Turn on soft wrapping and change language mode to javascript
  4. Resize the Window so it's soft wrapped
  5. On line 15 type a until the you reach the end of the window

  Expected: To not lose subpixel AA

  Issues we've seen: Loss of subpixel AA. Both when soft wrapping was enabled and disabled.

  

• Scrolling horizontally shift + scroll wheel (Upgrade Electron to v1.6.x #12696 (comment))

  • macOS
  • Windows 10
  • Ubuntu
  Repro steps

  1. Open a file that has long lines
  2. Turn off soft wrapping and resize the window so you have a horizontal scrollbar
  3. Press shift and scroll in both directions using the mouse wheel

  Expected: File to scroll horizontally in both directions

  Issues we've seen: File does not scroll horizontally at all in any direction

• Scrollbars misbehaving on the first file that is opened (Upgrade Electron to v1.6.x #12696 (comment))

  Repro steps

  1. Open Atom
  2. Open some file that has content enough to have a vertical scrollbar
  3. Close Atom
  4. Open Atom and look at the vertical scrollbar

  Expected: Scrollbar to be visible

  Issues we've seen: Scrollbar is not visible and is flickering when you are editing

• Middle clicking on unsaved tab (1.19, Linux: Middle-clicking an unsaved tab causes entire desktop to be unresponsive to clicks #15197)

  • macOS
  • Ubuntu
  Repro steps

  1. Open Atom and open a new unsaved file.
  2. Make edits to the unsaved file and middle click on the tab.

  Expected:
  Clicking save/cancel or the options on the dialog works.

  Issues we've seen:
  The UI and the dialog is unable to receive mouse clicks. You can still choose options via Keyboard, but not mouse.

• Linux: Atom scrolls even when not focused (Atom scrolls even when not focused #15482)

  Repro steps

  1. Open Atom with a large enough file that has a scrollbar
  2. Open a different application and place it over the Atom window
  3. Scroll with the mouse inside the other window
  4. Focus the Atom window

  Expected: Atom window should keep the original scroll position

  Issues we've seen: The Atom window scrolls after it is focused
  Note: this is a regression that has not been fixed since Electron 3, will leave it in the template to keep it on our radar once it's fixed

Other

• Supported Versions of OS (Atom 1.19.0 crashes instantly after launching #15297)

  • macOS
  • Windows 10
  • Ubuntu
  Repro steps

  1. Check the list of supported platforms Electron Master
  2. Check what version is listed as minimum for each platform on https://atom.io and https://atom.io/faq

  Expected: https://atom.io and https://atom.io/faq to display the same versions as the Electron documentation

  Issues we've seen: macOS version support changed and https://atom.io and https://atom.io/faq was not updated

• macOS : Native tabs (Native tabs on macOS #14711)

  Repro steps

  1. Open two atom windows on mac
  2. Go to View in menus and check Show Tab Bar
     
  3. Do this for both Atom windows.
  4. Drag one atom window to the other via tab bar

  Expected:
  Native Tab bar shows up and able to drag different windows to each other. Also make sure UI isn't messed up.

  Issues we've seen:
  Native Tab Bar doesn't work and messes up UI when enabled

• Ubuntu with KDE: Menu uses 100% CPU (Globalmenu consumes 100% CPU and does not work electron/electron#8455, Menus not working in plasma 5.9 with global menu enabled #13885)

  Repro steps

  1. Install Atom on Ubuntu KDE. [instructions]
  2. Enable KDE plasma global menu: Right-click on the desktop -> Add Panel => Application Menu Bar. [demo]
  3. Start Atom.
  4. Try to use the desktop menu.

  Expected:
  Desktop menu items work

  Issues we've seen:
  Desktop menu items do nothing when you click on things.

• macOS: Slovak QWERTZ (Slovak QWERTZ does not resolve keys correctly on macOS atom-keymap#223)

  Repro steps

  1. Install Slovak keyboard layout on mac
  2. Open keyboard viewer to see the keys are are suppose to show up and open keyboard resolver on atom.
  3. hit cmd+'.

  Expected: Resolves to cmd+' like it shows on keyboard viewer on mac.

  Issues we've seen: It resolves to ctrl+§

• macOS: IME jump (1.20.0: Popup jump when using macOS Chinese IME #15696)

  Repro steps

  1. Install an IME layout, for example Chinese pinyin
  2. Type one character

  Expected:
  Expected the IME to be positioned at under the character you entered
  Issues we've seen: The IME window is in the top left corner. When you enter the second character it jumps to be positioned under the text

[sadick254]

I don't think the tests would pass either. We first need to get the build passing and start working on the failing tests.

[DeeDeeG]

More Windows 32-bit problems.

I opened a PR for this one: atom/atom-keymap#250

It is a similar issue with keyboard-layout, although this was already fixed in atom/keyboard-layout#59 and released in keyboard-layout@2.0.17. But the new keyboard-layout package version still needs to be bumped to at least 2.0.17 in atom-keymap's package.json. So that's what my PR does.

Update: It's building for me on all platforms. Still issues with the tests, but at least it's building! https://dev.azure.com/DeeDeeG/b/_build/results?buildId=1186 (I had to fork tree-sitter to patch the version we use to compile as C++ 14.)

[sadick254]

@DeeDeeG I will be using your tree-sitter fork to get the macOS build passing for now.

[DeeDeeG]

Brief summary:

Okay. I recommend installing it as npm install --package-lock-only https://github.com/DeeDeeG/node-tree-sitter#bb298eaae66e0c4f11908cb6209f3e141884e88e.

(Which is this commit: DeeDeeG/node-tree-sitter@bb298ea)

Full comment below:

I think for best security, I would recommend putting the exact commit hash in the URL in package.json, rather than the GitHub branch name. Because the branch can be updated on GitHub, whereas at least forcing a repeated commit hash for altered content in git is super hard/unlikely. So the commit hash is stable and will only point to the intended code. Similar verifiability to the npm registry, I think.

(So, safe barring a rather impressive hash collision attack??? Related article: https://github.blog/2017-03-20-sha-1-collision-detection-on-github-com/. For the record, I am not attempting such a hash collision! I just think it pays to be safe about security and use good practices. Like not using branch names or tag names in git URLs in package.json. Only full commit hashes is the most secure option for git URLs.)

So I suppose it would be :

"tree-sitter": "https://github.com/DeeDeeG/node-tree-sitter#bb298eaae66e0c4f11908cb6209f3e141884e88e"

(One can do npm install --package-lock-only https://github.com/DeeDeeG/node-tree-sitter#bb298eaae66e0c4f11908cb6209f3e141884e88e)

That is my commit DeeDeeG/node-tree-sitter@bb298ea bumping tree-sitter@0.17.1 to compile as C++14. That (0.17.1) is the version of tree-sitter Atom is currently using: https://github.com/atom/atom/blob/v1.58.0-beta0/package.json#L164

(Thoughts for the future: Eventually, perhaps the Atom github org should fork the tree-sitter/node-tree-sitter repo as atom/node-tree-sitter, and publish it to the npm registry as @atom/tree-sitter? Or request the tree-sitter org members publish another release in the 0.17.x series? Or hope they merge tree-sitter/node-tree-sitter#83, and upgrade Atom to tree-sitter@0.19/newer.)

[mfonville]

@sadick254 how about going to v11.4.10 of Electron? It has some security fixes and should have same compatibility

[mfonville]

Electron 11.4.11 is now available #22810

[sadick254]

Thank you @mfonville for following up on the Electron upgrade and Electron releases. We would update to the latest 11.x.x after getting this working. We did start with using customElements instead of the deprecated document.registerElement

[mfonville]

Having the polyfill for compatibility is a great idea, but I think it'd be a good idea to add a warning/message to alarm the package maintainers that they should implement this change. You wouldn't want to maintain this work-around polyfill forever.

[DeeDeeG]

FYI/heads up:

Some packages have issues. I tried ide-python and ide-javascript and there is an issue with an assert. Perhaps it has to do with the remote module being deprecated and off by default.

I tried to install hydrogen, but it had some error. I didn't have time to look into that yet.

The usual reliable terminus works like a charm, at least on Windows. 👍

[sadick254]

@DeeDeeG We have enabled remote module

atom/src/main-process/atom-window.js

Line 61 in 10f7d45

enableRemoteModule: true,

The remote module has not been removed on electron version 11. It will be removed on version 12. I think the issue might be something else.

[mfonville]

Electron now went to 11.5.0 https://github.com/electron/electron/releases/tag/v11.5.0 with some security fixes

It will be the final release for the 11-series.

[the-j0k3r]

Holy mother of electrons, wut?

Congrats, keep it going

[darangi]

@DeeDeeG Thanks for the heads up 🙇🏾 , I went ahead to install about 41 ide packages(The most I could find), I can confirm only ide-javascript and ide-python throw those errors. Also, Installing hydrogen does not work, it is stuck at installing.

I will create issues in the repos responsible for those packages

[UziTech]

@darangi @sadick254 what is needed to get this over the line?

[sadick254]

@UziTech I thought ide-javascript and ide-python that had errors would be fixed. But that doesn't seem to be the case. I see the errors from ide-javascript and ide-python as non-blockers, those packages are external.

The errors are related to the deprecation of remote module. We are planning to start the work to move from electron.remote on another branch.

[the-j0k3r]

Congrats. @sadick254 great job on this.

Thanks =)

[DeeDeeG]

I'm glad to see this merged!!!

One small request: Would the Atom GitHub org be willing to fork tree-sitter? That way I don't have to keep my fork live just to ensure Atom's master branch can build... I will keep my fork up as long as needed, obviously... I love Atom and want it to be able to build. But I think it's good to not have my personal GitHub fork of tree-sitter still "in the loop" for building official Atom. Thank you!

Atom's master branch as of this PR needs my fork of tree-sitter to be available on GitHub in order to build. See:

atom/package.json

Line 164 in 7033a5a

"tree-sitter": "git+https://github.com/DeeDeeG/node-tree-sitter.git#bb298eaae66e0c4f11908cb6209f3e141884e88e",

Can an official maintainer of Atom, or the atom GitHub org, host the forked tree-sitter so I don't have to be "in the loop", and official Atom's master branch can build without my fork?

Example workflow to do this:

• Fork https://github.com/tree-sitter/node-tree-sitter under the atom GitHub org, using github.com
• Clone it locally git clone https://github.com/atom/node-tree-sitter AND cd node-tree-sitter
• Optional: add my fork as a remote git remote add deedeeg https://github.com/DeeDeeG/node-tree-sitter AND git remote update
• Optional: check out the commit with my patch as a new branch git switch -c [some-branch-name-here, for example "0.17.1-C++-14"] bb298eaae66e0c4f11908cb6209f3e141884e88e
  • (OR just manually recreate @sergei-dyshel's very tiny, one-line patch, see: DeeDeeG/node-tree-sitter@bb298ea... and commit that on some branch.)
• Push the branch with the patch to the atom GitHub org's fork... git push --set-upstream origin [whatever-branch-name-has-the-patch, for example "0.17.1-C++-14"]
• Then you can switch the tree-sitter dependency in Atom's package.json to the following...
  • "tree-sitter": "git+https://github.com/atom/node-tree-sitter.git#bb298eaae66e0c4f11908cb6209f3e141884e88e",
  • Or if you manually recreated the patch, then specify whatever your actual resulting commit hash is, instead of bb298ea...
• Optionally, since git dependencies in package.json are kind of troublesome to deal with, you could post a forked version of tree-sitter to the npm package registry and use that... but probably better to focus on Bump tree-sitter@0.19.0 #23283 or similar, instead of putting in that effort?

[baarkerlounger]

Will there be a new beta now that this has been merged or are there other things that need to be done first?

[iamunadike]

Please @sadick254 I understand aabout the archiving of at atom by microsoft, but I am among those that I already love the editor so much to give up on it anytime soon, most people say this and that about VScode and the likes but forget atom is the most hackable editor till date,
Infact it too precise,
With that in mind , I will like to appeal for your continuous work as a great team of developers are willing to work with you using atom fork to continue the development and releases. God bless you Sir for all the effort and work you have put in place to keep this atom project alive and others too who contributed
PLEASE lets not give up, we Have come this far. we shall PREVAIL