Declarative Shadow DOM

[dgp1130]

I've been struggling lately with a lot of CSS bundling and scoping challenges (see #37). With Declarative Shadow DOM on its way and already supported by Chrome, I think a better strategy might be to find a DX story around that and then find a way to gracefully fall back for browsers that don't support it,

I experimented with this in ref/declarative-shadow-dom-prototype and got something which reasonably works. It roughly looks like this:

export async function renderComponent(lightDom: string): Promise<string> {
    return `
        <!-- Host element. -->
        <div id="component">
            <!-- Shadow root. -->
            <template shadowroot="open">
                <!-- Polyfill declarative shadow DOM for browsers that don't support it yet. -->
                ${polyfillDeclarativeShadowDom()}

                <!-- Inline styles to apply them to the shadow root. -->
                ${await inlineStyle('rules_prerender/examples/declarative_shadow_dom/component.css')}

                <!-- Shadow DOM content, styled with the associated style sheet. -->
                <div>Shadow content</div>

                <!-- Slot to insert the associated light DOM. -->
                <slot></slot>
            </template>

            <!-- Light DOM content, not affected by above style sheet. -->
            ${lightDom}
        </div>
    `;
}

The new inlineStyle() function is similar to includeStyle() but instead of injecting an HTML comment which gets postprocessed into a singular <style /> tag for the page, it gets directly inlined in its own <style /> tag at the specified location. This allows users to put styles directly inside a shadow root, which otherwise wouldn't be possible without hard-coding the CSS. This scopes the CSS to only the DOM elements directly in the shadow DOM.

I also needed to add a polyfill for browsers which don't support it. This seems to work ok, but does require JavaScript, likely has a FOUC, and needs to be manually included anywhere a declarative shadow root is used.

This mostly seems to work and solves the expected problem, but there are a few open questions to resolve:

I suspect the implementation might still be a little buggy (unfortunately declarative shadow DOM doesn't work in Stackblitz so its hard to make a minimal reproduction for these):

• I can't get a declarative shadow root with an unnamed <slot /> element with default content to actually load. It works if I use a named <slot /> with default content, but I'm not sure why the name is required. This might be a quirk of the spec for complicated reasons or just a bug in Chrome's implementation.
• After adding about 100 shadow roots, each of which inlining a 100 line CSS file, the polyfill breaks in Chrome for some reason. It seems like the last shadow root doesn't get parsed correctly and doesn't get transformed into a real shadow root, then gets picked up by the polyfill, which fails. I couldn't find anything wrong with the syntax or anything unique about the last element. I'm also not sure why I need so many shadow roots before this bug reproduces. Definitely seems like a bug here.

I'm not sure about the best way to load CSS. AFAICT, the only way to apply CSS inside a declarative shadow root is to have a <style /> or <link /> tag within it. I don't want to use a <link /> tag because it means I need to bundle each component's shadow CSS in a separate file and serve it (which is very awkward and rules_prerender doesn't take that much liberty with file structure atm, the user is supposed to be in control of that). I also don't want to use inline a <style /> tag because it will be repeated every time an element is rendered, significantly increasing the bundle size.

For now I'm just inlining the <style /> tag each time. I was hoping that gzip would do a good job of compressing such highly repetitive content but my initial experiments aren't very promising. Even if the bundle size increase is compressed out, the styles still need to be parsed multiple times, take up extra memory, and aren't as easy to debug (modifying one component's styles in DevTools wouldn't affect other components of the same type).

What I would love to see is something like <style shadow-id="foo" />, then declare a shadow root with <template shadowroot="open" styles="foo" /> to reference the specific inline style that should be applied to that root. I suppose that's possible via runtime JavaScript, but its not built into the standard. I'll need to do more investigation and exploration here to understand what is the best way of loading styles in a repeating declarative shadow DOM structure.

One other question is about tree shaking CSS. This model provides no means of tree shaking unused styles. We could add PurgeCSS and run it on the output, though I'm not sure if it's smart enough to take declarative shadow DOM into account. Even if it does, we always run the risk of client-side JS applying a class at runtime when the style got erased. That's kind of an independent issue, but I want to make sure that whatever strategy we use for CSS is tree shakable, as that will undoubtedly be a necessary optimization.

[dgp1130]

I've been continuing to iterate on this and have something that I think is usable. Unfortunately I'm struggling a bit with tests. I would like to make some automated test for the declarative shadow DOM polyfill, but I don't have a set browser test setup just yet. I tried integrating Karma but after a few hours I'm declaring miserable failure.

I immediately ran into bazelbuild/rules_nodejs#2093 and had to upgrade rules_nodejs (and @bazel/*) to >= 3.4.2 to get a fix. This seems to break some other targets, but I haven't gotten to them yet.

After that, I got some errors like this which cause a timeout after 30 seconds:

24 07 2021 21:48:46.074:WARN [web-server]: 404: /jasmine.js
Chrome Headless 76.0.3809.0 (Linux x86_64) ERROR: 'There is no timestamp for jasmine.js!'

Which led me to bazelbuild/rules_nodejs#1872 and bazelbuild/rules_nodejs#1867, I don't fully understand everything, but it seems like the files need to be UMD bundles, which doesn't seem to be the case here.

I tried to fix this by using a tsconfig with module: "umd" and introducing a new js_library() with named_module_srcs (which appears to be undocumented, but is used in the Karma example?)

This is able to actually run the test, however it executes 0 of 0 specs. So the files aren't being loaded correctly for some reason. I made sure the files include an import / export so the files are interpreted as modules. I also added /// <amd-module name="..." /> comments as used in the example. However the test still doesn't execute anything.

Running the test in my own browser and opening DevTools, I see a temporary file with this content:

      // A simplified version of Karma's requirejs.config.tpl.js for use with Karma under Bazel.
      // This does an explicit `require` on each test script in the files, otherwise nothing will be loaded.
      (function(){
        var runtimeFiles = [
          // BEGIN RUNTIME FILES
          
          // END RUNTIME FILES
        ].map(function(file) { return file.replace(/\.js$/, ''); });
        var allFiles = [
            // BEGIN USER FILES
            
            // END USER FILES
        ];
        var allTestFiles = [];
        allFiles.forEach(function (file) {
          if (/[^a-zA-Z0-9](spec|test)\.js$/i.test(file) && !/\/node_modules\//.test(file)) {
            allTestFiles.push(file.replace(/\.js$/, ''))
          }
        });
        require(runtimeFiles, function() { return require(allTestFiles, window.__karma__.start); });
      })();

Note the // BEGIN USER FILES and // END USER FILES with no files between them. It definitely looks like my tests should be included in there, but they just aren't for no particular reason.

It's worth noting this quote from the @bazel/concatjs documentation:

Ultimately by using concatjs, you’re signing up for at least a superficial understanding of [UMD].

I definitely do not have that much understanding of UMD and don't particularly care to learn it just to get some Karma tests to work.

My prototype is in ref/karma.

[dgp1130]

After making no progress with Karma, I went looking to see if I could run browser tests any other way in Bazel. Jest has some support, but I'm personally not a fan of running browser tests on a fake environment built on Node. I would rather use a real browser if possible.

I took another look at @web/test-runner, which uses a real browser to test with ESM support and a lot less legacy than Karma or the karma_web_test_suite() Starlark rules. I couldn't find anyone who has actually tried to use @web/test-runner in Bazel, but I took a stab at it just to see if it would be possible. After a bit of fiddling, I was actually able to get something working, see ref/web-test-runner.

I'm not convinced that it's a good idea to use custom infrastructure here, but maybe it would be worth making a separate project to support @web/test-runner on Bazel? I'll keep playing around and thinking about whether or not I want to move forward with this.

[dgp1130]

I've been skipping the testing question for now as I don't think browser test infra would fully solve the problem here. I can't easily test the Declarative Shadow DOM polyfill anyways because it needs to run on an HTML page which has templates that were not processed and converted into shadow roots. That means I'd need to test on an old browser, and with a specific HTML file. While this probably possible, I'm not sure it's worth the effort to set up anyways.

Regardless, I've been stuck for a while trying to make the Declarative Shadow DOM component publishable to NPM (see #39). It is necessary to publish a component because users need to include the client-side script based on possibly conditional logic in their prerender code. I was planning to make something simple by publishing the TypeScript source code and letting users recompile it on demand. I was able to make this work within Bazel after tackling a few unexpected but related issues (alias() of a prerender_component() requires some more effort, and we need to publish the tsconfig.json the component is compiled with). However I'm not able to figure out the best way to import the component just yet. It can't be in the regular import { polyfillDeclarativeShadowDom } from rules_prerender, because it is built as a prerender_component(), not a regular ts_library(). If we allowed that import, the client side script would get dropped.

Instead, users have to depend upon a prerender_component() at @npm//rules_prerender:declarative_shadow_dom. That much works, but the big question is how to import it. The file is actually at packages/rules_prerender/declarative_shadow_dom.ts, which is a private path. It's also TypeScript, so the actual compiled JS is actually at:

dist/bin/out/k8-opt-exec-2B5CBBC6/bin/external/npm/rules_prerender/packages/rules_prerender/declarative_shadow_dom.js

That means we need some reasonable import statement to resolve to that path both in NodeJS at runtime, and in TypeScript (and tsserver in the IDE). I'm not really sure how to make either of those work. Adding to the complexity here is the fact that this path is in a non-standard configuration (2B5CBBC6). This value is not consistent and will vary based on user code, so we can't hard-code anything here in source.

I'm starting the think this "ship TS to NPM and compile it on the user's machine" isn't much of a workaround. It might be easier to just address the core problem and find a way to create a prerender_component() from JS by fixing #39.

[dgp1130]

I was finally able to solve the publishing problem in #39 and took another stab at this. I was able to develop a prototype which correctly publishes the full Declarative Shadow DOM component, including the client-side polyfill, and link / bundle everything correctly. I'll need to refine it a bit, but I'm pretty confident this can be landed without shaving too much more of a yak.

See the declarative-shadow-dom branch for current progress.

[dgp1130]

Merged Declarative Shadow DOM support to main. Example is here.

This will be included in the 0.0.10 release.