Diffing with unsafeHTML to preserve internal component's state?

[alienself]

Hi,

I implemented ssr using lit-ssr in my project and I would like the client to be able to receive server updates in the form of an html template that I can use in my page component's render function as the result of a POST request, similar to Next.js's server actions and revalidatePath.

I tried to send both raw html and pre-rendered html template (the json output of the html function) but i get various errors and issues.

Raw HTML:

existing html in page component

<h1>list</h1>
<div class="list">
    <my-post></my-post>
</div>

const serverUpdate = `
  <h1>list</h1>
  <div class="list">
      <my-post></my-post>
      <my-post></my-post>
  </div>`;
return html`${unsafeHTML(serverUpdate )}`;

The problem of unsafeHTML is that we loose the state contained in the custom elements such as my-post. The element is basically destroyed and recreated, which is absolutely not a possibility for my use case.

JSON template:

Instead of sending raw html we can also run Page.prototype.render() on the server to basically return the JSON output of the render function.

const serverUpdate ={
   "_$litType$": 1,
   "strings": [
       "\n    <h1>list</h1>\n    <div class=\"list\">\n        <my-post></my-post>\n        <my-post></my-post>\n    </div>"
   ],
   "values": []
}
serverUpdate.strings.raw = serverUpdate.strings; // allows unsafe html
return serverUpdate;

With this technique, I am able to basically the get same result as unsafeHTML
however with the same issue, the state of is not not maintained and the whole html tree is replaced.

Is there any way I could patch the template as if the server update was rendered on the client?

[augustjk]

Preserving state is difficult because we leverage actual DOM nodes rather than creating a separate virtual DOM with all state/props.

The HTML string output from SSR will only contain attributes so if you're able to encode all needed data there, it could work. But any non-attribute bindings will be lost and you need to hydrate them.

Even in regular SSR use case we recommend that the same template and data be shipped to the client for hydrating like in the small code snippet here: https://github.com/lit/lit/tree/main/packages/labs/ssr#server-only-templates

While serializing template result is another possibility, with a chance of preserving other bindings, you'd need the values to also be serializable. I'm not sure how these would work with things like directives and event bindings certainly aren't serializable as they're functions. There might be interesting optimizations possible if the template is already present in the client and the server could just provide the values array.. but you'd need a way to match that to a template.

Partial server render and update is indeed an interesting pursuit but it would require a lot of care and coordination.

Also looking at your raw html example, would nested custom elements need to be server rendered with declarative shadow dom? Then unsafeHTML won't work as the template elements representing DSD won't get parsed as such. You'd need to manually use DOMParser for that.

[alienself]

Thank you for your response, @augustjk.

I am currently exploring the replication of Next.Js versions 13 and 14 using only Lit. To achieve this, I have developed a ServerLitElement class. This class includes an asynchronous render() method, enabling direct database queries within the render method itself, as illustrated below:

class ServerLitElement extends LitElement {
  // ...
  async render()  {
      const todos = await db.select(...);
      return html`<ul>${todos.map(todo => ...)}</ul>`;   
  }
}

It's important to note that ServerLitElement does not support event bindings, akin to Next.Js. This characteristic significantly simplifies certain serialization challenges. Consequently, ServerLitElement should be regarded as akin to more static templates.

Like NextJs, ServerLitElement allows for nesting within other ServerLitElements or slotting into LitElement. These client elements are rendered on both the client and server sides.

I have successfully adapted lit-ssr to accommodate async render(), which has been functioning effectively. For hydration purposes, I adjusted ssr-client to avoid errors in case of digest discrepancies. This is due to ServerLitElement potentially containing server-specific code, which is not present on the client. Therefore, a dummy LitElement is created for each ServerLitElement on the client, rendering a precomputed version of their render function. After the main ssr pass, I generate the render functions for each ServerLitComponent used on the page. These render functions are initially embedded in the HTML page within a script tag. Notably, these ssr passes do not expand custom components within the template but rather "bake" the render function of a ServerLitComponent.

This system then utilizes these pre-rendered data in the dummy client ServerLitComponent's render method (which is not asynchronous on the client) and is primarily useful during the hydration phase.

This approach has been highly successful. However, challenges arise when updating the content of the render method for these dummy/client-side ServerLitComponents. For example, if a new todo item is added, it necessitates updating the Todo list ServerLitComponent. In such cases, we might invoke a GET request to /revalidate-path?page=/todos to refresh the server elements of our page with the new todo item.

We can re-run the ssr for the ServerLitComponents on our page to generate new render functions for the ServerLitComponent that the client will use. These are then sent to the client, prompting a render update on the affected dummy ServerLitComponents.

However, a significant drawback emerges at this point: although the ServerLitComponents update with new server data, their use of unsafeHTML or a similar raw string method for rendering leads to the recreation of elements from scratch. This results in the loss of all internal states.

I am seeking advice on how to resolve this issue. Is there an alternative to unsafeHTML that would allow us to preserve the component's state? Given that we trust the HTML returned from our server, it should be treated as a regular template.

[augustjk]

Thanks for the detailed explanation! We've been exploring some ideas around this too but never got to iron out the details so it's exciting to see you actually have working code.

The server component approach seems viable with limitations on bindings, but indeed updating a server rendered component is a challenge. We don't have a clear public way of doing this but the idea would to send serialized template results and be able to persist the same reference to the strings array so that client updates will recognize the same reference and do an update rather than recreate the DOM nodes. Lit-html uses the strings array reference for caching and detecting whether the template has changed.

The very first update on the client will have to be through hydrate() which sounds like you're doing with the "dummy" client render. At that point the HTML from the server and the render root will be associated with a template result. Calling render() on that render root with a template result with the same strings array reference and different values would then do an update.

I don't know what your client render/hydration template looks like but perhaps a new tag function that wraps the base html tag function to control what strings array goes in similar to our static html tag function, or maybe a directive that uses some key to look up templates could allow swapping that template result with one sent from a server?