How to Integrate the FormValidityObserver into lit

[ITenthusiasm]

I'm opening this discussion after talking briefly in lit/rfcs#8 with the hope of learning how to properly integrate the FormValidityObserver into Lit. For those who don't know, the FormValidityObserver (source code) is a tool in the @form-obser/core NPM package that enables devs to validate their form fields as users interact with them (and to display accessible error messages for those fields when they become invalid). It also exposes some helpers for manual error handling.

For those who are interested or who will find it helpful for the discussion, I'll give some Background and Goals for the Lit Integration. And, of course, I'll add Questions.

Background

The FormValidityObserver is a little unique in that it doesn't try to reinvent the browser's form features in a framework-specific way. Instead, it seeks to enhance the areas where the browser falls short, and it encourages the use of the existing browser features that already seem to work well. The enhancements include things like:

• Accessible Error Messaging
• HTML5 Validation Support
• Custom Validation Logic (including async logic)
• Progressive Enhancement
• Many other things

Because of the utility's approach, most of the heavy lifting is actually done in the Core, and much of the logic is easily portable to any JS Framework. So far, each of the JS framework integrations have only needed to do 3 things:

1. Provide a helper function (autoObserve) that can (optionally) be used to automatically setup/cleanup the FormValidityObserver.
2. Wrap the FormValidityObserver.configure method so that it can provide the right attributes to form controls during SSR in addition to doing what the raw configure method does. (More on that in a later question.)
3. Re-expose the FormValidityObserver's methods as destructure-able properties.

So my questions will primarily be related to items 1 and 2. But I'm open to any unrelated feedback that could make the FormValidityObserver better (and therefore improve DX for Lit users).

Goals

While building out this integration, these are the goals that I would like to achieve if possible:

1. Maintain the same API across JS frameworks. (Example API for Svelte with source code -- less than 100 LOC.)
2. Support SSR so that form controls are rendered with the proper constraint attributes (e.g., required, minlength, etc.).

So far, I've been able to achieve these goals with Svelte, Vue (source), Solid (source), and React (source). So I'm hoping that I can achieve them with Lit as well.

Questions

The OP felt really long when I included the full questions. So I've placed the original questions here, and I've put their details in separate comments.

• How to Integrate configure for Lit Apps?
• Is Automated Setup/Teardown Possible?
• Can RefCallback Be Tied to a Specific HTMLElement?
• Any Recommendations for Event Delegation and the Shadow DOM in Lit?

[ITenthusiasm]

1) How to Integrate configure for Lit Apps?

As I mentioned in the Background, every JS framework integration wraps the core configure method by adding some logic to provide the proper constraint attributes to fields during SSR (and CSR). All of these frameworks support some kind of "props spreading". So configure is able to take something like this:

<textarea
  id="comment"
  aria-describedby="comment-error"
  {...configure("comment", {
    required: "We covet your feedback.",
    minlength: { value: 80, message: (field) => `Say something based on ${field.value}` },
  })}
/>

and produce markup in SSR/CSR like this:

<textarea id="comment" aria-describedby="comment-error" name="comment" required minlength="80" />

Because Lit uses template strings, I naively thought that the configure method could create an "Attributes String" instead of creating a "Props Object". But my mindset was naive, because this:

html`<textarea id="comment" aria-describedby="comment-error" ${configure(/* ... */)}></textarea>

seems to ignore any attributes generated by configure, and instead it produces:

<textarea id="comment" aria-describedby="comment-error"></textarea>

I figured there was something that I was fundamentally misunderstanding regarding Lit, and I was hoping to get insight on a path forward.

[augustjk]

html`<textarea id="comment" aria-describedby="comment-error" ${configure(/* ... */)}></textarea>

If configure() returns a string, this is invalid as lit-html does not allow dynamic values where tag name or attribute names would appear. We call these element expressions and only directives are allowed to be present in element and you likely want to make configure a custom directive which sets attributes directly on the element.

A caveat is that directives that work on element part currently do not have support in Lit SSR and will do nothing.

[ITenthusiasm]

Ah, I see. That is indeed unfortunate/difficult. Does Lit currently have anything else that represents "props/attributes spreading"-like behavior? If not, are there any plans to introduce such a thing in the future (and would it be supported in SSR)?

How much adoption does the SSR tool have? I noticed that it's still experimental; so I'm wondering if it would just make sense to push forward with a custom directive for now and revisit the idea of SSR later.

[augustjk]

lit/rfcs#26 and there's also a community implementation of it here https://github.com/open-wc/open-wc/blob/master/docs/docs/development/lit-helpers.md#spread-directives

If you're only ever creating string attributes, it's possible to use static expressions to insert raw strings using unsafeStatic into the template. That'll work for SSR too. I wouldn't really recommend that though as this requires using a different html tag function by the user and leaves opening for injections.

Lit SSR has been around for a while with some integrations with other tools and frameworks like Astro, Eleventy, Next.js, and Nuxt. It shouldn't be completely discounted but support could be an add on later.

Custom directives are generally the recommended pattern for work like that.

[ITenthusiasm]

Okay. That's helpful to know. I guess I'll look into custom directives for those who are okay with just doing CSR.

Regarding SSR, since everyone is already in the same boat of not being able to spread attributes anyway, would something like this (for SSR support) be acceptable?

html`
  <form>
    <textarea
      id="comment"
      name="comment"
      minlength="80"
      required
      aria-describedby="comment-error"
      ${configure("comment", {
        required: "We covet your feedback",
        minlength: (field) => `Say something based on your ${field.value}`,
      })}
    ></textarea>
  </form>
`;

The core FormValidityObserver.configure method was written to work with pure HTML+JS applications. Thus, it does nothing on the server, but it configures the error messages to display on the client (and returns void). The core method does not need any references to HTMLElements (though it does need the field's name), so it should work just fine by itself in Lit. The reason that other JS framework integrations have special configure wrappers is so that developers can:

1. Collocate their error configurations with their fields.
2. Avoid redundantly mentioning the same prop/attribute multiple times.

With the approach that I just showed, Goal 1 is still attainable with SSR. But without support for spreading attributes, would it be safe to conclude that Goal 2 is impossible? If so, would this be a sufficient workaround that isn't too bad/inconvenient for DX? (If it is acceptable, is it still worth creating the custom directive?)

[ITenthusiasm]

2) Is Automated Setup/Teardown Possible? [Resolved]

As I mentioned in the Background, each JS framework integration exposes an (optional) autoObserve function that automates the process of setting up/tearing down the form validity observer. Here's an example in Svelte (9 LOC) that relies on their actions feature.

I thought I could accomplish something similar with a Ref Callback in Lit. But the following seems to do nothing:

// Somewhere in Lit's `createFormValidityObserver` ...
observer.autoObserve = function autoObserve(novalidate = true) {
  /** @type {HTMLFormElement | undefined} */
  let form;

  /** @param {typeof form} litRef @returns {void} */
  return (litRef) => {
    if (litRef) {
      form = litRef;
      observer.observe(form);
      if (novalidate) form.setAttribute("novalidate", "");
      return;
    }

    observer.unobserve(/** @type {HTMLFormElement} */ (form));
    form = undefined;
  };
};

export class TestForm extends LitElement {
  #observer = createFormValidityObserver("input");
  #formRef = this.#observer.autoObserve() as RefOrCallback<Element>;

  render() {
    return html`<form ${ref(this.#formRef)}> Other Markup...</form>`;
  }
}

So I was hoping for insight on how to proceed.

[augustjk]

The ref directive with a callback should be able to do that.
https://lit.dev/playground/#gist=bd27910c19328736ff6f94de1964049d

[ITenthusiasm]

Yes, that seems to be true. Apparently, while I was doing some debugging, I added a connectedCallback without calling super.connectedCallback(). I wasn't aware that doing so would break the application. 😅 The ref is functioning as expected now.

[ITenthusiasm]

3) Can RefCallback Be Tied to a Specific HTMLElement? [Resolved]

In each of my JS Framework integrations, I've been typing "Ref Callbacks" (or their equivalents) such that the reference argument expects an HTMLFormElement. But when I tried this in Lit, TypeScript complained, saying that it only accepts callbacks that expect an Element. Is there a way to use ref such that it expects an HTMLElement instead?

I know that making the ref function generic might not have that much value since the templating logic doesn't really care. But from a DX perspective, a generic ref function type can more clearly set expectations on what a given ref callback is expecting as an argument when it's used.

[ITenthusiasm]

https://stackblitz.com/edit/form-observer-lit-example?file=src%2Fmy-element.ts,src%2FcreateFormValidityObserver.js&terminal=dev

[augustjk]

Hmm yeah this might be a bug. Generics have some problems getting carried through to directives.

[ITenthusiasm]

Gotcha. Should I open an Issue for that or does it need more investigation? If yes, is there a specific repository?

[augustjk]

https://github.com/lit/lit/issues/new/choose with a minimal repro would be awesome. thanks!

[ITenthusiasm]

Opened #4441

[ITenthusiasm]

4) Any Recommendations for Event Delegation and the Shadow DOM in Lit? [Resolved]

When I first wrote the FormValidityObserver (which itself extends a much smaller class called the FormObserver), I didn't (fully) take the ShadowDOM into account. Currently, the observers use event delegation to listen for user interactions at the document level. This allows the observers to automatically account for fields added to or removed from the <form>'s list of form.elements. Only events belonging to an observed <form> field are acknowledged.

The problem with this is that Lit tends to create elements in the ShadowDOM by default. This causes event.target to become the Host Element instead of the actual field that emitted the event. It seems that developers could force a form-related Lit Component not to use a ShadowDOM, but not using the ShadowDOM doesn't seem to be recommended by the docs.

Some potential options (without having explored them in greater detail) could be to:

1. Look at event.composedPath() instead of event.target (though I'm not sure if there'd be any performance consequences for doing so.
2. See if I can use event delegation somewhere within the ShadowRoot itself.

If there are any thoughts on this, that would be appreciated.

---

But there's another great problem: If I understand correctly The ShadowDOM disrupts forms. I don't consider this a bug in the Web Standard. Since the ShadowDOM is intended to hide a set of elements from the outside, then it only makes since that these hidden elements would also be hidden from <form>s. This makes some things difficult in Lit, though.

The reason this can be problematic is that sometimes developers want to split their forms into multiple sub components. One component is one section of the form, another component is another section, etc. If each of these components are created with Lit (using the default settings), then that means that their fields will be invisible to the "owning" <form> (whether the form is in the Light DOM or its own ShadowDOM). This means that techniques like using FormData(form) and form.elements are no longer effective. It also means that progressive enhancement no longer works. But ironically, things are broken in a reversed fashion: Fields that would be properly submitted in forms without JS will not be submitted by default with JS (assuming I understand SSR correctly).

Has Lit considered how to deal with this dilemma? Is there a standardized way to resolve this?

[ITenthusiasm]

Sweet! I'll see what I can do with composedPath() then. Hopefully it doesn't have any negative performance implications -- though I guess each field will only ever be interacted with one at a time anyway.

---

Yes! I'm familiar with formAssociated components, and I'm all about them. 😄 My main concern was for developers who basically take this

<form>
  <input name="name" type="text" />
  <! -- Even more fields -->
  <input name="some-other-field" type="text" />
  <!-- Yet more fields! -->
</form>

and do this:

function MyForm() {
  return (
    <form>
      <FirstSection />
      <SecondSection />
    </form>
  );
}

function FirstSection() {
  return (
    <>
      <input name="name" type="text" />
      {/* Even More Fields */}
    </>
  );
}

function SecondSection() {
  return (
    <>
      <input name="some-other-field" type="text" />
      {/* Yet more fields! */}
    </>
  );
}

Although I don't necessarily recommend it, I have seen developers do this with frameworks like React. If they tried to replicate this behavior in Lit, they'd end up in a situation where all of the native fields in FirstSection and SecondSection would fail to be associated with the wrapping <form> element. (This is because FirstSection and SecondSection would have their own ShadowDOMs separate from the wrapping <form>.) I didn't know if I should try to account for this situation, or if I should just expect all developers to be aware of how the ShadowDOM works when they build out their forms and to write their code accordingly. (For instance, they'd either have to put everything inside Form or use createRenderRoot for nested groups of native elements.) I didn't know if Lit set any expectations on this either.

I could certainly set expectations in my library's documentation if needed.

[ITenthusiasm]

It seems that getRootNode can be used to grab the owning Document or ShadowRoot. So that might be an alternative to using event.composedPath, and it would only need to be called once (during form registration). That is, instead of attaching listeners to form.ownerDocument and looking at event.composedPath() as events occur, I could attach listeners to form.getRootNode() and look at event.target whenever events happen.

The only caveat here is that observe would always need to be called after an element is added to the DOM, not before. Additionally, <form> elements would not be allowed to move between root nodes (though that would be a very odd and unusual use case). Theoretically, if form elements really needed to be moved between root nodes, they could be unobserved, moved, then re-observed.

However, the benefit of this approach is that it would be compatible with closed ShadowDOMs (if a developer did not want their form to be exposed to the outside at all). event.composedPath requires developers to always use an open ShadowDOM. Curious to know if there are any other thoughts on that.

Of course, the issue that I described about the "Form, FirstSection, and Section" scenario remain. I don't know if there's a way to prevent that from becoming a problem for developers -- since it goes against the browser's design.

(I know this is a lot. No rush. Thanks so much for your time answering all of these questions and giving your input.)

[ITenthusiasm]

It looks like I have to use node.getRootNode() anyway because of some functionality in the FormValidityObserver. So I guess I'll use that over event.composedPath().

[ITenthusiasm]

I'm going to mark this question as resolved now. To recap, I think the main options (if not the only options) are:

1. Use event.composedPath() to find the form control in the ShadowDOM while an event handler is being called.
2. Register the FormValidityObserver with node.getRootNode() so that all ShadowDOM form controls are discoverable.

The big benefit of the 2nd option is that it enables the FormValidityObserver to work with closed ShadowDOMs (unlike option 1). However, it also requires knowledge of the registered form's root node. So the form needs to be attached before it is observed, and it needs to be unobserved before it is detached.

The benefit of the 1st option is that everything can work fine without having knowledge of a form's root node. However, this approach gives less flexibility to developers, and knowledge of the form's ownerDocument is still necessary.

So it basically comes down to tradeoffs. Due to the versatility of the 2nd Option, I'm going to see if I can work with that for the time being. We'll see how it goes. In the meantime, I'm marking this as resolved. That just leaves #4437 (reply in thread) to be answered.

[ITenthusiasm]

For Historical Purposes Only

So @augustjk and @sorvell are geniuses. I think the 2nd option is definitely the way to go. We can save developers from the burden of having to keep track of the root node themselves (or, more importantly, having to aggressively keep track of when unobserve is called) if we just have the FormObserver keep track of the root(s) associated with the observed form(s). This will require some additional testing for the FormObserver.

See https://discord.com/channels/1012791295170859069/1184292640477429860/1184292640477429860 for additional context.

[ITenthusiasm]

First Draft of Usage with Lit

@augustjk Not sure if you saw my latest comment on the last remaining open question, but I decided to go through with the idea since it would work with SSR. (I can add a client-only solution if it would be beneficial.) If Lit ever supports dynamic attributes in the future, the code would be a little less redundant.

The example Lit usage can be found here. The main action is happening in lit-example.ts. Any thoughts/feedback would be appreciated (if you or anyone else is able).

The FormValidityObserver mainly addresses validation and accessible error messaging for forms in applications (hence the utility's name). For storing form data, I still appeal to using FormData with the raw HTMLFormElement. I plan to add some documentation soon to give ideas on how to use FormData more effectively/easily for complex form data.