Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add new css parser - postcss #1458

Open
wants to merge 15 commits into
base: master
Choose a base branch
from

Conversation

daibhin
Copy link
Contributor

@daibhin daibhin commented Apr 24, 2024

rrweb-snapshot parses stylesheets to adapt the CSS for playback

In certain cases the parsing wasn't always correct:
PostHog/posthog#21427
#1379

Which led to improved CSS selector efforts first included in rrweb@2.0.0-alpha.13 (#1401)

Sadly this caused playback issues that consumed 100% CPU and froze playback
PostHog/posthog#21791
profiler


CSS parsing is performed in the https://github.com/rrweb-io/rrweb/blob/master/packages/rrweb-snapshot/src/css.ts which was pulled from https://github.com/reworkcss/css (which hasn't been updated in over 3 years)

I propose moving to https://github.com/csstree/csstree which creates an actual AST which can be used to effectively debug via https://astexplorer.net

I couldn't figure out some import issues in the test/integration.test.ts

Evaluation failed: ReferenceError: require is not defined

All test/css.test.ts tests still pass which confirms this is as good as the trickiest cases previously handled without any of the slowness introduced in #1401

Copy link

changeset-bot bot commented Apr 24, 2024

⚠️ No Changeset found

Latest commit: 4a48247

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

}
getSelectors(ast.stylesheet);
const mediaFeatures = css
.findAll(newAst, (node) => node.type === 'MediaFeature')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly some further optimizations we could even make here by using the css.walk method to only traverse the AST once (rather than twice like I assume findAll is doing)

@@ -1,6 +1,6 @@
{
"name": "rrweb-snapshot",
"version": "2.0.0-alpha.13",
"version": "2.0.0-alpha.12",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably be 13?

@eoghanmurray
Copy link
Contributor

This is great and something I thought we might get to 'some day' after the recent fixes to css.ts.

Would you be able to post a before/after of the built file sizes from this change; i.e.

$ ls -al packages/rrweb/dist/*.js packages/rrweb/dist/record/*.js packages/rrweb/dist/replay/*.js

One of the reasons css.ts was ported over was so that we could tree shake it a bit.

@daibhin
Copy link
Contributor Author

daibhin commented Apr 29, 2024

Something isn't right here because I'm getting the same values when running yarn build:dev && ls -al packages/rrweb/dist/*.js packages/rrweb/dist/record/*.js packages/rrweb/dist/replay/*.js on both branches.

Possibly need to fix the import errors I mentioned in the description first 🤔 🤷 Scratch that I got it working at the repo root

Before

-rw-r--r--@ 7 david  staff   26568 29 Apr 18:24 packages/rrweb/dist/record/rrweb-record-pack.js
-rw-r--r--@ 7 david  staff   10418 29 Apr 18:24 packages/rrweb/dist/record/rrweb-record-pack.min.js
-rw-r--r--@ 7 david  staff  165350 29 Apr 18:24 packages/rrweb/dist/record/rrweb-record.js
-rw-r--r--@ 7 david  staff   68811 29 Apr 18:24 packages/rrweb/dist/record/rrweb-record.min.js
-rw-r--r--@ 7 david  staff  209951 29 Apr 18:24 packages/rrweb/dist/replay/rrweb-replay-unpack.js
-rw-r--r--@ 7 david  staff   92849 29 Apr 18:24 packages/rrweb/dist/replay/rrweb-replay-unpack.min.js
-rw-r--r--@ 7 david  staff  193751 29 Apr 18:24 packages/rrweb/dist/replay/rrweb-replay.js
-rw-r--r--@ 7 david  staff   88049 29 Apr 18:24 packages/rrweb/dist/replay/rrweb-replay.min.js
-rw-r--r--@ 7 david  staff  401587 29 Apr 18:24 packages/rrweb/dist/rrweb-all.js
-rw-r--r--@ 7 david  staff  170294 29 Apr 18:24 packages/rrweb/dist/rrweb-all.min.js
-rw-r--r--@ 7 david  staff  351097 29 Apr 18:24 packages/rrweb/dist/rrweb.js
-rw-r--r--@ 7 david  staff  152936 29 Apr 18:24 packages/rrweb/dist/rrweb.min.js

After

-rwxrwxr-x@ 3 david  staff   26584 30 Apr 16:33 packages/rrweb/dist/record/rrweb-record-pack.js
-rwxrwxr-x@ 3 david  staff   10437 30 Apr 16:33 packages/rrweb/dist/record/rrweb-record-pack.min.js
-rwxrwxr-x@ 3 david  staff  157698 30 Apr 16:33 packages/rrweb/dist/record/rrweb-record.js
-rwxrwxr-x@ 3 david  staff   68728 30 Apr 16:33 packages/rrweb/dist/record/rrweb-record.min.js
-rwxrwxr-x@ 3 david  staff  191575 30 Apr 16:33 packages/rrweb/dist/replay/rrweb-replay-unpack.js
-rwxrwxr-x@ 3 david  staff   87904 30 Apr 16:33 packages/rrweb/dist/replay/rrweb-replay-unpack.min.js
-rwxrwxr-x@ 3 david  staff  175375 30 Apr 16:33 packages/rrweb/dist/replay/rrweb-replay.js
-rwxrwxr-x@ 3 david  staff   83086 30 Apr 16:33 packages/rrweb/dist/replay/rrweb-replay.min.js
-rwxrwxr-x@ 3 david  staff  375719 30 Apr 16:33 packages/rrweb/dist/rrweb-all.js
-rwxrwxr-x@ 3 david  staff  165248 30 Apr 16:33 packages/rrweb/dist/rrweb-all.min.js
-rwxrwxr-x@ 3 david  staff  325229 30 Apr 16:34 packages/rrweb/dist/rrweb.js
-rwxrwxr-x@ 3 david  staff  147850 30 Apr 16:34 packages/rrweb/dist/rrweb.min.js

@daibhin
Copy link
Contributor Author

daibhin commented Apr 30, 2024

In a significant update here I moved to using postcss once I figured out how their plugin system works. AST explorer was particularly useful in writing the plugins needed. Here's an example for the media selector: https://astexplorer.net/#/gist/244e2fb4da940df52bf0f4b94277db44/7ccd8e85c2a64282f021b34717b0076c418f61b1

This massively simplifies the adaptCssForReplay method and is extensible to the point that we can easily improve it by adding more custom plugins.

I've moved around some test files to better separate concerns. Some of the tests were "testing the framework" given they had been copied over from https://github.com/reworkcss/css/blob/master/test/parse.js. I removed any that were no longer relevant but left any of the cases added in https://github.com/rrweb-io/rrweb/pull/1440/files#diff-ca07555d1c1965f9c7ef3b5df3772a19953138b8eac62deb7786e549b25b455d as we know they've specifically broken in the past and we would like to avoid breaking them in the future

@eoghanmurray
Copy link
Contributor

Super! What sort of filesize numbers are there with the postcss version?

@daibhin
Copy link
Contributor Author

daibhin commented May 3, 2024

Sorry @eoghanmurray, probably wasn't clear but I updated my earlier comment with latest build numbers having run yarn build:dev && ls -al packages/rrweb/dist/*.js packages/rrweb/dist/record/*.js packages/rrweb/dist/replay/*.js

master

-rw-r--r--@ 7 david  staff   26568 29 Apr 18:24 packages/rrweb/dist/record/rrweb-record-pack.js
-rw-r--r--@ 7 david  staff   10418 29 Apr 18:24 packages/rrweb/dist/record/rrweb-record-pack.min.js
-rw-r--r--@ 7 david  staff  165350 29 Apr 18:24 packages/rrweb/dist/record/rrweb-record.js
-rw-r--r--@ 7 david  staff   68811 29 Apr 18:24 packages/rrweb/dist/record/rrweb-record.min.js
-rw-r--r--@ 7 david  staff  209951 29 Apr 18:24 packages/rrweb/dist/replay/rrweb-replay-unpack.js
-rw-r--r--@ 7 david  staff   92849 29 Apr 18:24 packages/rrweb/dist/replay/rrweb-replay-unpack.min.js
-rw-r--r--@ 7 david  staff  193751 29 Apr 18:24 packages/rrweb/dist/replay/rrweb-replay.js
-rw-r--r--@ 7 david  staff   88049 29 Apr 18:24 packages/rrweb/dist/replay/rrweb-replay.min.js
-rw-r--r--@ 7 david  staff  401587 29 Apr 18:24 packages/rrweb/dist/rrweb-all.js
-rw-r--r--@ 7 david  staff  170294 29 Apr 18:24 packages/rrweb/dist/rrweb-all.min.js
-rw-r--r--@ 7 david  staff  351097 29 Apr 18:24 packages/rrweb/dist/rrweb.js
-rw-r--r--@ 7 david  staff  152936 29 Apr 18:24 packages/rrweb/dist/rrweb.min.js

fix/css-selectors

-rwxrwxr-x@ 3 david  staff   26584 30 Apr 16:33 packages/rrweb/dist/record/rrweb-record-pack.js
-rwxrwxr-x@ 3 david  staff   10437 30 Apr 16:33 packages/rrweb/dist/record/rrweb-record-pack.min.js
-rwxrwxr-x@ 3 david  staff  157698 30 Apr 16:33 packages/rrweb/dist/record/rrweb-record.js
-rwxrwxr-x@ 3 david  staff   68728 30 Apr 16:33 packages/rrweb/dist/record/rrweb-record.min.js
-rwxrwxr-x@ 3 david  staff  191575 30 Apr 16:33 packages/rrweb/dist/replay/rrweb-replay-unpack.js
-rwxrwxr-x@ 3 david  staff   87904 30 Apr 16:33 packages/rrweb/dist/replay/rrweb-replay-unpack.min.js
-rwxrwxr-x@ 3 david  staff  175375 30 Apr 16:33 packages/rrweb/dist/replay/rrweb-replay.js
-rwxrwxr-x@ 3 david  staff   83086 30 Apr 16:33 packages/rrweb/dist/replay/rrweb-replay.min.js
-rwxrwxr-x@ 3 david  staff  375719 30 Apr 16:33 packages/rrweb/dist/rrweb-all.js
-rwxrwxr-x@ 3 david  staff  165248 30 Apr 16:33 packages/rrweb/dist/rrweb-all.min.js
-rwxrwxr-x@ 3 david  staff  325229 30 Apr 16:34 packages/rrweb/dist/rrweb.js
-rwxrwxr-x@ 3 david  staff  147850 30 Apr 16:34 packages/rrweb/dist/rrweb.min.js

@eoghanmurray eoghanmurray changed the title feat: add new css parser feat: add new css parser - postcss May 7, 2024
@@ -9,6 +9,9 @@ import {
} from './types';
import { isElement, Mirror, isNodeMetaEqual } from './utils';

// eslint-disable-next-line @typescript-eslint/no-unsafe-assignment, @typescript-eslint/no-var-requires
const postcss = require('postcss');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with the various import methods, but feel this needs a bit of an explanation as to why you are doing it this way

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is causing tests to fail with require is not defined

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it that we need a typescript version of https://github.com/giuseppeg/postcss-pseudo-classes
?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This require business might be fixed with this PR: #1033

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the heads up @Juice10. I've been meaning to get back to this PR but have had some vacation recently. The good news is that this seems to have resolved all known CSS issues for us. I can hold off until the Vite changes are merged if that is imminent

@eoghanmurray
Copy link
Contributor

eoghanmurray commented May 8, 2024

Hopefully the following doesn't derail the conversation, and even if we go ahead with the following, that doesn't detract from the worth of the current PR...

So okay I just had a thought; all this CSS processing is on the replay side, right?
I'm wondering why we don't just iterate over the final sheet.rules directly after the stylesheet has been rendered. We will be rendering it as an inline <style> so the .sheet.rules will be accessible. I don't think we'd need any 3rd party library for that approach, and it would have another advantage in that we could apply the hover classes to stylesheets that are only referenced in the recording (assuming the .sheet.rules are accessible). I'm planning to do this approach anyway so that we can apply our hover classes to CDN stylesheets that are currently recorded only by reference — if you replay the with crossorigin="anonymous" present, then it's possible to access and modify the rules for the sheet. If that succeeds, it could enable further paring back on what needs to be inlined at record time; we could avoid inlining anything that has a 'versioned' url and a permanent web presence.

===

edit: I've drafted this up in #1480

@pauldambra
Copy link
Contributor

a 'versioned' url and a permanent web presence.

I don't think we can know that something has a permanent web presence only because it has a versioned URL - e.g. I was just dealing with a customer bug report that centred around the assets only living for 15 days and not inlining at capture

We either need very robust inlining or asset capture at recording time (ideally both :)). Loading over the net back to the customer is only a fallback in a perfect world.

(I appreciate that's a (this) consumer of the library position and not necessarily a developer of the library position 😊)

@eoghanmurray
Copy link
Contributor

Yes, this could be configurable, and likely a robust solution will involve deploy of your own proxy/cache for assets. We replicate the '.hover' modification server side in our proxy cache so applying those modifications at replay on all .sheet (rather than only the captured cssText) would also make a web proxy easier to run.

@eoghanmurray
Copy link
Contributor

eoghanmurray commented May 23, 2024

Rebase or merge of master needed: there is a new test in #1481 which adds coverage to the css parser; hopefully it will pass (or fail) as-is without having to modify the test itself.

@eoghanmurray
Copy link
Contributor

Also re. rrweb-snapshot/test/rebuild.test.ts

if you revert the whitespace changes to the expected css, you could instead cherry-pick the following commit which should do loose matching by ignoring whitespace and semicolon:
85db6c2

@eoghanmurray
Copy link
Contributor

Let me know if you wish me to execute the above two for you.
Still awaiting someone to figure out how to pull in a typescript version of https://github.com/giuseppeg/postcss-pseudo-classes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants