[T-290] Markdown transformer #3936

DeMoorJasper · 2019-12-22T15:59:04Z

↪️ Pull Request

This PR adds a markdown transformer for Parcel 2, the aim of this PR is to get parcel to become a fully functional static site generator from markdown files, by allowing specification of a layout file and metadata through frontmatter.

We can probably use this to generate the Parcel 2 docs site?

For code highlighting this uses highlight.js and for markdown parsing it uses marked.

Loosely based on #2882

Closes #3357

Close T-290

✔️ PR Todo

Added/updated unit tests for this change
Filled out test instructions (In case there aren't any unit tests)
Included links to related issues/PRs

height · 2019-12-22T16:00:19Z

This pull request has been linked to T-290.

parcel-benchmark · 2019-12-22T16:12:24Z

Benchmark Results

Kitchen Sink ✅

Timings

Description	Time	Difference
Cold	5.46s	-30.00ms
Cached	2.06s	-26.00ms

Cold Bundles

Bundle	Size	Difference	Time	Difference
dist/modern/kitchen-sink.90de237b.js	1.16kb	+0.00b	452.00ms	+48.00ms ⚠️

Cached Bundles

Bundle	Size	Difference	Time	Difference
dist/legacy/parcel.d5807e82.webp	102.94kb	+0.00b	226.00ms	+66.00ms ⚠️
dist/modern/parcel.d5807e82.webp	102.94kb	+0.00b	227.00ms	+66.00ms ⚠️

React HackerNews ✅

Timings

Description	Time	Difference
Cold	8.21s	+58.00ms
Cached	1.98s	-78.00ms

Cold Bundles

No bundle changes detected.

Cached Bundles

Bundle	Size	Difference	Time	Difference
dist/main/index.js	66.80kb	+0.00b	241.00ms	+21.00ms ⚠️

AtlasKit Editor 🚨

Timings

Description	Time	Difference
Cold	FAILED	-0.00ms
Cached	FAILED	-0.00ms

Cold Bundles

No bundles found, this is probably a failed build...

Cached Bundles

No bundles found, this is probably a failed build...

Three.js x4 🚨

Timings

Description	Time	Difference
Cold	FAILED	-0.00ms
Cached	FAILED	-0.00ms

Cold Bundles

No bundles found, this is probably a failed build...

Cached Bundles

No bundles found, this is probably a failed build...

Click here to view a detailed benchmark overview.

packages/transformers/markdown/src/MarkedTransformer.js

mischnic · 2019-12-24T22:04:04Z

Related: #3384, (#3262)

packages/examples/markdown/src/template.html

devongovett · 2020-01-29T05:08:25Z

packages/transformers/markdown/src/MarkedTransformer.js

+// @flow
+import {Transformer} from '@parcel/plugin';
+import {promisify} from '@parcel/utils';
+import marked from 'marked';


We might want to look into using something like remark for markdown since it supports all kinds of plugins for different things. For example, syntax highlighting could be done by a plugin for remark however people like without needing Parcel to build it into the transformer. Thoughts?

Not sure, wouldn't that require a javasrcript based config?

packages/examples/markdown/src/template.html

packages/packagers/markdown-html/src/MarkdownHtmlTransformer.js

height · 2020-03-14T09:09:31Z

This pull request has been linked to and will mark 1 task as "Done" when merged:

T-290 Markdown Transformer (unlink task)

💡Tip: You can link multiple Height tasks to a pull request.

bradenhs · 2020-04-30T18:09:50Z

The ability to use parcel 2 as a static site generator with code highlighting built in would be awesome! What's the status of this PR?

DeMoorJasper · 2020-04-30T18:16:30Z

@bradenhs it's ready for a merge just awaiting a review

mischnic · 2020-05-03T11:48:56Z

Using a language (i.e. tsx that is supported with GitHub) fails at buildtime. I think we should rather fallback to plaintext like in the marked docs: https://marked.js.org/#/USING_ADVANCED.md
Should the dependencies (especially highlight.js and marked) be installed on demand instead (packageManager.require)? They are quite large given that most users wouldn't process markdown anyway.
Relative paths in the template don't work (Failed to resolve 'style.css' from './markdown/src/sub/other.md')
This makes it somewhat unusable for a SSG usecase.

src/
├── index.md       `template: ./template.html`
├── style.css
├── sub
│   └── other.md   `template: ../template.html`
└── template.html  `<link rel="stylesheet" href="style.css" />`

Has @devongovett changed his opinion regarding marked vs remark?
(More of a future roadmap: The only thing that is blocking me from using Parcel instead of eleventy is an auto-generated navigation/sitemap...)

DeMoorJasper · 2020-05-03T13:25:47Z

@mischnic I haven't really found a way around relative paths except running posthtml over the template and rewriting all relative imports to be relative to markdown instead of the template.

I agree with dynamically using the markdown and highlight libraries.

Will refactor everything and hack in the relative support once Devon has reviewed this. My main question about remark vs marked is there no point in changing except for the config but that might require a js config? Which we are actively trying to prevent everywhere.

mischnic · 2020-05-03T13:40:22Z

My main question about remark vs marked is there no point in changing except for the config but that might require a js config? Which we are actively trying to prevent everywhere.

True, but we could do use a .remarkrc just like .posthtmlrc (remark and PostHTML actually have the exact same node API). It seems like remark don't have an official config file either.

no point in changing

Do you mean "remark doesn't have any benefits" or "remark doesn't have any benefits if we don't allow a config file"

DeMoorJasper · 2020-05-03T15:09:20Z

@mischnic with no point in changing that I mean it would/should output the same as marked anyway if there's no config. So the only reason I see was the config as it's main selling point is the plugins

devongovett · 2020-05-03T17:10:19Z

I think we need to come up with something that's extensible, or people will just run into issues pretty quickly. Remark has a great plugin ecosystem around it that people can use to extend it. We just need a way of letting people specify those. I think things like syntax highlighting should also be done in plugins too so it can be customizable. We just need a way to run remark and load plugins inside parcel.

I'm also thinking that maybe part of this should be done in a packager too. As @mischnic said, we'll need a way of generating a list of links between pages and such. If we ran all of the pages through a packager at the end, that could inject the markdown into a template or something to generate the full HTML page with the list of links, which could come from the bundle graph. We're doing something similar at work for our docs website, except using MDX and React.

I think we'd want a way to specify layouts at the page level along with other metadata like title, description, etc. This could be done using frontmatter perhaps, or however plugins wanted, and stored in asset.meta. Then the packager could read the metadata for each bundle in the bundle graph while its generating the list of pages.

I'm not saying all that has to happen in this PR, but I think we should move in this direction. In my experience so far it's been awesome to use Parcel as a static site generator, so I'd like for it to be as extensible as possible.

DeMoorJasper · 2021-02-28T10:46:05Z

Might recreate this PR at some point in the future, for now gonna close it

markdown transformer

161e744

DeMoorJasper changed the title ~~markdown transformer~~ [T-290] Markdown transformer Dec 22, 2019

DeMoorJasper added 2 commits December 22, 2019 17:20

add template and frontmatter

3184a39

allow custom templates

6a91f57

DeMoorJasper commented Dec 22, 2019

View reviewed changes

packages/transformers/markdown/src/MarkedTransformer.js Outdated Show resolved Hide resolved

DeMoorJasper commented Dec 22, 2019

View reviewed changes

packages/transformers/markdown/src/MarkedTransformer.js Outdated Show resolved Hide resolved

update markdown test

a823f92

DeMoorJasper added 4 commits December 25, 2019 11:10

Return raw html by default

6a47cb5

enable all tests

5e3ac0f

Merge branch 'v2' into v2-marked

c6485a3

Merge branch 'v2' into v2-marked

8b6d6c4

DeMoorJasper mentioned this pull request Jan 12, 2020

WIP: Add Markdown Plugin #3384

Closed

3 tasks

mischnic added ✨ Parcel 2 🔌 Plugins labels Jan 13, 2020

DeMoorJasper added 2 commits January 15, 2020 13:34

Merge branch 'v2' into v2-marked

b48b107

Merge branch 'v2' into v2-marked

8e9859d

mischnic reviewed Jan 23, 2020

View reviewed changes

packages/examples/markdown/src/template.html Outdated Show resolved Hide resolved

DeMoorJasper added 3 commits January 23, 2020 16:33

Merge branch 'v2' into v2-marked

9e517d5

Merge branch 'v2' into v2-marked

d1ef956

Merge branch 'v2' into v2-marked

b5ef91f

devongovett reviewed Jan 29, 2020

View reviewed changes

DeMoorJasper added 6 commits January 29, 2020 16:30

Merge branch 'v2' into v2-marked

73fb09f

Merge branch 'v2' into v2-marked

4f53ed5

Merge branch 'v2' into v2-marked

e42633e

.

a08831b

attempt at magic

b1cb591

magic?

a7dc7b1

DeMoorJasper commented Feb 2, 2020

View reviewed changes

packages/examples/markdown/src/template.html Show resolved Hide resolved

DeMoorJasper commented Feb 2, 2020

View reviewed changes

packages/packagers/markdown-html/src/MarkdownHtmlTransformer.js Outdated Show resolved Hide resolved

DeMoorJasper added 4 commits February 8, 2020 18:40

Merge branch 'v2' into v2-marked

e9879f5

remove packager

9ac6d51

Merge branch 'v2' into v2-marked

892b1d9

Merge branch 'v2' into v2-marked

61bb1fa

mischnic removed the ✨ Parcel 2 label Mar 8, 2020

DeMoorJasper added 2 commits March 11, 2020 17:59

Merge branch 'v2' into v2-marked

ecce0b9

Merge branch 'v2' into v2-marked

de4bd88

DeMoorJasper added 3 commits March 18, 2020 17:19

Merge branch 'v2' into v2-marked

a33ab7d

Merge branch 'v2' into v2-marked

9aca617

Merge branch 'v2' into v2-marked

b8e22af

Merge branch 'v2' into v2-marked

2f9a886

DeMoorJasper requested a review from mischnic April 30, 2020 18:16

Merge branch 'v2' into v2-marked

9567787

DeMoorJasper requested a review from devongovett May 3, 2020 13:22

DeMoorJasper closed this Feb 28, 2021

mischnic deleted the v2-marked branch September 26, 2023 09:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[T-290] Markdown transformer #3936

[T-290] Markdown transformer #3936

DeMoorJasper commented Dec 22, 2019 •

edited

height bot commented Dec 22, 2019

parcel-benchmark commented Dec 22, 2019 •

edited

Timings

Cold Bundles

Cached Bundles

Timings

Cold Bundles

Cached Bundles

Timings

Cold Bundles

Cached Bundles

Timings

Cold Bundles

Cached Bundles

mischnic commented Dec 24, 2019

devongovett Jan 29, 2020

DeMoorJasper Feb 1, 2020

height bot commented Mar 14, 2020 •

edited

bradenhs commented Apr 30, 2020

DeMoorJasper commented Apr 30, 2020

mischnic commented May 3, 2020 •

edited

DeMoorJasper commented May 3, 2020

mischnic commented May 3, 2020 •

edited

DeMoorJasper commented May 3, 2020 •

edited

devongovett commented May 3, 2020 •

edited

DeMoorJasper commented Feb 28, 2021

[T-290] Markdown transformer #3936

[T-290] Markdown transformer #3936

Conversation

DeMoorJasper commented Dec 22, 2019 • edited

↪️ Pull Request

✔️ PR Todo

height bot commented Dec 22, 2019

parcel-benchmark commented Dec 22, 2019 • edited

Benchmark Results

Timings

Cold Bundles

Cached Bundles

Timings

Cold Bundles

Cached Bundles

Timings

Cold Bundles

Cached Bundles

Timings

Cold Bundles

Cached Bundles

mischnic commented Dec 24, 2019

devongovett Jan 29, 2020

Choose a reason for hiding this comment

DeMoorJasper Feb 1, 2020

Choose a reason for hiding this comment

height bot commented Mar 14, 2020 • edited

bradenhs commented Apr 30, 2020

DeMoorJasper commented Apr 30, 2020

mischnic commented May 3, 2020 • edited

DeMoorJasper commented May 3, 2020

mischnic commented May 3, 2020 • edited

DeMoorJasper commented May 3, 2020 • edited

devongovett commented May 3, 2020 • edited

DeMoorJasper commented Feb 28, 2021

DeMoorJasper commented Dec 22, 2019 •

edited

parcel-benchmark commented Dec 22, 2019 •

edited

height bot commented Mar 14, 2020 •

edited

mischnic commented May 3, 2020 •

edited

mischnic commented May 3, 2020 •

edited

DeMoorJasper commented May 3, 2020 •

edited

devongovett commented May 3, 2020 •

edited