mdx-js
Custom processing option for mdx loader #2134
-
|
Following a previous discussion on sections in mdx I would like to propose an option in mdx loader that allows for custom logic in the processing of MDX files. Let's assume the example of MDX with sections for different content blogs: {/* c:how-it-works.main */}
Some MDX here
...
{/* c:start */}
Some more MDX here
...One might want to split process the above file and later receive an object as a default export, that would look something like this: const res = {
"some.content.path": function() {
const {Fragment: _Fragment, jsx: _jsx} = arguments[0]
const no = 3.14
function _createMdxContent(props) { /* … */ }
function MDXContent(props = {}) { /* … */ }
return {no, default: MDXContent}
},
"some.other.content.path": function() {
//...
},
//...
};
export default res;To achieve this, I propose we add an option to mdx loader that allows custom processing of files (as implemented here): // ...
if(options.execute) {
options.execute.call(this, value, process, callback)
} else {
process({value, path: this.resourcePath}).then(
(file) => {
callback(null, file.value, file.map)
},
(/** @type VFileMessage */ e) => {
const fpath = path.relative(this.context, this.resourcePath);
e.message = `${fpath}:${e.name}: ${e.message}`;
callback(e);
}
)
}
// ...This gives the user the freedom to pass the following as config to the loader in webpack: function execute(value, process, callback) {
const splittedFile = value.split(/{[\n\t ]*\/\*\s*c:[\w.\-\\[\d\]]+\s*\*\/[\n\t ]*}/)
splittedFile.shift()
const matchingComments = value.match(/(?<={[\n\t ]*\/\*\s*c:)[\w.\-\\[\d\]]+(?=\s*\*\/[\n\t ]*})/g)
if(splittedFile.length) {
Promise.all(splittedFile.map(async (v, i) => {
const file = await process({value: v, path: this.resourcePath})
return file
})).then((arr) => {
const objElements = arr.map((file, i) => {
return `"${matchingComments[i]}": function() {
${file.value}
},`
})
const result = `
const res = {
${objElements.join("")}
};
export default res;
`
callback(null, result, arr[0].map)
return result
}, callback)
} else {
process({value, path: this.resourcePath}).then((file) => {
const result = `
export default function() {
${file.value}
}
`
callback(null, result, file.map)
return result
}, callback)
}
}
// ...
// webpack config excerpt:
{
test: /\.mdx$/,
use: [
{
loader: path.resolve(__dirname, 'loaders/mdx-extended-loader/index.cjs'),
options: {
execute: execute,
outputFormat: "function-body"
}
}
],
}
// ... |
Beta Was this translation helpful? Give feedback.
Replies: 6 comments 12 replies
-
|
Heya @benjaminpreiss! I'd second @wooorm's comment
This feels like something which may be better served working off of a syntax tree. |
Beta Was this translation helpful? Give feedback.
-
|
Ah, okey - sure! If that is possible with a remark plugin, could you give me a hint how? I'd be glad to implement a working alternative... Just to be very clear about what I want to achive: I want to be able to selectively import only certain sections of the MDX by their identifier (which is specified in specially formatted comments). These sections could then hold a variety of things, like exported variables or markdown. See this excerpt from the filecoin saturn page as an example of a MDX file: In that case I would like to be able to use only the "ctas" array from the "start" section... Would that be possible with a remark plugin? |
Beta Was this translation helpful? Give feedback.
-
For getting started with ASTs I'd recommend reading through https://unifiedjs.com/learn/ The gist of an AST approach would be to:
Some reference tools:
|
Beta Was this translation helpful? Give feedback.
-
Reading this, it sounds like you interpreted my statements as saying that you should write Because most of the work is turning separate sections, whether with tags, thematic breaks, or with comments, into different components. Splitting these two steps allows both your custom syntax and other syntaxes. The next steps are what @ChristianMurphy describes above. But I think it’s easier to perform those steps on an AST that‘s split in sections. |
Beta Was this translation helpful? Give feedback.
-
|
Okey, thanks for the clarification! Sounds good to me 😄 |
Beta Was this translation helpful? Give feedback.
-
|
Sweet :) Let us know if you need help! |
Beta Was this translation helpful? Give feedback.
-
|
As mentioned in #454, this would also be super useful for building slide decks as @benjaminpreiss maybe you could also add some content from #454 in your original description above as another use case for this? Would be great if this were published as a package with configurable delimiters! (eg. in presentations, having the ability to separate pages with a simple |
Beta Was this translation helpful? Give feedback.
-
|
Okey, I am thrilled to anounce that I am finally building it! Here is the future repo for reference: https://github.com/frontline-hq/recma-sections I have a question though - working with the estrees "id": {
"type": "Identifier",
"start": 87,
"end": 91,
"name": "blue"
}What are these properties for? Only for source maps? Cause in that case I wouldn't change them... |
Beta Was this translation helpful? Give feedback.
-
|
They are for positional info. |
Beta Was this translation helpful? Give feedback.
-
|
It’s used for various tools that work with the AST. Indeed, for source maps (I think), but also for the JSX automatic dev runtime, or potential MDX linters. Typically you ignore it. It should only exist for nodes that are based on content in the original document, so generated nodes should not include this. |
Beta Was this translation helpful? Give feedback.
-
|
Hmmm, okey! But what should I do with nodes that I replace with other nodes? A typical result would then be a path in the tree where in the middle of the path the positional information goes missing |
Beta Was this translation helpful? Give feedback.
-
|
The new node is still a generated node, so it shouldn’t have positional information. Its children may have been relocated, but should keep their original positional information, so they could potentially be mapped back to the source code. |
Beta Was this translation helpful? Give feedback.
-
|
Okey, first version is pushed... Please have a look and let me know what you think! |
Beta Was this translation helpful? Give feedback.
-
|
@karlhorky Is this sufficient for your usecase? |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for this! From a quick glance at the code and readme, it seems that it splits an MDX file into multiple MDX components, so it seems indeed helpful for our use case to create a new version of We'll probably want to configure the delimiter to be cc @ProchaLu @EricCote this may provide some basis for the |
Beta Was this translation helpful? Give feedback.
-
|
There is now an official npm release, typescript support and compilation to cjs and esm: https://www.npmjs.com/package/@frontline-hq/recma-sections |
Beta Was this translation helpful? Give feedback.
-
|
Hi! Great work! It’s cool that you got this working.
|
Beta Was this translation helpful? Give feedback.
-
|
Alright, sure! Let me create an issue in our repo for this :) |
Beta Was this translation helpful? Give feedback.
Heya @benjaminpreiss!
Exciting to see progress towards multiple documents in a file.
Using comments to provide metadata for the sections is interesting, could be a valuable addition.
I'd second @wooorm's comment
This feels like something which may be better served working off of a syntax tree.
Whether that be the initial markdown AST (MDAST/remark), the intermediate HTML AST (HAST/rehype), or the final JavaScript AST (ESAST/recma).
Which mdx already supports plugins for (https://github.com/mdx-js/mdx/tree/main/packages/mdx#optionsremarkplugins) which could make generating a proof of concept ea…