Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to Remark for markdown parsing #75

Closed
wants to merge 13 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
18 changes: 12 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -734,7 +734,10 @@ See below all configuration with default values:
+ "root": "/",
+ "doctype": "<!DOCTYPE html>",
+ "redirectReact": true,
+ "markdownRenderer": "",
+ "markdown": {
+ "headingIds": true,
+ "plugins": []
+ },
+ "watchTimeout": 400,
+ "browserSync": {},
+ "globalProp": {},
Expand Down Expand Up @@ -777,7 +780,10 @@ A breakdown:
"redirectReact": true, // You can disable redirecting `import` calls to the locally
// installed react instance of cuttlebelle rather than your
// local folder.
"markdownRenderer": "", // A path to a file that `module.exports` an Marked.Renderer()
"markdown": { // Markdown settings
"headingIds": true // Add id attributes to headings
"plugins": [] // A list of markdown plugin paths or NPM plugin names of
}, // Remark plugins you have installed locally.
// object. Learn more about it here:
// https://github.com/chjj/marked#renderer
// The only addition is the `preparse` key that will be run
Expand Down Expand Up @@ -981,14 +987,14 @@ _(_ 💡 _Please look at the coding style and work with it, not against it :smil

Cuttlebelle is being automatically tested on the below systems and node versions.

https://travis-ci.org/cuttlebelle/cuttlebelle.svg?branch=master
![Travis CI - Build Status](https://travis-ci.org/cuttlebelle/cuttlebelle.svg?branch=master)

| OS | Node version | Node version | Node version |
|-----------|--------------|--------------|--------------|
| `Linux` | `~8` | `~10` | `~12` |
| `OSX` | `~8` | `~10` | `~12` |
| `Linux` | `~10` | `~12` | `latest` |
| `OSX` | `~10` | `~12` | `latest` |

I got an [end-to-end test script](https://github.com/cuttlebelle/cuttlebelle/blob/master/tests/tester.js) that compares fixtures to what cuttlebelle
I've got an [end-to-end test script](https://github.com/cuttlebelle/cuttlebelle/blob/master/tests/tester.js) that compares fixtures to what cuttlebelle
generates. In each of those folders I test for [specific things](https://github.com/cuttlebelle/cuttlebelle/blob/master/tests/tester.js#L30) and make sure
the checksum of the generated files match the fixtures. In addition to that I created as many
[unit tests](https://github.com/cuttlebelle/cuttlebelle/tree/master/tests/__unit__) as I can via [Jest](https://facebook.github.io/jest/).
Expand Down
40 changes: 18 additions & 22 deletions man/build.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,36 +17,32 @@ const PGK = require('../package.json');
const Chalk = require('chalk');
const Path = require('upath');
const Fs = require('fs');
const Remark = require('remark');
const Man = require('remark-man');


//--------------------------------------------------------------------------------------------------------------------------------------------------------------
// Generating man page
//--------------------------------------------------------------------------------------------------------------------------------------------------------------
const args = [
`--version`,
`"${ PGK.version }"`,
`--manual`,
`"Cuttlebelle Help"`,
`--section`,
`1`,
`man.md`,
];
const manOptions = {
name: 'CUTTLEBELLE',
version: PGK.version,
manual: 'Cuttlebelle Help',
section: 1
};

const cmd = spawnSync( 'marked-man', args, { cwd: __dirname, encoding : 'utf8' } );
const md = new Remark().use( Man, manOptions );

if( cmd.stderr ) {
console.log(`An error occured when executing: ${ Chalk.yellow(`marked-man ${ args.join(' ') }`) } inside: ${ Chalk.yellow( __dirname ) }`);
console.error( Chalk.red( cmd.stderr.toString() ) );
md.process( Fs.readFileSync( Path.normalize(`${ __dirname }/man.md`) ), ( error, result ) => {
if( error ) {
console.log(`An error occured when building from: ${ Chalk.yellow('man.md') } inside: ${ Chalk.yellow( __dirname ) }`);
console.error( Chalk.red( error.toString() ) );

process.exit( 1 );
}
process.exit( 1 );
return;
}


//--------------------------------------------------------------------------------------------------------------------------------------------------------------
// Writing man page
//--------------------------------------------------------------------------------------------------------------------------------------------------------------
if( cmd.stdout ) {
const manpage = cmd.stdout.toString();
const manpage = String( result );
const manfile = Path.normalize(`${ __dirname }/cuttlebelle.1`);

Fs.writeFileSync( manfile, manpage, `utf-8`, ( error ) => {
Expand All @@ -60,4 +56,4 @@ if( cmd.stdout ) {
process.exit( 0 );
}
});
}
});
12 changes: 6 additions & 6 deletions man/man.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,4 @@
CUTTLEBELLE
===========

> Cuttlebelle - The react static site generator that separates editing and code concerns
> The react static site generator that separates editing and code concerns

To learn more about how to make a new website with Cuttlebelle, visit [https://cuttlebelle.com](https://cuttlebelle.com)

Expand All @@ -15,7 +12,7 @@ To learn more about how to make a new website with Cuttlebelle, visit [https://c

Cuttlebelle is a react static site generator that separates editing and code concerns.

## Website
## WEBSITE

[https://cuttlebelle.com](https://cuttlebelle.com)

Expand Down Expand Up @@ -84,7 +81,10 @@ Defaults:
"root": "/",
"doctype": "<!DOCTYPE html>",
"redirectReact": true,
"markdownRenderer": "",
"markdown": {
"headingIds": true,
"plugins": []
},
"watchTimeout": 400
},
"docs": {
Expand Down
25 changes: 15 additions & 10 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "cuttlebelle",
"description": "The react static site generator that separates editing and code concerns",
"version": "1.0.0-alpha.69",
"version": "1.0.0-alpha.70",
"homepage": "http://cuttlebelle.com",
"author": {
"name": "Dominik Wilkowski",
Expand Down Expand Up @@ -58,32 +58,37 @@
"babel-plugin-import-redirect": "^1.1.1",
"babel-plugin-transform-es2015-modules-commonjs": "^6.26.2",
"browser-sync": "^2.26.7",
"deepmerge": "^4.2.2",
"del": "^5.1.0",
"fs-extra": "^8.1.0",
"fs-extra": "^9.0.0",
"js-yaml": "^3.13.1",
"marked": "^0.8.0",
"node-notifier": "^6.0.0",
"node-notifier": "^7.0.0",
"prettify-html": "^0.0.2",
"prop-types": "^15.7.2",
"react": "^16.13.0",
"react-docgen": "^5.3.0",
"react-dom": "^16.13.0",
"remark": "^12.0.0",
"remark-html": "^11.0.2",
"require-from-string": "^2.0.2",
"slugify": "^1.4.0",
"traverse": "^0.6.6",
"unist-util-is": "^4.0.2",
"unist-util-visit": "^2.0.2",
"window-size": "^1.1.1"
},
"devDependencies": {
"@babel/cli": "^7.8.4",
"chalk": "^3.0.0",
"chalk": "^4.0.0",
"copy-dir": "^1.2.0",
"diff": "^4.0.2",
"dirsum": "^0.1.1",
"flow-bin": "^0.120.1",
"jest-cli": "^25.1.0",
"marked-man": "^0.7.0",
"onchange": "^6.1.0",
"replace-in-file": "^5.0.2"
"flow-bin": "^0.125.1",
"jest-cli": "^26.0.1",
"onchange": "^7.0.2",
"remark-dropcap": "^0.1.7",
"remark-man": "^7.0.0",
"replace-in-file": "^6.0.0"
},
"jest": {
"testEnvironment": "node",
Expand Down
4 changes: 2 additions & 2 deletions src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@ if( process.argv.includes('-w') || process.argv.includes('watch') ) {
// merging default settings with package.json
const pkgLocation = Path.normalize(`${ process.cwd() }/package.json`);
if( Fs.existsSync( pkgLocation ) ) {
const loacalPkg = require( pkgLocation );
SETTINGS.set( loacalPkg.cuttlebelle );
const localPkg = require( pkgLocation );
SETTINGS.set( localPkg.cuttlebelle );
}


Expand Down
129 changes: 102 additions & 27 deletions src/parse.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,19 @@
//--------------------------------------------------------------------------------------------------------------------------------------------------------------
// Dependencies
//--------------------------------------------------------------------------------------------------------------------------------------------------------------
import Marked from 'marked';
import Fs from 'fs';
import Remark from 'remark';
import HTML from 'remark-html';
import visit from 'unist-util-visit';
import is from 'unist-util-is';
import YAML from 'js-yaml';
import React from 'react';


//--------------------------------------------------------------------------------------------------------------------------------------------------------------
// Local
//--------------------------------------------------------------------------------------------------------------------------------------------------------------
import { Log, Style } from './helper';
import { Log, Style, Slug } from './helper';
import { SETTINGS } from './settings';
import { Path } from './path';

Expand Down Expand Up @@ -80,41 +84,112 @@ export const ParseContent = ( content, file = 'partial.md', props = {} ) => {
*/
export const ParseMD = ( markdown, file, props ) => {
if( typeof markdown === 'string' ) {
const mdSettings = SETTINGS.get().site.markdown;

const md = new Remark()
.use( HTML );

// Add IDs to headings
if( mdSettings.headingIds === true ) {
md.use(
() => ( tree ) => {
visit( tree, 'heading', node => {
if( !node.children || node.children.length < 1 ) {
return;
}

let text = [];
node.children.forEach( child => {
if( is( child, 'text' ) ) {
text.push( child.value );
}
} );

let data = node.data || ( node.data = {} );
let hProperties = data.hProperties || ( data.hProperties = {} );
let id = hProperties.id;

if( id ) {
data.id = id;
return;
}

if( !text ) {
return;
}

id = Slug( text.join(' ') );
data.id = id;
hProperties.id = id;
} );
}
);
}

let renderer = new Marked.Renderer();

if( SETTINGS.get().site.markdownRenderer ) {
const filePath = Path.normalize(`${ process.cwd() }/${ SETTINGS.get().site.markdownRenderer }`);
let plugins = mdSettings.plugins;
if( plugins ) {
if( typeof plugins === 'string' ) {
plugins = [ plugins ];
}

try {
const customRenderer = require( filePath );
renderer = customRenderer({ Marked: new Marked.Renderer(), ...props });
if( Array.isArray( plugins ) && plugins.length > 0 ) {
plugins.forEach( plugin => {
// First check the path
let pluginPath = Path.normalize( `${ process.cwd() }/${ plugin }` );

if( !Fs.existsSync( pluginPath ) ) {
// Null the path as it was not found
pluginPath = null;
}

// Only continue if there is a plugin file found, fail silently
if( pluginPath ) {
try {
// Require the plugin so that we can inspect it
const pluginRequire = require(pluginPath);
if( pluginRequire ) {
if( typeof pluginRequire === 'function' ) {
// Looks like a custom plugin, provide props to it
// If it's a simple NPM plugin, passing props shouldn't hurt
md.use(pluginRequire, props);
}
else if( Array.isArray(pluginRequire) && pluginRequire[0] && typeof pluginRequire[0] === 'function' ) {
// It's an array, and so is an NPM plugin with options provided
// Add the plugin using the handy `use(array)`` feature of Remark
md.use([pluginRequire]);
}
}
}
catch( error ) {
// Fail loudly if the plugin won't load properly
Log.error(`Using the plugin for markdown caused an error at ${ Style.yellow( pluginPath ) }`);
Log.error( error );

if( process.env.NODE_ENV === 'production' ) { // let’s die in a fiery death if something goes wrong in production
process.exit( 1 );
}
}
}
} );
}
catch( error ) {
Log.error(`Using the custom renderer for markdown caused an error at ${ Style.yellow( filePath ) }`);
Log.error( error );
}

let processed;
md.process( markdown, ( error, result ) => {
if( error ) {
Log.error(`Rendering markdown caused an error in ${ Style.yellow( file ) }`);
Log.error( String( error ) );

if( process.env.NODE_ENV === 'production' ) { // let’s die in a fiery death if something goes wrong in production
process.exit( 1 );
}
return;
}
}

try {
if( typeof renderer.preparse === 'function' ) {
markdown = renderer.preparse( markdown );
}
processed = String( result );
} );

return Marked( markdown, { renderer: renderer } );
}
catch( error ) {
Log.error(`Rendering markdown caused an error in ${ Style.yellow( file ) }`);
Log.error( error );

if( process.env.NODE_ENV === 'production' ) { // let’s die in a fiery death if something goes wrong in production
process.exit( 1 );
}
}
return processed;
}
else {
return markdown;
Expand Down
12 changes: 9 additions & 3 deletions src/settings.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
// Dependencies
//--------------------------------------------------------------------------------------------------------------------------------------------------------------
import Fs from 'fs';
import deepmerge from 'deepmerge';


//--------------------------------------------------------------------------------------------------------------------------------------------------------------
Expand Down Expand Up @@ -48,7 +49,10 @@ export type Settings = {
root: string,
doctype: string,
redirectReact: boolean,
markdownRenderer: string,
markdown: {
headingIds: boolean,
plugins: array
},
watchTimeout: number,
},
docs: {
Expand Down Expand Up @@ -92,7 +96,9 @@ export const SETTINGS = {
root: Path.normalize(`/`),
doctype: '<!DOCTYPE html>', // https://github.com/facebook/react/issues/1035
redirectReact: true,
markdownRenderer: '',
markdown: {
headingIds: true
},
watchTimeout: 400,
browserSync: {},
globalProp: {},
Expand Down Expand Up @@ -204,7 +210,7 @@ export const SETTINGS = {

newSettings.folder = Object.assign( SETTINGS.defaults.folder, localSettings.folder );
newSettings.layouts = Object.assign( SETTINGS.defaults.layouts, localSettings.layouts );
newSettings.site = Object.assign( SETTINGS.defaults.site, localSettings.site );
newSettings.site = deepmerge( SETTINGS.defaults.site, localSettings.site );
newSettings.docs = Object.assign( SETTINGS.defaults.docs, localSettings.docs );

SETTINGS.defaults = newSettings;
Expand Down