Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create chunks for virtual modules when preserving modules #2511

Merged
merged 9 commits into from Oct 30, 2018

Conversation

lukastaegert
Copy link
Member

This PR contains:

  • bugfix
  • feature
  • refactor
  • documentation
  • other

Are tests included?

  • yes (bugfixes and features will not be merged without tests)
  • no

Breaking Changes?

  • yes (breaking changes will not be merged unless absolutely necessary)
  • no

List any relevant issue numbers:
Resolves #2503

Description

When preserving modules, we also need to create chunks for virtual modules injected by plugins (e.g. babel helpers etc.). This PR solves this by doing the following:

  • When creating ids, \0 characters are replaced with _
  • When preserving modules,
    • non-relative ids are ignored when determining the common base directory
    • non-relative ids are prefixed with ROLLUP_ to reduce the danger of name clashes and turned into top-level modules (there is no check if such an id is already taken, though, therefore the prefix)

@lukastaegert lukastaegert force-pushed the virtual-preserved-modules branch 2 times, most recently from ca90a85 to fcf68aa Compare October 15, 2018 05:38
@lukastaegert lukastaegert changed the base branch from master to external-execution-order October 15, 2018 05:38
@lukastaegert lukastaegert force-pushed the virtual-preserved-modules branch 2 times, most recently from 8b20bcd to 9d26dd2 Compare October 15, 2018 06:18
@lukastaegert
Copy link
Member Author

Damn it, seems like Windows really does not like my tests. Will keep on investigating tomorrow.

description: 'Generates actual files for virtual modules when preserving modules',
options: {
input: ['main.js'],
experimentalCodeSplitting: true,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

arent those like conflicting settings? preserving modules & code splitting?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not in rollup < 1.0. experimentalCodeSplitting internally uses the "new" code-splitting pipeline which is a prerequisite for preserving modules, which is like the ultimate form of code-splitting. In rollup 1.0 of course, this is the only pipeline that exists.

.generate({ format: 'esm' })
.then(generated =>
assert.deepEqual(Object.keys(generated.output), [
'ROLLUP__virtualModule',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldnt it be better to generate hashed filename?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to disagree here. Two reasons:

  • It is unclear how the file name should be generated.
  • The point of preserving modules is to preserve as much as possible. Nothing else is hashed and thus there is no benefit in hashing just this file. On the contrary, this will cause friction as it is not a priori clear which file names will be generated.

Copy link
Contributor

@guybedford guybedford left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

src/Chunk.ts Outdated
this.id = normalize(
isAbsolute(this.entryModule.id)
? relative(preserveModulesRelativeDir, sanitizedId)
: 'ROLLUP_' + basename(sanitizedId)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps make these a folder - _virtual/..., or something like that, in case there are a lot of them?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea!

const inputBase = commondir(
chunks.filter(chunk => chunk.entryModule).map(chunk => chunk.entryModule.id)
chunks
.filter(chunk => chunk.entryModule && isAbsolute(chunk.entryModule.id))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't the ID generation for preserve modules now guarantee this is always true?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not if the id is generated by a plugin. And this extends to other workflows as well.

@@ -2,6 +2,7 @@ import * as path from './path';

// ported from https://github.com/substack/node-commondir
export default function commondir(files: string[]) {
if (files.length === 0) return '/';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sehr gut.

@lukastaegert lukastaegert force-pushed the virtual-preserved-modules branch 3 times, most recently from ba6cf4e to 4abb2e0 Compare October 26, 2018 05:06
@lukastaegert lukastaegert changed the base branch from external-execution-order to master October 28, 2018 09:18
@lukastaegert
Copy link
Member Author

I went for the _virtual/ folder approach now and also added a deconflicting logic. I.e. if one were to re-process the output of a previous run, it would be properly deconflicted.

@lukastaegert lukastaegert merged commit 7746e0f into master Oct 30, 2018
@lukastaegert lukastaegert deleted the virtual-preserved-modules branch October 30, 2018 11:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[bug - experimentalPreserveModules] Create chunk for virtual modules
3 participants