Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for text_blobs #228

Merged
merged 5 commits into from Mar 30, 2022
Merged

Add support for text_blobs #228

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
fab6135
???
Mar 30, 2022
fb81001
code review + add tests
Mar 30, 2022
a88f545
nits from code review
Mar 30, 2022
cb7ede0
fix priority in test
Mar 30, 2022
a3043e1
F
Mar 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
30 changes: 26 additions & 4 deletions packages/core/src/plugins/bindings.ts
View file
Open in desktop
Expand Up @@ -45,6 +45,7 @@ export interface BindingsOptions {
bindings?: Record<string, any>;
globals?: Record<string, any>;
wasmBindings?: Record<string, string>;
textBlobBindings?: Record<string, string>;
serviceBindings?: ServiceBindingsOptions;
}

Expand Down Expand Up @@ -164,6 +165,16 @@ export class BindingsPlugin
})
wasmBindings?: Record<string, string>;

@Option({
type: OptionType.OBJECT,
typeFormat: "NAME=PATH",
name: "text-blob",
description: "Text blob to bind",
logName: "Text Blob Bindings",
fromWrangler: ({ text_blobs }) => text_blobs,
})
textBlobBindings?: Record<string, string>;

@Option({
type: OptionType.OBJECT,
typeFormat: "NAME=MOUNT[@ENV]",
Expand Down Expand Up @@ -246,8 +257,9 @@ export class BindingsPlugin
// 1) Wrangler [vars]
// 2) .env Variables
// 3) WASM Module Bindings
// 4) Service Bindings
// 5) Custom Bindings
// 4) Text blob Bindings
// 5) Service Bindings
// 6) Custom Bindings

const bindings: Context = {};
const watch: string[] = [];
Expand Down Expand Up @@ -281,12 +293,22 @@ export class BindingsPlugin
}
}

// 4) Load service bindings
// 4) Load text blobs from files
if (this.textBlobBindings) {
// eslint-disable-next-line prefer-const
for (let [name, textPath] of Object.entries(this.textBlobBindings)) {
textPath = path.resolve(this.ctx.rootPath, textPath);
bindings[name] = await fs.readFile(textPath, "utf-8");
watch.push(textPath);
}
}

// 5) Load service bindings
for (const { name, service } of this.#processedServiceBindings) {
bindings[name] = new Fetcher(service, this.#getServiceFetch);
}

// 5) Copy user's arbitrary bindings
// 6) Copy user's arbitrary bindings
Object.assign(bindings, this.bindings);

return { globals: this.globals, bindings, watch };
Expand Down
9 changes: 9 additions & 0 deletions packages/core/test/fixtures/lorem-ipsum.txt
View file
Open in desktop
@@ -0,0 +1,9 @@
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Amet consectetur adipiscing elit duis tristique sollicitudin nibh sit amet. Parturient montes nascetur ridiculus mus mauris vitae ultricies leo integer. Nisi porta lorem mollis aliquam ut porttitor. Placerat duis ultricies lacus sed. Amet nisl suscipit adipiscing bibendum est ultricies integer quis auctor. Hendrerit dolor magna eget est. Sed nisi lacus sed viverra. Id cursus metus aliquam eleifend mi in nulla posuere. Donec ultrices tincidunt arcu non sodales neque sodales ut etiam. Lectus proin nibh nisl condimentum id. Vitae tempus quam pellentesque nec nam aliquam sem.

Ipsum dolor sit amet consectetur adipiscing elit pellentesque habitant. Pellentesque massa placerat duis ultricies. Ultrices gravida dictum fusce ut placerat orci nulla. Quam pellentesque nec nam aliquam sem et tortor consequat id. Laoreet sit amet cursus sit amet dictum sit amet justo. Cursus metus aliquam eleifend mi in nulla. Mattis molestie a iaculis at erat pellentesque adipiscing commodo. Nulla aliquet porttitor lacus luctus. Tristique senectus et netus et. Donec adipiscing tristique risus nec feugiat in fermentum posuere urna.

Hac habitasse platea dictumst quisque sagittis purus sit. Adipiscing commodo elit at imperdiet dui accumsan sit. Etiam erat velit scelerisque in dictum non consectetur a. Cras pulvinar mattis nunc sed blandit libero volutpat sed cras. Est velit egestas dui id ornare. Orci dapibus ultrices in iaculis. Quis vel eros donec ac odio tempor orci dapibus ultrices. Lectus arcu bibendum at varius vel. Eget egestas purus viverra accumsan in nisl nisi scelerisque eu. Elementum eu facilisis sed odio morbi quis. Etiam tempor orci eu lobortis elementum nibh tellus molestie. Enim nulla aliquet porttitor lacus luctus accumsan tortor posuere ac. Nunc lobortis mattis aliquam faucibus purus in massa. Quam id leo in vitae turpis massa. Mus mauris vitae ultricies leo integer malesuada nunc vel.

Et malesuada fames ac turpis egestas sed tempus urna et. Donec massa sapien faucibus et molestie ac feugiat. Amet cursus sit amet dictum sit amet justo donec enim. Ut etiam sit amet nisl purus. Risus in hendrerit gravida rutrum quisque non tellus. Nibh mauris cursus mattis molestie a. Elementum nisi quis eleifend quam adipiscing vitae proin. Mi quis hendrerit dolor magna eget est lorem ipsum. Ut morbi tincidunt augue interdum velit euismod in pellentesque massa. Proin sagittis nisl rhoncus mattis rhoncus urna neque. Ullamcorper sit amet risus nullam eget felis. Sit amet tellus cras adipiscing enim eu. Quis eleifend quam adipiscing vitae proin sagittis nisl rhoncus. Convallis posuere morbi leo urna molestie at elementum eu facilisis. Sed adipiscing diam donec adipiscing tristique risus nec. Rhoncus aenean vel elit scelerisque mauris pellentesque. Leo in vitae turpis massa sed elementum tempus egestas sed. In hac habitasse platea dictumst quisque sagittis purus sit amet. Eros donec ac odio tempor orci. Eleifend mi in nulla posuere sollicitudin aliquam ultrices.

Amet facilisis magna etiam tempor orci eu lobortis. In ornare quam viverra orci. Convallis convallis tellus id interdum velit laoreet id. Rutrum tellus pellentesque eu tincidunt tortor aliquam. Tincidunt dui ut ornare lectus sit amet est placerat in. Nibh sed pulvinar proin gravida hendrerit lectus. Ornare aenean euismod elementum nisi quis eleifend quam adipiscing vitae. Amet luctus venenatis lectus magna fringilla urna. Nec ultrices dui sapien eget mi proin sed. Magna ac placerat vestibulum lectus. Risus nec feugiat in fermentum posuere urna. Cursus turpis massa tincidunt dui. Turpis egestas integer eget aliquet nibh praesent. Tincidunt vitae semper quis lectus. Massa id neque aliquam vestibulum morbi blandit cursus risus. Egestas fringilla phasellus faucibus scelerisque. In massa tempor nec feugiat nisl pretium fusce. A lacus vestibulum sed arcu non odio. Adipiscing elit duis tristique sollicitudin nibh sit amet commodo nulla. Quam id leo in vitae turpis massa sed elementum.
45 changes: 42 additions & 3 deletions packages/core/test/plugins/bindings.spec.ts
View file
Open in desktop
@@ -1,4 +1,5 @@
import assert from "assert";
import { readFileSync } from "fs";
import fs from "fs/promises";
import path from "path";
import { setImmediate } from "timers/promises";
Expand Down Expand Up @@ -37,6 +38,9 @@ const fixturesPath = path.join(__dirname, "..", "..", "..", "test", "fixtures");
// its 2 integer parameters together and returns the result, it is from:
// https://webassembly.github.io/wabt/demo/wat2wasm/
const addModulePath = path.join(fixturesPath, "add.wasm");
// lorem-ipsum.txt is five paragraphs of lorem ipsum nonsense text
const loremIpsumPath = path.join(fixturesPath, "lorem-ipsum.txt");
const loremIpsum = readFileSync(loremIpsumPath, "utf-8");

test("BindingsPlugin: parses options from argv", (t) => {
let options = parsePluginArgv(BindingsPlugin, [
Expand All @@ -54,6 +58,10 @@ test("BindingsPlugin: parses options from argv", (t) => {
"MODULE1=module1.wasm",
"--wasm",
"MODULE2=module2.wasm",
"--text-blob",
"TEXT1=text-blob-1.txt",
"--text-blob",
"TEXT2=text-blob-2.txt",
"--service",
"SERVICE1=service1",
"--service",
Expand All @@ -64,6 +72,7 @@ test("BindingsPlugin: parses options from argv", (t) => {
bindings: { KEY1: "value1", KEY2: "value2" },
globals: { KEY3: "value3", KEY4: "value4" },
wasmBindings: { MODULE1: "module1.wasm", MODULE2: "module2.wasm" },
textBlobBindings: { TEXT1: "text-blob-1.txt", TEXT2: "text-blob-2.txt" },
serviceBindings: {
SERVICE1: "service1",
SERVICE2: { service: "service2", environment: "development" },
Expand Down Expand Up @@ -96,6 +105,10 @@ test("BindingsPlugin: parses options from wrangler config", async (t) => {
MODULE1: "module1.wasm",
MODULE2: "module2.wasm",
},
text_blobs: {
TEXT1: "text-blob-1.txt",
TEXT2: "text-blob-2.txt",
},
experimental_services: [
{ name: "SERVICE1", service: "service1", environment: "development" },
{ name: "SERVICE2", service: "service2", environment: "production" },
Expand All @@ -109,6 +122,7 @@ test("BindingsPlugin: parses options from wrangler config", async (t) => {
envPath: ".env.test",
globals: { KEY5: "value5", KEY6: false, KEY7: 10 },
wasmBindings: { MODULE1: "module1.wasm", MODULE2: "module2.wasm" },
textBlobBindings: { TEXT1: "text-blob-1.txt", TEXT2: "text-blob-2.txt" },
serviceBindings: {
SERVICE1: { service: "service1", environment: "development" },
SERVICE2: { service: "service2", environment: "production" },
Expand Down Expand Up @@ -141,6 +155,7 @@ test("BindingsPlugin: logs options", (t) => {
bindings: { KEY3: "value3", KEY4: "value4" },
globals: { KEY5: "value5", KEY6: "value6" },
wasmBindings: { MODULE1: "module1.wasm", MODULE2: "module2.wasm" },
textBlobBindings: { TEXT1: "text-blob-1.txt", TEXT2: "text-blob-2.txt" },
serviceBindings: {
SERVICE1: "service1",
SERVICE2: { service: "service2", environment: "development" },
Expand All @@ -152,6 +167,7 @@ test("BindingsPlugin: logs options", (t) => {
"Custom Bindings: KEY3, KEY4",
"Custom Globals: KEY5, KEY6",
"WASM Bindings: MODULE1, MODULE2",
"Text Blob Bindings: TEXT1, TEXT2",
"Service Bindings: SERVICE1, SERVICE2",
]);
logs = logPluginOptions(BindingsPlugin, { envPath: true });
Expand Down Expand Up @@ -259,13 +275,33 @@ test("BindingsPlugin: setup: loads WebAssembly bindings", async (t) => {
result = await plugin.setup();
t.not(result.bindings?.ADD, undefined);
});

test("BindingsPlugin: setup: loads text blob bindings", async (t) => {
let plugin = new BindingsPlugin(ctx, {
textBlobBindings: { LOREM_IPSUM: loremIpsumPath },
});
let result = await plugin.setup();
t.not(result.bindings?.LOREM_IPSUM, undefined);
caass marked this conversation as resolved.
Show resolved Hide resolved
t.is(result.bindings?.LOREM_IPSUM, loremIpsum);

// Check resolves text blob bindings path relative to rootPath
plugin = new BindingsPlugin(
{ log, compat, rootPath: path.dirname(loremIpsumPath) },
{ textBlobBindings: { LOREM_IPSUM: loremIpsumPath } }
caass marked this conversation as resolved.
Show resolved Hide resolved
);
result = await plugin.setup();
t.not(result.bindings?.LOREM_IPSUM, undefined);
caass marked this conversation as resolved.
Show resolved Hide resolved
t.is(result.bindings?.LOREM_IPSUM, loremIpsum);
});

test("BindingsPlugin: setup: loads bindings from all sources", async (t) => {
// Bindings should be loaded in this order, from lowest to highest priority:
// 1) Wrangler [vars]
// 2) .env Variables
// 3) WASM Module Bindings
// 4) Service Bindings
// 5) Custom Bindings
// 4) Text Blob Bindings
// 5) Service Bindings
// 6) Custom Bindings

// wranglerOptions should contain [kWranglerBindings]
const wranglerOptions = parsePluginWranglerConfig(BindingsPlugin, {
Expand All @@ -287,6 +323,9 @@ test("BindingsPlugin: setup: loads bindings from all sources", async (t) => {
B: addModulePath,
C: addModulePath,
},
textBlobBindings: {
D: loremIpsumPath,
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea with this test is that the lowest priority binding defines the most bindings, that are then overriden by subsequent higher priority bindings. For textBlobBindings, you'd want to define A, B, C as loremIpsumPath, for wasmBindings you'd define A, B, C, D, for envPath you'd define A=env\nB=env\nC=env\nD=env\nE=env and then for wrangler bindings you'd define A, B, C, D, E, F.

This is probably a little excessive, but it's nice to test every combination 馃檪

serviceBindings: { A: throws, B: throws },
bindings: { A: obj },
envPath,
Expand All @@ -295,7 +334,7 @@ test("BindingsPlugin: setup: loads bindings from all sources", async (t) => {
assert(result.bindings);

t.is(result.bindings.E, "w");
t.is(result.bindings.D, "env");
t.is(result.bindings.D, loremIpsum);
t.true(result.bindings.C instanceof WebAssembly.Module);
t.true(result.bindings.B instanceof Fetcher);
t.is(result.bindings.A, obj);
Expand Down
1 change: 1 addition & 0 deletions packages/shared/src/wrangler.ts
View file
Open in desktop
Expand Up @@ -35,6 +35,7 @@ export interface WranglerEnvironmentConfig {
}; // inherited
usage_model?: "bundled" | "unbound"; // inherited
wasm_modules?: Record<string, string>; // (probably) inherited
text_blobs?: Record<string, string>;
experimental_services?: {
name: string;
service: string;
Expand Down