Skip to content
/ loner Public

A transform stream for node and browserify that ensures that certain data sequences appear in their own, separate chunks

Notifications You must be signed in to change notification settings

regular/loner

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

loner

A transform stream for node and browserify that ensures that certain data sequences appear in their own, separate chunks

When progressively parsing streaming binary data, you often need to find magic sequences of bytes that act as separators or "keywords". This can be surprisingly hard to implement efficiently (i.e. without a lot of buffering and copying data around) because the magic sequences might be spread across an arbitrary amount of chunks and you must take care to implement all those edge cases correctly.

loner does that for you. It ensures that each magic sequence you specify will be in its own data chunk, while otherwise leaving the sequence of chunks as it is. Thus all you need to do downstream is to compare each incoming chunk against the sequence you are looking for, for example by using buffer-equal.

All the heavy lifting is actually done by binary-stream-replace.

Example

const loner = require('loner');

let PNGSignature = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);

let l = loner(PNGSignature);
l.on('data', (data) => console.log(data) );

l.write(Buffer.from('Hello\x89\x50'));
l.write(Buffer.from([0x4e, 0x47, 0x0d, 0x0a]));
l.write(Buffer.from("\x1a\x0a\nWorld"));
l.end();

// output =>
// <Buffer 48 65 6c 6c 6f c2>
// <Buffer 89 50 4e 47 0d 0a 1a 0a>
// <Buffer 0a 57 6f 72 6c 64>

API

loner(seq1, seq2, ..., seqN)

seq1 .. seqN Buffer objects containing sequences that you want isolated (on separate chunks). You can also pass Strings instead of Buffers, they will be converted internally.

NOTE If a sequence is fully contained within another seqeunce, as in loner('Hello','lo'), interesting things happen! The outcome then depends on the ordering of arguments. In this example, you'd get all the los separated, which obviously breaks each hello into hel and lo.

About

A transform stream for node and browserify that ensures that certain data sequences appear in their own, separate chunks

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published