How to create new ReadableStream in Node

[MarcoEstevez]

In version 5.0.0-1 I could just do this:

const readableStream = new openpgp.stream.ReadableStream({
    start(controller) {
        controller.enqueue(new Uint8Array([0x01, 0x02, 0x03]));
        controller.close();
    }
});

now in version 5.0.0-2 node throws an error ReadableStream is not a constructor,

On the documentation mentions to use standard ReadableStream:

const readableStream = new ReadableStream({
    start(controller) {
        controller.enqueue(new Uint8Array([0x01, 0x02, 0x03]));
        controller.close();
    }
});

But its not available on Node as far as I know, can you please advice here

[twiss]

Hey 👋 You could use a polyfill (e.g. web-streams-polyfill), or a built-in Node.js Readable stream (though the API for that one is different).

[twiss]

I don't mean to be rude, but why don't you try it rather than asking me? 🙃 I don't see a reason why it wouldn't work, in any case.

[MarcoEstevez]

I see :-), thanks @twiss

[MarcoEstevez]

For some reasons Readable interface does not work as expected and chunks are generated as Buffers (binary) which makes openpgp not happy

class StringStream extends stream.Readable {
    constructor(data) {
        super();
        this.data = data;
    }

    _read(size) {
        let chunk = this.data.slice(0, size);

        if (chunk) {
          this.str = this.data.slice(size);
          this.push(chunk);
        }
      
        else {
          this.push(null);
        }
    }
}

const content = fs.readFileSync('./root/kafka.txt.pgp', 'utf8');
const test = new StringStream ( content  )

const message = await openpgp.readMessage({
    armoredMessage: test
});

const { data: decrypted, signatures } = await openpgp.decrypt({
    message,
    publicKeys: publicKey, // for verification (optional)
    privateKeys: privateKey // for decryption
});

but then I realiced Node standard createdReadStream from fs worked fine

const readableStream = fs.createReadStream('./root/kafka.txt.pgp', 'utf8');

const message = await openpgp.readMessage({
    armoredMessage: readableStream // parse armored message
});

const { data: decrypted, signatures } = await openpgp.decrypt({
    message,
    publicKeys: publicKey, // for verification (optional)
    privateKeys: privateKey // for decryption
});

and I saw how chunks that createReadSteam generates were actually strings, so I tryed a new approach using generator functions and Readable.from(iterator) variant so that finally I managed to make it work

class Generator {
    constructor(data, size = 16384) {
        this.data = data;
        this.size = size;
    }

    * chunkGenerator() {
        let chunk = null
        do {
            chunk = this.data.slice(0, this.size)
            if(chunk) {
                this.data = this.data.slice(this.size)
                yield chunk
            } else {
                return null
            }
        }  while (chunk)     
    }
}

const content = fs.readFileSync('./root/kafka.txt.pgp', 'utf8');
const readableStream = new Readable.from( new Generator(content).chunkGenerator() )

const message = await openpgp.readMessage({
    armoredMessage: readableStream // parse armored message
});

const { data: decrypted, signatures } = await openpgp.decrypt({
    message,
    publicKeys: publicKey, // for verification (optional)
    privateKeys: privateKey // for decryption
});

decrypted.on('data', (cunck)=>{
    console.log(cunck)
}) 

[MarcoEstevez]

I wonder why ReadStream implementation has been dropped on the new version though, since Node lacks new ReadStream javascript api, it was quite handy :-P

[twiss]

The rationale is mentioned in #1291. If it's more convenient, you can also use the polyfill I linked, that's the same as what openpgp.stream.ReadableStream was on Node.js before anyway :)