Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce Buffer operations in EventSource #2630

Open
Uzlopak opened this issue Jan 20, 2024 · 4 comments
Open

Reduce Buffer operations in EventSource #2630

Uzlopak opened this issue Jan 20, 2024 · 4 comments
Labels
enhancement New feature or request

Comments

@Uzlopak
Copy link
Contributor

Uzlopak commented Jan 20, 2024

In the EventSourceStream code of the initial EventSource implementation, alot of Buffer Operations are used to process the incoming Stream. The Buffer operations are concentrated in the _transform and parseLine method of the EventSourceStream.

I am sure, we can reduce the overhead of the buffer operations by using some simple integer variables, marking start and end of the Buffer we want to extract the subarray and then pass it accordingly parseLine method.

Maybe the Buffer.concat operation to buffer the incoming chunks can be avoided. But be careful, as the incoming stream is containing utf8 code. So you have to concat the buffers at one point to avoid utf8 issues.

In the parseLine method the field and value are also processed via .subarray. Maybe first calling toString('utf8') and then processing it makes more sense? Anyhow, some benchmarking would be useful to make a informed decision.

@Uzlopak Uzlopak added the enhancement New feature or request label Jan 20, 2024
@KhafraDev
Copy link
Member

In websocket I was inspired by lpinca/ws to add in a utility that returned a buffer that contained n bytes, rather than concatenating every buffer stored. 5165d67

@Uzlopak
Copy link
Contributor Author

Uzlopak commented Jan 21, 2024

@KhafraDev

Maybe use similar logic as in eventsource npm package?
Especially EventSource/eventsource#239

@KhafraDev
Copy link
Member

KhafraDev commented Jan 21, 2024

It looks as though they behave similarly, with the benefit of ws' approach being able to lazily set the buffers and allocate an exact buffer size. Both would have to be benchmarked before I could give a definitive answer.

@metcoder95
Copy link
Member

The solution from the PR seems to be of help for the usage of concat, but a way to iterate over the buffer to consume it while avoiding subarrays might require the approach @KhafraDev suggests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants