Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stream() is very slow with large messages #45

Open
bringert opened this issue Dec 12, 2021 · 2 comments
Open

stream() is very slow with large messages #45

bringert opened this issue Dec 12, 2021 · 2 comments

Comments

@bringert
Copy link

In current HEAD, with Python 3.6.5, I found that stream() is extremely slow. It takes many seconds on a Raspberry Pi 3 to parse a few tens of kilobytes, and it gets progressively slower.

It looks like pyre_sseclient._event_complete() searches the message from the start for every character read, which makes detecting the end of a message quadratic in the length of the message.

@bringert
Copy link
Author

https://github.com/btubbs/sseclient/blob/master/sseclient.py doesn't seem to have this quadratic behavior, so one solution would be to change to that.

@bringert
Copy link
Author

Changing it so that the search for end_of_field just searches the end of the buffer speeds it up a lot, but it's still a lot slower than https://github.com/btubbs/sseclient/blob/master/sseclient.py.

It looks like it's because pyre_sseclient reads one character at a time, whereas sseclient.py uses a default chunk size of 1024.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant