Skip to content

Commit

Permalink
In readToByteBuffer, read to max, not the internal buffer size
Browse files Browse the repository at this point in the history
Fixes #1774. Regressed by #1671.
  • Loading branch information
jhy committed May 17, 2022
1 parent b681a29 commit ccbd65f
Show file tree
Hide file tree
Showing 3 changed files with 29 additions and 6 deletions.
6 changes: 6 additions & 0 deletions CHANGES
@@ -1,5 +1,11 @@
jsoup changelog

*** Release 1.15.2 [PENDING]
* Bugfix: when using the readToByteBuffer method, such as in Connection.Response.body(), if the document has not
already been parsed and must be read fully, and there is any maximum buffer size being applied, only the default
internal buffer size is read.
<https://github.com/jhy/jsoup/issues/1774>

*** Release 1.15.1 [2022-May-15]
* Change: removed previously deprecated methods and classes (including org.jsoup.safety.Whitelist; use
org.jsoup.safety.Safelist instead).
Expand Down
Expand Up @@ -81,17 +81,14 @@ public ByteBuffer readToByteBuffer(int max) throws IOException {
final ByteArrayOutputStream outStream = new ByteArrayOutputStream(bufferSize);

int read;
int remaining = bufferSize;

while (true) {
read = read(readBuffer, 0, remaining);
read = read(readBuffer, 0, bufferSize);
if (read == -1) break;
if (localCapped) { // this local byteBuffer cap may be smaller than the overall maxSize (like when reading first bytes)
if (read >= remaining) {
outStream.write(readBuffer, 0, remaining);
if (read >= max) {
outStream.write(readBuffer, 0, max);
break;
}
remaining -= read;
}
outStream.write(readBuffer, 0, read);
}
Expand Down
20 changes: 20 additions & 0 deletions src/test/java/org/jsoup/integration/ConnectTest.java
Expand Up @@ -671,4 +671,24 @@ public void maxBodySize() throws IOException {
assertEquals("Large HTML", doc1.title());
assertEquals("Large HTML", doc2.title());
}

@Test
public void maxBodySizeInReadToByteBuffer() throws IOException {
// https://github.com/jhy/jsoup/issues/1774
// when calling readToByteBuffer, contents were not buffered up
String url = FileServlet.urlTo("/htmltests/large.html"); // 280 K

Connection.Response defaultRes = Jsoup.connect(url).execute();
Connection.Response smallRes = Jsoup.connect(url).maxBodySize(50 * 1024).execute(); // crops
Connection.Response mediumRes = Jsoup.connect(url).maxBodySize(200 * 1024).execute(); // crops
Connection.Response largeRes = Jsoup.connect(url).maxBodySize(300 * 1024).execute(); // does not crop
Connection.Response unlimitedRes = Jsoup.connect(url).maxBodySize(0).execute();

int actualDocText = 280735;
assertEquals(actualDocText, defaultRes.body().length());
assertEquals(50 * 1024, smallRes.body().length());
assertEquals(200 * 1024, mediumRes.body().length());
assertEquals(actualDocText, largeRes.body().length());
assertEquals(actualDocText, unlimitedRes.body().length());
}
}

0 comments on commit ccbd65f

Please sign in to comment.