Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: cleanup core api default body limit #336

Merged
merged 2 commits into from
Mar 16, 2023
Merged

chore: cleanup core api default body limit #336

merged 2 commits into from
Mar 16, 2023

Conversation

malexandru-rdx
Copy link
Contributor

Update axum's version. Check breaking changelog.

Remove Tower RequestBodyLimitLayer middleware and use DefaultBodyLimit::max (see TODO and tokio-rs/axum#1397).

@github-actions
Copy link

github-actions bot commented Feb 27, 2023 •
edited

Docker tags
eu.gcr.io/dev-container-repo/babylon-node:pr-336
eu.gcr.io/dev-container-repo/babylon-node:sha-b35ec6f

@malexandru-rdx malexandru-rdx force-pushed the chore/core-api-default-body-limit branch from 2ba0021 to 0d0135d Compare March 3, 2023 07:39
@malexandru-rdx malexandru-rdx marked this pull request as ready for review March 3, 2023 09:09
@jakrawcz-rdx
Copy link
Contributor

Locally, I have set const LARGE_REQUEST_MAX_BYTES: usize = 13;, and then I run the most basic test I've found (test_core_api_can_submit_and_commit_transaction()):

on develop:

transactionSubmitPost call failed with: 413 - length limit exceeded
com.radixdlt.api.core.generated.client.ApiException: transactionSubmitPost call failed with: 413 - length limit exceeded
	at app//com.radixdlt.api.core.generated.api.TransactionApi.getApiException(TransactionApi.java:80)
	at app//com.radixdlt.api.core.generated.api.TransactionApi.transactionSubmitPostWithHttpInfo(TransactionApi.java:510)
	at app//com.radixdlt.api.core.generated.api.TransactionApi.transactionSubmitPost(TransactionApi.java:488)
	at app//com.radixdlt.api.core.NetworkSubmitTransactionTest.test_core_api_can_submit_and_commit_transaction(NetworkSubmitTransactionTest.java:89)

and here:

transactionSubmitPost call failed with: 400 - {"error_type":"Basic","code":400,"message":"BytesRejection(FailedToBufferBody(LengthLimitError(LengthLimitError(Error { inner: LengthLimitError }))))"}
com.radixdlt.api.core.generated.client.ApiException: transactionSubmitPost call failed with: 400 - {"error_type":"Basic","code":400,"message":"BytesRejection(FailedToBufferBody(LengthLimitError(LengthLimitError(Error { inner: LengthLimitError }))))"}
	at app//com.radixdlt.api.core.generated.api.TransactionApi.getApiException(TransactionApi.java:80)
	at app//com.radixdlt.api.core.generated.api.TransactionApi.transactionSubmitPostWithHttpInfo(TransactionApi.java:510)
	at app//com.radixdlt.api.core.generated.api.TransactionApi.transactionSubmitPost(TransactionApi.java:488)
	at app//com.radixdlt.api.core.NetworkSubmitTransactionTest.test_core_api_can_submit_and_commit_transaction(NetworkSubmitTransactionTest.java:89)

... so the limit works, but the error is surfaced rather differently (out of which 413 would be my preferred).
I think this would justify adding an actual unit test like throws_413_on_tx_limit_exceeded(). I imagine this might require a configurable LARGE_REQUEST_MAX_BYTES, and if that is too cumbersome, then I can live with just some manual verifications (like above).

dhedey
dhedey approved these changes Mar 15, 2023
View reviewed changes
Copy link
Contributor

@dhedey dhedey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. Thanks for updating things. A few minor tweaks needed before merging 👍

core-rust/core-api-server/src/core_api/extractors.rs Outdated
async fn from_request_parts(parts: &mut Parts, state: &S) -> Result<Self, Self::Rejection> {
match axum::Json::<T>::from_request_parts(parts, state).await {
Ok(value) => Ok(Self(value.0)),
Err(rejection) => Err(client_error(format!("{:?}", rejection))),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this is now throwing the error for request size, so we will probably want to handle it properly and turn it into a 403.

Similarly on line 47? Perhaps this logic can be unified into a method?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Removed the unused from_request_parts.

core-rust/core-api-server/src/core_api/extractors.rs Outdated
@@ -12,18 +14,35 @@ use super::{client_error, ResponseError};

#[derive(Debug)]
pub(crate) struct Json<T>(pub T);
pub use axum::Extension; // Re-export Extension so that it can be used easily
pub use axum::extract::State; // Re-export Extension so that it can be used easily
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment needs updating :)

core-rust/core-api-server/src/core_api/server.rs
@@ -93,8 +92,6 @@ pub async fn create_server<F>(
{
let core_api_state = CoreApiState { state_manager };

// TODO - Change to remove the Tower RequestBodyLimitLayer middleware and use DefaultBodyLimit::max
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove the tower dependency now? Or is it still used elsewhere?
Also could we update LARGE_REQUEST_MAX_BYTES now? 👍 I imagine it can be more like 3 MB now that the max transaction size is about 1MB?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@malexandru-rdx
Update axum and use DefaultBodyLimit instead of tower's workaround
be463ca 
Signed-off-by: Alexandru Murtaza <alexandru.murtaza@rdx.works>
@malexandru-rdx
address CR: remove unused dependencies, fix 413 error, reduce request…
d6e58f1 
… max size

Signed-off-by: Alexandru Murtaza <alexandru.murtaza@rdx.works>
@malexandru-rdx malexandru-rdx force-pushed the chore/core-api-default-body-limit branch from 0d0135d to d6e58f1 Compare March 15, 2023 14:47
@sonarcloud
Copy link

sonarcloud bot commented Mar 15, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@malexandru-rdx malexandru-rdx merged commit c25a100 into develop Mar 16, 2023
@malexandru-rdx malexandru-rdx deleted the chore/core-api-default-body-limit branch March 16, 2023 12:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dhedey dhedey dhedey approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@malexandru-rdx @jakrawcz-rdx @dhedey