Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolved a flaky error #1476

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

njain2208
Copy link

PR Overview:


This PR fixes the flaky/non-deterministic behavior of the following test because it assumes the ordering.

org.apache.tika.cli.TikaCLITest#testJsonMetadataOutput

Test Overview:


The output returned by getParamOutContent is flaky in nature and is non-deterministic. This results in the testJsonMetadataOutput failing.

This flakiness was identified by the nondex tool created by the researchers of UIUC.

[ERROR]   TikaCLITest.testJsonMetadataOutput:258 expected: <true> but was: <false>

You can reproduce the issue by running the following commands:

mvn install -pl tika-app -am -DskipTests
mvn test -pl tika-app  -Dtest=org.apache.tika.cli.TikaCLITest#testJsonMetadataOutput
mvn -pl tika-app edu.illinois:index-maven-plugin:2.1.1:nondex -Dtest=org.apache.tika.cli.TikaCLITest#testJsonMetadataOutput

Fix:


To fix the issue I decided to sort the JSON string. To sort the JSON string I used the Google extension "com.google.code.gson" in the tika-app/pom.xml file. This made the output deterministic and led the code to successfully passing.

https://github.com/njain2208/tika/blob/8fce6e9afcc6872e0d5cdfac08ccd2c7dfd7d0b0/tika-app/src/test/java/org/apache/tika/cli/TikaCLITest.java#L251-L271

@tballison
Copy link
Contributor

Thank you for opening this. As I look at the unit test, I don't think we should test for legacy sort there because, as you point out, there is no guarantee of order from Metadata#names(). I removed that portion of the test in the main branch. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants