Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky tests #353

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

anirudh711
Copy link

PR Overview

The PR proposes a fix for the following tests -
com.jsoniter.extra.TestPreciseFloat#test_indirect_encode
com.jsoniter.output.TestGson#test_serializeNulls
com.jsoniter.output.TestObject#test_omit_default
com.jsoniter.output.TestObject#test_omit_null
com.jsoniter.output.TestNested#test_map_of_objects
com.jsoniter.output.TestNested#test_object_of_array
com.jsoniter.output.TestGenerics#test_wildcard
com.jsoniter.output.TestObject#test_indention

Build Project

  • To build the project :
mvn clean install
  • To run the test :
mvn -pl . test
  • To run the nondex tool on this test :
mvn -pl .  edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=<path.to.test#testName>

Problem

This flakiness was identified by the nondex tool created by the researchers of UIUC. The above eight tests fail when run on the nondex tool.
The eight tests, although varied in notion of what is being tested are flaky for the same reason. A json with more than one field is being serialised and being checked with assertEquals. When the json is being serialised, the order is not maintained and assertEquals fails at times when it is different from the true order. This does not occur with json with one element since order is constant.

public void test_wildcard() throws IOException {
TestObject7 obj = new TestObject7();
ArrayList<Integer> list = new ArrayList<Integer>();
list.add(1);
obj.field = list;
HashMap<String, Object> map = new HashMap<String, Object>();
map.put("hello", 1);
obj.field2 = map;
assertEquals("{\"field\":[1],\"field2\":{\"hello\":1}}", JsonStream.serialize(obj));
}
}

This means for the test above, the json that is being serilaised is -

{ field:[1] , field2:{"hello":1} }

The serialized output can be either

"{\"field\":[1],\"field2\":{\"hello\":1}}"

"{\"field2\":{\"hello\":1},\"field\":[1]}"

This change in ordering presents itself in different instances in the above eight tests.

Fix:

The proposed fix to use JSONAssert.assertEquals from org.skyscreamer.jsonassert.JSONAssert package which checks the structure of the JSON in the equality and not focus on the order of the field. This provides more granularity and accuracy of how we check JSON value equalities.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant