From b3dce2dc17086c72bd1f8ccf508874849f9631bb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=89amonn=20McManus?= Date: Sun, 30 Jul 2023 09:20:32 -0700 Subject: [PATCH] Strict mode for JSON parsing, contributed by @marten-voorberg. (#2437) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Strict mode for JSON parsing (#2323) * Feat #6: Add strict flag to Gson and GsonBuilder * Test #2: Add failing tests for capitalized keywords * Feat #2: JsonReader does not read (partially) capitalized keywords if strict mode is used * Feat #3: Added implementation and tests for JSONReader not accepting specific escape sequence representing in strict mode * Test #3: Simplify test cases by removing unnecessary array * Feat #3: Improve error by including the illegal character * Feat #5: JsonReader does not allow unespaced control flow characters in strict mode * Test #5: Test unespaced control flow characters in strict mode * Feat #4: Disallow espaced newline character in strict mode * Test #4: Add tests for (dis)allowing newline character depensding on strictness * Test #5: Test case for unescaped control char in non-strict mode * Test #2: Simplify test cases * Feat #13: Change leniency API to Strictness enum in JsonReader, Gson, and GsonBuilder * Feat #15: Change JsonWriter API to also use Strictness * Test #15: Test Strictness in JsonWriter API * Doc #15: Add and update documentation for Strictness in JsonWriter API * refactor #12: Fixed typos and empty catch brackets in tests * refactor #12: Resolved importing wildcards, made some lines adhere to Google java style * #5 Add test case for unescaped control characters * Feat #5: add new lines to make JsonReader able to detect unescaped control characters (U+0000 through U+001F) and throw exceptions. * Feat #5: add new lines to make JsonReader able to detect unescaped control characters (U+0000 through U+001F) and throw exceptions. * Test #11: Added two tests for testing implementation of control character handling in strict mode and moved the implementation to nextQuotedValue * Test #11: Added two tests for testing implementation of control character handling in strict mode and moved the implementation to nextQuotedValue --------- Co-authored-by: LMC117 <2295699210@qq.com> Co-authored-by: Marten Voorberg * Doc #17: Add and change javadoc of public methods * Doc #17: Update JavaDoc in JsonReader and Strictness * Doc #17: Update JavaDoc in Gson and GsonBuilder * Test #34: Add tests for setting strictness through GsonBuilder * Fix: Add Fix broken test * Fix: Invalid JavaDoc in Gson.java * Doc #17: update outdated javadoc * #37: Resolve more PR feedback * Fix #37: Resolve various PR comments * Fix #37: Resolve various PR comments * Refactor #35: Refactor JsonReader#peekKeyword to reduce the amount of strictness checks (#39) * Doc #40: Update JavaDoc based on PR feedback * Doc #40: Update old RFC in GsonBuilder documentation * Doc #40: Fix formatting error in JavaDoc * Doc #40: Add tests for setting strictness and lenient to JsonReaderTest * Test #43: Changed tests to make use of assertThrows * test #43: Changed tests to make use of assertThrows as per feedback * Test #43: Update JsonWriterTest#testStrictnessNull to use assertThrows * Test #43: Update JsonWriterTest#testStrictnessNull to use assertThrows * test #43: Resolve PR recommendations * Test #43: Mini change to TC * Test #43: Mini change to TC --------- Co-authored-by: Marten Voorberg * doc #46: Resolved comments in main PR * Feat #45: Change Gson.fromJson and Gson.toJson to be strict when the provided writer/reader is strict * Fix #45: Small type * Update gson/src/test/java/com/google/gson/stream/JsonReaderTest.java Co-authored-by: Marcono1234 * Fix #45: Resolve various comments by Marcono1234 * Update gson/src/main/java/com/google/gson/GsonBuilder.java Co-authored-by: Marcono1234 * Fix #45: Resolve various comments by Marcono1234 * Fix #45: Resolve various comments by eamonmcmanus * Strictness mode follow-up * Update Troubleshooting.md and Gson default lenient mode documentation * Always use GSON strictness when set. * Rename Strictness.DEFAULT to Strictness.LEGACY_STRICT * Update JavaDoc with new strictness functionality * Replace default with legacy strict for JsonReader javadoc * Add JSONReader test cases for U2028 and U2029 * Refactor JSONReader#peekKeyWord() based on @eamonmcmanus's suggestion * Deprecate setLenient in favor of setStrictness --------- Co-authored-by: Carl Peterson Co-authored-by: Gustaf Johansson Co-authored-by: gustajoh <58432871+gustajoh@users.noreply.github.com> Co-authored-by: LMC117 <2295699210@qq.com> Co-authored-by: Marcono1234 * Strictness follow-up (#2408) * Strictness mode follow-up - Remove mentions of `null` Gson strictness; this is an implementation detail - Fix incorrect / outdated documentation - Reduce links to RFC; if there is already a link to it in a previous sentence don't link to it again - Extend and update tests - Minor punctuation changes in documentation for consistency * Deprecate `setLenient` methods * `strictness2` fixes & improvements (#2456) * Adjust ProGuard default rules and shrinking tests (#2420) * Adjust ProGuard default rules and shrinking tests * Adjust comment * Add shrinking test for class without no-args constructor; improve docs * Improve Unsafe mention in Troubleshooting Guide * Improve comment for `-if class *` * Bump com.google.guava:guava from 32.0.1-jre to 32.1.1-jre (#2444) Bumps [com.google.guava:guava](https://github.com/google/guava) from 32.0.1-jre to 32.1.1-jre. - [Release notes](https://github.com/google/guava/releases) - [Commits](https://github.com/google/guava/commits) --- updated-dependencies: - dependency-name: com.google.guava:guava dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump com.google.guava:guava-testlib from 32.0.1-jre to 32.1.1-jre (#2443) Bumps [com.google.guava:guava-testlib](https://github.com/google/guava) from 32.0.1-jre to 32.1.1-jre. - [Release notes](https://github.com/google/guava/releases) - [Commits](https://github.com/google/guava/commits) --- updated-dependencies: - dependency-name: com.google.guava:guava-testlib dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Support non-generic type for `TypeToken.getParameterized` for legacy reasons (#2447) This partially restores the behavior before a589ef20087b4b0f1ec3048d3ceaef1eedccd09d, except that back then for a non-generic type a bogus `TypeToken(ParameterizedType)` was created, whereas now a `TypeToken(Class)` is created instead. * Fixed Typo in GsonBuilder.java (#2449) * Make date-formatting tests less fragile with regular expressions. (#2450) * Make date-formatting tests less fragile with regular expressions. This is not great. We should really ensure that formatted dates are the same regardless of JDK version. There is code that attempts to do that but it is not really effective. So for now we fudge around the differences by using regular expressions to paper over the differences. * Temporarily add test-debugging code. * Another attempt at debugging a test failure. * Fix pattern in assertion. * Modification in test cases (#2454) * Fixed Typo in GsonBuilder.java * Suggestions on Test cases * Modified test cases using assertThrows method (JUnit) * Update gson/src/test/java/com/google/gson/JsonArrayAsListTest.java Co-authored-by: Marcono1234 * Update gson/src/test/java/com/google/gson/GsonTest.java Co-authored-by: Marcono1234 * Update gson/src/test/java/com/google/gson/JsonArrayAsListTest.java Co-authored-by: Marcono1234 * Update gson/src/test/java/com/google/gson/JsonStreamParserTest.java Co-authored-by: Marcono1234 * Update gson/src/test/java/com/google/gson/JsonStreamParserTest.java Co-authored-by: Marcono1234 * Update gson/src/test/java/com/google/gson/JsonStreamParserTest.java Co-authored-by: Marcono1234 * Update gson/src/test/java/com/google/gson/ToNumberPolicyTest.java Co-authored-by: Marcono1234 * Update gson/src/test/java/com/google/gson/TypeAdapterTest.java Co-authored-by: Marcono1234 * Update gson/src/test/java/com/google/gson/TypeAdapterTest.java Co-authored-by: Marcono1234 * Update gson/src/test/java/com/google/gson/ToNumberPolicyTest.java Co-authored-by: Marcono1234 * Update gson/src/test/java/com/google/gson/ToNumberPolicyTest.java Co-authored-by: Marcono1234 --------- Co-authored-by: Marcono1234 * Minor follow-up changes --------- Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: elevne <97422844+elevne@users.noreply.github.com> Co-authored-by: Éamonn McManus Co-authored-by: Wonil --------- Signed-off-by: dependabot[bot] Co-authored-by: Marten Co-authored-by: Gustaf Johansson Co-authored-by: gustajoh <58432871+gustajoh@users.noreply.github.com> Co-authored-by: LMC117 <2295699210@qq.com> Co-authored-by: Marcono1234 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: elevne <97422844+elevne@users.noreply.github.com> Co-authored-by: Wonil --- Troubleshooting.md | 11 +- gson/src/main/java/com/google/gson/Gson.java | 127 +++++--- .../java/com/google/gson/GsonBuilder.java | 57 +++- .../java/com/google/gson/JsonElement.java | 3 +- .../main/java/com/google/gson/JsonParser.java | 14 +- .../com/google/gson/JsonStreamParser.java | 4 +- .../main/java/com/google/gson/Strictness.java | 29 ++ .../java/com/google/gson/TypeAdapter.java | 33 +- .../com/google/gson/stream/JsonReader.java | 163 +++++++--- .../com/google/gson/stream/JsonWriter.java | 93 ++++-- .../gson/stream/MalformedJsonException.java | 3 +- .../java/com/google/gson/GsonBuilderTest.java | 30 ++ .../test/java/com/google/gson/GsonTest.java | 11 +- .../java/com/google/gson/JsonParserTest.java | 14 +- .../java/com/google/gson/MixedStreamTest.java | 2 + .../com/google/gson/ToNumberPolicyTest.java | 8 +- .../com/google/gson/functional/ArrayTest.java | 1 - .../google/gson/functional/LeniencyTest.java | 1 + .../gson/functional/PrettyPrintingTest.java | 3 +- .../gson/functional/UncategorizedTest.java | 1 - .../internal/bind/JsonElementReaderTest.java | 5 +- .../internal/bind/JsonTreeReaderTest.java | 2 +- .../internal/bind/JsonTreeWriterTest.java | 13 +- .../gson/stream/JsonReaderPathTest.java | 3 +- .../google/gson/stream/JsonReaderTest.java | 301 ++++++++++++++---- .../google/gson/stream/JsonWriterTest.java | 265 +++++++++------ 26 files changed, 864 insertions(+), 333 deletions(-) create mode 100644 gson/src/main/java/com/google/gson/Strictness.java diff --git a/Troubleshooting.md b/Troubleshooting.md index 184f19166e..6bf2b8579a 100644 --- a/Troubleshooting.md +++ b/Troubleshooting.md @@ -127,7 +127,7 @@ For example, let's assume you want to deserialize the following JSON data: } ``` -This will fail with an exception similar to this one: `MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 5 column 4 path $.languages[2]` +This will fail with an exception similar to this one: `MalformedJsonException: Use JsonReader.setStrictness(Strictness.LENIENT) to accept malformed JSON at line 5 column 4 path $.languages[2]` The problem here is the trailing comma (`,`) after `"French"`, trailing commas are not allowed by the JSON specification. The location information "line 5 column 4" points to the `]` in the JSON data (with some slight inaccuracies) because Gson expected another value after `,` instead of the closing `]`. The JSONPath `$.languages[2]` in the exception message also points there: `$.` refers to the root object, `languages` refers to its member of that name and `[2]` refers to the (missing) third value in the JSON array value of that member (numbering starts at 0, so it is `[2]` instead of `[3]`). The proper solution here is to fix the malformed JSON data. @@ -147,9 +147,12 @@ To spot syntax errors in the JSON data easily you can open it in an editor with **Reason:** Due to legacy reasons Gson performs parsing by default in lenient mode -**Solution:** See [`Gson` class documentation](https://www.javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/Gson.html#default-lenient) section "Lenient JSON handling" - -Note: Even in non-lenient mode Gson deviates slightly from the JSON specification, see [`JsonReader.setLenient`](https://www.javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/stream/JsonReader.html#setLenient(boolean)) for more details. +**Solution:** If you are using Gson 2.11.0 or newer, call [`GsonBuilder.setStrictness`](https://www.javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/GsonBuilder.html#setStrictness(com.google.gson.Strictness)), +[`JsonReader.setStrictness`](https://www.javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/stream/JsonReader.html#setStrictness(com.google.gson.Strictness)) +and [`JsonWriter.setStrictness`](https://www.javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/stream/JsonWriter.html#setStrictness(com.google.gson.Strictness)) +with `Strictness.STRICT` to overwrite the default lenient behavior of `Gson` and make these classes strictly adhere to the JSON specification. +Otherwise if you are using an older Gson version, see the [`Gson` class documentation](https://www.javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/Gson.html#default-lenient) +section "JSON Strictness handling" for alternative solutions. ## `IllegalStateException`: "Expected ... but was ..." diff --git a/gson/src/main/java/com/google/gson/Gson.java b/gson/src/main/java/com/google/gson/Gson.java index c6f8508ef1..0219c1a186 100644 --- a/gson/src/main/java/com/google/gson/Gson.java +++ b/gson/src/main/java/com/google/gson/Gson.java @@ -105,10 +105,14 @@ *

See the Gson User Guide * for a more complete set of examples.

* - *

Lenient JSON handling

+ *

JSON Strictness handling

* For legacy reasons most of the {@code Gson} methods allow JSON data which does not - * comply with the JSON specification, regardless of whether {@link GsonBuilder#setLenient()} - * is used or not. If this behavior is not desired, the following workarounds can be used: + * comply with the JSON specification when no explicit {@linkplain Strictness strictness} is set (the default). + * To specify the strictness of a {@code Gson} instance, you should set it through + * {@link GsonBuilder#setStrictness(Strictness)}. + * + *

For older Gson versions, which don't have the strictness mode API, the following + * workarounds can be used: * *

Serialization

*
    @@ -132,6 +136,10 @@ * to make sure there is no trailing data *
* + * Note that the {@code JsonReader} created this way is only 'legacy strict', it mostly adheres + * to the JSON specification but allows small deviations. See {@link JsonReader#setStrictness(Strictness)} + * for details. + * * @see TypeToken * * @author Inderjeet Singh @@ -140,7 +148,8 @@ */ public final class Gson { static final boolean DEFAULT_JSON_NON_EXECUTABLE = false; - static final boolean DEFAULT_LENIENT = false; + // Strictness of `null` is the legacy mode where some Gson APIs are always lenient + static final Strictness DEFAULT_STRICTNESS = null; static final FormattingStyle DEFAULT_FORMATTING_STYLE = FormattingStyle.COMPACT; static final boolean DEFAULT_ESCAPE_HTML = true; static final boolean DEFAULT_SERIALIZE_NULLS = false; @@ -184,7 +193,7 @@ public final class Gson { final boolean generateNonExecutableJson; final boolean htmlSafe; final FormattingStyle formattingStyle; - final boolean lenient; + final Strictness strictness; final boolean serializeSpecialFloatingPointValues; final boolean useJdkUnsafe; final String datePattern; @@ -231,13 +240,15 @@ public final class Gson { *
  • By default, Gson excludes transient or static fields from * consideration for serialization and deserialization. You can change this behavior through * {@link GsonBuilder#excludeFieldsWithModifiers(int...)}.
    • + *
  • No explicit strictness is set. You can change this by calling + * {@link GsonBuilder#setStrictness(Strictness)}.
    • * */ public Gson() { this(Excluder.DEFAULT, DEFAULT_FIELD_NAMING_STRATEGY, Collections.>emptyMap(), DEFAULT_SERIALIZE_NULLS, DEFAULT_COMPLEX_MAP_KEYS, DEFAULT_JSON_NON_EXECUTABLE, DEFAULT_ESCAPE_HTML, - DEFAULT_FORMATTING_STYLE, DEFAULT_LENIENT, DEFAULT_SPECIALIZE_FLOAT_VALUES, + DEFAULT_FORMATTING_STYLE, DEFAULT_STRICTNESS, DEFAULT_SPECIALIZE_FLOAT_VALUES, DEFAULT_USE_JDK_UNSAFE, LongSerializationPolicy.DEFAULT, DEFAULT_DATE_PATTERN, DateFormat.DEFAULT, DateFormat.DEFAULT, Collections.emptyList(), Collections.emptyList(), @@ -248,7 +259,7 @@ public Gson() { Gson(Excluder excluder, FieldNamingStrategy fieldNamingStrategy, Map> instanceCreators, boolean serializeNulls, boolean complexMapKeySerialization, boolean generateNonExecutableGson, boolean htmlSafe, - FormattingStyle formattingStyle, boolean lenient, boolean serializeSpecialFloatingPointValues, + FormattingStyle formattingStyle, Strictness strictness, boolean serializeSpecialFloatingPointValues, boolean useJdkUnsafe, LongSerializationPolicy longSerializationPolicy, String datePattern, int dateStyle, int timeStyle, List builderFactories, @@ -265,7 +276,7 @@ public Gson() { this.generateNonExecutableJson = generateNonExecutableGson; this.htmlSafe = htmlSafe; this.formattingStyle = formattingStyle; - this.lenient = lenient; + this.strictness = strictness; this.serializeSpecialFloatingPointValues = serializeSpecialFloatingPointValues; this.useJdkUnsafe = useJdkUnsafe; this.longSerializationPolicy = longSerializationPolicy; @@ -802,7 +813,7 @@ public void toJson(Object src, Appendable writer) throws JsonIOException { * 
        * Type typeOfSrc = new TypeToken<Collection<Foo>>(){}.getType();
    *
    - * @param writer Writer to which the JSON representation of src needs to be written. + * @param writer Writer to which the JSON representation of src needs to be written * @throws JsonIOException if there was a problem writing to the writer * @since 1.2 * @@ -822,24 +833,38 @@ public void toJson(Object src, Type typeOfSrc, Appendable writer) throws JsonIOE * Writes the JSON representation of {@code src} of type {@code typeOfSrc} to * {@code writer}. * - *

    The JSON data is written in {@linkplain JsonWriter#setLenient(boolean) lenient mode}, - * regardless of the lenient mode setting of the provided writer. The lenient mode setting - * of the writer is restored once this method returns. + *

    If the {@code Gson} instance has an {@linkplain GsonBuilder#setStrictness(Strictness) explicit strictness setting}, + * this setting will be used for writing the JSON regardless of the {@linkplain JsonWriter#getStrictness() strictness} + * of the provided {@link JsonWriter}. For legacy reasons, if the {@code Gson} instance has no explicit strictness setting + * and the writer does not have the strictness {@link Strictness#STRICT}, the JSON will be written in {@link Strictness#LENIENT} + * mode.
    + * Note that in all cases the old strictness setting of the writer will be restored when this method returns. * *

    The 'HTML-safe' and 'serialize {@code null}' settings of this {@code Gson} instance * (configured by the {@link GsonBuilder}) are applied, and the original settings of the * writer are restored once this method returns. * + * @param src the object for which JSON representation is to be created + * @param typeOfSrc the type of the object to be written + * @param writer Writer to which the JSON representation of src needs to be written + * * @throws JsonIOException if there was a problem writing to the writer */ public void toJson(Object src, Type typeOfSrc, JsonWriter writer) throws JsonIOException { @SuppressWarnings("unchecked") TypeAdapter adapter = (TypeAdapter) getAdapter(TypeToken.get(typeOfSrc)); - boolean oldLenient = writer.isLenient(); - writer.setLenient(true); + + Strictness oldStrictness = writer.getStrictness(); + if (this.strictness != null) { + writer.setStrictness(this.strictness); + } else if (writer.getStrictness() != Strictness.STRICT) { + writer.setStrictness(Strictness.LENIENT); + } + boolean oldHtmlSafe = writer.isHtmlSafe(); - writer.setHtmlSafe(htmlSafe); boolean oldSerializeNulls = writer.getSerializeNulls(); + + writer.setHtmlSafe(htmlSafe); writer.setSerializeNulls(serializeNulls); try { adapter.write(writer, src); @@ -848,7 +873,7 @@ public void toJson(Object src, Type typeOfSrc, JsonWriter writer) throws JsonIOE } catch (AssertionError e) { throw new AssertionError("AssertionError (GSON " + GsonBuildConfig.VERSION + "): " + e.getMessage(), e); } finally { - writer.setLenient(oldLenient); + writer.setStrictness(oldStrictness); writer.setHtmlSafe(oldHtmlSafe); writer.setSerializeNulls(oldSerializeNulls); } @@ -892,7 +917,10 @@ public void toJson(JsonElement jsonElement, Appendable writer) throws JsonIOExce *
  • {@link GsonBuilder#disableHtmlEscaping()}
    • *
  • {@link GsonBuilder#generateNonExecutableJson()}
    • *
  • {@link GsonBuilder#serializeNulls()}
    • - *
  • {@link GsonBuilder#setLenient()}
    • + *
  • {@link GsonBuilder#setStrictness(Strictness)}. If no + * {@linkplain GsonBuilder#setStrictness(Strictness) explicit strictness has been set} the created + * writer will have a strictness of {@link Strictness#LEGACY_STRICT}. Otherwise, the strictness of + * the {@code Gson} instance will be used for the created writer.
    • *
  • {@link GsonBuilder#setPrettyPrinting()}
    • *
  • {@link GsonBuilder#setFormattingStyle(FormattingStyle)}
    • * @@ -904,7 +932,7 @@ public JsonWriter newJsonWriter(Writer writer) throws IOException { JsonWriter jsonWriter = new JsonWriter(writer); jsonWriter.setFormattingStyle(formattingStyle); jsonWriter.setHtmlSafe(htmlSafe); - jsonWriter.setLenient(lenient); + jsonWriter.setStrictness(strictness == null ? Strictness.LEGACY_STRICT : strictness); jsonWriter.setSerializeNulls(serializeNulls); return jsonWriter; } @@ -914,35 +942,50 @@ public JsonWriter newJsonWriter(Writer writer) throws IOException { * *

    The following settings are considered: *

      - *
    • {@link GsonBuilder#setLenient()}
      • + *
    • {@link GsonBuilder#setStrictness(Strictness)}. If no + * {@linkplain GsonBuilder#setStrictness(Strictness) explicit strictness has been set} the created + * reader will have a strictness of {@link Strictness#LEGACY_STRICT}. Otherwise, the strictness of + * the {@code Gson} instance will be used for the created reader.
      • *
    */ public JsonReader newJsonReader(Reader reader) { JsonReader jsonReader = new JsonReader(reader); - jsonReader.setLenient(lenient); + jsonReader.setStrictness(strictness == null ? Strictness.LEGACY_STRICT : strictness); return jsonReader; } /** * Writes the JSON for {@code jsonElement} to {@code writer}. * - *

    The JSON data is written in {@linkplain JsonWriter#setLenient(boolean) lenient mode}, - * regardless of the lenient mode setting of the provided writer. The lenient mode setting - * of the writer is restored once this method returns. + *

    If the {@code Gson} instance has an {@linkplain GsonBuilder#setStrictness(Strictness) explicit strictness setting}, + * this setting will be used for writing the JSON regardless of the {@linkplain JsonWriter#getStrictness() strictness} + * of the provided {@link JsonWriter}. For legacy reasons, if the {@code Gson} instance has no explicit strictness setting + * and the writer does not have the strictness {@link Strictness#STRICT}, the JSON will be written in {@link Strictness#LENIENT} + * mode.
    + * Note that in all cases the old strictness setting of the writer will be restored when this method returns. * *

    The 'HTML-safe' and 'serialize {@code null}' settings of this {@code Gson} instance * (configured by the {@link GsonBuilder}) are applied, and the original settings of the * writer are restored once this method returns. * + * @param jsonElement the JSON element to be written + * @param writer the JSON writer to which the provided element will be written * @throws JsonIOException if there was a problem writing to the writer */ public void toJson(JsonElement jsonElement, JsonWriter writer) throws JsonIOException { - boolean oldLenient = writer.isLenient(); - writer.setLenient(true); + Strictness oldStrictness = writer.getStrictness(); boolean oldHtmlSafe = writer.isHtmlSafe(); - writer.setHtmlSafe(htmlSafe); boolean oldSerializeNulls = writer.getSerializeNulls(); + + writer.setHtmlSafe(htmlSafe); writer.setSerializeNulls(serializeNulls); + + if (this.strictness != null) { + writer.setStrictness(this.strictness); + } else if (writer.getStrictness() != Strictness.STRICT) { + writer.setStrictness(Strictness.LENIENT); + } + try { Streams.write(jsonElement, writer); } catch (IOException e) { @@ -950,7 +993,7 @@ public void toJson(JsonElement jsonElement, JsonWriter writer) throws JsonIOExce } catch (AssertionError e) { throw new AssertionError("AssertionError (GSON " + GsonBuildConfig.VERSION + "): " + e.getMessage(), e); } finally { - writer.setLenient(oldLenient); + writer.setStrictness(oldStrictness); writer.setHtmlSafe(oldHtmlSafe); writer.setSerializeNulls(oldSerializeNulls); } @@ -1169,9 +1212,12 @@ private static void assertFullConsumption(Object obj, JsonReader reader) { *

    Unlike the other {@code fromJson} methods, no exception is thrown if the JSON data has * multiple top-level JSON elements, or if there is trailing data. * - *

    The JSON data is parsed in {@linkplain JsonReader#setLenient(boolean) lenient mode}, - * regardless of the lenient mode setting of the provided reader. The lenient mode setting - * of the reader is restored once this method returns. + *

    If the {@code Gson} instance has an {@linkplain GsonBuilder#setStrictness(Strictness) explicit strictness setting}, + * this setting will be used for reading the JSON regardless of the {@linkplain JsonReader#getStrictness() strictness} + * of the provided {@link JsonReader}. For legacy reasons, if the {@code Gson} instance has no explicit strictness setting + * and the reader does not have the strictness {@link Strictness#STRICT}, the JSON will be written in {@link Strictness#LENIENT} + * mode.
    + * Note that in all cases the old strictness setting of the reader will be restored when this method returns. * * @param the type of the desired object * @param reader the reader whose next JSON value should be deserialized @@ -1198,9 +1244,12 @@ public T fromJson(JsonReader reader, Type typeOfT) throws JsonIOException, J *

    Unlike the other {@code fromJson} methods, no exception is thrown if the JSON data has * multiple top-level JSON elements, or if there is trailing data. * - *

    The JSON data is parsed in {@linkplain JsonReader#setLenient(boolean) lenient mode}, - * regardless of the lenient mode setting of the provided reader. The lenient mode setting - * of the reader is restored once this method returns. + *

    If the {@code Gson} instance has an {@linkplain GsonBuilder#setStrictness(Strictness) explicit strictness setting}, + * this setting will be used for reading the JSON regardless of the {@linkplain JsonReader#getStrictness() strictness} + * of the provided {@link JsonReader}. For legacy reasons, if the {@code Gson} instance has no explicit strictness setting + * and the reader does not have the strictness {@link Strictness#STRICT}, the JSON will be written in {@link Strictness#LENIENT} + * mode.
    + * Note that in all cases the old strictness setting of the reader will be restored when this method returns. * * @param the type of the desired object * @param reader the reader whose next JSON value should be deserialized @@ -1220,8 +1269,14 @@ public T fromJson(JsonReader reader, Type typeOfT) throws JsonIOException, J */ public T fromJson(JsonReader reader, TypeToken typeOfT) throws JsonIOException, JsonSyntaxException { boolean isEmpty = true; - boolean oldLenient = reader.isLenient(); - reader.setLenient(true); + Strictness oldStrictness = reader.getStrictness(); + + if (this.strictness != null) { + reader.setStrictness(this.strictness); + } else if (reader.getStrictness() != Strictness.STRICT) { + reader.setStrictness(Strictness.LENIENT); + } + try { JsonToken unused = reader.peek(); isEmpty = false; @@ -1244,7 +1299,7 @@ public T fromJson(JsonReader reader, TypeToken typeOfT) throws JsonIOExce } catch (AssertionError e) { throw new AssertionError("AssertionError (GSON " + GsonBuildConfig.VERSION + "): " + e.getMessage(), e); } finally { - reader.setLenient(oldLenient); + reader.setStrictness(oldStrictness); } } diff --git a/gson/src/main/java/com/google/gson/GsonBuilder.java b/gson/src/main/java/com/google/gson/GsonBuilder.java index c72c411f07..68eb7d718b 100644 --- a/gson/src/main/java/com/google/gson/GsonBuilder.java +++ b/gson/src/main/java/com/google/gson/GsonBuilder.java @@ -21,14 +21,15 @@ import static com.google.gson.Gson.DEFAULT_ESCAPE_HTML; import static com.google.gson.Gson.DEFAULT_FORMATTING_STYLE; import static com.google.gson.Gson.DEFAULT_JSON_NON_EXECUTABLE; -import static com.google.gson.Gson.DEFAULT_LENIENT; import static com.google.gson.Gson.DEFAULT_NUMBER_TO_NUMBER_STRATEGY; import static com.google.gson.Gson.DEFAULT_OBJECT_TO_NUMBER_STRATEGY; import static com.google.gson.Gson.DEFAULT_SERIALIZE_NULLS; import static com.google.gson.Gson.DEFAULT_SPECIALIZE_FLOAT_VALUES; +import static com.google.gson.Gson.DEFAULT_STRICTNESS; import static com.google.gson.Gson.DEFAULT_USE_JDK_UNSAFE; import com.google.errorprone.annotations.CanIgnoreReturnValue; +import com.google.errorprone.annotations.InlineMe; import com.google.gson.annotations.Since; import com.google.gson.annotations.Until; import com.google.gson.internal.$Gson$Preconditions; @@ -71,12 +72,16 @@ * .create(); * * - *

    NOTES: + *

    Notes: *

      - *
    • the order of invocation of configuration methods does not matter.
      • - *
    • The default serialization of {@link Date} and its subclasses in Gson does + *
    • The order of invocation of configuration methods does not matter.
      • + *
    • The default serialization of {@link Date} and its subclasses in Gson does * not contain time-zone information. So, if you are using date/time instances, * use {@code GsonBuilder} and its {@code setDateFormat} methods.
      • + *
    • By default no explicit {@link Strictness} is set; some of the {@link Gson} methods + * behave as if {@link Strictness#LEGACY_STRICT} was used whereas others behave as + * if {@link Strictness#LENIENT} was used. Prefer explicitly setting a strictness + * with {@link #setStrictness(Strictness)} to avoid this legacy behavior. *
    * * @author Inderjeet Singh @@ -100,7 +105,7 @@ public final class GsonBuilder { private boolean escapeHtmlChars = DEFAULT_ESCAPE_HTML; private FormattingStyle formattingStyle = DEFAULT_FORMATTING_STYLE; private boolean generateNonExecutableJson = DEFAULT_JSON_NON_EXECUTABLE; - private boolean lenient = DEFAULT_LENIENT; + private Strictness strictness = DEFAULT_STRICTNESS; private boolean useJdkUnsafe = DEFAULT_USE_JDK_UNSAFE; private ToNumberStrategy objectToNumberStrategy = DEFAULT_OBJECT_TO_NUMBER_STRATEGY; private ToNumberStrategy numberToNumberStrategy = DEFAULT_NUMBER_TO_NUMBER_STRATEGY; @@ -130,7 +135,7 @@ public GsonBuilder() { this.generateNonExecutableJson = gson.generateNonExecutableJson; this.escapeHtmlChars = gson.htmlSafe; this.formattingStyle = gson.formattingStyle; - this.lenient = gson.lenient; + this.strictness = gson.strictness; this.serializeSpecialFloatingPointValues = gson.serializeSpecialFloatingPointValues; this.longSerializationPolicy = gson.longSerializationPolicy; this.datePattern = gson.datePattern; @@ -521,18 +526,40 @@ public GsonBuilder setFormattingStyle(FormattingStyle formattingStyle) { } /** - * Configures Gson to allow JSON data which does not strictly comply with the JSON specification. + * Sets the strictness of this builder to {@link Strictness#LENIENT}. * - *

    Note: Due to legacy reasons most methods of Gson are always lenient, regardless of - * whether this builder method is used. + * @deprecated This method is equivalent to calling {@link #setStrictness(Strictness)} with + * {@link Strictness#LENIENT}: {@code setStrictness(Strictness.LENIENT)} * - * @return a reference to this {@code GsonBuilder} object to fulfill the "Builder" pattern - * @see JsonReader#setLenient(boolean) - * @see JsonWriter#setLenient(boolean) + * @return a reference to this {@code GsonBuilder} object to fulfill the "Builder" pattern. + * @see JsonReader#setStrictness(Strictness) + * @see JsonWriter#setStrictness(Strictness) + * @see #setStrictness(Strictness) */ + @Deprecated + @InlineMe(replacement = "this.setStrictness(Strictness.LENIENT)", imports = "com.google.gson.Strictness") @CanIgnoreReturnValue public GsonBuilder setLenient() { - lenient = true; + return setStrictness(Strictness.LENIENT); + } + + /** + * Sets the strictness of this builder to the provided parameter. + * + *

    This changes how strict the + * RFC 8259 JSON specification is enforced when parsing or + * writing JSON. For details on this, refer to {@link JsonReader#setStrictness(Strictness)} and + * {@link JsonWriter#setStrictness(Strictness)}.

    + * + * @param strictness the new strictness mode. May not be {@code null}. + * @return a reference to this {@code GsonBuilder} object to fulfill the "Builder" pattern. + * @see JsonReader#setStrictness(Strictness) + * @see JsonWriter#setStrictness(Strictness) + * @since $next-version$ + */ + @CanIgnoreReturnValue + public GsonBuilder setStrictness(Strictness strictness) { + this.strictness = Objects.requireNonNull(strictness); return this; } @@ -711,7 +738,7 @@ public GsonBuilder registerTypeHierarchyAdapter(Class baseType, Object typeAd } /** - * Section 2.4 of JSON specification disallows + * Section 6 of JSON specification disallows * special double values (NaN, Infinity, -Infinity). However, * Javascript * specification (see section 4.3.20, 4.3.22, 4.3.23) allows these values as valid Javascript @@ -804,7 +831,7 @@ public Gson create() { return new Gson(excluder, fieldNamingPolicy, new HashMap<>(instanceCreators), serializeNulls, complexMapKeySerialization, - generateNonExecutableJson, escapeHtmlChars, formattingStyle, lenient, + generateNonExecutableJson, escapeHtmlChars, formattingStyle, strictness, serializeSpecialFloatingPointValues, useJdkUnsafe, longSerializationPolicy, datePattern, dateStyle, timeStyle, new ArrayList<>(this.factories), new ArrayList<>(this.hierarchyFactories), factories, diff --git a/gson/src/main/java/com/google/gson/JsonElement.java b/gson/src/main/java/com/google/gson/JsonElement.java index 23e5654aac..1b440d0532 100644 --- a/gson/src/main/java/com/google/gson/JsonElement.java +++ b/gson/src/main/java/com/google/gson/JsonElement.java @@ -321,7 +321,8 @@ public String toString() { try { StringWriter stringWriter = new StringWriter(); JsonWriter jsonWriter = new JsonWriter(stringWriter); - jsonWriter.setLenient(true); + // Make writer lenient because toString() must not fail, even if for example JsonPrimitive contains NaN + jsonWriter.setStrictness(Strictness.LENIENT); Streams.write(this, jsonWriter); return stringWriter.toString(); } catch (IOException e) { diff --git a/gson/src/main/java/com/google/gson/JsonParser.java b/gson/src/main/java/com/google/gson/JsonParser.java index 20d3750cea..557d00c8e0 100644 --- a/gson/src/main/java/com/google/gson/JsonParser.java +++ b/gson/src/main/java/com/google/gson/JsonParser.java @@ -41,7 +41,7 @@ public JsonParser() {} * An exception is thrown if the JSON string has multiple top-level JSON elements, * or if there is trailing data. * - *

    The JSON string is parsed in {@linkplain JsonReader#setLenient(boolean) lenient mode}. + *

    The JSON string is parsed in {@linkplain JsonReader#setStrictness(Strictness) lenient mode}. * * @param json JSON text * @return a parse tree of {@link JsonElement}s corresponding to the specified JSON @@ -57,7 +57,7 @@ public static JsonElement parseString(String json) throws JsonSyntaxException { * An exception is thrown if the JSON string has multiple top-level JSON elements, * or if there is trailing data. * - *

    The JSON data is parsed in {@linkplain JsonReader#setLenient(boolean) lenient mode}. + *

    The JSON data is parsed in {@linkplain JsonReader#setStrictness(Strictness) lenient mode}. * * @param reader JSON text * @return a parse tree of {@link JsonElement}s corresponding to the specified JSON @@ -87,8 +87,8 @@ public static JsonElement parseReader(Reader reader) throws JsonIOException, Jso * Unlike the other {@code parse} methods, no exception is thrown if the JSON data has * multiple top-level JSON elements, or if there is trailing data. * - *

    The JSON data is parsed in {@linkplain JsonReader#setLenient(boolean) lenient mode}, - * regardless of the lenient mode setting of the provided reader. The lenient mode setting + *

    The JSON data is parsed in {@linkplain JsonReader#setStrictness(Strictness) lenient mode}, + * regardless of the strictness setting of the provided reader. The strictness setting * of the reader is restored once this method returns. * * @throws JsonParseException if there is an IOException or if the specified @@ -97,8 +97,8 @@ public static JsonElement parseReader(Reader reader) throws JsonIOException, Jso */ public static JsonElement parseReader(JsonReader reader) throws JsonIOException, JsonSyntaxException { - boolean lenient = reader.isLenient(); - reader.setLenient(true); + Strictness strictness = reader.getStrictness(); + reader.setStrictness(Strictness.LENIENT); try { return Streams.parse(reader); } catch (StackOverflowError e) { @@ -106,7 +106,7 @@ public static JsonElement parseReader(JsonReader reader) } catch (OutOfMemoryError e) { throw new JsonParseException("Failed parsing JSON source: " + reader + " to Json", e); } finally { - reader.setLenient(lenient); + reader.setStrictness(strictness); } } diff --git a/gson/src/main/java/com/google/gson/JsonStreamParser.java b/gson/src/main/java/com/google/gson/JsonStreamParser.java index cbc2883ca8..7d2629368b 100644 --- a/gson/src/main/java/com/google/gson/JsonStreamParser.java +++ b/gson/src/main/java/com/google/gson/JsonStreamParser.java @@ -28,7 +28,7 @@ /** * A streaming parser that allows reading of multiple {@link JsonElement}s from the specified reader * asynchronously. The JSON data is parsed in lenient mode, see also - * {@link JsonReader#setLenient(boolean)}. + * {@link JsonReader#setStrictness(Strictness)}. * *

    This class is conditionally thread-safe (see Item 70, Effective Java second edition). To * properly use this class across multiple threads, you will need to add some external @@ -66,7 +66,7 @@ public JsonStreamParser(String json) { */ public JsonStreamParser(Reader reader) { parser = new JsonReader(reader); - parser.setLenient(true); + parser.setStrictness(Strictness.LENIENT); lock = new Object(); } diff --git a/gson/src/main/java/com/google/gson/Strictness.java b/gson/src/main/java/com/google/gson/Strictness.java new file mode 100644 index 0000000000..f3bd3fe08f --- /dev/null +++ b/gson/src/main/java/com/google/gson/Strictness.java @@ -0,0 +1,29 @@ +package com.google.gson; + +import com.google.gson.stream.JsonReader; +import com.google.gson.stream.JsonWriter; + +/** + * Modes that indicate how strictly a JSON {@linkplain JsonReader reader} or + * {@linkplain JsonWriter writer} follows the syntax laid out in the + * RFC 8259 JSON specification. + * + *

    You can look at {@link JsonReader#setStrictness(Strictness)} to see how the strictness + * affects the {@link JsonReader} and you can look at + * {@link JsonWriter#setStrictness(Strictness)} to see how the strictness + * affects the {@link JsonWriter}.

    + * + * @see JsonReader#setStrictness(Strictness) + * @see JsonWriter#setStrictness(Strictness) + * @since $next-version$ + */ +public enum Strictness { + /** Allow large deviations from the JSON specification. */ + LENIENT, + + /** Allow certain small deviations from the JSON specification for legacy reasons. */ + LEGACY_STRICT, + + /** Strict compliance with the JSON specification. */ + STRICT +} diff --git a/gson/src/main/java/com/google/gson/TypeAdapter.java b/gson/src/main/java/com/google/gson/TypeAdapter.java index 5fdea225a5..d15b1e080c 100644 --- a/gson/src/main/java/com/google/gson/TypeAdapter.java +++ b/gson/src/main/java/com/google/gson/TypeAdapter.java @@ -134,10 +134,10 @@ public TypeAdapter() { /** * Converts {@code value} to a JSON document and writes it to {@code out}. - * Unlike Gson's similar {@link Gson#toJson(JsonElement, Appendable) toJson} - * method, this write is strict. Create a {@link - * JsonWriter#setLenient(boolean) lenient} {@code JsonWriter} and call - * {@link #write(JsonWriter, Object)} for lenient writing. + * The strictness {@link Strictness#LEGACY_STRICT} is used for writing the JSON data. + * To use a different strictness setting create a {@link JsonWriter}, call its + * {@link JsonWriter#setStrictness(Strictness)} method and then use + * {@link #write(JsonWriter, Object)} for writing. * * @param value the Java object to convert. May be null. * @since 2.2 @@ -207,10 +207,11 @@ public final TypeAdapter nullSafe() { } /** - * Converts {@code value} to a JSON document. Unlike Gson's similar {@link - * Gson#toJson(Object) toJson} method, this write is strict. Create a {@link - * JsonWriter#setLenient(boolean) lenient} {@code JsonWriter} and call - * {@link #write(JsonWriter, Object)} for lenient writing. + * Converts {@code value} to a JSON document. + * The strictness {@link Strictness#LEGACY_STRICT} is used for writing the JSON data. + * To use a different strictness setting create a {@link JsonWriter}, call its + * {@link JsonWriter#setStrictness(Strictness)} method and then use + * {@link #write(JsonWriter, Object)} for writing. * * @throws JsonIOException wrapping {@code IOException}s thrown by {@link #write(JsonWriter, Object)} * @param value the Java object to convert. May be null. @@ -253,10 +254,10 @@ public final JsonElement toJsonTree(T value) { public abstract T read(JsonReader in) throws IOException; /** - * Converts the JSON document in {@code in} to a Java object. Unlike Gson's - * similar {@link Gson#fromJson(Reader, Class) fromJson} method, this - * read is strict. Create a {@link JsonReader#setLenient(boolean) lenient} - * {@code JsonReader} and call {@link #read(JsonReader)} for lenient reading. + * Converts the JSON document in {@code in} to a Java object. The strictness + * {@link Strictness#LEGACY_STRICT} is used for reading the JSON data. To use a different + * strictness setting create a {@link JsonReader}, call its {@link JsonReader#setStrictness(Strictness)} + * method and then use {@link #read(JsonReader)} for reading. * *

    No exception is thrown if the JSON data has multiple top-level JSON elements, * or if there is trailing data. @@ -270,10 +271,10 @@ public final T fromJson(Reader in) throws IOException { } /** - * Converts the JSON document in {@code json} to a Java object. Unlike Gson's - * similar {@link Gson#fromJson(String, Class) fromJson} method, this read is - * strict. Create a {@link JsonReader#setLenient(boolean) lenient} {@code - * JsonReader} and call {@link #read(JsonReader)} for lenient reading. + * Converts the JSON document in {@code json} to a Java object. The strictness + * {@link Strictness#LEGACY_STRICT} is used for reading the JSON data. To use a different + * strictness setting create a {@link JsonReader}, call its {@link JsonReader#setStrictness(Strictness)} + * method and then use {@link #read(JsonReader)} for reading. * *

    No exception is thrown if the JSON data has multiple top-level JSON elements, * or if there is trailing data. diff --git a/gson/src/main/java/com/google/gson/stream/JsonReader.java b/gson/src/main/java/com/google/gson/stream/JsonReader.java index de7aef5ff5..0d106a1ccc 100644 --- a/gson/src/main/java/com/google/gson/stream/JsonReader.java +++ b/gson/src/main/java/com/google/gson/stream/JsonReader.java @@ -16,6 +16,7 @@ package com.google.gson.stream; +import com.google.gson.Strictness; import com.google.gson.internal.JsonReaderInternalAccess; import com.google.gson.internal.TroubleshootingGuide; import com.google.gson.internal.bind.JsonTreeReader; @@ -27,7 +28,7 @@ import java.util.Objects; /** - * Reads a JSON (RFC 7159) + * Reads a JSON (RFC 8259) * encoded value as a stream of tokens. This stream includes both literal * values (strings, numbers, booleans, and nulls) as well as the begin and * end delimiters of objects and arrays. The tokens are traversed in @@ -181,7 +182,7 @@ *

    Prefixing JSON files with ")]}'\n" makes them non-executable * by {@code