Consider annotations and compilation of .json files

[NoelAbrahams]

Motivation

The shape of JSON stored in files can be validated using JSON schema. However, JSON schema cannot be used to ensure the stored JSON corresponds to a specific TypeScript interface.

Understandably, most JSON tends to be stored in databases. However, there are many cases where small documents are stored on the file system in order to permit quick editing. Two common examples are config and resource files. The JSON in these files is loaded at runtime and accessed by JavaScript code, thereby necessitating they correspond to specific TypeScript interfaces.

We have a small but significant use case for this scenario.

Proposal

Interface declarations file user.d.ts

declare module user {

     interface Address {
           house: number;
           street: string;
    }

    interface User  {
          name: string;
          address:Address;
          age: number;
    }
}

TypeScript JSON file user.json.ts

/// <reference path="user.d.ts" />
user.User: {            <----------- Annotation
   "name": "Joe Bloggs", /*Comment*/    // Permit comments
   "address": {
       "house": "1", // Error: Number expected
       street: "London"  // Error: Names must be quoted
    },
   "age": 55     // Error: Missing comma
    "id": 101 // Error: Invalid property
}

Compiled output (after errors are corrected): File user.json

{
   "name": "Joe Bloggs",
   "address": {
       "house": 1
       "street": "London"
    },
   "age": 55    
}

[ahejlsberg]

This is an idea we've discussed before and I think it has a lot of merit. I would propose that the type name be specified as part of the /// <reference/> comment so that the core JSON syntax doesn't have to be extended. For example:

/// <reference path="user.d.ts" type="user.User"/>
{
    "name": "Joe Bloggs",
    ...
}

A number of JSON processing tools already tolerate comments (but definitely wouldn't tolerate actual identifiers before the object literal).

[ahejlsberg]

I should clarify... The thing I find useful here is the type checking and associated IDE services such as statement completion. I'm less sure about producing "corrected" output.

[NoelAbrahams]

@ahejlsberg, no I meant "after the user has corrected the errors", not for the compiler to correct it.

If I understand you correctly, with the /// <reference/> option, just dropping that into an existing .json file would cause the compiler to include it in the compilation run.

This is cleaner than having to generate emit, but would require stripping out the /// <reference/> manually. Furthermore, one cannot simply require the file in node.

var json = require('user.json');  // SyntaxError: user.json:  Unexpected token /

I like the idea of adding the type annotation in the reference, but would like user.json.ts to emit user.json with the <reference> and other comments stripped out.

[duanyao]

Currently I use type assertion syntax to enforce type check on a object literal:

<user.User>{
   "name": "Joe Bloggs",
   "address": {
       "house": 1,
       "street": "London",
    },
   "age": 55    
}

Is this what you want?

[NoelAbrahams]

@duanyao, that is not valid syntax in a .json file.

[duanyao]

After compilation, it is a valid json. You presented an example of user.json.ts, and was compiled to user.json. Did I miss something?

[NoelAbrahams]

Are you suggesting that as an alternative syntax? It's not clear from the statement "Is that what you want?"

The problem with the type assertion syntax is that here we are attempting to annotate JSON - not coerce it into something.

[duanyao]

No. I suggest that you write .json.ts files in TS (of cause), and compile them to .json files.

[NoelAbrahams]

In that case there is nothing new being suggested.

[duanyao]

Why need something new if existing tools can solve the problem? Or are there problems I overlooked?

[NoelAbrahams]

A JSON file is not equivalent to a JavaScript file. There is no top-level variable. That's probably what you're missing.

[duanyao]

Top-level variable is not the issue. I just tested tsc:

<user.User>{
   "name": "Joe Bloggs",
   "address": {
       "house": 1,
       "street": "London",
    },
   "age": 55    
}

was compiled into

{
    "name": "Joe Bloggs",
    "address": {
        "house": 1,
        "street": "London"
    },
    "age": 55
};

You see, the only problem is the trailing ;. I think tsc can be improved to support a "json mode" to remove these artifact.

[NoelAbrahams]

That's interesting. But still won't do. JSON is a subset of JavaScript and things that are permissible in JavaScript is not valid JSON.

<any>{
   'name': 'Joe Bloggs', // e.g. single quotes
   'bar': () => 10
}

[duanyao]

These format issues should also be addressed by "json mode" of tsc. Data type issues like () => 10 can be caught by the type assertion or "json mode".

[NoelAbrahams]

Like I said there is nothing new being suggested.

[NoelAbrahams]

@mhegazy, @RyanCavanaugh any chance of adding this to the discussion list now that .js files are being compiled?

[NoelAbrahams]

Hi, guys, is this on the radar?

[mhegazy]

We have not discussed this in a while. I believe we still need a detailed proposal of how this can be accomplished.

One other issue that seems more appealing is #7071, this requires no annotation, it just allows the compiler to load these files and infer their types. you would get an error if you were to assign the contents of the file to a variable of an unmatching type. obviously this only works for node, so that might not be what you are looking for.

[NoelAbrahams]

@mhegazy, #7071 is a special case that unfortunately doesn't make things any better from our perspective.

The use-case I outlined is to do with loading JSON resources via HTTP and using them on a client application. We just want to ensure the types used in the client correspond to the JSON file stored on the server. I'd say this is a common requirement.

The only way I can think of to accomplish this at the moment is to have a new file type foo.json.ts and have the compiler emit foo.json as outlined above.

[RyanCavanaugh]

I think the use case here is well-handled by a few things now:

• JSDoc type annotations in JS files
• import of json "works" as "expected" now (with a smaller inference surface)
• io-ts exists

Given lack of other feedback here, I don't see us proceeding anytime soon.

[matthew-dean]

Note that JSON type inference doesn't always work as expected, and can mis-match types, even when the data in a JSON exactly matches an interface. See: #54488