Customize toJSON() for custom serialization logic

[davidecaroselli]

Issue Creation Checklist

• I understand that my issue will be automatically closed if I don't fill in the requested information
• I have read the contribution guidelines

Feature Description

Describe the feature you'd like to see implemented

Customize the logic of Model.toJSON() in order to account for most-common use cases.
Via an additional decorator @JsonField we should be able, for example, to decide a custom serializer or being able to hide a field in the resulting JSON.

export class User extends JsonModel<User> {
    @Attribute(DataTypes.INTEGER)
    @PrimaryKey
    declare id: number;

    @Attribute(DataTypes.STRING)
    @JsonField({serializer: (value: string) => value.substring(0, 8) + "********"})
    declare email: string;

    @Attribute(DataTypes.STRING)
    @JsonField(false)
    declare secret: string;

}

Describe why you would like this feature to be added to Sequelize

Sequelize is an amazing tool when it comes to map objects to/from a database, however I feel that adding the "last mile", that is the JSON serialization, will make this project truly powerful and sort of complete.

Is this feature dialect-specific?

• No. This feature is relevant to Sequelize as a whole.
• Yes. This feature only applies to the following dialect(s):

Would you be willing to resolve this issue by submitting a Pull Request?

• Yes, I have the time and I know how to start.
• Yes, I have the time but I will need guidance.
• No, I don't have the time, but my company or I are supporting Sequelize through donations on OpenCollective.
• No, I don't have the time, and I understand that I will need to wait until someone from the community or maintainers is interested in implementing my feature.

---

Indicate your interest in the addition of this feature by adding the 👍 reaction. Comments such as "+1" will be removed.

[davidecaroselli]

I have already implemented a Proof-of-concept that can be integrated with a PR, however I'm very new to the project and I don't know where to start and the best place to inject this logic.

In the meantime, this is the additional module I created in my project (a test script can be found at the bottom as well):

import {InferAttributes, InferCreationAttributes, Model} from "@sequelize/core";
import {camelCase, snakeCase} from "case-anything";

export type ToJsonOptions = {
    keepNulls?: boolean;  // Whether to keep null values or not (default is false).
    case?: "camel" | "snake";  // The case to use for field names, default is to keep field name as is.
}

export type JsonFieldOptions = {
    visible?: boolean; // Whether the column is visible or not (default is true).
    serializer?: (value: any) => any;  // The custom serializer function for the column.
}

class JsonSchema {
    hiddenFields: Set<string> = new Set();
    serializers: Map<string, (value: any) => any> = new Map();

    static getOrCreate(target: Function): JsonSchema {
        return target["__json_schema"] = target["__json_schema"] || new JsonSchema();
    }

    static get(target: unknown): JsonSchema | undefined {
        return (target instanceof Function) ? target["__json_schema"] : undefined;
    }
}

export const JsonField = function (visible?: boolean | JsonFieldOptions, options?: JsonFieldOptions): Function {
    return function JsonField(target: Object, propertyName: PropertyKey): void {
        options = options || {};

        if (typeof visible === "object") {
            options = visible;
        } else if (typeof visible === "boolean") {
            options.visible = visible;
        }

        const key = propertyName.toString();
        const schema = JsonSchema.getOrCreate(target.constructor);
        if (options.visible === false) schema.hiddenFields.add(key);
        if (options.serializer) schema.serializers.set(key, options.serializer);
    }
}

export default abstract class JsonModel<M extends Model> extends Model<InferAttributes<M>, InferCreationAttributes<M>> {
    toJSON(options?: ToJsonOptions): object {
        return toJSON(this.get(), this.constructor, options);
    }
}

function isNull(value: any): boolean {
    return value === null || value === undefined;
}

function toJSON(modelData: object, modelClass: Function, options?: ToJsonOptions): { [key: string]: any } {
    const schema = JsonSchema.get(modelClass);
    const json: { [key: string]: any } = {};

    for (const name in modelData) {
        if (schema?.hiddenFields.has(name)) continue;

        let value: any = modelData[name];
        let key = name;

        switch (options?.case) {
            case "camel":
                key = camelCase(name);
                break;
            case "snake":
                key = snakeCase(name);
                break;
        }

        if (isNull(value)) {
            if (!options?.keepNulls) continue;
            json[key] = null;
        } else {
            const serializer = schema?.serializers.get(name);

            if (serializer) {
                json[key] = serializer(value);
            } else if (Array.isArray(value)) {
                json[key] = value.map(obj => toJSON(obj, obj?.constructor, options));
            } else if (value instanceof Model) {
                json[key] = toJSON(value.get(), value.constructor, options);
            } else {
                json[key] = value;
            }
        }
    }

    return json;
}

Here's how you can use it currently:

class User extends JsonModel<User> {
    @Attribute(DataTypes.STRING)
    @JsonField({serializer: (value: string) => value.substring(0, 9) + "********"})
    declare email: string;

    @Attribute(DataTypes.STRING)
    @JsonField(false)
    declare password: string;
}

[...]

const user = await User.create({
    email: "john.doe@gmail.com",
    password: "top-secret-password"
});

console.log(user.get({plain: true}));
// {
//   id: 1,
//   email: 'john.doe@gmail.com',
//   password: 'top-secret-password',
//   updatedAt: 2024-04-12T15:38:49.740Z,
//   createdAt: 2024-04-12T15:38:49.740Z
// }
console.log(user.toJSON());
// {
//   id: 1,
//   email: 'john.doe@********',
//   updatedAt: 2024-04-12T15:38:49.740Z,
//   createdAt: 2024-04-12T15:38:49.740Z
// }

[ephys]

Hi! Thank you for the proof of concept

Unfortunately I still don't think JSON serialization of classes is an ORM concern, and as such does not belong in Sequelize.

This is the sort of feature that should be implemented as its own generic library that works for any class.

If it's necessary to override toJSON, users can easily create a base class, or a plugin system, similar to what is being considered for #15497, could be implemented

[davidecaroselli]

Hi @ephys !

Sure, i see your point about not having to include a JSON serialization logic to Sequelize too..

However.. 😄 Still a Model class has the toJSON() method, which must indicate that the JSON serialization feature is very interesting to the common user of this library, and for sure it was it for me!
While I was going through the documentation, it felt to me just like the ability to customize serialization was there somewhere, yet just out of reach, so close yet needing custom implementation to make it work properly.

With that said, I totally understand if you don't want to integrate such feature, and I find the suggestion to make it a plugin very interesting. Do you have any practical hints for where to "inject" my logic without requiring the user to extend a custom class?

I think it should be as simple as just adding the decorators and that's it! The serialization magically works, like this:

class User extends Model<InferAttributes<User>, InferCreationAttributes<User>> {
    @Attribute(DataTypes.STRING)
    @JsonField({serializer: (value: string) => value.substring(0, 9) + "********"})
    declare email: string;

    @Attribute(DataTypes.STRING)
    @JsonField(false)
    declare password: string;
}

However in order to do this I need to be able to modify the Model class to intercept the toJSON() method, and add an option parameter to it (ToJsonOptions in my example implementation).

Do you see an easy way to do it I'm missing? Is there a plugin architecture in Sequelize that allows to customize its internals from outside?

Thanks!

[ephys]

I agree that serialization is important, what we do not want is to have a sequelize-specific solution that doesn't work for anything else. There are also multiple possible approaches: toJSON is simple but fully synchronous. Other approaches that don't use toJSON could benefit from being asynchronous. In a way, graphql's resolvers are also a way to serialize

If there are libraries that do this, we're happy to link to them from our documentation

There is no plugin system yet, but you could provide a install(sequelize) function that your users need to call once after having initialized Sequelize

In that function, you can hook the afterDefine (or beforeDefine) event in which you replace the toJSON method of the model you reveive

[davidecaroselli]

Thank you for your response!

Indeed, I followed your feedback and I have just created a separate library called Jsonthis that can be very easily connected with Sequelize!

Here's a quick example on how to use it:

function maskEmail(value: string): string {
    return value.replace(/(?<=.).(?=[^@]*?.@)/g, "*");
}

class User extends Model<InferAttributes<User>, InferCreationAttributes<User>> {
    @Attribute(DataTypes.INTEGER)
    @PrimaryKey
    declare id: number;

    @Attribute(DataTypes.STRING)
    @NotNull
    @JsonField({serializer: maskEmail})
    declare email: string;

    @Attribute(DataTypes.STRING)
    @NotNull
    @JsonField(false)
    declare password: string;
}

const jsonthis = new Jsonthis({sequelize});  // Here's where all the magic happens!

const user = await User.create({
    id: 1,
    email: "john.doe@gmail.com",
    password: "s3cret"
});

console.log(user.toJSON());
// {
//   id: 1,
//   email: 'j******e@gmail.com',
//   updatedAt: 2024-04-13T18:00:20.909Z,
//   createdAt: 2024-04-13T18:00:20.909Z
// }