feat: migrate updateQuery to typescript #17063

lohart13 · 2024-02-07T11:23:00Z

Pull Request Checklist

Have you added new tests to prevent regressions?
If a documentation update is necessary, have you opened a PR to the documentation repository?
Did you update the typescript typings accordingly (if applicable)?
Does the description below contain a link to an existing issue (Closes #[issue]) or a description of the issue you are solving?
Does the name of your PR follow our conventions?

Description Of Change

This PR migrates updateQuery methods to typescript.

ephys

Preliminary review but the few comments I posted already request some pretty big changes to update

packages/core/src/dialects/abstract/index.ts

packages/core/src/dialects/abstract/query-generator-internal.ts

packages/core/src/dialects/abstract/query-generator-typescript.ts

packages/core/src/dialects/abstract/query-interface-typescript.ts

lohart13 · 2024-02-12T07:32:45Z

Looking into merging update and bulkUpdate it seems there is some work required to remove the Model instance building from query.js and only return the raw database values as the model.save method needs some work in order to work with raw values rather than an instance.

I'd propose to fix these methods once #16988 has been merged.

I'll update #16988 to exclude the changes to the queryInterface

packages/core/src/utils/iterators.ts

ephys

As always, thank you so much for your work!

I have yet to review sqlite & mssql but I already have a few comments so I'll take a break here

ephys · 2024-02-18T10:39:45Z

packages/core/src/dialects/abstract/query-generator-internal.ts

+    const returnFields: string[] = [];
+
+    if (Array.isArray(options.returning)) {
+      returnFields.push(...options.returning.map(field => {
+        if (typeof field === 'string') {
+          return this.queryGenerator.quoteIdentifier(field);
+        } else if (field instanceof BaseSqlExpression) {
+          return this.queryGenerator.formatSqlExpression(field, options);
+        }
+
+        throw new Error(`Unsupported value in "returning" option: ${inspect(field)}. This option only accepts true, false, an array of strings, or a sql expression.`);
+      }));
+    } else if (modelDefinition) {
+      for (const [name, attribute] of modelDefinition.physicalAttributes) {
+        returnFields.push(this.queryGenerator.quoteIdentifier(attribute.columnName ?? name));
+      }
+    }


Small optimization, as this function is called often:

assigned returnFields to the result of options.returning.map to avoid creating an unused array

used .values() instead of iterating the physicalAttributes map directly, as columnName is guaranteed to be provided, and creating an array for each entry of the map is expensive

Suggested change

const returnFields: string[] = [];

if (Array.isArray(options.returning)) {

returnFields.push(...options.returning.map(field => {

if (typeof field === 'string') {

return this.queryGenerator.quoteIdentifier(field);

} else if (field instanceof BaseSqlExpression) {

return this.queryGenerator.formatSqlExpression(field, options);

}

throw new Error(`Unsupported value in "returning" option: ${inspect(field)}. This option only accepts true, false, an array of strings, or a sql expression.`);

}));

} else if (modelDefinition) {

for (const [name, attribute] of modelDefinition.physicalAttributes) {

returnFields.push(this.queryGenerator.quoteIdentifier(attribute.columnName ?? name));

}

}

let returnFields: string[];

if (Array.isArray(options.returning)) {

returnFields = options.returning.map(field => {

if (typeof field === 'string') {

return this.queryGenerator.quoteIdentifier(field);

} else if (field instanceof BaseSqlExpression) {

return this.queryGenerator.formatSqlExpression(field, options);

}

throw new Error(`Unsupported value in "returning" option: ${inspect(field)}. This option only accepts true, false, an array of strings, or a sql expression.`);

});

} else {

returnFields = [];

if (modelDefinition) {

for (const attribute of modelDefinition.physicalAttributes.values()) {

returnFields.push(this.queryGenerator.quoteIdentifier(attribute.columnName));

}

}

}

I've updated this now, LMK if it looks OK.

packages/core/src/dialects/abstract/index.ts

packages/core/src/dialects/abstract/query-generator-typescript.ts

ephys · 2024-02-18T10:53:40Z

packages/core/src/dialects/abstract/query-generator-typescript.ts

+    }
+
+    const bind = Object.create(null);
+    const valueHash = removeNullishValuesFromHash(attrValueHash, this.options.omitNull ?? false);


I'd like to slowly phase out removeNullishValuesFromHash and the omitNull option

Could you move its use to the places that use updateQuery instead of keeping it here? That will make it possible to not include it in the model method rewrite that's in progress, while keeping it in the old model methods

I've moved the removeNullishValuesFromHash logic to the model methods.

ephys · 2024-02-18T11:19:19Z

packages/core/src/dialects/mssql/query-generator-internal.ts

+        } else if (field instanceof Literal) {
+          // Due to how the mssql query is built, using a literal would never result in a properly formed query.
+          // It's better to warn early.
+          throw new Error(`literal() cannot be used in the "returning" option array in ${this.dialect.name}. Use col(), or a string instead.`);


This check is a bit outdated. The logic should accept instances of Col, and strings, but ban all other values, not just Literal

Updated the logic now

ephys · 2024-02-18T11:19:44Z

packages/core/src/dialects/mssql/query-generator-internal.ts

+    const returnFields: string[] = [];
+
+    if (Array.isArray(options.returning)) {
+      returnFields.push(...options.returning.map(field => {


Comments made on the parent method apply here too

Implementation update now.

ephys · 2024-02-18T11:20:35Z

packages/core/src/dialects/mssql/query-generator-internal.ts

+
+  generateOutputTableFragment(returnFields: string[], modelDefinition?: ModelDefinition | null): string {
+    if (!(modelDefinition instanceof ModelDefinition)) {
+      throw new Error('Using returning with triggers requires specifying a model or model definition.');


This check is specific to where this method was used. modelDefinition should be made mandatory here, and this check should be moved to where this method is called

Updated now.

ephys · 2024-02-18T11:29:36Z

packages/core/src/dialects/mssql/query-generator-internal.ts

+
+    const columnsToExclude = new Set<string>();
+    const attributes = map(filter(returnFields, field => field !== '*'), field => {
+      const unquotedColumn = field.replace(/^\[/, '').replace(/\]$/, '');


Quoting then unquoting is a bad idea and will break for identifiers that require escaping

The issue is that getReturnFields handles both formatting returning, & setting its default value.

You need to first use a function called normalizeReturning which:

if returning is true: sets it to the list of model columns when a model is available, or * otherwise

if returning is false: sets it to EMPTY_ARRAY

if returning is an array: leaves it as-is

Then getReturnFields can become formatReturnField, which only handles serialization

I agree and have updated the implementation now.

packages/core/src/dialects/ibmi/query-generator-typescript.ts

WikiRik · 2024-02-18T18:21:20Z

Heads up for these and your other open PRs; we've moved to prettier in #17101 which causes a lot of merge conflicts. When resolving you can just accept your changes first and then run yarn format to fix the differences

ephys · 2024-02-18T18:54:26Z

To add to this, very sorry for all the conflicts I've been introducing recently

WikiRik · 2024-03-22T15:29:05Z

@lohart13 @ephys just so that we are on the same page; is this order of when we want to review/merge PRs correct?

fix: unify returning queries #17157 and fix: make BaseSqlExpression a unique class #17158 (these are interchangeable from each other)
feat: migrate updateQuery to typescript #17063
feat: migrate bulkInsertQuery & insertQuery to ts #16988
feat: migrate index to typescript #16227

Your other PRs are on hold while we try to focus efforts on releasing v7(-beta) soonish. Read more about our plans here; https://github.com/sequelize/meetings/wiki/Meeting-2024%E2%80%9003%E2%80%9022

lohart13 · 2024-03-23T00:47:01Z

@lohart13 @ephys just so that we are on the same page; is this order of when we want to review/merge PRs correct?

fix: unify returning queries #17157 and fix: make BaseSqlExpression a unique class #17158 (these are interchangeable from each other)

feat: migrate updateQuery to typescript #17063

feat: migrate bulkInsertQuery & insertQuery to ts #16988

feat: migrate index to typescript #16227

Your other PRs are on hold while we try to focus efforts on releasing v7(-beta) soonish. Read more about our plans here; https://github.com/sequelize/meetings/wiki/Meeting-2024%E2%80%9003%E2%80%9022

@WikiRik, yes, this order is correct.

Out of interest, when would it be likely to have PR feat: add DataTypes.DATE.PLAIN #15565 reviewed/merged?

lohart13 · 2024-03-23T01:15:11Z

@WikiRik I've also add PR #17191 which can be merged at a similar time to #17158 and #17157

lohart13 force-pushed the update-ts branch 2 times, most recently from 6c58304 to 1c4e81d Compare February 8, 2024 09:55

ephys mentioned this pull request Feb 8, 2024

feat: implement Model._UNSTABLE_destroyMany #17031

Merged

5 tasks

lohart13 force-pushed the update-ts branch 2 times, most recently from 2cf0c07 to 91da630 Compare February 10, 2024 01:39

lohart13 changed the title ~~feat: migrate updateQuery to typescript~~ feat: migrate update to typescript Feb 10, 2024

lohart13 force-pushed the update-ts branch from 91da630 to f643234 Compare February 10, 2024 03:32

ephys requested changes Feb 10, 2024

View reviewed changes

ephys mentioned this pull request Feb 11, 2024

feat: add ModelRepository#_UNSTABLE_bulkDestroy and manual ON DELETE handling #17078

Merged

5 tasks

lohart13 force-pushed the update-ts branch from f643234 to 7225cdd Compare February 12, 2024 07:21

lohart13 changed the title ~~feat: migrate update to typescript~~ feat: migrate updateQuery to typescript Feb 12, 2024

lohart13 mentioned this pull request Feb 12, 2024

feat: migrate bulkInsertQuery & insertQuery to ts #16988

Open

6 tasks

ephys reviewed Feb 18, 2024

View reviewed changes

packages/core/src/utils/iterators.ts Outdated Show resolved Hide resolved

ephys requested changes Feb 18, 2024

View reviewed changes

lohart13 force-pushed the update-ts branch from 7225cdd to cc6ca13 Compare February 26, 2024 01:37

lohart13 requested a review from a team as a code owner February 26, 2024 01:37

lohart13 requested review from ephys and WikiRik February 26, 2024 01:37

lohart13 force-pushed the update-ts branch from cc6ca13 to 81656ae Compare February 26, 2024 21:57

lohart13 mentioned this pull request Mar 11, 2024

fix: make BaseSqlExpression a unique class #17158

Merged

5 tasks

lohart13 mentioned this pull request Mar 23, 2024

fix: update bulkDeleteQuery supported options #17191

Merged

5 tasks

sequelize-bot bot added the conflicted This PR has merge conflicts and will not be present in the list of PRs to review label Apr 10, 2024

lohart13 force-pushed the update-ts branch from 81656ae to bc5fcbf Compare April 14, 2024 23:37

sequelize-bot bot removed the conflicted This PR has merge conflicts and will not be present in the list of PRs to review label Apr 14, 2024

lohart13 added 4 commits April 17, 2024 11:49

fix: prefer BaseSqlExpression

338cc6e

fix(mysql): cannot bind LIMIT or OFFSET

0fb8166

feat: migrate updateQuery to ts

9205ce7

meta: update tests

aa43c56

lohart13 force-pushed the update-ts branch from bc5fcbf to aa43c56 Compare April 16, 2024 23:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: migrate updateQuery to typescript #17063

feat: migrate updateQuery to typescript #17063

lohart13 commented Feb 7, 2024 •

edited

ephys left a comment •

edited

lohart13 commented Feb 12, 2024

ephys left a comment

ephys Feb 18, 2024

lohart13 Feb 26, 2024

ephys Feb 18, 2024

lohart13 Feb 26, 2024

ephys Feb 18, 2024

lohart13 Feb 26, 2024

ephys Feb 18, 2024

lohart13 Feb 26, 2024

ephys Feb 18, 2024

lohart13 Feb 26, 2024

ephys Feb 18, 2024

lohart13 Feb 26, 2024

WikiRik commented Feb 18, 2024

ephys commented Feb 18, 2024

WikiRik commented Mar 22, 2024

lohart13 commented Mar 23, 2024

lohart13 commented Mar 23, 2024

feat: migrate updateQuery to typescript #17063

Are you sure you want to change the base?

feat: migrate updateQuery to typescript #17063

Conversation

lohart13 commented Feb 7, 2024 • edited

Pull Request Checklist

Description Of Change

ephys left a comment • edited

Choose a reason for hiding this comment

lohart13 commented Feb 12, 2024

ephys left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WikiRik commented Feb 18, 2024

ephys commented Feb 18, 2024

WikiRik commented Mar 22, 2024

lohart13 commented Mar 23, 2024

lohart13 commented Mar 23, 2024

lohart13 commented Feb 7, 2024 •

edited

ephys left a comment •

edited