Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: convert utils to typescript #13782

Merged
merged 57 commits into from
Feb 25, 2022
Merged

refactor: convert utils to typescript #13782

merged 57 commits into from
Feb 25, 2022

Conversation

sdepold
Copy link
Member

@sdepold sdepold commented Dec 17, 2021

Pull Request Checklist

Please make sure to review and check all of these items:

  • Have you added new tests to prevent regressions?
  • Does npm run test or npm run test-DIALECT pass with this change (including linting)?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
  • Did you update the typescript typings accordingly (if applicable)?
  • Does the description below contain a link to an existing issue (Closes #[issue]) or a description of the issue you are solving?
  • Did you follow the commit message conventions explained in CONTRIBUTING.md?

Description Of Change

Conversion of lib/utils to typescript and splitting into smaller chunks.

ephys
ephys previously requested changes Dec 23, 2021
Copy link
Member

@ephys ephys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The majority of my review is about making the types stricker or add hints for TypeScript.

Some of them are based on my understanding of the methods, so a double review is welcome

lib/utils/array.ts Outdated Show resolved Hide resolved
lib/utils/check.ts Outdated Show resolved Hide resolved
lib/utils/check.ts Outdated Show resolved Hide resolved
lib/utils/dialect.ts Outdated Show resolved Hide resolved
lib/utils/dialect.ts Outdated Show resolved Hide resolved
lib/utils/object.ts Outdated Show resolved Hide resolved
lib/utils/sequelize-method.ts Outdated Show resolved Hide resolved
lib/utils/sequelize-method.ts Outdated Show resolved Hide resolved
lib/utils/sequelize-method.ts Outdated Show resolved Hide resolved
lib/utils/object.ts Outdated Show resolved Hide resolved
@ephys ephys added the type: typescript For issues and PRs. Things that involve typescript, such as typings and intellisense. label Jan 3, 2022
@ephys
Copy link
Member

ephys commented Jan 12, 2022

I've resolved the issue you had with this PR being dependant on the DataTypes PR.

Solution was the same as my own TS conversion PR:

  • Never import anything from /types. If you need something from /types, import from index.js instead. Example
  • Same logic if something in /types needs to import from /lib. Example
  • I set allowJs to false in tsconfig. If you need to import from a JS file, use require instead of import

If you're ok with it I can push my changes to this branch (edit: done). They include a resolution for the merge conflict + these changes. It compiles now :)

package.json Outdated Show resolved Hide resolved
yarn.lock Show resolved Hide resolved
@ephys ephys self-assigned this Jan 13, 2022
@ephys
Copy link
Member

ephys commented Jan 13, 2022

Working a bit more on this currently

I'm changing part of mapWhereFieldNames because this piece of code is going to cause subtle bugs for a model that looks like this:

sequelize.define('User', { 
  firstName: {
    type: DataTypes.STRING,
    field: 'name' // this is also the name of another attribute but it should work because User.name uses "name2" as a column name!
  },
  name: {
    type: DataTypes.STRING,
    field: 'name2'
  }
});

@ephys
Copy link
Member

ephys commented Jan 14, 2022

Damnit. I broke TypeScript < 4.4 because they finally allowed symbol to be used as keys in 4.4.

And it also broke in TS < 4.1 because I used template string types again but I'll wait until sequelize/meetings#6 (comment) has been decided to change it

@ephys
Copy link
Member

ephys commented Feb 11, 2022

There is something fishy going on in this PR. Some of our tests are randomly failing. I've seen sqlite fail, then postgres in a re-run.

@ephys
Copy link
Member

ephys commented Feb 11, 2022

For future me: https://github.com/sequelize/sequelize/pull/13992/files#diff-3274f1a37032fb0ae4e2823def0007c634e869ae0dfc304ff6a12c36513c3a52R271 this is the code that I need to include in the new utils when resolving the new conflict

@sdepold
Copy link
Member Author

sdepold commented Feb 11, 2022

There is something fishy going on in this PR. Some of our tests are randomly failing. I've seen sqlite fail, then postgres in a re-run.

In also some failing tests With mariadb and than something else on my other pr

@ephys
Copy link
Member

ephys commented Feb 11, 2022

So probably every dialect, but at different times. Very weird.

I'll keep a list of failing suites here so we can investigate whether they have something in common:
https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true

# postgres 9.5 (node 12) (minified aliases)
  1) [POSTGRES Specific] DAO
       enums
         ARRAY(ENUM)
           should be able to insert a new record even with an array of enums in a schema:
     schema "special_schema" does not exist
  Error
      at Query.run (lib/dialects/postgres/query.js:55:25)
      at /home/runner/work/sequelize/sequelize/lib/sequelize.js:315:28
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (internal/process/task_queues.js:97:5)
      at async /home/runner/work/sequelize/sequelize/lib/dialects/postgres/query-interface.js:104:76
      at async /home/runner/work/sequelize/sequelize/lib/dialects/postgres/query-interface.js:104:96
      at async PostgresQueryInterface.ensureEnums (lib/dialects/postgres/query-interface.js:104:20)
      at async PostgresQueryInterface.createTable (lib/dialects/abstract/query-interface.js:85:5)
      at async Function.sync (lib/model.js:938:5)
      at async Context.<anonymous> (test/integration/dialects/postgres/dao.test.js:490:11)

https://github.com/sequelize/sequelize/runs/5152023145?check_suite_focus=true

This one I've seen fail in other PRs, so unrelated to this one

# sqlite (node 12)
 1) [SQLITE] Transaction
       fails with SQLITE_BUSY when retry.match is changed:
     AssertionError: expected promise to be rejected with an error including 'SQLITE_BUSY: database is locked' but it was fulfilled with [ undefined, undefined ]
      at assertIfNotNegated (node_modules/chai-as-promised/lib/chai-as-promised.js:68:19)
      at /home/runner/work/sequelize/sequelize/node_modules/chai-as-promised/lib/chai-as-promised.js:183:17
      at async Context.<anonymous> (test/integration/transaction.test.js:739:9)

https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true

# mariadb 10.5 (node 12)

 1) [MARIADB] Model
       scope
         complex merge
           should merge complex scopes with findAll options correctly regardless of their order:

      AssertionError: expected { id: 1, name: 'foo1', bars: [ …(2) ] } to deeply equal { id: 1, name: 'foo1', bars: [ …(2) ] }
      + expected - actual

           {
             "bazs": [
               {
                 "barId": 1
      -          "id": 2
      +          "id": 1
                 "quxes": [
                   {
      -              "bazId": 2
      -              "id": 1
      -              "name": "qux3"
      +              "bazId": 1
      +              "id": 3
      +              "name": "qux1"
                   }
                   {
      -              "bazId": 2
      -              "id": 2
      -              "name": "qux4"
      +              "bazId": 1
      +              "id": 4
      +              "name": "qux2"
                   }
                 ]
               }
               {
                 "barId": 1
      -          "id": 1
      +          "id": 2
                 "quxes": [
                   {
      -              "bazId": 1
      -              "id": 3
      -              "name": "qux1"
      +              "bazId": 2
      +              "id": 1
      +              "name": "qux3"
                   }
                   {
      -              "bazId": 1
      -              "id": 4
      -              "name": "qux2"
      +              "bazId": 2
      +              "id": 2
      +              "name": "qux4"
                   }
                 ]
               }
             ]
      
      at Proxy.assertEql (node_modules/chai/lib/chai/core/assertions.js:1096:10)
      at Proxy.methodWrapper (node_modules/chai/lib/chai/utils/addMethod.js:57:25)
      at doAsserterAsyncAndAddThen (node_modules/chai-as-promised/lib/chai-as-promised.js:289:22)
      at Proxy.<anonymous> (node_modules/chai-as-promised/lib/chai-as-promised.js:255:20)
      at Proxy.overwritingMethodWrapper (node_modules/chai/lib/chai/utils/overwriteMethod.js:78:33)
      at Proxy.assertEqual (node_modules/chai/lib/chai/core/assertions.js:1035:12)
      at Proxy.methodWrapper (node_modules/chai/lib/chai/utils/addMethod.js:57:25)
      at doAsserterAsyncAndAddThen (node_modules/chai-as-promised/lib/chai-as-promised.js:289:22)
      at Proxy.<anonymous> (node_modules/chai-as-promised/lib/chai-as-promised.js:255:20)
      at Proxy.overwritingMethodWrapper (node_modules/chai/lib/chai/utils/overwriteMethod.js:78:33)
      at Context.<anonymous> (test/integration/model/scope/merge.test.js:185:43)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (internal/process/task_queues.js:97:5)

@WikiRik
Copy link
Member

WikiRik commented Feb 11, 2022

I'll keep a list of failing suites here so we can investigate whether they have something in common: https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true https://github.com/sequelize/sequelize/runs/5152023145?check_suite_focus=true https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true

The logs are only available for a limited period (I thought 1 week but haven't checked) so download them before they're gone (or fix it before ofc)

sdepold and others added 2 commits February 12, 2022 17:56
* test: cover isColString with tests

* test: cover remaining check util functions

* refactor: improve importing of check module

Co-authored-by: Sascha Depold <sdepold@ebay.com>
@ephys
Copy link
Member

ephys commented Feb 22, 2022

I've copy-pasted the relevant CI logs in my previous comment to make sure we don't lose them

I've also resolved the merge conflict

@sdepold if you have time, do you mind if I let you take over the remaining of this PR now that the TS part is done?

@sdepold
Copy link
Member Author

sdepold commented Feb 23, 2022

1) [SQLITE] Transaction
[2387](https://github.com/sequelize/sequelize/runs/5152023145?check_suite_focus=true#step:6:2387)
       fails with SQLITE_BUSY when retry.match is changed:
[2388](https://github.com/sequelize/sequelize/runs/5152023145?check_suite_focus=true#step:6:2388)
     AssertionError: expected promise to be rejected with an error including 'SQLITE_BUSY: database is locked' but it was fulfilled with [ undefined, undefined ]
[2389](https://github.com/sequelize/sequelize/runs/5152023145?check_suite_focus=true#step:6:2389)
      at assertIfNotNegated (node_modules/chai-as-promised/lib/chai-as-promised.js:68:19)
[2390](https://github.com/sequelize/sequelize/runs/5152023145?check_suite_focus=true#step:6:2390)
      at /home/runner/work/sequelize/sequelize/node_modules/chai-as-promised/lib/chai-as-promised.js:183:17
[2391](https://github.com/sequelize/sequelize/runs/5152023145?check_suite_focus=true#step:6:2391)
      at async Context.<anonymous> (test/integration/transaction.test.js:739:9)
[2392](https://github.com/sequelize/sequelize/runs/5152023145?check_suite_focus=true#step:6:2392)
1) [POSTGRES Specific] DAO
[2865](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2865)
       enums
[2866](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2866)
         ARRAY(ENUM)
[2867](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2867)
           should be able to insert a new record even with an array of enums in a schema:
[2868](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2868)
     schema "special_schema" does not exist
[2869](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2869)
  Error
[2870](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2870)
      at Query.run (lib/dialects/postgres/query.js:55:25)
[2871](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2871)
      at /home/runner/work/sequelize/sequelize/lib/sequelize.js:315:28
[2872](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2872)
      at runMicrotasks (<anonymous>)
[2873](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2873)
      at processTicksAndRejections (internal/process/task_queues.js:97:5)
[2874](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2874)
      at async /home/runner/work/sequelize/sequelize/lib/dialects/postgres/query-interface.js:104:76
[2875](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2875)
      at async /home/runner/work/sequelize/sequelize/lib/dialects/postgres/query-interface.js:104:96
[2876](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2876)
      at async PostgresQueryInterface.ensureEnums (lib/dialects/postgres/query-interface.js:104:20)
[2877](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2877)
      at async PostgresQueryInterface.createTable (lib/dialects/abstract/query-interface.js:85:5)
[2878](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2878)
      at async Function.sync (lib/model.js:938:5)
[2879](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2879)
      at async Context.<anonymous> (test/integration/dialects/postgres/dao.test.js:490:11)
[2880](https://github.com/sequelize/sequelize/runs/5155830293?check_suite_focus=true#step:9:2880)
1) [MARIADB] Model
[2585](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2585)
       scope
[2586](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2586)
         complex merge
[2587](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2587)
           should merge complex scopes with findAll options correctly regardless of their order:
[2588](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2588)

[2589](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2589)
      AssertionError: expected { id: 1, name: 'foo1', bars: [ …(2) ] } to deeply equal { id: 1, name: 'foo1', bars: [ …(2) ] }
[2590](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2590)
      + expected - actual
[2591](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2591)

[2592](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2592)
           {
[2593](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2593)
             "bazs": [
[2594](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2594)
               {
[2595](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2595)
                 "barId": 1
[2596](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2596)
      -          "id": 2
[2597](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2597)
      +          "id": 1
[2598](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2598)
                 "quxes": [
[2599](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2599)
                   {
[2600](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2600)
      -              "bazId": 2
[2601](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2601)
      -              "id": 1
[2602](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2602)
      -              "name": "qux3"
[2603](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2603)
      +              "bazId": 1
[2604](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2604)
      +              "id": 3
[2605](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2605)
      +              "name": "qux1"
[2606](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2606)
                   }
[2607](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2607)
                   {
[2608](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2608)
      -              "bazId": 2
[2609](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2609)
      -              "id": 2
[2610](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2610)
      -              "name": "qux4"
[2611](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2611)
      +              "bazId": 1
[2612](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2612)
      +              "id": 4
[2613](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2613)
      +              "name": "qux2"
[2614](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2614)
                   }
[2615](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2615)
                 ]
[2616](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2616)
               }
[2617](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2617)
               {
[2618](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2618)
                 "barId": 1
[2619](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2619)
      -          "id": 1
[2620](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2620)
      +          "id": 2
[2621](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2621)
                 "quxes": [
[2622](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2622)
                   {
[2623](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2623)
      -              "bazId": 1
[2624](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2624)
      -              "id": 3
[2625](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2625)
      -              "name": "qux1"
[2626](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2626)
      +              "bazId": 2
[2627](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2627)
      +              "id": 1
[2628](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2628)
      +              "name": "qux3"
[2629](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2629)
                   }
[2630](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2630)
                   {
[2631](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2631)
      -              "bazId": 1
[2632](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2632)
      -              "id": 4
[2633](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2633)
      -              "name": "qux2"
[2634](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2634)
      +              "bazId": 2
[2635](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2635)
      +              "id": 2
[2636](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2636)
      +              "name": "qux4"
[2637](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2637)
                   }
[2638](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2638)
                 ]
[2639](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2639)
               }
[2640](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2640)
             ]
[2641](https://github.com/sequelize/sequelize/runs/5156282992?check_suite_focus=true#step:8:2641)

@ephys
Copy link
Member

ephys commented Feb 24, 2022

The MariaDB test has failed in another PR https://github.com/sequelize/sequelize/runs/5320675964?check_suite_focus=true, which means it's also unrelated to this PR

Only the postgres one remains

@WikiRik
Copy link
Member

WikiRik commented Feb 24, 2022

I have seen the Postgres one as well I believe. At least the 'schema does not exist' in DAO tests. Not sure about this specific test

@ephys
Copy link
Member

ephys commented Feb 24, 2022

Then I think we can merge this PR. I'll resolve the merge conflict

Documented here: #14161

Copy link
Member

@WikiRik WikiRik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few small remarks/questions before we merge this

package.json Outdated Show resolved Hide resolved
src/utils/dialect.ts Outdated Show resolved Hide resolved
src/utils/format.ts Show resolved Hide resolved
src/utils/sequelize-method.ts Show resolved Hide resolved
src/utils/sequelize-method.ts Show resolved Hide resolved
Copy link
Member

@WikiRik WikiRik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@ephys ephys merged commit 5e92477 into main Feb 25, 2022
@ephys ephys deleted the ts-utils-index branch February 25, 2022 16:16
@github-actions
Copy link
Contributor

🎉 This PR is included in version 7.0.0-alpha.11 🎉

The release is available on:

Your semantic-release bot 📦🚀

vanthome pushed a commit to vanthome/sequelize that referenced this pull request Jun 12, 2022
Co-authored-by: ephys <zoe@ephys.dev>
Co-authored-by: Rik Smale <13023439+WikiRik@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
released on @v7 type: typescript For issues and PRs. Things that involve typescript, such as typings and intellisense.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants