Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Mocha and use new max diff option #1338

Merged
merged 2 commits into from Feb 21, 2022
Merged

Update Mocha and use new max diff option #1338

merged 2 commits into from Feb 21, 2022

Conversation

mattseddon
Copy link
Member

@mattseddon mattseddon commented Feb 21, 2022

Following on from this update in Mocha: mochajs/mocha#4799. This is an attempt at improving the usefulness of the output produced by our Mocha test suite.

The following test:

    it.only('should produce a massive diff', async () => {
      const fun = cloneDeep(staticPlotsFixture)
      delete fun.plots['logs/loss.tsv']

      expect(fun).to.deep.equal(staticPlotsFixture)
    })

Previously produced:

Test
     1) should produce a massive diff:

      AssertionError: expected { Object (plots, sectionName, ...) } to deeply equal { Object (plots, sectionName, ...) }
      + expected - actual

And now produces this output:

Test
     1) should produce a massive diff

  0 passing (2s)

  1 failing

  1) Plots Test Suite
       Plots
         should produce a massive diff:

      AssertionError: expected { Object (plots, sectionName, ...) } to deeply equal { Object (plots, sectionName, ...) }
      + expected - actual

               ]
               "type": "vega"
             }
           ]
      +    "logs/loss.tsv": [
      +      {
      +        "content": {
      +          "$schema": "https://vega.github.io/schema/vega-lite/v4.json"
      +          "data": {
      +            "values": [
      +              {
      +                "loss": "2.298783302307129"
      +                "rev": "workspace"
      +                "step": "0"
      +                "timestamp": "1641966224600"
      +              }
      +              {
      +                "loss": "2.2779736518859863"
      +                "rev": "workspace"
      +                "step": "1"
      +                "timestamp": "1641966239108"
      +              }
      +              {
      +                "loss": "2.2572131156921387"
      +                "rev": "workspace"
      +                "step": "2"
      +                "timestamp": "1641966253176"
      +              }
      +              {
      +                "loss": "2.238112688064575"
      +                "rev": "workspace"
      +                "step": "3"
      +                "timestamp": "1641966267354"
      +              }
      +              {
      +                "loss": "2.212251901626587"
      +                "rev": "workspace"
      +                "step": "4"
      +                "timestamp": "1641966285745"
      +              }
      +              {
      +                "loss": "2.1894900798797607"
      +                "rev": "workspace"
      +                "step": "5"
      +                "timestamp": "1641966303339"
      +              }
      +              {
      +                "loss": "2.165510654449463"
      +                "rev": "workspace"
      +                "step": "6"
      +                "timestamp": "1641966320693"
      +              }
      +              {
      +                "loss": "2.135964870452881"
      +                "rev": "workspace"
      +                "step": "7"
      +                "timestamp": "1641966335781"
      +              }
      +              {
      +                "loss": "2.114135265350342"
      +                "rev": "workspace"
      +                "step": "8"
      +                "timestamp": "1641966351758"
      +              }
      +              {
      +                "loss": "2.298783302307129"
      +                "rev": "main"
      +                "step": "0"
      +                "timestamp": "1641966224600"
      +              }
      +              {
      +                "loss": "2.2779736518859863"
      +                "rev": "main"
      +                "step": "1"
      +                "timestamp": "1641966239108"
      +              }
      +              {
      +                "loss": "2.2572131156921387"
      +                "rev": "main"
      +                "step": "2"
      +                "timestamp": "1641966253176"
      +              }
      +              {
      +                "loss": "2.238112688064575"
      +                "rev": "main"
      +                "step": "3"
      +                "timestamp": "1641966267354"
      +              }
      +              {
      +                "loss": "2.212251901626587"
      +                "rev": "main"
      +                "step": "4"
      +                "timestamp": "1641966285745"
      +              }
      +              {
      +                "loss": "2.1894900798797607"
      +                "rev": "main"
      +                "step": "5"
      +                "timestamp": "1641966303339"
      +              }
      +              {
      +                "loss": "2.165510654449463"
      +                "rev": "main"
      +                "step": "6"
      +                "timestamp": "1641966320693"
      +              }
      +              {
      +                "loss": "2.135964870452881"
      +                "rev": "main"
      +                "step": "7"
      +                "timestamp": "1641966335781"
      +              }
      +              {
      +                "loss": "2.114135265350342"
      +                "rev": "main"
      +                "step": "8"
      +                "timestamp": "1641966351758"
      +              }
      +              {
      +                "loss": "2.0380799770355225"
      +                "rev": "4fb124a"
      +                "step": "0"
      +                "timestamp": "1642041230991"
      +              }
      +              {
      +                "loss": "2.0002100467681885"
      +                "rev": "4fb124a"
      +                "step": "1"
      +                "timestamp": "1642041244696"
      +              }
      +              {
      +                "loss": "1.9573605060577393"
      +                "rev": "4fb124a"
      +                "step": "2"
      +                "timestamp": "1642041257185"
      +              }
      +              {
      +                "loss": "1.91573965549469"
      +                "rev": "4fb124a"
      +                "step": "3"
      +                "timestamp": "1642041270652"
      +              }
      +              {
      +                "loss": "1.8714964389801025"
      +                "rev": "4fb124a"
      +                "step": "4"
      +                "timestamp": "1642041284801"
      +              }
      +              {
      +                "loss": "1.8267308473587036"
      +                "rev": "4fb124a"
      +                "step": "5"
      +                "timestamp": "1642041301919"
      +              }
      +              {
      +                "loss": "1.7825157642364502"
      +                "rev": "4fb124a"
      +                "step": "6"
      +                "timestamp": "1642041318814"
      +              }
      +              {
      +                "loss": "1.7360031604766846"
      +                "rev": "4fb124a"
      +                "step": "7"
      +                "timestamp": "1642041335775"
      +              }
      +              {
      +                "loss": "1.6929490566253662"
      +                "rev": "4fb124a"
      +                "step": "8"
      +                "timestamp": "1642041350855"
      +              }
      +              {
      +                "loss": "1.6454246044158936"

      +                "rev": "42b8736"
      +                "step": "0"
      +                "timestamp": "1642041785966"
      +              }
      +              {
      +                "loss": "1.6063436269760132"
      +                "rev": "42b8736"
      +                "step": "1"
      +                "timestamp": "1642041804111"
      +              }
      +              {
      +                "loss": "1.5570942163467407"
      +                "rev": "42b8736"
      +                "step": "2"
      +                "timestamp": "1642041820386"
      +              }
      +              {
      +                "loss": "1.5230435132980347"
      +                "rev": "42b8736"
      +                "step": "3"
      +                "timestamp": "1642041836358"
      +              }
      +              {
      +                "loss": "1.473145842552185"
      +                "rev": "42b8736"
      +                "step": "4"
      +                "timestamp": "1642041851764"
      +              }
      +              {
      +                "loss": "1.444159984588623"
      +                "rev": "42b8736"
      +                "step": "5"
      +                "timestamp": "1642041866838"
      +              }
      +              {
      +                "loss": "1.3941730260849"
      +                "rev": "42b8736"
      +                "step": "6"
      +                "timestamp": "1642041881837"
      +              }
      +              {
      +                "loss": "1.370380163192749"
      +                "rev": "42b8736"
      +                "step": "7"
      +                "timestamp": "1642041897483"
      +              }
      +              {
      +                "loss": "1.3217320442199707"
      +                "rev": "42b8736"
      +                "step": "8"
      +                "timestamp": "1642041912764"
      +              }
      +              {
      +                "loss": "2.273470401763916"
      +                "rev": "1ba7bcd"
      +                "step": "0"
      +                "timestamp": "1642041482186"
      +              }
      +              {
      +                "loss": "2.20936918258667"
      +                "rev": "1ba7bcd"
      +                "step": "1"
      +                "timestamp": "1642041500577"
      +              }
      +              {
      +                "loss": "2.153379201889038"
      +                "rev": "1ba7bcd"
      +                "step": "2"
      +                "timestamp": "1642041519065"
      +              }
      +              {
      +                "loss": "2.0221104621887207"
      +                "rev": "1ba7bcd"
      +                "step": "3"
      +                "timestamp": "1642041543481"
      +              }
      +              {
      +                "loss": "2.024623155593872"
      +                "rev": "1ba7bcd"
      +                "step": "4"
      +                "timestamp": "1642041565772"
      +              }
      +              {
      +                "loss": "1.8110722303390503"
      +                "rev": "1ba7bcd"
      +                "step": "5"
      +                "timestamp": "1642041586986"
      +              }
      +              {
      +                "loss": "1.7324824333190918"
      +                "rev": "1ba7bcd"
      +                "step": "6"
      +                "timestamp": "1642041609569"
      +              }
      +              {
      +                "loss": "1.6054636240005493"
      +                "rev": "1ba7bcd"
      +                "step": "7"
      +                "timestamp": "1642041631783"
      +              }
      +              {
      +                "loss": "1.5145071744918823"
      +                "rev": "1ba7bcd"
      +                "step": "8"
      +                "timestamp": "1642041648829"
      +              }
      +            ]
      +          }
      +          "encoding": {
      +            "color": {
      +              "legend": {
      +                "disable": true
      +              }
      +              "scale": {
      +                "domain": [
      +                  "workspace"
      +                  "main"
      +                  "4fb124a"
      +                  "42b8736"
      +                  "1ba7bcd"
      +                ]
      +                "range": [
      +                  "#945dd6"
      +                  "#13adc7"
      +                  "#f14c4c"
      +                  "#3794ff"
      +                  "#cca700"
      +                ]
      +              }
      +            }
      +          }
      +          "height": 300
      +          "layer": [
      +            {
      +              "encoding": {
      +                "color": {
      +                  "field": "rev"
      +                  "type": "nominal"
      +                }
      +                "x": {
      +                  "field": "step"
      +                  "title": "step"
      +                  "type": "quantitative"
      +                }
      +                "y": {
      +                  "field": "loss"
      +                  "scale": {
      +                    "zero": false
      +                  }
      +                  "title": "loss"
      +                  "type": "quantitative"
      +                }
      +              }
      +              "layer": [
      +                {
      +                  "mark": "line"
      +                }
      +                {
      +                  "encoding": {
      +                    "opacity": {
      +                      "condition": {
      +                        "selection": "label"
      +                        "value": 1
      +                      }
      +                      "value": 0
      +                    }
      +                  }
      +                  "mark": "point"
      +                  "selection": {
      +                    "label": {
      +                      "clear": "mouseout"
      +                      "empty": "none"
      +                      "encodings": [
      +                        "x"
      +                      ]
      +                      "nearest": true
      +                      "on": "mouseover"
      +                      "type": "single"
      +                    }
      +                  }
      +                }
      +              ]
      +     
Error: 1 tests failed.
        at /Users/mattseddon/PP/vscode-dvc/extension/dist/test/suite/index.js:71:40
        at done (/Users/mattseddon/PP/vscode-dvc/extension/node_modules/mocha/lib/mocha.js:1068:7)
        at processTicksAndRejections (node:internal/process/task_queues:96:5)

       }
      +            {
      +              "layer": [
      +                {
      +                  "encoding": {
      +                    "x": {
      +                      "field": "step"
      +                      "type": "quantitative"
      +                    }
      +                  }
      +                  "mark": {
      +                    "color": "gray"
      +                    "type": "rule"
      +                  }
      +                }
      +                {
      +                  "encoding": {
      +                    "text": {
      +                      "field": "loss"
      +                      "type": "quantitative"
      +                    }
      +                    "x": {
      +                      "field": "step"
      +                      "type": "quantitative"
      +                    }
      +                    "y": {
      +                      "field": "loss"
      +                      "type": "quantitative"
      +                    }
      +                  }
      +                  "layer": [
      +                    {
      +                      "encoding": {
      +                        "color": {
      +                          "field": "rev"
      +                          "type": "nominal"
      +                        }
      +                      }
      +                      "mark": {
      +                        "align": "left"
      +                        "dx": 5
      +                        "dy": -5
      +                        "type": "text"
      +                      }
      +                    }
      +                  ]
      +                }
      +              ]
      +              "transform": [
      +                {
      +                  "filter": {
      +                    "selection": "label"
      +                  }
      +                }
      +              ]
      +            }
      +          ]
      +          "title": ""
      +          "width": 300
      +        }
      +        "multiView": false
      +        "revisions": [
      +          "workspace"
      +          "main"
      +          "4fb124a"
      +          "42b8736"
      +          "1ba7bcd"
      +        ]
      +        "type": "vega"
      +      }
      +    ]
           "predictions.json": [
             {
               "content": {
                 "$schema": "https://vega.github.io/schema/vega-lite/v4.json"
      
        at /Users/mattseddon/PP/vscode-dvc/extension/dist/test/suite/plots/index.test.js:47:45
        at Generator.next (<anonymous>)
        at /Users/mattseddon/PP/vscode-dvc/extension/dist/test/suite/plots/index.test.js:8:71
        at new Promise (<anonymous>)
        at __awaiter (dist/test/suite/plots/index.test.js:4:12)
        at Context.<anonymous> (dist/test/suite/plots/index.test.js:44:64)
        at processImmediate (node:internal/timers:464:21)


----------|---------|----------|---------|---------|-------------------

File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 

----------|---------|----------|---------|---------|-------------------

All files |       0 |        0 |       0 |       0 |                   

----------|---------|----------|---------|---------|-------------------

Note: 0 could potentially cause the test suite to hang if the diff gets too big but we can cross that bridge when we get to it.

@mattseddon mattseddon self-assigned this Feb 21, 2022
@mattseddon mattseddon marked this pull request as ready for review February 21, 2022 05:49
@codeclimate
Copy link

codeclimate bot commented Feb 21, 2022

Code Climate has analyzed commit 32d3feb and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (85% is the threshold).

This pull request will bring the total coverage in the repository to 96.3% (0.0% change).

View more on Code Climate.

@mattseddon mattseddon merged commit 9284b57 into master Feb 21, 2022
@mattseddon mattseddon deleted the update-mocha branch February 21, 2022 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants