TypeError at end of NER training

[kinghuang]

How to reproduce the behaviour

I upgraded from spaCy 2.2.3 to 2.2.4, and I'm getting an error at the end of spacy train. Here's the command output including the error traceback.

root@1c02094b4938:/project# spacy train \
> --base-model en_core_oi_lg/en_core_oi_lg-0.0.3 \
> --pipeline ner \
> --n-iter 30 \
> --n-early-stopping 5 \
> en \
> models \
> data/ner/train \
> data/ner/eval
✔ Created output directory: models
Training pipeline: ['ner']
Starting with base model 'en_core_oi_lg/en_core_oi_lg-0.0.3'
Extending component from base model 'ner'
Counting training words (limit=0)

Itn  NER Loss   NER P   NER R   NER F   Token %  CPU WPS
---  ---------  ------  ------  ------  -------  -------
  1   4857.485  67.342  58.462  62.588   92.721    17608                                                                                           
  2   2574.152  74.643  68.901  71.657   92.721    19069                                                                                           
  3   1822.187  77.081  74.286  75.658   92.721    19331                                                                                           
…
 15    246.849  80.291  78.791  79.534   92.721    19075                                                                                           
 16    242.276  80.518  78.571  79.533   92.721    18501                                                                                           
 17    217.591  79.911  78.681  79.291   92.721    18839                                                                                           
Early stopping, best iteration is: 11
Best score = 79.919691339363; Final iteration score = 79.29442811710254
✔ Saved model to output directory
models/model-final
⠙ Creating best model...
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.8/site-packages/spacy/__main__.py", line 33, in <module>
    plac.call(commands[command], sys.argv[1:])
  File "/usr/local/lib/python3.8/site-packages/plac_core.py", line 367, in call
    cmd, result = parser.consume(arglist)
  File "/usr/local/lib/python3.8/site-packages/plac_core.py", line 232, in consume
    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
  File "/usr/local/lib/python3.8/site-packages/spacy/cli/train.py", line 583, in train
    best_model_path = _collate_best_model(final_meta, output_path, best_pipes)
  File "/usr/local/lib/python3.8/site-packages/spacy/cli/train.py", line 643, in _collate_best_model
    bests[component] = _find_best(output_path, component)
  File "/usr/local/lib/python3.8/site-packages/spacy/cli/train.py", line 666, in _find_best
    return max(accuracies)[1]
TypeError: '>' not supported between instances of 'dict' and 'dict'

Your Environment

• spaCy version: 2.2.4
• Platform: Linux-4.19.76-linuxkit-x86_64-with-glibc2.2.5
• Python version: 3.8.2

[adrianeboyd]

Sorry, adding the per_type metrics to the final output added dicts of values that can't be compared here. It only tries to compare them in cases where the higher priority metrics (ents_f/p/r) are the same for two iterations, which is why it only happens occasionally.

A quick fix for that is in a PR here:

https://github.com/explosion/spaCy/pull/5186/files#diff-ef11bc255afe9cb7ee81d55f7d35bbbcR667-R668

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.