Skip to content

Commit

Permalink
Convert single-byte charset probers to use nested dicts for language …
Browse files Browse the repository at this point in the history
…models (#121)

* Convert single byte charset modules to use dicts of dicts for language modules

- Also provide conversion script

* Fix debug logging check

* Keep Hungarian commented out until we retrain
  • Loading branch information
dan-blanchard committed Jun 8, 2017
1 parent 9b8c5c2 commit ec3bce7
Show file tree
Hide file tree
Showing 15 changed files with 33,145 additions and 1,606 deletions.
6 changes: 4 additions & 2 deletions chardet/compat.py
Expand Up @@ -25,10 +25,12 @@
if sys.version_info < (3, 0):
PY2 = True
PY3 = False
base_str = (str, unicode)
string_types = (str, unicode)
text_type = unicode
iteritems = dict.iteritems
else:
PY2 = False
PY3 = True
base_str = (bytes, str)
string_types = (bytes, str)
text_type = str
iteritems = dict.items

0 comments on commit ec3bce7

Please sign in to comment.