Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(dict): Handle cases from Linux #252

Merged
merged 2 commits into from May 18, 2021
Merged

fix(dict): Handle cases from Linux #252

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fb0dac4
refactor(dict): Allow 0..n corrections in BuiltIn
May 16, 2021
639e65b
fix(dict): Handle cases from Linux
May 16, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 2 additions & 3 deletions crates/typos-dict/assets/words.csv
View file
Open in desktop
Expand Up @@ -13021,7 +13021,6 @@ handelbars,handlebars
handicaped,handicapped
handwritng,handwriting
harasments,harassments
hardlinked,hardline
harmoniacs,harmonic
harmonisch,harmonic
harrasment,harassment
Expand Down Expand Up @@ -23557,7 +23556,7 @@ referens,references
referere,referee
referign,refering
refering,referring
refernce,references
refernce,reference
reffered,referred
refilles,refills
refillls,refills
Expand Down Expand Up @@ -27976,7 +27975,7 @@ tast,taste
tath,that
tehy,they
tghe,the
ther,there
ther,there,their,the
thge,the
thna,than
thne,then
Expand Down
1 change: 1 addition & 0 deletions crates/typos-dict/codegen/Cargo.toml
View file
Open in desktop
Expand Up @@ -18,6 +18,7 @@ codecov = { repository = "crate-ci/typos" }
phf = { version = "0.8", features = ["unicase"] }
phf_codegen = "0.8"
csv = "1.1"
itertools = "0.10"
unicase = "2.5"
codegenrs = "1.0"
structopt = "0.3"
14 changes: 10 additions & 4 deletions crates/typos-dict/codegen/src/main.rs
View file
Open in desktop
Expand Up @@ -18,20 +18,26 @@ fn generate<W: std::io::Write>(file: &mut W) {

writeln!(
file,
"pub static WORD_DICTIONARY: phf::Map<unicase::UniCase<&'static str>, &'static str> = "
"pub static WORD_DICTIONARY: phf::Map<unicase::UniCase<&'static str>, &'static [&'static str]> = "
)
.unwrap();
let mut builder = phf_codegen::Map::new();
let records: Vec<_> = csv::ReaderBuilder::new()
.has_headers(false)
.flexible(true)
.from_reader(DICT)
.records()
.map(|r| r.unwrap())
.collect();
for record in &records {
smallest = std::cmp::min(smallest, record[0].len());
largest = std::cmp::max(largest, record[0].len());
let value = format!(r#""{}""#, &record[1]);
let mut record_fields = record.iter();
let key = record_fields.next().unwrap();
smallest = std::cmp::min(smallest, key.len());
largest = std::cmp::max(largest, key.len());
let value = format!(
"&[{}]",
itertools::join(record_fields.map(|field| format!(r#""{}""#, field)), ", ")
);
builder.entry(unicase::UniCase::new(&record[0]), &value);
}
let codegenned = builder.build();
Expand Down