Feature/debullet

[thegoatherder]

• New: added debullet method to one > change > api. This debullets some text. A bullet is any of these prefixing a sentence: -–—*•
• New: tests for the new debullet method
• Fix: found an issue where • was being stored as a term with pre: '', str: '•'. Added • as notWord in the terms tokenizer to prevent this.

Example Code

import nlp from './src/one.js'

let arr = [
  '- it is-cool - he is nice',
  `- it is-cool 
- he is nice
- oooh yeah, baby!`,
  `-it is-cool 
-he is nice
-oooh yeah, baby!`,
  `* it is-cool 
* he is nice
* oooh yeah, baby!`,
  `• it is-cool 
• he is nice
• oooh yeah, baby!`,
  `•it is-cool 
•he is nice
•oooh yeah, baby!`,
]

console.log('================')
console.log('Without Debullet')
console.log('================')
arr.forEach(txt => {
  console.log(nlp(txt).out())
  console.log('----------------')
})

console.log('\n\n\n=============')
console.log('With Debullet')
console.log('=============')
arr.forEach(txt => {
  console.log(nlp(txt).debullet().out())
  console.log('----------------')
})

console.log('\n\n\n======================')
console.log('Unaffected by Debullet')
console.log('======================')
;[
  'it is-cool - he is nice',
  'it is-cool. he is- nice-',
  `i was -
feeling weird -
were you too?`,
].forEach(txt => {
  console.log(nlp(txt).debullet().out())
  console.log('----------------')
})

** Example Output **

================
Without Debullet
================
- it is-cool - he is nice
----------------
- it is-cool
- he is nice
- oooh yeah, baby!
----------------
-it is-cool
-he is nice
-oooh yeah, baby!
----------------
* it is-cool
* he is nice
* oooh yeah, baby!
----------------
• it is-cool 
• he is nice
• oooh yeah, baby!
----------------
•it is-cool
•he is nice
•oooh yeah, baby!
----------------



=============
With Debullet
=============
it is-cool - he is nice
----------------
it is-cool
he is nice
oooh yeah, baby!
----------------
it is-cool
he is nice
oooh yeah, baby!
----------------
it is-cool
he is nice
oooh yeah, baby!
----------------
it is-cool
he is nice
oooh yeah, baby!
----------------
it is-cool
he is nice
oooh yeah, baby!
----------------



======================
Unaffected by Debullet
======================
it is-cool - he is nice
----------------
it is-cool. he is- nice-
----------------
i was -
feeling weird -
were you too?
----------------

Test output

> node tests\one\change\debullet.test.js
Debugger attached.
TAP version 13
# dont-debullet
ok 1 [one/bullets] don't debullet hyphens
ok 2 [one/bullets] don't debullet hyphens with multi-line
# do-debullet
ok 3 [one/bullets] do debullet line beginning hyphen
ok 4 [one/bullets] do debullet multiline beginning hyphen
ok 5 [one/bullets] do debullet multiline beginning hyphen no-space
ok 6 [one/bullets] do debullet multiline beginning asterisk
ok 7 [one/bullets] do debullet multiline beginning bullet char
ok 8 [one/bullets] do debullet multiline beginning bullet char nospace

1..8
# tests 8
# pass  8

# ok

All tests pass with npm run test

16.76s
   9,239  ✔️

Sorry - have not updated API docs. Give me any pointers on this and I can do it.

[spencermountain]

works for me!
I may change it to operate only on terms[0], if that works with you

[spencermountain]

moving this to sentences().debullet() for 14.9.0
should be the same
cheers

[spencermountain]

last adjustment - this is now .normalize({ debullet: true }) and part of normalize('medium') and released in 14.9.0