Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NLP fix #2: arithmetics support, complete CFG features (suppressed now) #78

Merged
merged 45 commits into from
Mar 13, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
a53cbd7
remove lodash entirely, implement alternatives in subdash
kengz Mar 4, 2016
56622b8
add CFG util, symbol, tokenize
kengz Mar 4, 2016
9d97073
update CFG maps
kengz Mar 4, 2016
6e33dc4
add norm. complete CFG import from kengz/nlp-time
kengz Mar 4, 2016
95fda53
make norm output format to be input, output, diff, using normal dates…
kengz Mar 4, 2016
0a14e0f
allow norm to take offset; refactor n remove comments
kengz Mar 4, 2016
0bdadae
make norm return null on fail instead of throwing err
kengz Mar 4, 2016
82ea5ef
remove ISO, introduce stdT format
kengz Mar 4, 2016
48450d6
fix nTnredistribute, 0 token parsing
kengz Mar 4, 2016
b81486d
allow signed number parsing
kengz Mar 4, 2016
1a3f2cb
fix hhcmm 24hour subnormal parsing
kengz Mar 4, 2016
5b9aa7c
add more customs in maps
kengz Mar 4, 2016
4a8702b
add fortnight as dt
kengz Mar 4, 2016
7010a16
autohour modding only mod on >12
kengz Mar 4, 2016
71046c3
add next avail method
kengz Mar 4, 2016
c9b38de
finish ooOp, ToTOp, nextAvail time
kengz Mar 4, 2016
425cb81
add month
kengz Mar 4, 2016
36372ac
add back 2gram parsing in tokenize
kengz Mar 4, 2016
b7bfa77
fix LoR precedence by t
kengz Mar 4, 2016
c84ecda
fix return order
kengz Mar 5, 2016
4ff65af
add week handelrs
kengz Mar 5, 2016
cc1cb58
more responsible data carrying in tokenize
kengz Mar 5, 2016
ce7fe3a
refactoring
kengz Mar 5, 2016
dc2c9c6
refactor n order util
kengz Mar 5, 2016
a480e05
finish preprocess norm; fix nTnredist
kengz Mar 5, 2016
18ce516
debug nTn
kengz Mar 5, 2016
4eefeee
debug nnOp in nTnredist
kengz Mar 5, 2016
1ad7cc4
add canon to lemma and symbol
kengz Mar 5, 2016
9da2ec9
fix lemma canon to set on value being azAZ
kengz Mar 5, 2016
cd7bb7f
"this" is not zero
kengz Mar 5, 2016
21c1e11
debug canon in restore
kengz Mar 5, 2016
50dcf27
split num from az both ways
kengz Mar 5, 2016
870a4a7
parsenormal1 acts only on string length >6
kengz Mar 5, 2016
26bcfcf
canon opt for T.canon
kengz Mar 5, 2016
b8d5410
n canon uses token if already numeric
kengz Mar 5, 2016
af9c08f
restore to using existing T.canon; fix canon via map and ordering
kengz Mar 5, 2016
668cbf1
fix gen nd day of month
kengz Mar 5, 2016
fbd77fa
first complete merger, all tests pass
kengz Mar 5, 2016
c7f167a
proper merge of norm into parser
kengz Mar 5, 2016
55029ae
some refactoring
kengz Mar 5, 2016
fcde65f
add removeTnPlus for default, quickfix to not parsing AND
kengz Mar 6, 2016
6b2081e
arithmetics support, fixes #70,62,21; suppressing full CFG features f…
kengz Mar 6, 2016
69787d3
add tests for time extraction, normal forms
kengz Mar 6, 2016
a75f0fc
update npm keywords for discovery
kengz Mar 6, 2016
5e24d94
remove UTC test due to different results in different timezone
kengz Mar 6, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 2 additions & 3 deletions History.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
### Issues currently under work:
- 71: "3.99" is interpreted as 4.39am: justified
- 66, 64, 28, 21, 18, 16, 15, 11, 5, 4: normalization: natural language for numbers and time
- 70, 66, 63, 62, 21, 18, 15: support arithmetic, in <n><T>, with <op=and> etc.
- 70, 66, 63, 18: full CFG and unit-specific arithmetics
- 27, 26, 13: default, e.g. tonight, tomorrow at 11 -> at 11am, at midnight etc
- 55, 52, 25, 2: causality, before, after, ago, from now, <bar> ops
- 32: ranges
- fix decimal number parsing
- CFG pending takeover: month-specific carry into days, defaults (tomorrow = 9am), leap year

0.3.1 / 2016-03-02
==================
Expand Down
70 changes: 50 additions & 20 deletions lib/maps.json
Original file line number Diff line number Diff line change
@@ -1,22 +1,20 @@
{
"op": {
"plus": ["plus", "+", "add", "and"],
"plus": ["and", "plus", "+", "add", "on"],
"minus": ["minus", "subtract"],
"times": ["times", "multiply"],
"divide": ["divide"]
},
"c": {
"c": ["every", "per", "repeat"]
},
"r": {
"to": ["to", "thru", "through", "til", "till", "until", "-", "~"]
"o": {
"plus": ["at", "in", "past", "late", "later", "after", "next", "from", "start", "starting", "since", "coming"],
"minus": ["last", "minus", "subtract", "ago", "before", "from"]
},
"n": {
"0.25": ["quarter"],
"0.5": ["half", "1/2", "half a", "half an"],
"0.5": ["half", "1/2", "half an"],
"0": ["zero"],
"1": ["one", "a", "an", "first"],
"2": ["two"],
"2": ["two", "second"],
"3": ["three", "third"],
"4": ["four", "fourth"],
"5": ["five", "fifth"],
Expand Down Expand Up @@ -46,20 +44,52 @@
"1000": ["thousand", "thousands", "thousandth", "k", "K"]
},
"t": {
"th": ["st", "nd", "rd", "th", "st day", "nd day", "rd day", "th day"],
"2 week": ["fortnight", "next fortnight", "a fortnight"]
},
"dt": {
"ms": ["ms", "millisecond", "milliseconds"],
"second": ["s", "sec", "secs", "second", "seconds"],
"minute": ["m", "min", "mins", "minute", "minutes"],
"hour": ["h", "hr", "hrs", "hour", "hours"],
"day": ["d", "day", "days", "dai"],
"week": ["w", "wk", "wks", "week", "weeks"],
"month": ["M", "MM", "mo", "moon", "moons", "month", "months"],
"year": ["y", "yr", "yrs", "year", "years"],
"tomorrow": ["tmr", "tom", "tomorrow"],
"yesterday": ["ytd", "yesterday"]
"s": ["second", "seconds", "s", "sec", "secs"],
"m": ["minute", "minutes", "m", "min", "mins"],
"h": ["hour", "hours", "h", "hr", "hrs"],
"d": ["day", "days", "d", "dai"],
"w": ["week", "weeks", "w", "wk", "wks"],
"M": ["month", "months", "monthes", "M", "mo", "moon", "moons"],
"y": ["year", "years", "y", "yr", "yrs"]
},
"T": {
"t:,dt:=3h": ["later", "soon"],
"t:=1d,dt:": ["st", "nd", "rd", "th", "st day", "nd day", "rd day", "th day"],
"t:,dt:1w": ["st week", "nd week", "rd week", "th week"],
"t:,dt:14d": ["day", "fortnight"],
"t:=0h=0m=0s1mer,dt:": ["pm", "p.m", "p.m.", "noon"],
"t:,dt:1d": ["tomorrow", "tmr"],
"t:,dt:-1d": ["yesterday", "ytd"],
"t:,0dt:": ["today"],
"t:=2h=0m=0s1mer,dt:": ["afternoon"],
"t:=6h=0m=0s0mer,dt:": ["dawn"],
"t:=7h=0m=0s0mer,dt:": ["am", "a.m", "a.m."],
"t:=7h=0m=0s1mer,dt:": ["evening"],
"t:=8h=0m=0s0mer,dt:": ["morning"],
"t:=9h=0m=0s1mer,dt:": ["tonight", "night"],
"t:=0h=0m=0s0mer,dt:1d": ["midnight"],
"t:,dt:=0w0wd": ["sunday", "sun"],
"t:,dt:=0w1wd": ["monday", "mon"],
"t:,dt:=0w2wd": ["tuesday", "tue", "tues"],
"t:,dt:=0w3wd": ["wednesday", "wed"],
"t:,dt:=0w4wd": ["thursday", "thu", "thur", "thurs"],
"t:,dt:=0w5wd": ["friday", "fri"],
"t:,dt:=0w6wd": ["saturday", "sat"],
"t:1M=1d,dt:": ["january", "jan"],
"t:2M=1d,dt:": ["february", "feb"],
"t:3M=1d,dt:": ["march", "mar"],
"t:4M=1d,dt:": ["april", "apr"],
"t:5M=1d,dt:": ["may"],
"t:6M=1d,dt:": ["june", "jun"],
"t:7M=1d,dt:": ["july", "jul"],
"t:8M=1d,dt:": ["august", "aug"],
"t:9M=1d,dt:": ["september", "sept", "sep"],
"t:10M=1d,dt:": ["october", "oct"],
"t:11M=1d,dt:": ["november", "nov"],
"t:12M=1d,dt:": ["december", "dec"],
"t:12M25d,dt:": ["christmas"]
},
"f": {
"1": ["once"],
Expand Down