forked from k-takata/Onigmo
/
README
252 lines (176 loc) · 7.69 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
README 2016/11/30
Onigmo (Oniguruma-mod) -- (C) K.Takata <kentkt AT csc DOT jp>
https://github.com/k-takata/Onigmo
Onigmo is a regular expressions library forked from Oniguruma.
It focuses to support new expressions like `\K`, `\R`, `(?(cond)yes|no)`
and etc. which are supported in Perl 5.10+.
Since Onigmo is used as the default regexp library of Ruby 2.0 or later,
many patches are backported from Ruby 2.x.
See also the Wiki page:
https://github.com/k-takata/Onigmo/wiki
Main New features:
Regular Expressions (depends on the syntax):
\K, \R, \X, (?(cond)yes|no)
(?adlu), \g{name}, \g{n}, (?&name), (?n), (?R), (?0)
(?P<name>...), (?P=name), (?P>name)
API:
onig_search_gpos (for Perl-compatible \G)
Encoding:
CP932, CP1250, CP1251, CP1252, CP1253, CP1254, CP1257
Syntax:
Python
New Source Files:
enc/jis/props.h JIS character properties data.
enc/jis/props.kwd JIS character properties data.
enc/unicode/casefold.h Unicode case folding data.
enc/unicode/name2ctype.h Unicode properties data.
enc/windows_*.c CP* encoding
onigmo.py onigmo.dll/libonigmo.so loader.
testpy.py test program.
tool/download-ucd.sh downloads Unicode Character Database (UCD).
tool/case-folding.rb generates casefold.h from UCD.
tool/convert-jis-props.sh converts props.kwd to props.h.
tool/convert-name2ctype.sh converts name2ctype.kwd to name2ctypes.h.
tool/enc-unicode.rb generates name2ctype.kwd from UCD.
win32/Makefile.mingw Makefile for Win32 (MinGW)
win32/makedef.py creates onigmo.def.
win32/onigmo.rc resource file for onigmo.dll.
ToDo:
* Reduce the size of Unicode Character Data.
* (?|...)
* Improve (?(cond)yes|no). (support look-ahead/behind assertions.)
Oniguruma's README follows:
======================================================================
README 2007/05/31
Oniguruma ---- (C) K.Kosako <sndgk393 AT ybb DOT ne DOT jp>
http://www.geocities.jp/kosako3/oniguruma/
Oniguruma is a regular expressions library.
The characteristics of this library is that different character encoding
for every regular expression object can be specified.
Supported character encodings:
ASCII, UTF-8, UTF-16BE, UTF-16LE, UTF-32BE, UTF-32LE,
EUC-JP, EUC-TW, EUC-KR, EUC-CN,
Shift_JIS, Big5, GB18030, KOI8-R, CP1251,
ISO-8859-1, ISO-8859-2, ISO-8859-3, ISO-8859-4, ISO-8859-5,
ISO-8859-6, ISO-8859-7, ISO-8859-8, ISO-8859-9, ISO-8859-10,
ISO-8859-11, ISO-8859-13, ISO-8859-14, ISO-8859-15, ISO-8859-16
* GB18030: contributed by KUBO Takehiro
* CP1251: contributed by Byte
------------------------------------------------------------
License
BSD license.
Install
Case 1: Unix and Cygwin platform
1. ./autogen.sh (If `configure` doesn't exist.)
2. ./configure
3. make
4. make install
* uninstall
make uninstall
* test (ASCII/EUC-JP)
make atest
* configuration check
onig-config --cflags
onig-config --libs
onig-config --prefix
onig-config --exec-prefix
Case 2: Win32 platform (VC++)
1. copy win32\Makefile Makefile
2. copy win32\config.h config.h
3. nmake
onig_s.lib: static link library
onig.dll: dynamic link library
* test (ASCII/Shift_JIS)
4. copy win32\testc.c testc.c
5. nmake ctest
Regular Expressions
See doc/RE (or doc/RE.ja for Japanese).
Usage
Include oniguruma.h in your program. (Oniguruma API)
See doc/API for Oniguruma API.
If you want to disable UChar type (== unsigned char) definition
in oniguruma.h, define ONIG_ESCAPE_UCHAR_COLLISION and then
include oniguruma.h.
If you want to disable regex_t type definition in oniguruma.h,
define ONIG_ESCAPE_REGEX_T_COLLISION and then include oniguruma.h.
Example of the compiling/linking command line in Unix or Cygwin,
(prefix == /usr/local case)
cc sample.c -L/usr/local/lib -lonig
If you want to use static link library(onig_s.lib) in Win32,
add option -DONIG_EXTERN=extern to C compiler.
Sample Programs
sample/simple.c example of the minimum (Oniguruma API)
sample/names.c example of the named group callback.
sample/encode.c example of some encodings.
sample/listcap.c example of the capture history.
sample/posix.c POSIX API sample.
sample/sql.c example of the variable meta characters.
(SQL-like pattern matching)
Test Programs
sample/syntax.c Perl, Java and ASIS syntax test.
sample/crnl.c --enable-crnl-as-line-terminator test
Source Files
oniguruma.h Oniguruma API header file. (public)
onig-config.in configuration check program template.
regenc.h character encodings framework header file.
regint.h internal definitions
regparse.h internal definitions for regparse.c and regcomp.c
regcomp.c compiling and optimization functions
regenc.c character encodings framework.
regerror.c error message function
regext.c extended API functions. (deluxe version API)
regexec.c search and match functions
regparse.c parsing functions.
regsyntax.c pattern syntax functions and built-in syntax definitions.
regtrav.c capture history tree data traverse functions.
regversion.c version info function.
st.h hash table functions header file
st.c hash table functions
oniggnu.h GNU regex API header file. (public)
reggnu.c GNU regex API functions
onigposix.h POSIX API header file. (public)
regposerr.c POSIX error message function.
regposix.c POSIX API functions.
enc/mktable.c character type table generator.
enc/ascii.c ASCII encoding.
enc/euc_jp.c EUC-JP encoding.
enc/euc_tw.c EUC-TW encoding.
enc/euc_kr.c EUC-KR, EUC-CN encoding.
enc/sjis.c Shift_JIS encoding.
enc/big5.c Big5 encoding.
enc/gb18030.c GB18030 encoding.
enc/koi8.c KOI8 encoding.
enc/koi8_r.c KOI8-R encoding.
enc/cp1251.c CP1251 encoding.
enc/iso8859_1.c ISO-8859-1 encoding. (Latin-1)
enc/iso8859_2.c ISO-8859-2 encoding. (Latin-2)
enc/iso8859_3.c ISO-8859-3 encoding. (Latin-3)
enc/iso8859_4.c ISO-8859-4 encoding. (Latin-4)
enc/iso8859_5.c ISO-8859-5 encoding. (Cyrillic)
enc/iso8859_6.c ISO-8859-6 encoding. (Arabic)
enc/iso8859_7.c ISO-8859-7 encoding. (Greek)
enc/iso8859_8.c ISO-8859-8 encoding. (Hebrew)
enc/iso8859_9.c ISO-8859-9 encoding. (Latin-5 or Turkish)
enc/iso8859_10.c ISO-8859-10 encoding. (Latin-6 or Nordic)
enc/iso8859_11.c ISO-8859-11 encoding. (Thai)
enc/iso8859_13.c ISO-8859-13 encoding. (Latin-7 or Baltic Rim)
enc/iso8859_14.c ISO-8859-14 encoding. (Latin-8 or Celtic)
enc/iso8859_15.c ISO-8859-15 encoding. (Latin-9 or West European with Euro)
enc/iso8859_16.c ISO-8859-16 encoding.
(Latin-10 or South-Eastern European with Euro)
enc/utf8.c UTF-8 encoding.
enc/utf16_be.c UTF-16BE encoding.
enc/utf16_le.c UTF-16LE encoding.
enc/utf32_be.c UTF-32BE encoding.
enc/utf32_le.c UTF-32LE encoding.
enc/unicode.c Unicode information data.
win32/Makefile Makefile for Win32 (VC++)
win32/config.h config.h for Win32
ToDo
? case fold flag: Katakana <-> Hiragana.
? add ONIG_OPTION_NOTBOS/NOTEOS. (\A, \z, \Z)
?? \X (== \PM\pM*)
?? implement syntax behavior ONIG_SYN_CONTEXT_INDEP_ANCHORS.
?? transmission stopper. (return ONIG_STOP from match_at())
and I'm thankful to Akinori MUSHA.
Mail Address: K.Kosako <sndgk393 AT ybb DOT ne DOT jp>