Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem with tokenize or regexs #635

Closed
rrthomas opened this issue Apr 22, 2024 · 2 comments · Fixed by #638
Closed

Problem with tokenize or regexs #635

rrthomas opened this issue Apr 22, 2024 · 2 comments · Fixed by #638

Comments

@rrthomas
Copy link
Contributor

I ran into this issue when the functx:lines function didn't seem to work for me, giving me an extra blank line after each line.

But I can find a simple reproducer using an example from: https://www.altova.com/xpath-xquery-reference/fn-tokenize which says:

For example:
fn:tokenize("abracadabra", "(ab)|(a)") returns ("", "r", "c", "d", "r", "")

But with fontoxpath:

var fontoxpath = require("fontoxpath")

console.log(fontoxpath.evaluateXPathToStrings(
'fn:tokenize("abracadabra", "(ab)|(a)")',
 null, 
undefined, 
undefined,
 {language: fontoxpath.evaluateXPath.XQUERY_3_1_LANGUAGE}))

Output:

[
  '',          'ab',
  'undefined', 'r',
  'undefined', 'a',
  'c',         'undefined',
  'a',         'd',
  'ab',        'undefined',
  'r',         'undefined',
  'a',         ''
]

Looks like the captures are being incorrectly returned as part of the results of tokenize.
rrthomas added a commit to rrthomas/ruth that referenced this issue Apr 22, 2024
@rrthomas
FunctX: add a workaround to functx:lines for a fontoxpath bug
4de590f 
See FontoXML/fontoxpath#635
@DrRataplan
Copy link
Collaborator

Hey Reuben,

Sorry for the long wait! Many changes: I'm no longer with Fonto, but I'm still involved in development!

Got it: we use regular JS regexes here, which indeed output capture groups... I made a fix, which I'll PR shortly!

Kind regards,

Martin

@DrRataplan DrRataplan mentioned this issue May 14, 2024
Merged
@rrthomas
Copy link
Contributor Author

Many thanks @DrRataplan!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix tokenize with capture groups DrRataplan/fontoxpath
2 participants
@rrthomas @DrRataplan