Dict(asdict=True) with items created/modified with parse action can't be nested in Each

[grv87]

from pyparsing import *
s = """
1 2 3
"""

def get_name_lambda(name):
    return lambda: name

POINT = Dict(And([Group(Empty().add_parse_action(get_name_lambda(name)) + pyparsing_common.signed_integer) for name in ["x", "y", "z"]]))
# 1 This works: [['x', 1], ['y', 2], ['z', 3]]
print(Each(POINT).parse_string(s))

POINT_AS_DICT = Dict(And([Group(Empty().add_parse_action(get_name_lambda(name)) + pyparsing_common.signed_integer) for name in ["x", "y", "z"]]), asdict=True)
# 2 This works: [{'x': 1, 'y': 2, 'z': 3}]
print(Each(POINT_AS_DICT).parse_string(s))

s = """
1 2 3
4 5 6
"""

POINTS = Dict(And([Group(Empty().add_parse_action(get_name_lambda(name)) + POINT) for name in ["point1", "point2"]]))
# 3 This works: [['point1', ['x', 1], ['y', 2], ['z', 3]], ['point2', ['x', 4], ['y', 5], ['z', 6]]]
print(POINTS.parse_string(s).as_list())
# 4 This works: [['point1', ['x', 1], ['y', 2], ['z', 3]], ['point2', ['x', 4], ['y', 5], ['z', 6]]]
print(Each(POINTS).parse_string(s))

POINTS_AS_DICTS = Dict(And([Group(Empty().add_parse_action(get_name_lambda(name)) + POINT_AS_DICT) for name in ["point1", "point2"]]))
# 5 This works: [['point1', {'x': 1, 'y': 2, 'z': 3}], ['point2', {'x': 4, 'y': 5, 'z': 6}]]
print(POINTS_AS_DICTS.parse_string(s))
# 6 This gives an error
print(Each(POINTS_AS_DICTS).parse_string(s))

The last case gives the error:

This gives error:
Traceback (most recent call last):
  File "…\pyparsing_spikes\issue_YYY.py", line 20, in <module>
    print(Each(POINTS).parse_string(s))
  File "C:\root\Python\3.10\w64\lib\site-packages\pyparsing\core.py", line 1131, in parse_string
    loc, tokens = self._parse(instring, 0)
  File "C:\root\Python\3.10\w64\lib\site-packages\pyparsing\core.py", line 817, in _parseNoCache
    loc, tokens = self.parseImpl(instring, pre_loc, doActions)
  File "C:\root\Python\3.10\w64\lib\site-packages\pyparsing\core.py", line 4296, in parseImpl
    tmpLoc = e.try_parse(instring, tmpLoc, raise_fatal=True)
  File "C:\root\Python\3.10\w64\lib\site-packages\pyparsing\core.py", line 880, in try_parse
    return self._parse(instring, loc, doActions=False)[0]
  File "C:\root\Python\3.10\w64\lib\site-packages\pyparsing\core.py", line 823, in _parseNoCache
    tokens = self.postParse(instring, loc, tokens)
  File "C:\root\Python\3.10\w64\lib\site-packages\pyparsing\core.py", line 5524, in postParse
    tokenlist[ikey] = _ParseResultsWithOffset("", i)
  File "C:\root\Python\3.10\w64\lib\site-packages\pyparsing\results.py", line 202, in __setitem__
    self._tokdict[k] = self._tokdict.get(k, list()) + [v]
TypeError: unhashable type: 'dict'

So, the problem occurs with asdict=True only. And only inside Each (or maybe other containers that call try_parse).
Debugging shows that case # 2 actually has the same behavior/problem. So, the depth of nesting doesn't matter. It's just that '1' is hashable, so it doesn't crash.

Is there any reason why try_parse should call postParse at all?

[ptmcg]

Of the 4 ParseExpression classes, Each and Or have this issue, And and MatchFirst do not. I'll have to dig deeper this evening.

[grv87]

@ptmcg, any news?

Looks that my issue can be easily fixed with

.add_parse_action(…, call_during_try = True)

So, it's again a documentation/misunderstanding problem.

[ptmcg]

Basil, Thanks for the time and effort in your thorough testing (where were you when I was doing the 3.0.0 pre releases??!!). I’ve been thinking about some of these, and you are right, I may just change the documentation to reflect the current behavior. That said, there *is* an odd bug that it looks like I introduced last October that breaks some uses of transform_string, which is a pretty major deal. Also annoying that my tests didn’t catch it. (You can see it if you run the cuneiform_python.py example.) However, I have some personal/family business that will completely occupy my time until mid next week, I will try to look into your reports if I can steal some time. So please bear with me. And thanks again for submitting all this feedback, it just makes pyparsing better.

…

-- Paul From: Basil Peace ***@***.***> Sent: Tuesday, July 19, 2022 6:33 PM To: pyparsing/pyparsing ***@***.***> Cc: Paul McGuire ***@***.***>; Mention ***@***.***> Subject: Re: [pyparsing/pyparsing] Dict(asdict=True) with items created/modified with parse action can't be nested in Each (Issue #432) @ptmcg <https://github.com/ptmcg> , any news? Looks that my issue can be easily fixed with .add_parse_action(…, call_during_try = True) So, it's again a documentation/misunderstanding problem. — Reply to this email directly, view it on GitHub <#432 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAC5XAZFYEPB7NKYL4WM6MDVU43LVANCNFSM53226XGQ> . You are receiving this because you were mentioned. <https://github.com/notifications/beacon/AAC5XA46UHUDNKCQFWUDWDDVU43LVA5CNFSM53226XG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOI3UJVLI.gif> Message ID: ***@***.*** ***@***.***> >

[ptmcg]

Basil -

And here it is November, much further than just "mid next week". Working on a book (with hard deadlines) was more intense than I expected.

I have been holding off the next release of pyparsing because of this open item. I may actually have to break some current behavior, and if I do that, I'll probably not do that in the 3.0.x versions, but release as a 3.1.0, and probably do some early release candidates too.

Thanks for your patience!
-- Paul