list sourceMap end row includes trailing blank lines

[dwelle]

When generating sourcemap for lists (both ul,li), the trailing blank lines (\s*) are included thus the end row of the following example isn't 1 as usual, but 3:

- aaa


bbb

see demo, where both bullet_list_open and list_item_open have [ 0, 9 ]

[puzrin]

I think that may be a bug. Usually only top level blocks are used to sync with preview scoll. So, nobody checked what happens in lists. But i don't remember details, need to investigate.

[rlidwka]

Brief technical reasoning for the current (possibly incorrect) behavior:

End line in map is a line when the rule terminates itself (i.e. is able to determine that the next line for sure can't be parsed by the current rule).

List stops on bbb because it's the first non-empty line to have indentation less than 2 (therefore first line that couldn't possibly be in the list).

Paragraph stops on a second empty line, because it's guaranteed to terminate a paragraph no matter how many empty lines are after it.

I'm not sure what should be in the map, there's a good chance all rules need to be re-checked for this.

[rlidwka]

P.S.: this behavior was unintentionally changed in 8.0.0 following CM spec changes.

Looking into fixing it now.

[dwelle]

Hmph.. checking CM spec, it probably should behave like this:

0 - foo
1 
2 - bar
3 
4 
5 - baz

should result in [0,2] and [2,5] maps because lists can have * empty lines between the bullet points.

But the following:

0 - foo
1 
2 - bar
3 
4 
5 xxxx

should yield [0,2], [2,3] coz the line 5 is not a continuation of that list, thus it terminates early...

If the above is correct, then:

0 - foo
1   - aaa
2
3 - bar

line 1 should still result in [1,2] (as opposed to current [1,3]) because the list at line 3 is not a continuation of the nested <ul> that starts at line 1

[rlidwka]

should result in [0,2] and [2,5] maps because lists can have * empty lines between the bullet points.

It doesn't seem consistent, as we should either count empty lines in all cases or not count them anywhere.

CM spec doesn't exactly define any of this because series of empty lines have no effect on the output.

[rlidwka]

I've been checking CM reference implementation. They do have source maps, and their behavior matches ours in this particular case.

0 - foo
1   - bar
2
3
4 - baz
5
6 - quux
7
8
9
10 xxxx
11
12
13 yyyy

<ul data-sourcepos="1:1-10:0">
  <li data-sourcepos="1:1-4:0">
    <p data-sourcepos="1:3-1:5">foo</p>
    <ul data-sourcepos="2:3-4:0">
      <li data-sourcepos="2:3-4:0">bar</li>
    </ul>
  </li>
  <li data-sourcepos="5:1-6:0">
    <p data-sourcepos="5:3-5:5">baz</p>
  </li>
  <li data-sourcepos="7:1-10:0">
    <p data-sourcepos="7:3-7:6">quux</p>
  </li>
</ul>
<p data-sourcepos="11:1-11:4">xxxx</p>
<p data-sourcepos="14:1-14:4">yyyy</p>

As you can see, list has [0,10] which counts empty lines, but paragraphs don't.

I think I'll just ask clarification for it on CM issue tracker.

[rlidwka]

Just created a new topic here: https://talk.commonmark.org/t/sourcepos-in-lists/2498

[geyang]

Wait, we have sourcemap now???

🎆 🍺 🍻

[puzrin]

Very limited implementation, restricted by current architecture

[geyang]

Oh still the block-level sourcemap ha 😃

[Meloneat]

It is like the plugin render mdfile to the sentence one by one ,so how could i use it to realize the enty file out from xx.md to xx.html ..enn,could i through it?