custom ReactiveHTML unable to render and process MouseEvent correctly

[derrickmcs]

panel version 1.1.1 - Cytoscape.js ver. 3.25.0 rendered using ReactiveHTML, MacOS12.1, Safari 15.2

When using Panel v0.14.1, the component rendered correctly. Mouse clicks were functional and on target.
When using Panel v1.1.1, the component can only render on certain layouts and mouse click did not work until manually resize the window. However, when the window scrolled away from original position, the mouse clicks were off target (usually by the same amount of pixels from the original position.

Complete, minimal, self-contained example code that reproduces the issue

from json.tool import main
import param
import panel as pn
from panel.reactive import ReactiveHTML

layouts = {'cose': {'name': 'cose'},
    'grid': {'name': 'grid'},
    'circle': {'name': 'circle'},
    'concentric': {'name': 'concentric'},
    'breadthfirst': {'name': 'breadthfirst'},
    'random': {'name': 'random'},
    'preset': {'name': 'preset'}
}

class Cytoscape(ReactiveHTML):
    layout_names = param.ObjectSelector(default='cose', doc="Layout Options to choose from.", objects=list(layouts.keys()))
    graph_layouts = param.Dict(default=layouts)
    style = param.List(doc="Use to set the styles of the nodes/edges")
    data = param.List(doc="Use to send node's data/attributes to Cytoscape")

    _template = """
        <div id="cy" style="width: 100%; height: 100%; position: relative; border: 1px solid"></div>
    """
    __javascript__ = ['assets/cytoscape.min.js']

    _scripts = {
        'render': """
            state.cy = cytoscape({
              container: cy,
              layout: {name: 'cose'}, //these layout will fail: grid, circle, concentric
              elements: [{data:{id:'A', label:'A'}},{data:{id:'B', label:'B'}}, {data:{source:'A', target:'B'}}],
              zoom: 1,
              pan: {x: 0, y: 0},
            });
            state.cy.on('click', function (evt) {
                console.log(evt.originalEvent, evt)
            });
        """,
        'remove': """
        delete state.cy
        """,
        'layout_names': """
            const layout = state.cy.layout(data.graph_layouts[data.layout_names])
            layout.run()
        """
    }

    _extension_name = 'cytoscape'

pn.extension('cytoscape', template='fast')

my_graph = Cytoscape(width=800, height=400)

widget_container = pn.Column(pn.pane.Markdown("## Title"))
app = pn.Column(widget_container).servable(target='main')
widget_container.append(pn.Row(my_graph.controls(['layout_names']), my_graph))

Initial render - Click on the top node (blue) - note the X-Y coord

Scroll the window the right - Click on the same node - It is off target to the left (note the x-y coord).

[MarcSkovMadsen]

My hypothesis is that cytoscape.js does not work well inside shadow root. I've created an issue with cytoscape.js in 3133. You will find more detail and a working example there.

[derrickmcs]

I don't know too much about ShadowDOM, but when review the mouse click event, I noticed the Target Element was on the outer div instead of the canvas,; thus, the coordinate information were not the same as in the other version of panel(v0.14.1).

[MarcSkovMadsen]

Could you share a screenshot of that click event? With the Target element?

And also share it in the issue I created with Cytoscape (link in post above)

[derrickmcs]

I apologize - after review again, I was using Cytoscape version 3.23.0 to produce the above screenshot. I saw you used version 3.25.0, which I could NOT get it render if I create the object in the "render" block. You got it to render by creating it in the "after_layout" block. Version 3.25 had introduced new problems... However I was able to get the screenshot of the 'target element", as you can see in the log, it's pointing at <div class="bk-Column">, which is the outer div contains the "Title, "controls", and Cytoscape graph.

[derrickmcs]

Can the srcElement and target be the canvas instead of its parent element? Is this within panel's control or it's has to do with bokeh web framework?

[MarcSkovMadsen]

Maybe this article is useful. Something about events inside shadowroot being retargetted to the "host". This can maybe be avoided if put inside a "slot" https://javascript.info/shadow-dom-events

[derrickmcs]

@MarcSkovMadsen awesome find - I wrapped the div with <span slot="cy_slot"> and it worked. Also no errors when creating the "cy" object in the "render" block, which is great (creating the object in the "after_layout" block causes the graph to refresh too often). Thank you for your help!

[MarcSkovMadsen]

I'm so happy to hear that @derrickmcs

Could you post a minimum, working example such that its easy for any one to learn from? Thanks

[MarcSkovMadsen]

@philippjfr

I would suggest

Documenting the use of "slot" for events when using ReactiveHTML.

[philippjfr]

Wow indeed, great find.

[derrickmcs]

I'm so sorry about this. I've spoken too soon. I was testing on the older version of panel (0.14.2) and that's why it worked. Using slot doesn't solve the problem.
With shadow DOM aside, If Cytoscape could base its position calculation on the "offsetX" and "offsetY", that might solve a lot of problem. I will try to post this on the Cytoscape ticket...

[derrickmcs]

I'm so sorry about this. I've spoken too soon. I was testing on the older version of panel (0.14.2) and that's why it worked. Using slot doesn't solve the problem.
With shadow DOM aside, If Cytoscape could base its position calculation on the "offsetX" and "offsetY", that might solve a lot of problem. I will try to post this on the Cytoscape ticket...

Nevermind on that, for whatever reason, Cytoscape received the mouseEvent from the outer layer, thus the offsetX and offsetY are incorrect.
However, when I wrapped the Cytoscape div with <user-card>, the user-card's offsetX and Y appears to be correct.
Example:
<user-card id="userCard"> <div id="cy" style="width: 100%; height: 100%; position: relative; border: 1px solid"></div> </user-card>

state.cy.on('click', function (evt) {
                console.log(evt.originalEvent.offsetX, evt.originalEvent.offsetY)
                console.log(evt.originalEvent)
            });

userCard.onclick = e => console.log('userCard', e.offsetX, e.offsetY  )

[derrickmcs]

I found a way around this issue, which is to trigger Cytoscape to recalculate its positions in the event of a scroll. (NOTE: not all browsers support scroll event)
Here's what needed:

               const mainEle = document.querySelector("div#main")

                mainEle.addEventListener("scrollend", (event) => {
                    console.log("scrollend event")
                    state.cy.resize()
                })

Here's a working code:

from json.tool import main
from turtle import width
import param
import panel as pn
from panel.reactive import ReactiveHTML

layouts = {'cose': {
    'name': 'cose',
            'idealEdgeLength': 100,
            'nodeOverlap': 20,
            'refresh': 20,
            'fit': 'true',
            'padding': 30,
            'randomize': False,
            'componentSpacing': 100,
            'nodeRepulsion': 400000,
            'edgeElasticity': 100,
            'nestingFactor': 5,
            'gravity': 80,
            'numIter': 1000,
            'initialTemp': 200,
            'coolingFactor': 0.95,
            'minTemp': 1.0
},
    'grid': {'name': 'grid',
             # 'columns': 4 makes 4 colums
             },
    'circle': {'name': 'circle'},
    'concentric': {'name': 'concentric'},
    'breadthfirst': {'name': 'breadthfirst'},
    'random': {'name': 'random'},
    'preset': {'name': 'preset'}
}

class Cytoscape(ReactiveHTML):
    layout_names = param.ObjectSelector(default='cose', doc="Layout Options to choose from.", objects=list(layouts.keys()))
    graph_layouts = param.Dict(default=layouts)
    style = param.List(doc="Use to set the styles of the nodes/edges")
    data = param.List(doc="Use to send node's data/attributes to Cytoscape")

    _template = """
        <div id="cy" style="width: 100%; height: 100%; position: relative; border: 1px solid"></div>
    """
    __javascript__ = ['https://cdnjs.cloudflare.com/ajax/libs/cytoscape/3.23.0/cytoscape.umd.js']#['assets/cytoscape.min323.js']#['https://cdnjs.cloudflare.com/ajax/libs/cytoscape/3.25.0/cytoscape.umd.js']

    _scripts = {
        'after_layout': """
            self.create()
        """,
        "create": """
            if (state.cy == undefined){
                state.cy = cytoscape({
                container: cy,
                layout: data.graph_layouts[data.layout_names], 
                elements: [{data:{id:'A', label:'A'}},{data:{id:'B', label:'B'}}, {data:{source:'A', target:'B'}}],
                zoom: 1,
                pan: {x: 0, y: 0},
                });
                state.cy.on('click', function (evt) {
                    console.log(evt.originalEvent.offsetX, evt.originalEvent.offsetY, evt.position)
                    console.log(evt.originalEvent)
                });

                const mainEle = document.querySelector("div#main")

                mainEle.addEventListener("scrollend", (event) => {
                    console.log("scrollend event")
                    state.cy.resize()
                })
                }; 
        """,
        'remove': """
        state.cy.destroy()
        delete state.cy
        """,
        'layout_names': """
            const layout = state.cy.layout(data.graph_layouts[data.layout_names])
            layout.run()
        """
    }

    _extension_name = 'cytoscape'

pn.extension('cytoscape', template='fast', sizing_mode='stretch_width')

my_graph = Cytoscape(width=800, height=600)

widget_container = pn.Column(pn.pane.Markdown("## Title"), min_height=600)
app = pn.Column(widget_container, pn.pane.Markdown(" ## this is the footer")).servable(target='main')
widget_container.append(pn.Row(my_graph.controls(['layout_names']), my_graph))

[MarcSkovMadsen]

Would it work if you set a timeout to trigger the resize after 100ms or something?

[derrickmcs]

Would it work if you set a timeout to trigger the resize after 100ms or something?

I'm not sure it will buy me much. I think trigger on a "scrollend" event is OK.

[MarcSkovMadsen]

Now I also think scrollend is better @derrickmcs . I've finetuned your example and added it as a reference example to #5448.

import param
import panel as pn
from panel.reactive import ReactiveHTML

class Cytoscape(ReactiveHTML):
    object = param.List()

    layout = param.Selector(default="cose", objects=["breadthfirst", "circle", "concentric", "cose", "grid", "preset", "random"])
    style = param.String("", doc="Use to set the styles of the nodes/edges")
    
    zoom = param.Number(1, bounds=(0,100))
    pan = param.Dict({"x": 0, "y": 0})

    data = param.List(doc="Use to send node's data/attributes to Cytoscape")

    selected_nodes = param.List()
    selected_edges = param.List()

    _template = """
        <div id="cy" style="width: 100%; height: 100%; position: relative; border: 1px solid"></div>
    """
    __javascript__ = ['https://cdnjs.cloudflare.com/ajax/libs/cytoscape/3.23.0/cytoscape.umd.js']

    _scripts = {
        'render': """
            self.create()
        """,
        "create": """
            if (state.cy == undefined){
                state.cy = cytoscape({
                container: cy,
                layout: {name: data.layout}, 
                elements: data.object,
                zoom: data.zoom,
                pan: data.pan,
                });
                state.cy.on('select unselect', function (evt) {
                  data.selected_nodes = state.cy.elements('node:selected').map((el)=>{return el.id()})
                  data.selected_edges = state.cy.elements('edge:selected').map((el)=>{return el.id()})
                });
                self.style()
                const mainEle = document.querySelector("body")
                mainEle.addEventListener("scrollend", (event) => {state.cy.resize().fit()})
                }; 
        """,
        'remove': """
        state.cy.destroy()
        delete state.cy
        """,
        "object": "state.cy.json({elements: data.object});state.cy.resize().fit()",
        'layout': "state.cy.layout({name: data.layout}).run()",
        "zoom": "state.cy.zoom(data.zoom)",
        "pan": "state.cy.pan(data.pan)",
        "style": """
state.cy.style().resetToDefault().append(data.style).update()
""",
    }

    _extension_name = 'cytoscape'

pn.extension('cytoscape', sizing_mode='stretch_width')

elements =  [{"data":{"id":'A', "label":'A'}},{"data":{"id":'B', "label":'B'}}, {"data":{"id": "A-B", "source":'A', "target":'B'}}]
graph = Cytoscape(object=elements, sizing_mode="stretch_width", height=600)
app = pn.Row(
    pn.Param(graph, parameters=["object", "zoom", "pan", "layout", "style", "selected_nodes", "selected_edges"], sizing_mode="fixed", width=300),
    graph
).servable()