Deserialization warning with Bokeh 2.1

[mrocklin]

The dashboard broke under 2.1. It has been resolved in #3904

However, we're still getting a deserialization error like the following:

 message: Message 'PATCH-DOC' content: {'references': [], 'events': [{'kind': 'ModelChanged', 'model': {'id': '1683'}, 'attr': 'text_align', 'new': {'value': 'right'}}, {'kind': 'ModelChanged', 'model': {'id': '1684'}, 'attr': 'text_align', 'new': {'value': 'right'}}, {'kind': 'ModelChanged', 'model': {'id': '1628'}, 'attr': 'reset_start', 'new': -8}, {'kind': 'ModelChanged', 'model': {'id': '1628'}, 'attr': 'reset_end', 'new': 0}]} 
 error: DeserializationError("Enum('left', 'right', 'center') expected str, got {'value': 'right'} of type dict")
Traceback (most recent call last):
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/server/protocol_handler.py", line 90, in handle
    work = await handler(message, connection)
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/server/session.py", line 67, in _needs_document_lock_wrapper
    result = func(self, *args, **kwargs)
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/server/session.py", line 261, in _handle_patch
    message.apply_to_document(self.document, self)
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/protocol/messages/patch_doc.py", line 100, in apply_to_document
    doc._with_self_as_curdoc(lambda: doc.apply_json_patch(self.content, setter))
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/document/document.py", line 1150, in _with_self_as_curdoc
    return f()
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/protocol/messages/patch_doc.py", line 100, in <lambda>
    doc._with_self_as_curdoc(lambda: doc.apply_json_patch(self.content, setter))
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/document/document.py", line 411, in apply_json_patch
    patched_obj.set_from_json(attr, value, models=references, setter=setter)
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/core/has_props.py", line 341, in set_from_json
    descriptor.set_from_json(self, json, models, setter)
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/core/property/descriptors.py", line 611, in set_from_json
    self.property.from_json(json, models),
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/core/property/bases.py", line 463, in from_json
    raise DeserializationError("%s expected %s, got %s of type %s" % (self, expected, json, type(json).__name__))
bokeh.core.property.bases.DeserializationError: Enum('left', 'right', 'center') expected str, got {'value': 'right'} of type dict

This comes about from the TaskProgress plot available here:

distributed/distributed/dashboard/components/scheduler.py

Lines 1488 to 1662 in 1d7640b

	class TaskProgress(DashboardComponent):
	""" Progress bars per task type """
	def __init__(self, scheduler, **kwargs):
	self.scheduler = scheduler
	data = progress_quads(
	dict(all={}, memory={}, erred={}, released={}, processing={})
	)
	self.source = ColumnDataSource(data=data)
	x_range = DataRange1d(range_padding=0)
	y_range = Range1d(-8, 0)
	self.root = figure(
	id="bk-task-progress-plot",
	title="Progress",
	name="task_progress",
	x_range=x_range,
	y_range=y_range,
	toolbar_location=None,
	tools="",
	**kwargs,
	)
	self.root.line( # just to define early ranges
	x=[0, 0.9], y=[-1, 0], line_color="#FFFFFF", alpha=0.0
	)
	self.root.quad(
	source=self.source,
	top="top",
	bottom="bottom",
	left="left",
	right="right",
	fill_color="#aaaaaa",
	line_color="#aaaaaa",
	fill_alpha=0.1,
	line_alpha=0.3,
	)
	self.root.quad(
	source=self.source,
	top="top",
	bottom="bottom",
	left="left",
	right="released-loc",
	fill_color="color",
	line_color="color",
	fill_alpha=0.6,
	)
	self.root.quad(
	source=self.source,
	top="top",
	bottom="bottom",
	left="released-loc",
	right="memory-loc",
	fill_color="color",
	line_color="color",
	fill_alpha=1.0,
	)
	self.root.quad(
	source=self.source,
	top="top",
	bottom="bottom",
	left="memory-loc",
	right="erred-loc",
	fill_color="black",
	fill_alpha=0.5,
	line_alpha=0,
	)
	self.root.quad(
	source=self.source,
	top="top",
	bottom="bottom",
	left="erred-loc",
	right="processing-loc",
	fill_color="gray",
	fill_alpha=0.35,
	line_alpha=0,
	)
	self.root.text(
	source=self.source,
	text="show-name",
	y="bottom",
	x="left",
	x_offset=5,
	text_font_size=value("10pt"),
	)
	self.root.text(
	source=self.source,
	text="done",
	y="bottom",
	x="right",
	x_offset=-5,
	text_align="right",
	text_font_size=value("10pt"),
	)
	self.root.ygrid.visible = False
	self.root.yaxis.minor_tick_line_alpha = 0
	self.root.yaxis.visible = False
	self.root.xgrid.visible = False
	self.root.xaxis.minor_tick_line_alpha = 0
	self.root.xaxis.visible = False
	hover = HoverTool(
	point_policy="follow_mouse",
	tooltips="""
	<div>
	<span style="font-size: 14px; font-weight: bold;">Name:</span>&nbsp;
	<span style="font-size: 10px; font-family: Monaco, monospace;">@name</span>
	</div>
	<div>
	<span style="font-size: 14px; font-weight: bold;">All:</span>&nbsp;
	<span style="font-size: 10px; font-family: Monaco, monospace;">@all</span>
	</div>
	<div>
	<span style="font-size: 14px; font-weight: bold;">Memory:</span>&nbsp;
	<span style="font-size: 10px; font-family: Monaco, monospace;">@memory</span>
	</div>
	<div>
	<span style="font-size: 14px; font-weight: bold;">Erred:</span>&nbsp;
	<span style="font-size: 10px; font-family: Monaco, monospace;">@erred</span>
	</div>
	<div>
	<span style="font-size: 14px; font-weight: bold;">Ready:</span>&nbsp;
	<span style="font-size: 10px; font-family: Monaco, monospace;">@processing</span>
	</div>
	""",
	)
	self.root.add_tools(hover)
	@without_property_validation
	def update(self):
	with log_errors():
	state = {
	"memory": {},
	"erred": {},
	"released": {},
	"processing": {},
	"waiting": {},
	}
	for tp in self.scheduler.task_prefixes.values():
	active_states = tp.active_states
	if any(active_states.get(s) for s in state.keys()):
	state["memory"][tp.name] = active_states["memory"]
	state["erred"][tp.name] = active_states["erred"]
	state["released"][tp.name] = active_states["released"]
	state["processing"][tp.name] = active_states["processing"]
	state["waiting"][tp.name] = active_states["waiting"]
	state["all"] = {
	k: sum(v[k] for v in state.values()) for k in state["memory"]
	}
	if not state["all"] and not len(self.source.data["all"]):
	return
	d = progress_quads(state)
	update(self.source, d)
	totals = {
	k: sum(state[k].values())
	for k in ["all", "memory", "erred", "released", "waiting"]
	}
	totals["processing"] = totals["all"] - sum(
	v for k, v in totals.items() if k != "all"
	)
	self.root.title.text = (
	"Progress -- total: %(all)s, "
	"in-memory: %(memory)s, processing: %(processing)s, "
	"waiting: %(waiting)s, "
	"erred: %(erred)s" % totals
	)

cc @bryevdv @jsignell

[jsignell]

It looks to me that if you remove the text_align = "right" line you will be good, but I don't know if that is desirable.

[mrocklin]

Is there a new way to align text?

[bryevdv]

No there is some bug: bokeh/bokeh#10181

…

On Tuesday, June 16, 2020, Matthew Rocklin ***@***.***> wrote: Is there a new way to align text? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3905 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIHJMBAU4NSNSD34OSW7MLRW6ARXANCNFSM4N7UINNQ> .

[mrocklin]

OK great. It sounds like the thing for us to do is to sit tight on the Dask side. Correct?

[bryevdv]

Yes seems we broke a few eggs with this release unfortunately.

[mrocklin]

It's ok, you all make a mean omelet.

[bryevdv]

@mrocklin @jsignell Bokeh 2.1.1 is available on PyPI and the bokeh channel on anaconda.org (presumably soon on C-F also) It should resolve this issue (but please test in your actual usage first!)

Tangentially: how can we help augment the downstream tests? I plan to raise the visibility of the adivsory downstream tests soon, but that said, they were all still passing even with both issues 2.1 introduced

[jsignell]

Would it make sense for dask to mark some tests as visualization tests and then bokeh could run them on some schedule? Probably released dask + latest bokeh is good enough for testing right?

[bryevdv]

@jsignell That's what we do already:

https://github.com/bokeh/bokeh/runs/798587187?check_suite_focus=true

But evidently not enough already, since it did not catch either of the issues with the 2.1 release.

[jsignell]

Yes sorry should have circled back after the dask dev meeting yesterday. I am going to look over the tests in dask and see if there are any holes in the testing that would be easy to fix.

[jakirkham]

Did we wind up resolving this or is this still an issue?

[bryevdv]

@jakirkham AFAIK Bokeh 2.1.1 resolved the immediate problem on the Bokeh side, I am not sure if version 2.1 can/was specifically excluded from requirements on the Dask side. Another option would be to set Bokeh 2.1.1 a a new min version which I would certainly endorse 😄 (it is ~18 months old at this point which seem like a lot in dask-time)

[jakirkham]

Guessing we would want to update this then

@jrbourbeau feel free to point any other spots I've missed :)

[bryevdv]

RIght, there's at least that and some docs spots IIRC, and there may be some old version-detection code that could be chucked out. If there's agreement to set 2.1.1 as a new minimum, I am happy to take a comprehensive look and submit a PR.

[jsignell]

Yeah I'd be happy to have the min be 2.1.1

[jrbourbeau]

Yeah, if bumping the minimum supported version to 2.1.1. will cut down on user pain or maintainer burden, then let's do it

[bryevdv]

See

• Bump Bokeh min version to 2.1.1 dask#8431
• Bump Bokeh min version to 2.1.1 #5548

[jakirkham]

Thanks Bryan! 😄