Missing (NaT) timestamps appear as 1970-01-01 in tooltips rather than "null"

[quantoid]

Tooltips show missing values (NaT) in a Timestamp column as "Jan 01, 1970" (the epoch) rather than "null" used for other types.

This makes it hard to show timestamps in tooltips because the missing values are misleading for users not familiar with the idea of an epoch; they think "wtf, is this data over 50 years old?!".

If this is a "feature" of Vega then Altair could at least document the issue and provide a way to work around it, e.g. make tooltip conditional on the datum being valid in the generated Vega JSON spec.

---

Please follow these steps to make it more efficient to solve your issue:

• [/] Since Altair is a Python wrapper around the Vega-Lite visualization grammar, most bugs should be reported directly to Vega-Lite. You can click the Action Button of your Altair chart and "Open in Vega Editor" to create a reproducible Vega-Lite example and see if you get the same error in the Vega Editor.
• [/] Search for duplicate issues.
• [/] Use the latest version of Altair.
• [/] Describe how to reproduce the bug and include the full code and data to reproduce it, ideally using a sample data set from vega_datasets.

[mattijn]

I understand what you mean and I agree that if a NaT is returned as a datetime from the past (albeit being Unix epoch) it leads to confusion, something we should avoid.

We'll have to figure out if we can solve this within Altair, or if this needs to be reported upstream Vega(-lite/-tooltip) repository.

[binste]

I also agree with the desired behaviour but I don't think we can do something on the Altair level as Altair properly uses a null in the Vega-Lite spec. The issue is reported upstream at vega/vega-lite#6417 so I'd suggest to close it here. What do you think @mattijn?

[mattijn]

Yes, it is unfortunate we cannot do something about this here. I think the behavior of the tooltip is quite confusing and even though it is already prioritized as P1 in the referred vega-lite issue we can try to bring it up again and highlight it.

I made a mistake in the following specification, that highlights another issue:

import pandas as pd
import altair as alt

df = pd.DataFrame({'a': ['A', 'B', 'C'], 'b': [28, 55, 43], 'c': [None, "2020-04-27", None], 'd': [0, 1000, None]})
alt.Chart(df).mark_bar().encode(x='a:O', y='b:Q', tooltip=['a:O', 'b:Q', 'c:T', 'd:Q'], color='d:N')

For the color encoding channel I used d:N (nominal), but only quantitative and temporal field types will filter None values, so currently there is a null legend item. So far so good, but by coincident I used d:Q (quantitative) in the tooltip and in that case the None values are interpret as zeroes. Even though there is already another real 0 for field d.

We have https://altair-viz.github.io/user_guide/transform/impute.html describing how to fill-in missing entries, but for me this feels as second step. I somehow miss the first step in the documentation with a clear section how None / NaN / NaT values are interpreted within Altair (Vega-Lite) like:

• sometimes filtered, depending on the the type
• when not filtered, interpret as zeroes.
• how to filter invalid values? Is this a good method: .transform_filter(alt.FieldValidPredicate(field='c', valid=True))?