Dagster 1.7.5 - 'Value is not JSON serializable' returning metadata

[the4thamigo-uk]

Dagster version

dagster, version 1.7.5

What's the issue?

I have an asset that obtains the metadata for an upstream asset (via DagsterInstance.get_event_records()), and attaches this upstream metadata as a subkey in its own Output metadata.

When I obtain the upstream metadata via AssetMaterialization.metadata, it has the form :

upstream_metadata = {
  "data": JsonMetadataValue(...)
}

I pass this metadata on in the following way, and in dagster 1.7.4 it used to work :

return Output(None, metadata = {  
  "upstream": {
    "metadata": upstream_metadata
  }
})

But in dagster 1.7.5 I get :

dagster._core.errors.DagsterInvalidMetadata: Could not resolve the metadata value for "upstream" to a known type. Value is not JSON serializable.

It seems that the regression is due to code in Output because it looks like the metadata I retrieve from the upstream AssetMaterialization has not changed in form between 1.7.4 and 1.7.5 .

What did you expect to happen?

It used to work in 1.7.4 but not in 1.7.5

How to reproduce?

No response

Deployment type

None

Deployment details

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

[kubiani]

Thinking this might be caused by #21576