Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JSONMarshal to go sdk #11609

Merged
merged 1 commit into from Dec 10, 2022
Merged

Add JSONMarshal to go sdk #11609

merged 1 commit into from Dec 10, 2022

Conversation

Frassle
Copy link
Member

@Frassle Frassle commented Dec 9, 2022

Description

This is not as useful for Go as it is for other languages, but it's consistent and it saves an Apply call.

Ideally we would handle nested Output types during marshalling, but to do that correctly we need some way to get access to per-call-to-JSONMarshal flags to note if we see any unknowns, or secrets.

Go just doesn't (AFAIK) have any way to support doing that.

So we do the next best thing which is we at least error that nested Outputs can't be marshalled. Note that the Output interface doesn't embed the Marshal interface because there may be users who have wrote their own Output types and that would be a breaking change.

Checklist

  • I have added tests that prove my fix is effective or that my feature works
  • I have run make changelog and committed the changelog/pending/<file> documenting my change
  • Yes, there are changes in this PR that warrants bumping the Pulumi Service API version

@Frassle Frassle requested a review from iwahbe December 9, 2022 15:03
@pulumi-bot
Copy link
Contributor

Changelog

[uncommitted] (2022-12-09)

Features

  • [sdk/go] Add JSONMarshal to go sdk.
    #11609

@iwahbe
Copy link
Member

iwahbe commented Dec 9, 2022

Ideally we would handle nested Output types during marshalling, but to do that correctly we need some way to get access to per-call-to-JSONMarshal flags to note if we see any unknowns, or secrets.

Can you provide an example here? I have other issues with the encoding/json custom marshaler interface, but I don't understand how it relates to unknowns or secrets.

sdk/go/pulumi/types.go Outdated Show resolved Hide resolved
@Frassle
Copy link
Member Author

Frassle commented Dec 9, 2022

Can you provide an example here? I have other issues with the encoding/json custom marshaler interface, but I don't understand how it relates to unknowns or secrets.

Yup firstly how would you make TestJSONMarshalNested work but with a successful json result of "[0, 1]" rather than the current code where it errors.

Secondly how would you make that work if one of the nested secrets was unknown? The overall json parse should then be unknown. Likewise for secrets.

This is not as useful for Go as it is for other languages, but it's
consistent and it saves an Apply call.

Ideally we would handle nested Output types during marshalling, but to
do that correctly we need some way to get access to
per-call-to-JSONMarshal flags to note if we see any unknowns, or
secrets.

Go just (AFAIK) have any way to support doing that.

So we do the next best thing which is we at least error that nested
Outputs can't be marshalled. Note that the `Output` interface doesn't
embed the `Marshal` interface because there _may_ be users who have
wrote their own `Output` types and that would be a breaking change.
@iwahbe
Copy link
Member

iwahbe commented Dec 9, 2022

Can you provide an example here? I have other issues with the encoding/json custom marshaler interface, but I don't understand how it relates to unknowns or secrets.

Yup firstly how would you make TestJSONMarshalNested work but with a successful json result of "[0, 1]" rather than the current code where it errors.

Taking another look, I think this would require leaving encoding/json framework. 😒 You would need to manually marshal into map[string]any and []any, then call json.Marshal. Never mind then.

Secondly how would you make that work if one of the nested secrets was unknown? The overall json parse should then be unknown. Likewise for secrets.

If the value is secret, I would mark the whole thing as secret. If its unknown, I would abort, and mark the returned value as unknown.

@iwahbe iwahbe self-requested a review December 9, 2022 18:21
@Frassle
Copy link
Member Author

Frassle commented Dec 9, 2022

If the value is secret, I would mark the whole thing as secret. If its unknown, I would abort, and mark the returned value as unknown.

Yup that's what this method does in dotnet and python. The fun is how you track that in Go where you don't have any async context to grab inside the marshal methods.

I think your right we could do better here if we didn't use encoding/json. That might be worth doing.

@iwahbe
Copy link
Member

iwahbe commented Dec 9, 2022

We could replicate the encoding/json interface, observing json tags and json.Marshaller. We would just allow ourselves a better internal API to handle outputs. We can consider that later. Improving JSONMarshal internal API will be a backward compatible change, allowing strictly more values to be serialized.

@Frassle
Copy link
Member Author

Frassle commented Dec 10, 2022

bors merge

@bors
Copy link
Contributor

bors bot commented Dec 10, 2022

Build succeeded:

@bors bors bot merged commit caf5e17 into master Dec 10, 2022
@bors bors bot deleted the fraser/goJsonMarshal branch December 10, 2022 01:11
@justinvp justinvp mentioned this pull request Jan 21, 2023
31 tasks
@lukehoban lukehoban mentioned this pull request Mar 27, 2023
19 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants