[sdk/go] pulumi.JSONMarshal doesn't support unknowns.

[dixler]

What happened?

I'm writing the documentation for inputs-outputs and adding the json support and I'm currently on Go. Go seems like the prickliest of our SDKs and I've run into some issues (potentially because I'm not familiar). It looks like from the PR description there is some outstanding work to make the experience with JSONMarshal better.
#11609

Expected Behavior

Doesn't crash.

Steps to reproduce

package main

import (
	"github.com/pulumi/pulumi-random/sdk/v4/go/random"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		pet, err := random.NewRandomPet(ctx, "parent", &random.RandomPetArgs{},
			pulumi.Protect(true))
		_ = err
		petOut := pulumi.JSONMarshal(pet.ToRandomPetOutput()) // offending line

		ctx.Export("petOut", petOut)
		return nil
	})
}

I'm getting the following error:

  pulumi:pulumi:Stack (prog-aws-go-dev):
    error: an unhandled error occurred: program failed:
    waiting for RPCs: json: error calling MarshalJSON for type pulumi.MapOutput: Outputs can not be marshaled to JSON

Output of pulumi about

% pulumi about
CLI          
Version      3.56.1-dev.0
Go Version   go1.20
Go Compiler  gc

Plugins
NAME    VERSION
go      unknown
random  4.8.2

Host     
OS       arch
Version  22.0.4
Arch     x86_64

This project is written in go: executable='/usr/bin/go' version='go version go1.20 linux/amd64'

Current Stack: pulumi/prog-aws-go/dev

TYPE                              URN
pulumi:pulumi:Stack               urn:pulumi:dev::prog-aws-go::pulumi:pulumi:Stack::prog-aws-go-dev
pulumi:providers:random           urn:pulumi:dev::prog-aws-go::pulumi:providers:random::default
random:index/randomPet:RandomPet  urn:pulumi:dev::prog-aws-go::random:index/randomPet:RandomPet::parent


Found no pending operations associated with pulumi/dev

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[abhinav]

We don't have a solution yet, but copying part of the Slack discussion between me and @Frassle here:

A: [one] option is: don't convert [to map[string]any] or try to replicate json.Marshal behavior. Instead, traverse the struct with reflection once, track all Outputs, build a pulumi.All from them, and inside its Apply, call json.Marshal. All the MarshalJSON methods just take the actual value and serialize it (erroring if the output is not done, in case someone used json.Marshal instead of pulumi.JSONMarshal).

F: So have JSONMarshal handle the secretness/unknownness via reflection walk, and then have the standard Marshal calls just be blocking calls that either return the value marshalled or error if it was unknown (at which point our JSONMarshal wouldn't have called it at all).
F: Bit odd that this gives you an "out" from output space via json.Marshal if you happen to only have known values, I can see that tripping some users up.

A: If we want extra safety around it, OutputState could have a bit on it indicating that it had been touched by pulumi.JSONMarshal, only after which will it serialize to MarshalJSON.
A: Long-term, it wouldn't be the worst idea to have an Output-aware JSON marshaller. It could be a [fork] of encoding/json's serialization code, or something else.

[justinvp]

As part of fixing this, also need to remove the note for the Go example in the docs at https://www.pulumi.com/docs/intro/concepts/inputs-outputs/#converting-outputs-to-json