pulumi convert to Go generates incorrect code and go.mod for component usage

[lukehoban]

Creating the default pulumi new serverless-aws-yaml and then pulumi convert --language go --out ./go fails with this:

Installing dependencies...

go: finding module for package github.com/pulumi/pulumi-aws-apigateway/sdk/apigateway/index
go: finding module for package github.com/pulumi/pulumi-aws-apigateway/sdk/apigateway
demo2 imports
        github.com/pulumi/pulumi-aws-apigateway/sdk/apigateway: module github.com/pulumi/pulumi-aws-apigateway/sdk@latest found (v0.0.10), but does not contain package github.com/pulumi/pulumi-aws-apigateway/sdk/apigateway
demo2 imports
        github.com/pulumi/pulumi-aws-apigateway/sdk/apigateway/index: module github.com/pulumi/pulumi-aws-apigateway/sdk@latest found (v0.0.10), but does not contain package github.com/pulumi/pulumi-aws-apigateway/sdk/apigateway/index
error: installing dependencies failed; rerun manually to try again, then run `pulumi up` to perform an initial deployment: `go mod tidy` failed to install dependencies: exit status 1

The code it generates is also wrong in referencing github.com/pulumi/pulumi-aws-apigateway/sdk/apigateway/index and in using unqualified references to types from apigateway like RouteArgs.

package main

import (
	"encoding/json"

	"github.com/pulumi/pulumi-aws-apigateway/sdk/apigateway"
	"github.com/pulumi/pulumi-aws-apigateway/sdk/apigateway/index"
	"github.com/pulumi/pulumi-aws/sdk/v5/go/aws"
	"github.com/pulumi/pulumi-aws/sdk/v5/go/aws/iam"
	"github.com/pulumi/pulumi-aws/sdk/v5/go/aws/lambda"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		tmpJSON0, err := json.Marshal(map[string]interface{}{
			"Version": "2012-10-17",
			"Statement": []map[string]interface{}{
				map[string]interface{}{
					"Action": "sts:AssumeRole",
					"Effect": "Allow",
					"Principal": map[string]interface{}{
						"Service": "lambda.amazonaws.com",
					},
				},
			},
		})
		if err != nil {
			return err
		}
		json0 := string(tmpJSON0)
		role, err := iam.NewRole(ctx, "role", &iam.RoleArgs{
			AssumeRolePolicy: pulumi.String(json0),
			ManagedPolicyArns: pulumi.StringArray{
				pulumi.String("arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"),
			},
		})
		if err != nil {
			return err
		}
		fn, err := lambda.NewFunction(ctx, "fn", &lambda.FunctionArgs{
			Runtime: pulumi.String("python3.8"),
			Handler: pulumi.String("handler.handler"),
			Role:    role.Arn,
			Code:    pulumi.NewFileArchive("./function"),
		})
		if err != nil {
			return err
		}
		api, err := apigateway.NewRestAPI(ctx, "api", &apigateway.RestAPIArgs{
			Routes: []RouteArgs{
				&RouteArgs{
					Path:      "/",
					LocalPath: "www",
				},
				&RouteArgs{
					Path:         "/date",
					Method:       index.MethodGET,
					EventHandler: fn.ID(),
				},
			},
		})
		if err != nil {
			return err
		}
		ctx.Export("url", api.Url)
		return nil
	})
}

The YAML source code from the default template is this:

name: demo2
description: A serverless app on AWS API Gateway and Lambda
runtime: yaml
resources:
    # An execution role to use for the Lambda function
    role:
        type: aws:iam:Role
        properties:
            assumeRolePolicy:
                fn::toJSON:
                    Version: 2012-10-17
                    Statement:
                        - Action: sts:AssumeRole
                          Effect: Allow
                          Principal:
                            Service: lambda.amazonaws.com
            managedPolicyArns:
                - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
    # A Lambda function to invoke
    fn:
        type: aws:lambda:Function
        properties:
            runtime: python3.8
            handler: handler.handler
            role: ${role.arn}
            code:
                fn::fileArchive: ./function
    # A REST API to route requests to HTML content and the Lambda function
    api:
        type: aws-apigateway:RestAPI
        properties:
            routes:
                - path: /
                  localPath: www
                - path: /date
                  method: GET
                  eventHandler: ${fn}
outputs:
    # The URL at which the REST API will be served.
    url: ${api.url}

[iwahbe]

I've transferred the issue from pulumi-yaml to pu/pu. The generated PCL looks correct, and other languages generate valid code. I think this is a problem with the go language generator.

Generated PCL:

resource role "aws:iam/role:Role" {
	__logicalName = "role"
	assumeRolePolicy = toJSON({
		"Version" = "2012-10-17",
		"Statement" = [{
			"Action" = "sts:AssumeRole",
			"Effect" = "Allow",
			"Principal" = {
				"Service" = "lambda.amazonaws.com"
			}
		}]
	})
	managedPolicyArns = ["arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"]
}

resource fn "aws:lambda/function:Function" {
	__logicalName = "fn"
	runtime = "python3.9"
	handler = "handler.handler"
	role = role.arn
	code = fileArchive("./function")
}

resource api "aws-apigateway:index:RestAPI" {
	__logicalName = "api"
	routes = [
		{
			path = "/",
			localPath = "www"
		},
		{
			path = "/date",
			method = "GET",
			eventHandler = fn
		}
	]
}

output url {
	__logicalName = "url"
	value = api.url
}

[iwahbe]

Progress so far:

• Mark external modules as forign #11437 makes sure that all the external types are appropriately qualified.
• Cleanup import generation #11440 removes the erroneous "github.com/.../apigateway/index".
• Don't import types referenced from local variables #11442 removes the erroneous "github.com/pulumi/pulumi-aws/sdk/v5/go/aws"

[iwahbe]

Filed issue: pulumi/pulumi-aws-apigateway#64.
The following error is because pulumi-aws-apigateway sets the wrong languages.go.importBasePath.

go: finding module for package github.com/pulumi/pulumi-aws-apigateway/sdk/apigateway
demo2 imports
        github.com/pulumi/pulumi-aws-apigateway/sdk/apigateway: module github.com/pulumi/pulumi-aws-apigateway/sdk@latest found (v0.0.10), but does not contain package github.com/pulumi/pulumi-aws-apigateway/sdk/apigateway

It sets the base path as github.com/pulumi/pulumi-aws-apigateway/sdk/apigateway when it should set it as github.com/pulumi/pulumi-aws-apigateway/sdk/go/apigateway (notice the /go between the /sdk and /apigateway:

https://github.com/pulumi/pulumi-aws-apigateway/blob/d8a5e8d6264c51d0b979628e6b6d641bd1f0f97c/schema.yaml#L444

We can see a correct example in the pulumi-aws package:

https://github.com/pulumi/pulumi-aws/blob/b9f8fd1106181730aa644641950c1707525656bf/provider/cmd/pulumi-resource-aws/schema.json#L202

[iwahbe]

As each of the underlying problems were fixed, we can now close the issue.