Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inclusion of a / in a datasources module name produces incorrect tokens in schema.json for children properties #611

Closed
phillipedwards opened this issue Sep 22, 2022 · 2 comments · Fixed by pulumi/pulumi#10935, #622 or #624
Assignees
Labels
customer/feedback Feedback from customers kind/bug Some behavior is incorrect or out of spec resolution/fixed This issue was fixed
Milestone

Comments

@phillipedwards
Copy link
Member

What happened?

In resources.go when a datasource's module name includes a 2 parts separated by a / the resulting token, in schema.json, is correctly produced for the corresponding get function, however, the tokens for the function's children properties are incorrectly produced, if the properties are non-primitives.

For example, the datasource aws_iam_policy_document when given a module name of x/iam gets a corresponding token in schema.json of aws:x/iam/getPolicyDocument:getPolicyDocument. This is correct, however, if you examine the statements property, which is a non-primitive, you'll notice the token is missing the iam portion of the module. Eg- aws:x/getPolicyDocumentStatement:getPolicyDocumentStatement.

Steps to reproduce

  1. git clone git@github.com:phillipedwards/pulumi-aws.git
  2. cd pulumi-aws
  3. make tfgen
  4. Open schema.json and examine the get function aws:x/iam/getPolicyDocument:getPolicyDocument note: we've introduced the x/ to detail the function is experimental.
  5. make build_python
  6. Open sdk/python/pulumi_aws/x/iam/get_policy_document.py and go to line 105.
  7. The def statements... function references a non-existent type of _x.outputs.GetPolicyDocumentStatement

Expected Behavior

  1. In schema.json the tokens for the non-primitive children of getPolicyDocument should have tokens that begin with aws:x/iam/.... Eg- aws:x/iam/getPolicyDocumentStatement:getPolicyDocumentStatement
  2. The generated python sdk function pulumi_aws.x.iam.get_policy_document.py should be referencing GetPolicyDocumentStatementResult as the return type for def statements...

Actual Behavior

  1. The non-primitive children tokens incorrectly start with aws:x/..... See, #/types/aws:x/getPolicyDocumentStatement:getPolicyDocumentStatement.
  2. The statements function signature is def statements(self) -> Optional[Sequence['_x.outputs.GetPolicyDocumentStatement']]:

Output of pulumi about

No response

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

@phillipedwards phillipedwards added needs-triage Needs attention from the triage team kind/bug Some behavior is incorrect or out of spec labels Sep 22, 2022
@danielrbradley danielrbradley removed the needs-triage Needs attention from the triage team label Sep 23, 2022
@viveklak viveklak self-assigned this Oct 4, 2022
@viveklak viveklak added this to the 0.79 milestone Oct 4, 2022
@viveklak viveklak added the customer/feedback Feedback from customers label Oct 4, 2022
@t0yv0
Copy link
Member

t0yv0 commented Oct 5, 2022

There is an underlying codegen bug here. pulumi/pulumi#10935 makes codegen generate the right type reference (GetPolicyDocumentStatementResult).

@t0yv0
Copy link
Member

t0yv0 commented Oct 5, 2022

Now looking into why the tokens are incorrect.

bors bot added a commit to pulumi/pulumi that referenced this issue Oct 6, 2022
10935: [codegen/python] Fix incorrect ObjectType name generated under tfbridge20 compat r=t0yv0 a=t0yv0

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

Python codegen gets confused about which modContext a given ObjectType belongs to, which may result in dangling type references being generated (when incorrect typeDetails being looked up for the ObjectType).

This PR comes from investigating user-reported pulumi/pulumi-terraform-bridge#611 although it does not in itself completely solve it. 

The minified example has the following:

- a function is declared

- one of the supplemental types in this function comes from a different module in the same package (by mistake - to fix pulumi/pulumi-terraform-bridge#611 this needs to change also), which is supported in theory

- function code is generated thinking this supplemental type belongs to module "x/iam" module

- but the type itself is generated thinking it belongs to "x" module

- the modContext for "x/iam" and "x" do not agree on "typeDetails", which under "tfbridge20" compatibility influences type name generation; therefore the code generates a dangling type reference `_x.outputs.GetPolicyDocumentStatement` where `_x.outputs.GetPolicyDocumentStatementResult` is expected


<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Fixes # (issue)

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Anton Tayanovskyy <anton@pulumi.com>
iwahbe pushed a commit to pulumi/pulumi that referenced this issue Oct 10, 2022
10935: [codegen/python] Fix incorrect ObjectType name generated under tfbridge20 compat r=t0yv0 a=t0yv0

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

Python codegen gets confused about which modContext a given ObjectType belongs to, which may result in dangling type references being generated (when incorrect typeDetails being looked up for the ObjectType).

This PR comes from investigating user-reported pulumi/pulumi-terraform-bridge#611 although it does not in itself completely solve it. 

The minified example has the following:

- a function is declared

- one of the supplemental types in this function comes from a different module in the same package (by mistake - to fix pulumi/pulumi-terraform-bridge#611 this needs to change also), which is supported in theory

- function code is generated thinking this supplemental type belongs to module "x/iam" module

- but the type itself is generated thinking it belongs to "x" module

- the modContext for "x/iam" and "x" do not agree on "typeDetails", which under "tfbridge20" compatibility influences type name generation; therefore the code generates a dangling type reference `_x.outputs.GetPolicyDocumentStatement` where `_x.outputs.GetPolicyDocumentStatementResult` is expected


<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Fixes # (issue)

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Anton Tayanovskyy <anton@pulumi.com>
@t0yv0 t0yv0 mentioned this issue Oct 12, 2022
@pulumi-bot pulumi-bot added the resolution/fixed This issue was fixed label Oct 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment