Ability to fully control the name of resources #1518
Comments
|
Note that you should be able to apply the I'll leave this issue open to track the idea of having some sort of global |
|
Uh? How do I do that? |
Sorry for the delay. In general, you can do this with: let x = new Resource("foo", {
name: "somethingotherthanfoo",
}); |
|
There are three ways I can imagine supporting this:
|
|
@lukehoban This feels like something we should get a head start on, at least design-wise, since it's been reported by several users. Is it worth assigning to someone to come up with a design proposal to review with the community during M19, even if implementation is deferred until M20? |
|
Any update on this? I don't understand why it defaults to mangling the names, and then having to provide it twice seems strange. Specifically on kubernetes this is really irritating. |
|
Some really good additional feedback on this topic at pulumi/pulumi-azure#209 (comment) which should be considered as part of changes here. |
|
We are planning on doing something here soon. As this has come up several times, I wanted to at least write down the reason Pulumi auto-names resources by appending a suffix, and refer to the documentation. We understand this is confusing and plan to explore options for what we can do here, including opt-in, opt-out, and explicit controls both globally and at the individual resource level. By auto-appending random suffixes (usually a string of hex), the engine is able to
These are non-trivial benefits, so whatever we do will need to weigh the tradeoffs involved here. For what it's worth, AWS CloudFormation does something very similar to this, for similar reasons. More complete documentation can be found at https://www.pulumi.com/docs/reference/programming-model/#autonaming. Thanks for the patience as we figure out the right path forward! |
I agree, and not all resources are allowed to customize, azure_native.containerservice.ManagedCluster didn't work. |
@joserafaelterra You should be able to set it with the |
|
I’m wondering if there’s a way to make this work without needing to touch all of our providers. The larger picture here is that it’s not pulumi per se that is in charge of autonaming. Instead, autonaming is handled on a per-resource basis with each provider. An important aspect of this is that pulumi does not know which properties of which resources are subject to autonaming. As a consequence of this approach, there’s no easy way to change autonaming within pulumi itself--these changes must happen within the providers. We have made a some recent progress towards making autonames more predictable and presentable: #8631. The title is somewhat misleading, as those changes alone do not make autonames deterministic. We’re still working on consuming these changes from the providers, but once they have been consumed, updated providers will be able to produce friendlier autonames using sequence numbers. If we want to be more flexible here (ala the changes suggested in #1518 (comment)), we’ll probably need to change the provider interface even further, e.g. by adding autoname information to Pulumi Package schemas and using that to handle autonaming within pulumi itself. If we want to make adjustments here without touching all of the providers, one approach might be to add a pulumi.autoname() API to our SDKs. This API could leave a sentinel value in the resource’s properties that pulumi is able to detect and then replace with an autoname. Parameterizing that API would allow users to control autonaming on a very fine-grain basis at the cost of needing (oddly enough) to explicitly autoname resources. |
|
Just want to add another example that is not directly Pulumi issue, but if the auto-naming could have been configured, I could have avoided this issue as well: GoogleCloudPlatform/buildpacks#172. TL;DR: some builders require name extension to have e.g. |
If providers could tell us which property was a name as part of the schema we wouldn't even need a special sentinel value here, we could just make it an optional property and if the engine sees it's null it could generate the autoname before passing off to the provider to create. N.B Even if we changed autonames to work like above I think we still want sequence numbers on Check because providers might want other pseudo-random properties and we don't want the engine to know about every possible one of them. |
There are different auto-naming strategies across providers. Azure Native would define whether to add a random suffix based on the resource type (e.g. Cosmos Account is globally unique and thus randomized, but Cosmos Database is unique per server and thus is just named with a straight logical name); Google Native uses composite names with slashes for some resources, etc. If we want to do this in the engine, we'd need to negotiate the autonaming format between the engine, the provider, and (potentially) the user configuration. I guess the schema could have some templated way to describe the auto-name format. |
|
@mikhailshilkov thanks for that didn't realize it was so varying, makes a lot of sense to just leave this in the providers then and deal with that we're going to need to do some updates. |
|
One option here that is possible today, though not quite as expressive as what is being discussed in other comments above, is to use stack transformations to change the naming strategy for some or all resources. For example: const prefix = `${pulumi.getProject()}-${pulumi.getStack()}-`;
pulumi.runtime.registerStackTransformation(args => {
if (args.props.name === undefined) {
return {
props: {...args.props, name: prefix + args.name },
opts: args.opts,
};
}
return undefined;
});This will construct the physical name for all resources to be a standard prefix followed by the logical name. This transformation could also choose to do different things for different resource types (or only apply this for certain resource types). The one big limitation to this approach is that it cannot gain access to the random value that is part of the current autonaming approach. Because of that, this approach will lead to delete-before-create replacement semantics (since the physical name is the same, and thus the resource must be deleted before the name can be reused). |
|
Is there any chance to have this issue resolved ? I see is stays unresolved for 4 years already. Auto naming consumes 8 characters, and if the cloud provider has limitation on naming length, it can become a issue to set any meaningful naming. |
|
@amuntean-godaddy There are a few tradeoffs around this issue, so it keeps slipping through as under-specified. Apologies for delays here. We are planning to make another attempt to suggest a design proposal for this issue within Q3'22. We will post it here for discussion, and then, hopefully, the actual implementation will follow shortly after. |
|
@lukehoban Here is the naming restrictions and rules on Azure we were talking about the other day. Having these rules hard coded in the Azure Native provider would be great to have valid random names automatically. |
Thank you for your work, but please don't make decisions for the client, give us a global switch, and everyone is happy. |
This isn't really about us making decisions for the client, this is about us trying to work out how to even do this from an engineering perspective. We fully understand that this would be a useful feature, we're trying to work out how to do add it without this being overly expensive/complicated to implement.
A global switch to turn off autoname suffixes? I don't think that will make everyone happy, there are other requests around naming that are more than just not adding the random suffixes. That's a lot of engineering effort, and it's not clear it's worth it and I probably haven't even thought of all the other complexities involved here. |
It's really hard, and you deserve respect. |
Have you considered using stack outputs to do that? https://www.pulumi.com/learn/building-with-pulumi/stack-outputs/ It's not deterministic, but it might still solve your information sharing problems. |
I posted with the above code and it was ok the first time, if I run it again I get the following, actually nothing has changed, I think it's caused by " auto-naming", the google brought me here. |
may be, It's a way, but why can't it be done according to the agreed documentation, some people don't want to write extra code, and more importantly, some development teams don't have permissions to list all S3 buckets. |
For sure it was just a suggestion that might have been applicable in your case. I totally understand that some teams need the deterministic names that they then share via other channels. Scenarios like that are why we understand this issue to be valuable to fix.
I would suggest raising that issue at https://github.com/pulumi/pulumi-kubernetes/ it might be an issue with helm specifically. |
|
Is overriding auto-naming as supported not sufficient? |
That assumes that there's no atomic rename-and-reconfigure operation, or that a separate renaming operation (similar to temporary file naming algorithms) is not a feasible strategy.
If Resources are created in the context of a Resource Component's constructor, then they could auto-name differently. This is a software engineering challenge, not a fundamental problem.
Okay, so I understand some people don't want to supply the name as resource constructor argument twice. But you can use variables for that, if DRY is the concern. Personally, the most pressing concern I have is the arbitrary and limiting naming rules that cloud providers impose and how that will interact with any naming algorithm Pulumi devises. For example, Azure limits some resources such as Storage Accounts to 24 characters. In terms of declarative style, idempotency etc. a lot can be said for predictable resource names within the provider's namespace. @oplancelot's professional collaboration requirements are of course also entirely realistic, I have them too. Even if the ‘collaboration’ is very advanced, in the form of automated interoperability. Or of the ability for other tools to statically analyze Pulumi source code (as in, resource specs created by users). All of this becomes harder by Pulumi auto-naming opaquely. I think the whole concept of Stacks is problematic and more trouble than it's worth. Instead of Stacks, the caller could supply a name prefix when standing up, much to the same effect, just without facilities for Stack-specific configuration. Because of Stacks, you have had to introduce the concept of ‘fully-qualified Stack names’ (#11387) as well because of collisions. I think Pulumi currently muddies its functionality between an interactive configuration tool and an infra-as-code-library. It is normal for infra-as-code that things like different environments and their configuration, naming etc. have to be solved by the user (developer). There are many approaches to that depending on the use case. Either static naming, querying some namespace, generating names either human-readable or machine-readable or both, guaranteed unique or just random. |
|
By the way, another interesting problem. Some resources get cloud provider names even as they're not first-class resources in Pulumi. For example, |
|
To add to the complexity here I would like to adjust the auto-naming strategy. My current example is an Azure web app with multiple environments with one stack per environment. I would like to use auto-naming so that I get the random suffixes that makes replacement easier, but I would also like to include the stack name in the auto naming so that globally unique resources such as Azure ContainerRegistry doesn't collide between stacks. At the moment I include the application name and stack name in each resource name to guide the auto naming, like this: const webAppRegistry = new azure.containerregistry.Registry(`MyWebAppRegistry${stack}`, {
resourceGroupName: resourceGroup.name,
...
}But this gets rather tedious and means that all resources have longer names than are necessary. I would rather control it via a callback or format, something like this: const stack = pulumi.getStack()
pulumi.setAutoNamingFormat("MyWebApp{resourceName}" + stack + "{randomSuffix}");
const webAppRegistry = new azure.containerregistry.Registry(`Registry`, {
resourceGroupName: resourceGroup.name,
...
}Where Pulumi would substitute {resourceName} and {randomSuffix} with proper values. |
|
I love @Raniz85 suggestion, because I would use a hyphen in the auto naming scheme (to adhere to our naming standards) but right now I need to finish the pulumi object name with a dangling hyphen in the end to achieve this, which is terrible visually. Also there are some object in some libraries (like azure native app gateway) which needs to refer to it's own name in it's definition, but with the autonaming in place, it becomes impossible. I used an workaround using @pulumi/random but this needs me manually controlling when to rotate the name whenever the resource needs to be replaced. Having a way to refer to the name after autonaming is in place would help a lot on these cases. |
|
Nobody else mentioned it here, so I am going to drop my $0.02. This might be controversial: I think the Logical and Physical names of resources are two completely different things and should rarely be in any way the same. I will admit that forcing of suffixes to generate physical names of resources based on the logical name does solve some problems, but it also introduces other problems:
A perfect example of this is an Aurora cluster. I have a project that creates Aurora clusters and RDS instances (there is enough commonality here) and the logical names of the cluster and instances look like this:
If I were to let auto-naming take over from here I would get resource names like this:
Logging into the console and seeing this you would have no idea what DB was what and tools like Teleport would be unusable for our end users (constantly needing to look up the ID to know which DB they are logging into). Of course there are many compromises like including those identifiers in the logical name (makes reading the Pulumi output just that bit more annoying and less consistent between stacks) but then leaving in superfluous logical items (like I appreciate there are |
|
I have been using Azure for over 10 years and 4+ years of using Terraform professionally. I thought I would try Pulumi out personally to see how I would like it. I just stared a little bit ago and already; I found this automatic naming policy to be incredibly irritating. So much so that I don't think I'll bother with Pulumi in the future. I understand putting on the Mickey Mouse gloves for new users because as a new user I want to "fall into the pit of success". But if I want to take off the gloves, please let me manage the automatic naming policy globally. And the inconsistency is also baffling. The docs say...
Well that would be better if they all had a |
I get your point and the documentation does just say |
|
Well four years later, the first issue I encountered when starting with pulumi today ;-) |
and others
I've already filed pulumi/pulumi-cloud#504 but this issue is more general.
When I provision Azure resources, they get created with postfixes (random strings appended at the end), e.g.
pulumifunc1sacd7c2fa0.I would like to have the ability to define the full name of any resource created by pulumi in full:
@joeduffy's initial comment:
The text was updated successfully, but these errors were encountered: