Not able to pass agent's environment variable [ env()] to array in variable default definition

[trlvsk]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

When filing a bug, please include the following headings if possible. Any
example text in this template can be deleted.

Overview of the Issue

I'm working with bamboo pipelines and using it's agents with a lot of environment variables.
So far I've been passing multiple of those using env() function such as names, architecture, technology, dynamic paths, versions, and many other which to this point was working like a charm - Example below.

variable buildNumber {
  type    = string
  default = env("bamboo_buildNumber")
}

variable branchName {
  type    = string
  default = env("bamboo_shortPlanBranchName")
  validation {
    condition     = length(var.branchName) > 1 && length(var.branchName) < 73
    error_message = "The length of branchName variable must be between 2 and 72 characters long."
  }
}

Recently due to redefinition on my organization AWS accounts i need to pass to ami_users variable 3 separate ID's (Dev / Pre-Prod / Prod)
There are more than 20 different accounts and each of those 20 have their own (Dev / Pre-Prod / Prod).
ID's for those 3 accounts are always stored within 3 separate bamboo variables.
I'm having trouble though to add them into the default array while defining ami_users variable
ami_users (or rather list of shared accounts) after creation of AMI is empty

The ultimate question is if Packer allows adding default array using env() function or not as documentation does not specify that. (If that is possible I'd love to see where am I wrong below.)

Reproduction Steps

When pipeline is ran with AWS account ID's hardcoded either in packer.hcl or in the packer-config then AMI is created perfectly fine and shared to those accounts.
When I run it with defaults only the AMI is not shared with no accounts.
Bamboo variables itself are not the problem in this case because they work like a charm for every other variable.
The issue is when I need to pack those into an array.

Below pasting two approaches that passed the builds but in the end did not share the ami to desired accounts.
In other words array passed is empty and AMI is not shared at all.

##Attempt 1
variable ami_users {
  type        = list(string)
  default     = [env("bamboo_AWS_ACCOUNT_ID_DEV"), env("bamboo_AWS_ACCOUNT_ID_PREPROD"), env("bamboo_AWS_ACCOUNT_ID_PROD")]
}

##Attempt 2
variable ami_users {
  type        = list(string)
  default     = ["${env("bamboo_AWS_ACCOUNT_ID_DEV")}", "${env("bamboo_AWS_ACCOUNT_ID_PREPROD")}", "${env("bamboo_AWS_ACCOUNT_ID_PROD")}"]
}

Packer Version

Packer 1.9.4

Operating system and Environment details

Debian Bookworm and/or AL2023

[lbajolet-hashicorp]

Hey @trlvsk,

I guess coalesce may do the trick for you? Something like coalesce(env("bamboo_AWS_ACCOUNT_ID_DEV"), "<default_value>") for example.

This would return the first non-empty string value among the arguments passed to coalesce, so from my understanding that'd work for you as if the env doesn't define the value you're looking for, you'll use the next one which would be your chosen default.

Please correct me if I missed something, and we can discuss this more.

[trlvsk]

Hello @lbajolet-hashicorp ,

Thanks for your time and reply.
That is not quite what i want to achieve though as coalesce would only return single non-empty value.
What i expect is to always have 3 ID's as default.
ami_users = ["123", "456", "789"]
Where:

• 123 - Dev ID (stored in variable AWS_ACCOUNT_ID_DEV)
• 456 - Pre-Prod ID (stored in variable AWS_ACCOUNT_ID_PREPROD)
• 789 - PROD ID (stored in variable AWS_ACCOUNT_ID_PROD)

When those values are hardcoded (either in main packer.hcl or in packer-config.hcl) then everything is working just fine.
AMI is created and in AWS console under Permissions -> Shared Accounts those ID's are there.
The end goal here is to fetch those from env variable as those scripts are ran for multiple different projects and pipelines that are ran in different AWS accounts so default dev/prod/pre-prod ID's will differ between projects

The problem only occurs while trying to pass env variable with env() to the list ( Variable type = list(string))
There was never an issue when it was just a string.

Thanks in advance :)

[lbajolet-hashicorp]

Hi again @trlvsk,

I think I misunderstood your question in the first place, you're wondering if you can use expressions for a default value, is that it?

I've justed tested this in my environment for reference:

variable test_values {
	type = list(string)
	default = [env("SHELL"), env("PWD")]
}

source "null" "test" {
	communicator = "none"
}

build {
	sources = ["null.test"]

	provisioner "shell-local" {
		inline = ["echo 'SHELL=${var.test_values[0]}'", "echo 'PWD=${var.test_values[1]}'"]
	}
}

Running the build yields the following:

$ packer build test.pkr.hcl
null.test: output will be in this color.

==> null.test: Running local shell script: /tmp/packer-shell3082975496
    null.test: SHELL=/bin/bash
    null.test: PWD=/home/[...]/packer
Build 'null.test' finished after 3 milliseconds 589 microseconds.

==> Wait completed after 3 milliseconds 618 microseconds

==> Builds finished. The artifacts of successful builds are:
--> null.test: Did not export anything. This is the null builder

So yes, you can use simple expressions, including env in a list variable.

Does that answer your question?

[trlvsk]

Hello again @lbajolet-hashicorp,

Thanks for the above answer.
I tested couple more things and made it work eventually.

It turns out that issue was not in the variable definition but in the way variable was called.
Not sure if that was my fault and there was a limitation that i was not aware of or there is some bug here but let me show you below.

Before I was calling variable as below which was failing as explained in previous comments.
Variable var.share_ami was just a pipeline specific bool which determined if AMI should be shared with different accounts or OU's/ORG's

source "amazon-ebs" "service_ami" {
  assume_role {
    role_arn     = var.assume_role
  }
  ami_ou_arns             = var.share_ami ? var.ami_ou_arns : []
  ami_users               = var.share_ami ? var.ami_users :[]
. . .

I removed the condition of var.share_ami from the source and it finally worked the way it should.
Since it's not really critical for me to have this condition i got rid of it as it wont be used that much anyways.
For OU's and ORG's condition does not break as they are hardcoded

source "amazon-ebs" "service_ami" {
  assume_role {
    role_arn     = var.assume_role
  }
  ami_ou_arns             = var.share_ami ? var.ami_ou_arns : []
  ami_users               = var.ami_users
. . .

All in all what i have is enough for me to work with but would like to know if it's expected to behave like this or that's some bug.
Nevertheless thanks for help so far.

[lbajolet-hashicorp]

Hi @trlvsk,

To be clear, I see you're using a ternary op for both ami_ou_arns and ami_users, do both return an empty array all the time? Did you make sure share_ami is not always false?

I've tried to replicate something like what you're trying to do, I don't see why that should be different, but it could have something to do with the context so I'm not ruling out definitely the possibility of a bug here.

For reference, here's the template I run my test on:

variable check {
	type = bool
	default = false
}

variable test_values {
	type = list(string)
	default = [env("SHELL"), env("PWD")]
}

local "end_arr" {
	expression = var.check ? var.test_values : ["abc", "def"]
}

source "null" "test" {
	communicator = "none"
}

build {
	sources = ["null.test"]

	provisioner "shell-local" {
		inline = ["echo 'SHELL=${local.end_arr[0]}'", "echo 'PWD=${local.end_arr[1]}'"]
	}
}

And here's what I get depending on the value of var.check:

$ packer build test.pkr.hcl 
null.test: output will be in this color.

==> null.test: Running local shell script: /tmp/packer-shell2280554638
    null.test: SHELL=abc
    null.test: PWD=def
Build 'null.test' finished after 3 milliseconds 954 microseconds.

==> Wait completed after 3 milliseconds 979 microseconds

==> Builds finished. The artifacts of successful builds are:
--> null.test: Did not export anything. This is the null builder
$ packer build -var check=true test.pkr.hcl 
null.test: output will be in this color.

==> null.test: Running local shell script: /tmp/packer-shell384017213
    null.test: SHELL=/bin/bash
    null.test: PWD=/home/[...]/packer
Build 'null.test' finished after 3 milliseconds 531 microseconds.

==> Wait completed after 3 milliseconds 549 microseconds

==> Builds finished. The artifacts of successful builds are:
--> null.test: Did not export anything. This is the null builder

At first glance this looks like what you're doing besides the location of the expression being in a local instead of a source, but the mechanisms that handle this are the same so I'd be surprised if it behaved differently.
Could you confirm?

[trlvsk]

Hello again.

Apologies for time it took to reply.
I've reviewed the pipeline and the code and in fact we've been passing the condition as a false (or rather overwriting it to false) by accident.
Hence why it was failing.

I've tested it again with the right condition which is share_ami set to true and indeed it works.
Thank you for your help.
And apologies for confusion.

Closing the issue :)