(aws_elasticloadbalancingv2): (error when adding application load balancer to NetworkTargetGroup)

[Vitaliy-Fanin]

Describe the bug

I am trying to create a network load balancer that has an application load balancer as its target. According to this link, this should be possible.

I am able to create the application load balancer and all the actions and targets groups for it. I even figured out how to get around this documented issue.

When I create the NetworkTargetGroup, if I add the alb as a sequence to the targets parameter, I get an error.

jsii.errors.JSIIError: target.attachToNetworkTargetGroup is not a function

Expected Behavior

I should be able to add the application load balancer as a target of the Network Load Balancer.

Current Behavior

jsii.errors.JavaScriptError:
TypeError: target.attachToNetworkTargetGroup is not a function
at NetworkTargetGroup.addTarget (C:....\Temp\jsii-kernel-utIllv\node_modules\aws-cdk-lib\aws-elasticloadbalancingv2\lib\nlb\network-target-group.js:1:2135)
at new NetworkTargetGroup (C:....\Temp\jsii-kernel-utIllv\node_modules\aws-cdk-lib\aws-elasticloadbalancingv2\lib\nlb\network-target-group.js:1:1348)
at NetworkListener.addTargets (C:....\Temp\jsii-kernel-utIllv\node_modules\aws-cdk-lib\aws-elasticloadbalancingv2\lib\nlb\network-listener.js:1:5081)
at C:....\Temp\tmpf0b001vr\lib\program.js:5453:144
at exports.Kernel._ensureSync (C:....\Temp\tmpf0b001vr\lib\program.js:5871:28)
at exports.Kernel.invoke (C:....\Temp\tmpf0b001vr\lib\program.js:5453:73)
at exports.KernelHost.processRequest (C:....\Temp\tmpf0b001vr\lib\program.js:6618:36)
at exports.KernelHost.run (C:....\Temp\tmpf0b001vr\lib\program.js:6592:48)
at Immediate._onImmediate (C:....\Temp\tmpf0b001vr\lib\program.js:6593:46)
at processImmediate (node:internal/timers:466:21)

The above exception was the direct cause of the following exception:

File "C:....\Programs\Python\Python310\lib\site-packages\jsii_runtime.py", line 86, in call
inst = super().call(*args, **kwargs)
File "C:...._app_servers_stack2.py", line 188, in init
app_nlb_listener.add_targets("NLB_Target-ALB",
File "C:....l\Programs\Python\Python310\lib\site-packages\aws_cdk\aws_elasticloadbalancingv2_init_.py", line 11454, in add_targets
return typing.cast("NetworkTargetGroup", jsii.invoke(self, "addTargets", [id, props]))
File "C:....\Programs\Python\Python310\lib\site-packages\jsii_kernel_init_.py", line 148, in wrapped
return recursize_dereference(kernel, fn(kernel, *args, **kwargs))
File "C:....\Programs\Python\Python310\lib\site-packages\jsii_kernel_init.py", line 386, in invoke
response = self.provider.invoke(
File "C:....\Programs\Python\Python310\lib\site-packages\jsii_kernel\providers\process.py", line 362, in invoke
return self._process.send(request, InvokeResponse)
File "C:....\Programs\Python\Python310\lib\site-packages\jsii_kernel\providers\process.py", line 329, in send
raise JSIIError(resp.error) from JavaScriptError(resp.stack)

jsii.errors.JSIIError: target.attachToNetworkTargetGroup is not a function

Reproduction Steps

from aws_cdk import (
Stack,
aws_ec2 as ec2,
aws_iam as iam,
aws_autoscaling as autoscaling,
aws_elasticloadbalancingv2 as elbv2,
Duration
)
from constructs import Construct

class AppServerStack3(Stack):

def __init__(self, scope: Construct, construct_id: str, parameters: dict, **kwargs) -> None:
    super().__init__(scope, construct_id, **kwargs)
    
    vpc = ec2.Vpc.from_lookup(self, "vpc", vpc_name=parameters["vpc_name"])

    amzn_linux = ec2.MachineImage.latest_amazon_linux(
        generation=ec2.AmazonLinuxGeneration.AMAZON_LINUX_2,
        edition=ec2.AmazonLinuxEdition.STANDARD,
        virtualization=ec2.AmazonLinuxVirt.HVM,
        storage=ec2.AmazonLinuxStorage.GENERAL_PURPOSE
        )

    # Instance Role and SSM Managed Policy
    role = iam.Role(self, "App-Server-Role2", assumed_by=iam.ServicePrincipal("ec2.amazonaws.com"))      
    role.add_managed_policy(iam.ManagedPolicy.from_aws_managed_policy_name("AmazonSSMManagedInstanceCore"))
    role.add_managed_policy(iam.ManagedPolicy.from_aws_managed_policy_name("SecretsManagerReadWrite"))
    role.add_managed_policy(iam.ManagedPolicy.from_aws_managed_policy_name("AmazonRDSFullAccess"))
    role.add_managed_policy(iam.ManagedPolicy.from_aws_managed_policy_name("AmazonS3FullAccess"))

    multipart_user_data = ec2.MultipartUserData()
    commands_user_data = ec2.UserData.for_linux()
    multipart_user_data.add_user_data_part(commands_user_data, ec2.MultipartBody.SHELL_SCRIPT, True)
    commands_user_data.add_commands("sudo yum -y update")
    commands_user_data.add_commands("sudo yum install -y wget")
    commands_user_data.add_commands("sudo amazon-linux-extras install -y java-openjdk11")

    asg_security_group = ec2.SecurityGroup(self, "AppServerSG2",
        vpc=vpc,
        description="Firewall for App Servers",
        security_group_name="AppServerSG2",
        allow_all_outbound=True)

    app_tier_private_subnets = [ec2.Subnet.from_subnet_attributes(self,'subnetid1', 
                            availability_zone = 'us-west-2a', 
                            subnet_id = parameters["private_subnet-App_AZ_A"]),
                            ec2.Subnet.from_subnet_attributes(self,'subnetid2', 
                            availability_zone = 'us-west-2b', 
                            subnet_id = parameters["private_subnet-App_AZ_B"]),
                            ec2.Subnet.from_subnet_attributes(self,'subnetid3', 
                            availability_zone = 'us-west-2c', 
                            subnet_id = parameters["private_subnet-App_AZ_C"])]

    app_servers_asg = autoscaling.AutoScalingGroup(self, "App-Server-ASG",
        auto_scaling_group_name = "App-Server-ASG2",
        vpc=vpc,
        instance_type=ec2.InstanceType(parameters["app_server_instance_type"]),
        machine_image=amzn_linux,
        key_name=parameters["ec2_key"],
        user_data=multipart_user_data,
        role=role,
        security_group=asg_security_group,
        min_capacity=parameters["min_capacity"],
        max_capacity=parameters["max_capacity"],
        desired_capacity=parameters["desired_capacity"],
        vpc_subnets=ec2.SubnetSelection(subnets = app_tier_private_subnets)
        )
    #Try to set ALB as target of NLB
    app_alb_security_group = ec2.SecurityGroup(self, "AppServerALB-SG",
        vpc=vpc,
        description="Firewall for App ALB",
        security_group_name="App-Server-ALB-SG",
        allow_all_outbound=True)
    
    app_alb2 = elbv2.ApplicationLoadBalancer(
        self, "ALB2",
        load_balancer_name="App-ALB2",
        vpc=vpc,
        vpc_subnets=ec2.SubnetSelection(subnets = app_tier_private_subnets),
        internet_facing=False,
        http2_enabled=True,
        idle_timeout=Duration.seconds(60),
        ip_address_type=elbv2.IpAddressType.IPV4,
        security_group=app_alb_security_group)

    app_alb2_listener = app_alb2.add_listener("ALB_Listener2", 
        certificates=[elbv2.ListenerCertificate(
                      certificate_arn=parameters["Cert_Arn"])], 
        protocol=elbv2.ApplicationProtocol.HTTPS,
        open=True,
        ssl_policy=elbv2.SslPolicy.FORWARD_SECRECY_TLS12_RES_GCM,
        port=443,
    )

    app_alb2_tg = app_alb2_listener.add_targets("ALB2_Collab-API-Target", 
        target_group_name="ALB2-ASG-collab", 
        port=8080, 
        protocol=elbv2.ApplicationProtocol.HTTP,
        stickiness_cookie_duration=Duration.minutes(15),
        priority=10,
        conditions=[elbv2.ListenerCondition.path_patterns(["/collab"])]
    )

    app_alb2_tg3 = app_alb2_listener.add_targets("ALB2_Events-API-Target", 
        target_group_name="App-ALB-ASG-events", 
        port=8081, 
        protocol=elbv2.ApplicationProtocol.HTTP,
        stickiness_cookie_duration=Duration.minutes(15),
        priority=20,
        conditions=[elbv2.ListenerCondition.path_patterns(["/events"])]
    )
    
    app_alb2_tg2 = app_alb2_listener.add_targets("ALB2-Default-Action", 
        target_group_name="App-ALB2-ASG-default", 
        port=8080, 
        protocol=elbv2.ApplicationProtocol.HTTP,
        stickiness_cookie_duration=Duration.minutes(15)
    )

    # https://github.com/aws/aws-cdk/issues/5667
    # Due to the issue described above, we have to use CDK escape hatch to allow multiple
    # target groups to attach themselves to one autoscaling group.
    temp_asg = app_servers_asg.node.default_child
    temp_asg.target_group_arns = [
        app_alb2_tg.target_group_arn, 
        app_alb2_tg2.target_group_arn, 
        app_alb2_tg3.target_group_arn]

    app_servers_asg.connections.allow_from(ec2.Connections(
                                            security_groups=[app_alb_security_group]), 
                                            port_range=ec2.Port.tcp(8080), 
                                            description="Allow App Server ALB access to App Server SG")

    app_servers_asg.connections.allow_from(ec2.Connections(
                                            security_groups=[app_alb_security_group]), 
                                            port_range=ec2.Port.tcp(8081), 
                                            description="Allow App Server ALB access to App Server SG")

    app_nlb = elbv2.NetworkLoadBalancer(self, "NLB-4-ALB",
        vpc=vpc,
        internet_facing=False,
        vpc_subnets=ec2.SubnetSelection(subnets = app_tier_private_subnets),
        load_balancer_name="APP-NLB-4-ALB"
    )

    NLB_TG = elbv2.NetworkTargetGroup(self, "APP_NLB_TG-4-ALB", 
        port=443,
        protocol=elbv2.Protocol.TCP,
        targets=[app_alb2],
        target_group_name="App-ALB-4-NLB-TG",
        target_type=elbv2.TargetType.ALB,
        vpc=vpc
        )

    app_nlb_listener = app_nlb.add_listener("NLB_Listener", port=443)
    app_nlb_listener.add_target_groups("NLB-TG", NLB_TG)
    
    # This approach also fails with the same error!
    '''app_nlb_listener.add_targets("NLB_Target-ALB", 
        target_group_name="TPA-BO-App-ALB-4-NLB", 
        port=443, 
        protocol=elbv2.Protocol.TCP,
        targets=[app_alb2])'''

Possible Solution

No response

Additional Information/Context

I have several spring webservices running on private EC2 instances that are set up in an auto scaling group. We've got API Gateway connected to the EC2 instances through a VPC Link. The VPC Link requires an NLB. The problem is, NLBs don't route traffic based on conditions like ALBs do. So I'm using the NLB to send traffic to the ALB and uses the ALB's conditions parameters to route traffic to the right port of the ASG so that the right web service receives the API call and responds to it.

CDK CLI Version

2.55.0

Framework Version

No response

Node.js Version

v16.17.0

OS

Windows 10

Language

Python

Language Version

Python 3.10.7

Other information

No response

[tim-finnigan]

Hi @Vitaliy-Fanin thanks reaching out. Regarding the error jsii.errors.JSIIError: target.attachToNetworkTargetGroup is not a function that appears to be caused by the code starting with app_nlb_listener.add_targets("NLB_Target-ALB",. As suggested in this Stack Overflow post I think you need to use add_target_group there instead of add_targets. If there's any more feedback on your use case you can share or updates on what you've tried please let us know.

[Vitaliy-Fanin]

I read that post before creating this issue, however .add_target_group() is not a valid function in python code. The closest is .add_target_groups().

It's been almost 2 months, but I saved the stack in case I ever got a response. I just dug into it again. The problem is specifically setting an ALB as the target on the Network Load balancer's target group.

If the end of my code looks like below, with #targets=[app_alb2], commented out, everything builds and deploys fine, but the NLB's Target Group does not have a target. If I have that targets line in, I get the error jsii.errors.JSIIError: target.attachToNetworkTargetGroup is not a function.

     app_nlb = elbv2.NetworkLoadBalancer(self, "TPA_BO_API_GW_NLB-4-ALB2",
        vpc=vpc,
        internet_facing=False,
        vpc_subnets=ec2.SubnetSelection(subnets = app_tier_private_subnets),
        load_balancer_name="TPA-BO-APP-NLB-4-ALB2"
    )

    NLB_TG = elbv2.NetworkTargetGroup(self, "TPA_BO_APP_NLB_TG-4-ALB2", 
        port=80,
        protocol=elbv2.Protocol.TCP,
        #targets=[app_alb2],
        target_group_name="TPA-BO-App-ALB-4-NLB-TG2",
        target_type=elbv2.TargetType.ALB,
        vpc=vpc
    )

    app_nlb_listener = app_nlb.add_listener("NLB_Listener2", port=80)
    app_nlb_listener.add_target_groups("TPA-NLB-TG2", NLB_TG)

add_target_groups is fine to add the target group to the listener, but it doesn't have a target inside of it.

If I update my code slightly, by NLB_TG.add_target(app_alb2) to the end, I get the same error again.

    app_nlb = elbv2.NetworkLoadBalancer(self, "TPA_BO_API_GW_NLB-4-ALB2",
        vpc=vpc,
        internet_facing=False,
        vpc_subnets=ec2.SubnetSelection(subnets = app_tier_private_subnets),
        load_balancer_name="TPA-BO-APP-NLB-4-ALB2"
    )

    NLB_TG = elbv2.NetworkTargetGroup(self, "TPA_BO_APP_NLB_TG-4-ALB2", 
        port=80,
        protocol=elbv2.Protocol.TCP,
        #targets=[app_alb2],
        target_group_name="TPA-BO-App-ALB-4-NLB-TG2",
        target_type=elbv2.TargetType.ALB,
        vpc=vpc
    )

    app_nlb_listener = app_nlb.add_listener("NLB_Listener2", port=80)
    app_nlb_listener.add_target_groups("TPA-NLB-TG2", NLB_TG)
    NLB_TG.add_target(app_alb2)

Bottom line, how do I get the application load balancer set as the target of the network load balancer? Any attempt, whether setting targets=[app_alb2] in the target group creation or by trying to add it as a target after the target group has been created , NLB_TG.add_target(app_alb2), gets me the same jsii.errors.JSIIError: target.attachToNetworkTargetGroup is not a function error.

Once the target group has been created, I can go into the console and add it manually without a problem, but we are using these stacks in multiple AWS accounts and that manual step has been forgotten several times already. I'd like for it to be fixed in the code.

Am I missing something or is there a bug here?

[tim-finnigan]

Hi @Vitaliy-Fanin thanks for following up. I found some documentation that may help: https://docs.aws.amazon.com/cdk/api/v2/python/aws_cdk.aws_elasticloadbalancingv2/README.html#defining-an-application-load-balancer Specifically this part of the code example using the AlbTarget helper class, with your target plugged in:

listener = nlb.add_listener("listener", port=80)

listener.add_targets("NLB_Target-ALB",
    targets=[targets.AlbTarget(app_alb2 80)],
    port=80
)

I think that upon creating the NLB listener you need to use add_targets() to add the ALB to it. Does that make sense?

[Vitaliy-Fanin]

Yes, that worked!

Wow. This is the first time I've had to go outside the core library to something else to get something done. I didn't realize there was such a library as aws_elasticloadbalancingv2_targets. I don't know if there is any way to improve the findability of these instructions. I searched a lot to figure this out. Hopefully this ticket will help others too, because this was a frustrating problem for me and there is very little guidance out there about this scenario. Thank you for the help, it is much appreciated.

[tim-finnigan]

Great! Glad to hear that worked, thanks for confirming (also credit to @peterwoodworth for helping me find that solution). I think this issue can be closed now but the topics covered here might be a good candidate for the aws-cdk-examples repository going forward.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.