Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: ServiceCatalog #6672

Draft
wants to merge 21 commits into
base: master
Choose a base branch
from

Conversation

davidconnor-ont
Copy link

@davidconnor-ont davidconnor-ont commented Aug 15, 2023

This is an early draft of the service catalog API endpoint.

These endpoints should provide enough coverage to run the core provisioning tasks of creating portfolios, products, linking them and provisioning a stack through the moto cloudformation module.

I'm providing this draft PR as it's already ballooned into quite a large PR even without the full unit test coverage needed for testing exception responses. I wanted to get any early feedback that might be needed before going down too many wrong paths. So far the code is not particularly complex as it's mostly managed the session state of the created resources.

Initial implementation covering:

  • associate_product_with_portfolio
  • create_constraint
  • create_portfolio
  • create_product
  • describe_portfolio
  • describe_product
  • describe_product_as_admin
  • describe_provisioned_product
  • get_provisioned_product_outputs
  • list_launch_paths
  • list_portfolios
  • list_portfolios_for_product
  • list_provisioning_artifacts
  • provision_product
  • search_products
  • search_provisioned_products
  • terminate_provisioned_product
  • update_portfolio
  • update_product

The aim of the initial version is to be able to run some of the simpler tutorials like this one from AWS:
https://www.youtube.com/watch?v=egZroVhDF4M

This terraform script should run completely and create all resources correctly including provisioning from the cloudformation template provided (EDIT: the provison_product step fails at the moment which is the part I'm currently working on.)

resource "aws_s3_bucket" "cf_templates" {
    bucket = "cf_templates"
}
locals {
  object_source = "${path.module}/simple.yaml"
}

resource "aws_s3_object" "file_upload" {
  bucket = aws_s3_bucket.cf_templates.id
  key         = "main_template"
  source      = local.object_source
  source_hash = filemd5(local.object_source)
}

resource "aws_servicecatalog_portfolio" "organization_portfolio" {
  name          = "Organization Portfolio"
  description   = "List of organization's services"
  provider_name = "CCOE"
}

resource "aws_servicecatalog_product" "s3" {
  name  = "S3 Bucket"
  owner = "CCOE"
  type  = "CLOUD_FORMATION_TEMPLATE"

  provisioning_artifact_parameters {
    template_url = "https://${aws_s3_bucket.cf_templates.id}.s3.${aws_s3_bucket.cf_templates.region}.amazonaws.com/${aws_s3_object.file_upload.key}"
    name = "Simple S3 Bucket"
    description = "v1.0"
    type = "CLOUD_FORMATION_TEMPLATE"
  }
}

resource "aws_servicecatalog_product_portfolio_association" "product_portfolio_association" {
  portfolio_id = aws_servicecatalog_portfolio.organization_portfolio.id
  product_id   = aws_servicecatalog_product.s3.id
}

resource "aws_servicecatalog_principal_portfolio_association" "associate_role" {
  portfolio_id  = aws_servicecatalog_portfolio.organization_portfolio.id
  principal_arn = "arn:aws:iam::444422324444:role/Admin-OneClick"
}

data "aws_servicecatalog_launch_paths" "s3_product_path" {
  product_id = aws_servicecatalog_product.s3.id
}

resource "aws_servicecatalog_provisioned_product" "provisioned_s3" {
  name = "Simple S3"
  product_id = aws_servicecatalog_product.s3.id
  path_id = data.aws_servicecatalog_launch_paths.s3_product_path.summaries[0].path_id
  provisioning_artifact_name = aws_servicecatalog_product.s3.provisioning_artifact_parameters[0].name
}

cloudformation template:

Resources:
  LocalBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: cfn-quickstart-bucket

Copy link
Collaborator

@bblommers bblommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @davidconnor-ont, welcome to Moto!

This all looks good to me so far. It all follows the same approach/setup as our other services, which is quite important to keep things readable.

Same goes for the tests - they only test a single operation, making them very readable, so I'm happy with that.

One possible suggestion:
There seem to be a lot of tests that share the same setup. Is it worth converting them to separate TestClass-es, so that there is less deduplication?
Example: https://github.com/getmoto/moto/blob/master/tests/test_autoscaling/test_autoscaling_groups.py

Don't worry about the size of the PR.
it's indeed ballooning, but because it's mostly session state management, its all easy to review.

@davidconnor-ont
Copy link
Author

Thanks for the feedback.

One possible suggestion: There seem to be a lot of tests that share the same setup. Is it worth converting them to separate TestClass-es, so that there is less deduplication? Example: https://github.com/getmoto/moto/blob/master/tests/test_autoscaling/test_autoscaling_groups.py

Definitely. I hadn't had the chance to refactor the tests into a more clean setup - it was repetitive out of laziness/convenience to get the functions semi-working. I'll take a look at that example when I'm updating them.

@davidconnor-ont
Copy link
Author

davidconnor-ont commented Aug 16, 2023

@bblommers Service catalog seems to implement filters mostly the same as EC2 and other filters that have the pattern:
[{"filterName":["filterValue"]}]
but has a slightly different inputFormat.

{"SearchQuery":["field:value", "anotherfield:value,value2]}

The docs indicate that "SearchQuery" is the only key, so really the values in SearchQuery map to the EC2 way of filtering under-the-hood.

I've added a commit where I map the input to match the excepted input of generic_filter and have implemented get_filter_value on the specific object.

Before I go any further:

  1. is there a standard way of filtering objects in moto like there is for tagging?
  2. If not, is importing the generic_filter code from EC2 ok across the service boundry (servicecatalog importing from ec2).

@bblommers
Copy link
Collaborator

Hi @davidconnor-ont,

There is no standard way to filter things - it's all a bit disjointed unfortunately

It sounds like the best solution to use generic_filter, so I'm OK with importing this from EC2 for now.

Importing EC2 does come at a price - it will add another 2/300 ms to the import time, because it automatically generates all kinds of default objects (AMI's, VPC's, etc.). So later on I will just refactor the filter-methods into a separate service-agnostic utilities-file to remedy this.

@davidconnor-ont
Copy link
Author

Hello, I'm still working on this. I'm testing it with our real world usage of service catalog so it's being adjusted as I add the unit testing to our service catalog calls. The main area I need to address is the provisioning a product when a passed in path_id is used - those are the artificially failing test placeholders in the last test run.

@bblommers
Copy link
Collaborator

@davidconnor-ont We've just released Moto v5 with quite a few breaking changes. Let me know if you need any help bringing this PR up-to-date, I'd be happy to help out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants