Skip to content

Commit

Permalink
[PLAT-11996][PLAT-13576]Create YAML/JSON template for create universe…
Browse files Browse the repository at this point in the history
… input

Summary:
Introduce `--config-template` to create universe to allow users to specify the universe spec via a JSON or YAML file.
Allow flags to override values if provided explicitly example : `yba universe create --config-template <file-path> --name dkumar`

Change input value of master-gflags to accept map of json or yaml type. CLI input for yaml map would look like:
```
./yba universe create -n dkumar-master-gflags-yaml-cli --provider-code gcp --provider-name dkumar-cli \
> --yb-db-version 2.23.0.0-b293 \
> --master-gflags "cdc_wal_retention_time_secs: 1
> log_min_segments_to_retain: 1
> log_cache_size_limit_mb: 0
> global_log_cache_size_limit_mb: 0
> log_stop_retaining_min_disk_mb: 9223372036854775807" \
> --num-nodes 1 --replication-factor 1 --user-tags yb_owner=dkumar,yb_task=dev,yb_dept=eng --debug
```

and json type would look like:
```./yba universe create -n dkumar-master-gflags --provider-code gcp --provider-name dkumar-cli \
--yb-db-version 2.23.0.0-b293 \
--master-gflags "{\"cdc_wal_retention_time_secs\": \"1\",\"log_min_segments_to_retain\": \"1\",\"log_cache_size_limit_mb\": \"0\",\"global_log_cache_size_limit_mb\": \"0\",\"log_stop_retaining_min_disk_mb\": \"9223372036854775807\"}" \
--num-nodes 1 --replication-factor 1 --user-tags yb_owner=dkumar,yb_task=dev,yb_dept=eng
```

Change input of Server Gflags:
JSON:
```
./yba universe create -n dkumar-tserver-cli-new --yb-db-version 2.23.0.0-b293 \
--provider-code gcp --provider-name dkumar-cli --add-read-replica --num-nodes 1,1 --replication-factor 1,1 \
--tserver-gflags "{\"primary\": {\"log_min_segments_to_retain\": \"1\",\"log_cache_size_limit_mb\": \"0\",\"global_log_cache_size_limit_mb\": \"0\",\"log_stop_retaining_min_disk_mb\": \"9223372036854775807\"},\"async\": {\"log_min_segments_to_retain\": \"2\",\"log_cache_size_limit_mb\": \"0\",\"global_log_cache_size_limit_mb\": \"0\"}}"
```
YAML:
```
./yba universe create -n dkumar-tserver-cli-new --yb-db-version 2.23.0.0-b293 \
--provider-code gcp --provider-name dkumar-cli --add-read-replica --num-nodes 1,1 --replication-factor 1,1 \
--tserver-gflags "primary:
    log_min_segments_to_retain: 1
    log_cache_size_limit_mb: 0
    global_log_cache_size_limit_mb: 0
    log_stop_retaining_min_disk_mb: 9223372036854775807
async:
    log_min_segments_to_retain: 2
    log_cache_size_limit_mb: 0
    global_log_cache_size_limit_mb: 0"
```

Template files would have it in map format according to file type

Test Plan:
Testing the fields via config file and flags independently. Using the flags to override values provided in the config file.
YAML and JSON templates has been added in the /templates directory to guide users.

Reviewers: sneelakantan

Reviewed By: sneelakantan

Subscribers: yugaware

Differential Revision: https://phorge.dev.yugabyte.com/D34514
  • Loading branch information
Deepti-yb committed May 13, 2024
1 parent eac0c26 commit 0097219
Show file tree
Hide file tree
Showing 9 changed files with 461 additions and 279 deletions.
1 change: 1 addition & 0 deletions .arclint
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
"(^managed/yba-cli/NOTICE$)",
"(^managed/yba-cli/internal/formatter/tabwriter/*)",
"(^managed/yba-cli/internal/formatter/templates/*)",
"(^managed/yba-cli/templates/*)",
"(^[.]clang-tidy)",
"(.*/py.typed)",
"(^troubleshoot/backend/gradlew)",
Expand Down
11 changes: 6 additions & 5 deletions managed/yba-cli/GNUmakefile
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,14 @@ ${BIN_DIR}/${OS_ARCH}/${BIN_NAME}: ${GO_SOURCE_FILES}
-X 'github.com/yugabyte/yugabyte-db/managed/yba-cli/pkg/components/yba.BuildID=${BUILD_ID}' \
-X 'main.Version=${VERSION}'"

.PHONY: doc
doc:
YBA_FF_TOOLS=true go run main.go tools gen-doc --format markdown
sed -i 's/\x1B\[32m//g' docs/*
sed -i 's/\x1B\[0m//g' docs/*

package: ${PACKAGE_NAME}
${PACKAGE_NAME}: yba
${PACKAGE_NAME}: yba doc
@mkdir -p $(PACKAGE_DIR)
ifneq ("$(shell cmp -s ${BIN_DIR}/${OS_ARCH}/${BIN_NAME} ${PACKAGE_DIR}/${BIN_NAME}; echo $$?)", "0")
@echo "Packaging ${BIN_NAME} into ${PACKAGE_DIR}"
Expand Down Expand Up @@ -83,10 +88,6 @@ updateclient:
build:
go build -ldflags="-X 'main.Version=v${VERSION}'" -o ${BIN_NAME}

.PHONY: doc
doc:
YBA_FF_TOOLS=true go run main.go tools gen-doc --format markdown

.PHONY: license
license:
./licenses/generate-license-file.sh

0 comments on commit 0097219

Please sign in to comment.