Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ibek command line namespaces #106

Closed
gilesknap opened this issue Sep 19, 2023 · 11 comments
Closed

ibek command line namespaces #106

gilesknap opened this issue Sep 19, 2023 · 11 comments

Comments

@gilesknap
Copy link
Member

gilesknap commented Sep 19, 2023

We have decided to bring into ibek the set of commands for building support modules at container build time, these currently reside in
https://github.com/epics-containers/ibek-support/blob/ibek-j20-changes/_global/support.py

To do so we need to intelligently namespace the command line interface to avoid confusion.

Here is a proposed hierarchy for criticism @coretl @GDYendell :-

schema  
                get-support-schema          # output the SUPPORT YAML schema
                build-ioc-schema            # take a list of SUPPORT YAML and generate an IOC YAML schema
instance  
                generate-scripts            # generates the ioc startup and subst files from an IOC YAML ( and list of SUPPORT YAML)
support    
                git_clone                   # clones a support module
                global_patches              # applies global patches to  a support module (patches that all modules require) 
                add_macro                   # adds a macro to the global support/RELEASE file used by all support in a generic IOC 
                add_support_module          # adds a support module to the list for compilation in a generic IOC
                add_libs                    # declares the lib files for a support module
                add_dbds                    # declares the dbd files for a support module
                generate_makefile           # creates a generic IOC makefile (based on sequence of calls to  add_support_module)
                create_links                # symlinks bob, pvi, ibek YAML to a known folder
                compile                     # compiles a support module after prep with the above commands
ioc                
                generate_makefile           # creates a generic IOC makefile (based on sequence of calls to support functions)
                compile                     # compiles the generic IOC after prep with above

One of the problems I see with this is that in some cases we have just written python code to wrap calling back out bash. This is certainly the case for the compile functions. I can use GitPython for the clone - but to what advantage - it might just get in the way when you need to do something obscure with git. Having said this - whenever such issues arise we can just directly call the command from install.sh and eschew ibek. Then we see ibek as something with some legible help that guides you to the most common actions.

@coretl
Copy link
Contributor

coretl commented Sep 19, 2023

Firstly, we could ditch the subcommands and group only in the help:
tiangolo/typer#514

@coretl
Copy link
Contributor

coretl commented Sep 19, 2023

Although maybe the namespacing is nice...

@gilesknap
Copy link
Member Author

Well we need longer names for the individual functions if we don't namespace methinks.
subcommands are helpful for context with auto complete
most of the use of this is in scripts anyway so length of commands is not an issue

@coretl
Copy link
Contributor

coretl commented Sep 19, 2023

Suggestion:

support
    generate-schema
    git-clone
    register  # global patch and add to global configure/RELEASE
    add-libs
    add-dbds
    add-macro
    compile
    generate-links
ioc
    generate-schema
    generate-makefile
    compile
startup
    generate

@gilesknap
Copy link
Member Author

This is what we have so far:

(venv) (dev) [hgv27681@pc0116 ibek]$ ibek --help
                                                                                                              
 Usage: ibek [OPTIONS] COMMAND [ARGS]...                                                                      
                                                                                                              
 IOC Builder for EPICS and Kubernetes                                                                         
 Provides support for building generic EPICS IOC container images and for running IOC instances in a          
 Kubernetes cluster.                                                                                          
                                                                                                              
╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --version                     Print the version of ibek and exit                                           │
│ --install-completion          Install completion for the current shell.                                    │
│ --show-completion             Show completion for the current shell, to copy it or customize the           │
│                               installation.                                                                │
│ --help                        Show this message and exit.                                                  │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ─────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ioc         Commands for building generic IOCs during container build                                      │
│ startup     Commands for building IOC instance startup files at container runtime                          │
│ support     Commands for building support modules during container build                                   │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ioc
╭─ Commands ─────────────────────────────────────────────────────────────────────────────────────────────────╮
│ compile            Compile a generic IOC once all support modules are registered and compiled              │
│ generate-makefile  get the dbd and lib files from all support modules and generate iocApp/src/Makefile     │
│                    from iocApp/src/Makefile.jinja                                                          │
│ generate-schema    Create a json schema from a <support_module>.ibek.support.yaml file                     │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
startup
╭─ Commands ─────────────────────────────────────────────────────────────────────────────────────────────────╮
│ generate            Build a startup script for an IOC instance                                             │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
support
╭─ Commands ─────────────────────────────────────────────────────────────────────────────────────────────────╮
│ add-dbds         declare the dbd files for this support module for inclusion in IOC Makefile               │
│ add-libs         declare the libraries for this support module for inclusion in IOC Makefile               │
│ add-macro        add or replace a macro in a RELEASE file                                                  │
│ compile          compile a support module after preparation with `ibek support register` etc.              │
│ generate-links   generate symlinks to the bob, pvi and support YAML for a compiled IOC                     │
│ generate-schema  Produce JSON global schema for all <support_module>.ibek.support.yaml files               │
│ git-clone        clone a support module from a remote repository                                           │
│ register         prepare the configure RELEASE files to build a support module inside an epics-containers  │
│                  build                                                                                     │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

@coretl
Copy link
Contributor

coretl commented Sep 19, 2023

If we want to maintain order, would this work?
https://skeptric.com/typer-command-order/

@coretl
Copy link
Contributor

coretl commented Sep 19, 2023

Apart from the order I think the names are right

@GDYendell
Copy link
Member

GDYendell commented Sep 19, 2023

https://skeptric.com/typer-command-order/

Neat. I think for the support subcommands especially having them in the order they should be run would be really useful.

@gilesknap
Copy link
Member Author

Will try to achieve the order. In the mean time. The install.sh scripts came out pretty (and are working - with a few caveats)

#!/bin/bash
##########################################################################
##### boilerplate install script for support modules #####################
##########################################################################

# ARGUMENTS:
#  $1 VERSION to install (must match repo tag)
VERSION=${1}

# get the name of this folder, i.e. the name of the support module
NAME=$(basename $(dirname ${0}))

ibek support git-clone ${NAME} ${VERSION}
ibek support register ${NAME}
ibek support add-libs ${NAME} devIocStats
ibek support add-dbds ${NAME} devIocStats.dbd

##########################################################################
##### put patch commands here if needed ##################################
##########################################################################

ibek support compile ${NAME}
ibek support generate-links ${NAME}

@gilesknap
Copy link
Member Author

I think we have this in good shape now.

Here are the commands and they now appear in the typical order of usage:

(venv) (ibek-j20-changes) [hgv27681@pc0116 ibek]$ ibek --help
                                                                                                          
 Usage: ibek [OPTIONS] COMMAND [ARGS]...                                                                  
                                                                                                          
 IOC Builder for EPICS and Kubernetes                                                                     
 Provides support for building generic EPICS IOC container images and for running IOC instances in a      
 Kubernetes cluster.                                                                                      
                                                                                                          
╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────╮
│ --version                     Print the version of ibek and exit                                       │
│ --install-completion          Install completion for the current shell.                                │
│ --show-completion             Show completion for the current shell, to copy it or customize the       │
│                               installation.                                                            │
│ --help                        Show this message and exit.                                              │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ─────────────────────────────────────────────────────────────────────────────────────────────╮
│ support     Commands for building support modules during container build                               │
│ ioc         Commands for building generic IOCs during container build                                  │
│ startup     Commands for building IOC instance startup files at container runtime                      │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────╯

(venv) (ibek-j20-changes) [hgv27681@pc0116 ibek]$ ibek support --help
                                                                                                          
 Usage: ibek support [OPTIONS] COMMAND [ARGS]...                                                          
                                                                                                          
 Commands for building support modules during container build                                             
                                                                                                          
╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────╮
│ --help          Show this message and exit.                                                            │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ─────────────────────────────────────────────────────────────────────────────────────────────╮
│ apt-install         Install debian packages into the container. If they have an http:// or https://    │
│                     prefix then they will be downloaded and installed from file.                       │
│ git-clone           clone a support module from a remote repository                                    │
│ register            prepare the configure RELEASE files to build a support module inside an            │
│                     epics-containers build                                                             │
│ add-libs            declare the libraries for this support module for inclusion in IOC Makefile        │
│ add-dbds            declare the dbd files for this support module for inclusion in IOC Makefile        │
│ add-release-macro   add or replace a macro the global RELEASE file                                     │
│ add-config-macro    add or replace a macro in CONFIG_SITE.linux-x86_64.Common file                     │
│ add-to-config-site  add some text to a support module's CONFIG_SITE file                               │
│ compile             compile a support module after preparation with `ibek support register` etc.       │
│ generate-links      generate symlinks to the bob, pvi and support YAML for a compiled IOC              │
│ generate-schema     Produce JSON global schema for all <support_module>.ibek.support.yaml files        │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────╯

(venv) (ibek-j20-changes) [hgv27681@pc0116 ibek]$ ibek ioc --help
                                                                                                          
 Usage: ibek ioc [OPTIONS] COMMAND [ARGS]...                                                              
                                                                                                          
 Commands for building generic IOCs during container build                                                
                                                                                                          
╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────╮
│ --help          Show this message and exit.                                                            │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ─────────────────────────────────────────────────────────────────────────────────────────────╮
│ generate-makefile  get the dbd and lib files from all support modules and generate iocApp/src/Makefile │
│                    from iocApp/src/Makefile.jinja                                                      │
│ compile            Compile a generic IOC after support modules are registered and compiled             │
│ build              EXPERIMENTAL: Attempt to interpret the Dockerfile and run it's commands inside the  │
│                    devcontainer. For internal, incremental builds of the Dockerfile.                   │
│ generate-schema    Create a json schema from a number of support_module.ibek.support.yaml files        │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────╯

(venv) (ibek-j20-changes) [hgv27681@pc0116 ibek]$ ibek startup --help
                                                                                                          
 Usage: ibek startup [OPTIONS] COMMAND [ARGS]...                                                          
                                                                                                          
 Commands for building IOC instance startup files at container runtime                                    
                                                                                                          
╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────╮
│ --help          Show this message and exit.                                                            │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ─────────────────────────────────────────────────────────────────────────────────────────────╮
│ generate           Build a startup script for an IOC instance                                          │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────╯

gilesknap added a commit that referenced this issue Sep 22, 2023
changes to add new support functions as per #106
@gilesknap
Copy link
Member Author

Done in #107

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants