Skip to content
/ myke Public

make with yaml: development tasks made simple with golang, yaml and many ingredients

License

Notifications You must be signed in to change notification settings

omio-labs/myke

Repository files navigation

myke Latest Release Build Status Go Report Card codecov

myke makes it easy to write development tasks

Development scripts usually begin as a simple shell wrapper with switch cases (service.sh start|stop|etc), and then aggregate multiple scripts, add arguments, discovery/listing, environment variable handling, then easy overriding and defaults, and soon manipulating files based on these variables, sed|awk|envsubst, then proceed to python/ruby/etc with some real templating, then start adding dependencies, then become projects themselves with a checkout/setup process :trollface:

myke solves all these problems in a single tiny binary, to avoid reinventing the same stuff over and over again.

Features

  • Define tasks in simple .yml files
  • Tasks execute in a predictable environment irrespective of which folder they are invoked from
  • Nice aggregation and discovery with tag-based grouping, suitable for few and many tasks, organizing into subfolders/submodules/repos/projects
  • Robust environment handling - Can be defined as keys in the YML or as dotenv files, overridden by dotenv.local files, PATH is always prepended, shell always takes precedence
  • Built-in templating using golang text/template and 50+ functions provided by sprig
  • Mixin ymls to share tasks, envvars, etc
  • Runtime arguments like myke task1 --key1=val1 task2 --key2=val2 ...
  • before/after/error hooks to perform cleanups, chains with mixins, etc
  • retry support with max and delay for failing tasks
  • Tiny, cross-platform binaries
  • and a lot of small utilities packed in

Usage

Create myke.yml with tasks. For example, running myke on this folder prints:

  PROJECT  |    TAGS    |             TASKS
+----------+------------+-------------------------------------+
  myke     |            | test
  example  |            | build
  env      |            | env
  tags1    | tagA, tagB | tag
  tags2    | tagB, tagC | tag
  depends  |            | after, before, before_after, itself
  template |            | args, file
  mixin    |            | task2, task3, task1

Using the above myke.yml, you can invoke tasks like:

  • myke build runs build in all projects
  • myke <project>/build runs build in that specific <project>
  • myke <tag>/build runs build in all projects tagged <tag>
  • myke <tagA>/<tagB>/.../build can match tasks by many tags (AND)
  • myke task1 --key1=val1 task2 --key2=val2 ... passes arguments to individual tasks

Installation

Examples

Explore the self documenting examples folder.

Task Execution Environment

  • tasks always run with cwd set to the folder where the task is defined
  • cwd/bin is always added to PATH
  • environment variables are loaded from:
    • env property in yml
    • dotenv files from env_files
    • for every dotenv file, the corresponding dotenv .local file is also loaded if present
  • same is done for every mixin that the yml uses
    • So, if you mixin <some-other-folder>/myke.yml, then that yml's cwd/bin is also added to the PATH, that yml's env/env_files/env_files.local are also loaded, and so on
  • shell exported environment variables take precedence
  • additional variables: $MYKE_PROJECT, $MYKE_TASK, $MYKE_CWD are always set
    • $myke is set to full path of myke itself to easily nest myke calls (e.g. $myke do_something will become myke.exe do_something in windows)
  • command is templated using golang text/template and sprig
    • environment and task arguments are passed in as variables
  • command is run using sh -exc

FAQs

How do I share common logic in tasks?

There are multiple ways including:

  • Place shared scripts in bin folder (remember that CWD/bin is always added to the PATH). If the scripts are complex, you can write them in any scripting language of your choice
  • If multiple projects need to share the same scripts, then use a common mixin folder (remember that for mixin ymls - the same CWD/bin is added to PATH, same env files are loaded, etc, refer Task Execution Environment above)

For example,

  • java-mixin
    • myke.yml - project template with tasks
    • myke.env - environment vars, can be overridden by extending projects
    • bin - gets added to the PATH of extending projects
      • any shared scripts that you want
  • kubernetes-mixin
    • ...
    • ...

Why use myke?

Deferring higher order build logic (like reading scm history for changelogs, updating scm tags/branches, generating version numbers, etc) to a meta-build tool (like a task runner or aggregator), restricting build tools to do only simple source builds, and having a shared build vocabulary across projects is a generally good idea. There are millions of such meta-build tools or task aggregators out there, we just wanted something fast, zero-dependency and language-agnostic while still helping us manage multiple components across repositories with ease.

In that sense, myke is never a build or deployment tool, its just a task aggregator. Its not designed to be an alternative for source build tools, rather it just augments them. The comparison below is on that same perspective.

  • maven is a lifecycle reactor and/or project management tool that does a lot of things (compilation/scm/release/lifecycle/build/etc), except its hard to use it as a simple task runner. myke focuses only on the latter
  • bazel buck pants gradle ... replace your current buildchain by giving you a totally new DSL to compile your programs (java_binary, etc). myke simply acts as a yml-based interface to your existing tools and workflows, thereby not needing to change your project and IDE setup
  • grunt gulp pyinvoke rake sake thor ... myke is zero-dependency, language agnostic, uses simple yml and allows aggregation of tasks through hierarchies, templates and tags
  • make scons ninja ... they are low-level build tools with a crux of file-based dependencies. Most buildchains today are already intelligent enough to process only changed files, so myke completely bypasses file tracking and only focuses on task aggregation and discoverability
  • capistrano fabric ... myke is not a deployment tool for remote machines, and does not do anything over SSH
  • ansible salt ... myke is not a configuration management tool, its a task runner
  • robo is the closest relative to myke, you should check it out as well

Development

Use docker/docker-compose to develop. You don't need to have golang installed.

  • docker-compose build Builds and runs tests
  • docker-compose up Produces bin folder with executables
  • docker-compose run --rm default /bin/bash Gives you a terminal inside the container, from where you can run go commands like:
    • go test ./... Runs all tests
    • go run main.go Compiles and runs main