yplatform

yplatform supports various cycles of software development, through a canonical set of commands.

yplatform aims to diminish cognitive load 💆, time to first build ⏱️ and time to first deployment 🚀 . Why?

yplatform was conceived within Tobii Pro's Cloud Services team 👋, previously known as support-firecloud .

---

Table of Contents

• Install standalone
• Install in a project
• Why?
• Uniform interface
• Cross-platform
• Cross-provider
• Show me!
• Repository structure
• Documentation

---

Install standalone

git clone https://github.com/ysoftwareab/yplatform.git and you will gain access to:

• a bootstrap process for local-development based on GNU and Homebrew
  • cross-platform.
    • No more worries about this is how it works on a developer machine (e.g. MacOS/Windows) and this is how it works in the CI (e.g. Linux).
  • brings common GNU utilities (bash, make, grep, sed, find, etc), version locked.
    • No more worries about different flags, different behaviour or missing features!
  • brings Homebrew.
    • No more worries about different versions available in distros' repositositories, or libraries/utilities not being available at all.
• sane per-user configuration e.g. git
• various utility scripts, 99% GNU Bourne Again Shell (Bash)
• documentation and best common practices

---

Install in a project

bash -c "$(curl -qfsSL https://raw.githubusercontent.com/ysoftwareab/yplatform/master/bin/yp-install)" to install the latest yplatform (as a git submodule) to your project and you will gain access to:

• a bootstrap process for CI machines based on GNU and Homebrew
  • cross-provider. No more worries about running pipelines with multiple CI providers, or switching to a new CI provider.
• a build system based on GNU Make, both robust and flexible
• sane per-repo configurations e.g. vscode
• various utility scripts, 99% Bourne Again Shell (Bash)
• a cloud infrastructure-as-code system based on GNU Make, both robust and flexible

---

Why?

yplatform aims to diminish cognitive load 💆, time to first build ⏱️ and time to first deployment 🚀 .

This means removing out-of-band information about system dependencies. How many times a repository's README mentions "something version x required"? How many times it doesn't mention anything, and you end up looking for a CI/Docker configuration?

This means removing differences in installing system dependencies. How many times did you look up "how to install X on Y?" How many times did you discover that you get different system dependencies, depending on the version of your Linux distribution?

This means removing differences in core utilities. How many times do you get something working on your MacOS development machine (BSD), only to see it fail on a CI machine, say running Ubuntu (GNU)?

This means removing differences in installing local dependencies. How many times a repository's README mentions npm install or yarn install or pip install or ./configure --prefix=/usr; make? Why should you care if this is a node/python/erlang/etc codebase, or what package manager it uses?

This means removing differences in triggering a build, checks, tests, a deployment, etc between codebases and their main programming language. How many times did you check how to build a codebase, what language it is using, what package manager or test framework it is using, what services need to be up, etc? How many times did you realize that checks are impossible to run locally, and that they run only in the CI, or worse - as a service reviewing/commenting your PR making it impossible to get early local feedback instead of feedback/noise in a Github pull request? How many times did you return years after to an old codebase, of your own above all!, only to realize you have no clue what's needed to build it again?

This means removing differences between what runs locally and what runs in the CI. How many times did you realize that you're instrumenting the CI ever-so-slightly-different than your local runs? Because maybe you duplicated code in the scripts section of your package.json and your GitHub workflow YAML?

This means removing differences between CI providers. How many times did you postpone trying out another CI provider? How many times did you choose a provider based on previous experience, popularity, how easy it is to integrate, rather than based on whether it is the most performant or the most appropriate for the job at hand?

This means consolidating best current practices. How many times did you look up how to set up encryption in a git repository? How many times did you reconfigure git with sane defaults or bump into CRLF/LF issues between system/developers? How many times did you configure or postpone altogether configuring a gitops flow? Env branches, tags, changelog creation, etc?

This means trying to make everything work the same everywhere. Emphasis on trying. In reality, 100% the same is not possible. But it is possible to achieve say 80%. And 80% is much better than 0% i.e. not trying at all.

---

Uniform interface

Starting fresh with a new project or switching between projects doesn't mean you need to switch development context, and instead you can focus on the business-logic context.

This mindset is useful for junior and senior developers alike. It makes onboarding new people as frictionless as possible.

Others have also been writing about this as early as 2016 and as late as 2021 and 2022.

Why use make you might ask. A few reasons actually:

• make it's old (1976) but still under development, which makes it omnipresent, reliable and durable. In 5, 10, 20 years from now, expect it to still work!
• lower cognitive load since chances are much higher that someone has already encountered make in other projects

The entrypoint to all projects that use yplatform is

Just type make !

make will fetch local dependencies, build the project and check that everything looks alright. That's all you need to have a fully functional development environment. Sometimes if your project has specific system dependencies, you may need to run make bootstrap.

"Then what?" you might ask.

Just type make help !

And you get a nice list of self-documented targets.

Want to see all targets? Type make help-all.

Want to fetch dependencies alone? Type make deps.

Want to build and skip dependencies and checks? Type make build.

What to run checks alone (i.e. linters)? Type make check.

Want to run tests? Type make test.

Want to release a new version as a git tag with automatic release notes based on git commits? Type make release.

Want to see commits since last release? Type make unreleased.

Want to promote a version to an environment branch? A branch that is coupled to a CI that deploys your software to a staging environment for example? Type make promote-env/staging/v1.2.3 for example.

Do you have secrets checked in and want to decrypt them transparently via git clean/smudge filters? Type make decrypt.

Do you want to reset your local git repository as if it was freshly cloned? Type make nuke.

And the list goes on and on and on. Fully extensible. Preloaded with minimal best current practices.

---

Cross-platform

Similarly yplatform itself runs across these platforms with very little effort and no duplication. Bootstrapped Docker images are also available.

Platform	minimal	common	minimal (Docker)	common (Docker)
Apple MacOS 11	✔️	✔️
Apple MacOS 12	✔️	✔️
Apple MacOS 13	✔️	✔️
Linux (Alpine 3.15)	✔️	✔️	yp-alpine-3.15-minimal	yp-alpine-3.15-common
Linux (Amazonlinux 2)	✔️	✔️	yp-amzn-2-minimal	yp-amzn-2-common
Linux (Archlinux)	✔️	✔️	yp-arch-0-minimal	yp-arch-0-common
Linux (Centos Stream 8)	✔️	✔️	yp-centos-8-minimal	yp-centos-8-common
Linux (Debian 9)	✔️	✔️	yp-debian-9-minimal	yp-debian-9-common
Linux (Debian 10)	✔️	✔️	yp-debian-10-minimal	yp-debian-10-common
Linux (RHEL 8.5)			yp-rhel-8.5-minimal	yp-rhel-8.5-common
Linux (Ubuntu 18.04)	✔️	✔️	yp-ubuntu-18.04-minimal	yp-ubuntu-18.04-common
Linux (Ubuntu 20.04)	✔️	✔️	yp-ubuntu-20.04-minimal	yp-ubuntu-20.04-common
Linux (Ubuntu 22.04)	✔️	✔️	yp-ubuntu-22.04-minimal	yp-ubuntu-22.04-common
Microsoft Windows 2019 (WSL)	✔️	💤
Microsoft Windows 2022 (WSL)	✔️	💤

NOTE RHEL needs entitlement certificates in the CI.

NOTE common install is paused for Microsoft Windows (WSL) because the process takes more than 1 hour on Github Actions CI (a combination of machine specs and being restricted to WSL v1), but we expect no problems.

NOTE it should even possible to create WSL images, and even OCI/podman images. Tracked in #213.

---

Cross-provider

yplatform itself currently runs across these providers with very little effort and no duplication.

Provider	yplatform status	master	provider-branches	version tags	PR	config	env
AppVeyor		✔️	appveyor*			config	env
Bitrise		💤	bitrise*			config	env
Buddy		✔️	buddy*			config	env
CircleCI		✔️	circle*	✔️	✔️	config	env
Cirrus CI		✔️	cirrus*		✔️	config	env
Github Actions CI		✔️	github*	✔️	✔️	config	env
Gitlab CI		💤	gitlab*			config	env
Semaphore		✔️	semaphore*		✔️	config	env
Sourcehut		✔️	sourcehut*		✔️	config	env

NOTE Bitrise builds for the master branch run only once a day due to freemium limits. NOTE Gitlab builds for the master branch run only once a week due to freemium limits.

We have also integrated in the past with more CI providers

Provider	yplatform status	master	provider-branches	version tags	PR	config	env
Codeship		💤	codeship*			config	env
Drone CI		💤	drone*			config	env
Travis CI		💤	travis*			config	env

NOTE Codeship builds are paused because Cloudbees Codeship stopped offering their free plan. NOTE Drone CI requires explicit runners. NOTE Travis CI builds for the master branch are paused because one has to constantly ask for more open-source credits.

• Integration with Travis CI is unstable since it now requires a subscription.
• The new pricing model for travis-ci.com
• Travis CI's new pricing plan threw a wrench in my open source works
• Travis CI is no longer providing CI minutes for open source projects

---

Show me!

Show me macOS and Ubuntu 20.04 bootstrapping

This is a bootstrapping of a fresh macOS developer machine.

Had to input my password (e.g. to install Homebrew) and decide post-installation if I optionally want to switch my shell from zsh to bash, something that is useful for those that are not techies. All done in 10 minutes. Set for life. Watch it at 10x, then it's only 1 minute. 😎

Second video is a bootstrapping of a fresh Ubuntu 20.04 developer machine. Had to input my password (e.g. to install some aptitude packages and Homebrew) and input my timezone. All done in 10 minutes. Set for life. Watch it at 10x, then it's only 1 minute. 😎

Show me ways-of-working with a project with yplatform

This is a cloning of a project on a macOS developer machine, and installing project-specific system dependencies.

Second video shows opening the same project, and playing around without actually having to know anything about it.

---

Repository structure

• /bin has executable scripts, 99% GNU Bourne Again Shell (Bash)
• /build.mk has build-system makefiles
• /bootstrap has scripts that help bootstrap a machine
• /ci has scripts that help steer the CI pipelines
• /dev has scripts that help bootstrap a developer machine
• /dockerfiles has scripts for the bootstrapped Docker images
• /gitconfig has git configuration
• /sshconfig has ssh configuration
• /repo has configuration that is repo-specific, for those repositories bootstrapped with yplatform
• /mk has common include makefiles e.g. sourced from /build.mk makefiles
• /sh has common include shell scripts e.g. sourced from /bin shell scripts

---

Documentation

• newcomer
  • bootstrap
• daily work
  • code of conduct
  • contributing via git (and Github Pull Requests)
  • working with make
  • working with a local npm dependency
  • how to release
  • style
    • Use common sense and BE CONSISTENT. (Google)
    • Less than 120 cpl, preferably less than 80
    • shell
    • Makefile
    • JavaScript/TypeScript
    • SASS
• set up a new git repository
  • new git (Github) repositories
  • how to license
  • integrate Travis CI
  • .ci.sh
  • how to manage secrets
• Amazon Web Services
  • CloudFormation
  • Identity Access Management

---

Writing

• https://developers.google.com/tech-writing/overview
• https://github.com/google/styleguide/blob/gh-pages/docguide/philosophy.md
• https://www.julian.com/guide/write/intro

---

License

Apache-2.0