testing.html.md.erb

---
title: Testing your service tiles
owner: Services
---

You can use the following tile testing practices for <%= vars.platform_name %>.

## <a id="testing"></a> Service tile testing

Good testing assures tile developers that their product installs and runs properly on diverse platforms
and assures you that the tile you install can provide the service on your platform.

Arrange for a pyramid structure for testing, starting with unit tests and moving up to successively broader and
more automated levels of integration.

Use [Concourse](./concourse.html) for creating build pipelines that follow this test structure.
Other continuous integration tools must support a pyramid testing approach.

### <a id="pyramid"></a> Tile test pyramid

For <%= vars.platform_name %> tiles, a typical test pyramid progresses as follows:

1. Unit tests for each tile **component**. For example, service components, broker, adapter, and
metrics emitter, manual by developer and in an automated pipeline.

1. System tests of the tile's **BOSH release**, including:
  - **Functional tests** that cover the main features of the service.
      The main features typically interact with almost all important external integration points, because
      the tests confirm product capability.
  - **Smoke tests** or life cycle tests for service instances that create and bind a service instance.
      You call it from a test app, verify the logs it generates, and delete it.
      For a typical end-to-end test sequence, see [Smoke Tests](#smoke).

1. System tests of **tile** operation within Tanzu Operations Manager.
  - These include:
      * **Configuration checks** that test every external configurable integration point and connection to remote servers using configured credentials.
      * **Default checks** that confirm "happy path" capability.
  - Use the Tanzu Operations Manager API to verify that property blueprints in the tile metadata are correct
    and that they translate correctly to the BOSH manifest that Tanzu Operations Manager generates.
  - Use the [om](https://github.com/pivotal-cf/om) tool to call the Tanzu Operations Manager API programmatically from Go.
    Avoid the unsupported opsmgr gem that called the Tanzu Operations Manager API from Ruby.
  - Confirm manually that the tile wires property blueprints to the expected pane and form controls in the UI.
  - Test your environment using one of the environments described in [Development Environments](./environments.html).

System tests might incur costs from using third-party services, IaaS resources, and others.

### <a id="smoke"></a> Smoke tests

Smoke tests are end-to-end life cycle tests for service instances that you can include as [post-deploy errands](./tile-errands.html#post-deploy)
within a tile and also automate in [Concourse](./concourse.html) or other integration platforms.

A typical smoke test runs as follows:

1. Create an org and space for the test to run in.

1. Register the tile service broker.

1. Activate service access for the created org.

1. Go through all service plans (or a subset of them):
  1. Create a service instance for the plan.
  1. Push a test app.
  1. Bind the service instance to the app.
  1. Use the app in a way that exercises the service instance. For a data service, for example, write and read from the service instance.
  1. Unbind the service instance.
  1. Delete the service instance.
  1. Delete the test app.

1. Delete the service broker.

1. Delete the test org and space.

### <a id="general-recs"></a> General recommendations

These are general recommendations for designing and running tests on <%= vars.platform_name %> tiles:

* Leave the environment exactly as it was before the test was run.

* Generate verbose logging with lots of contextual data to make troubleshooting easier.

* Design test suites for re-usability by making them highly parameterizable.
Important parameters include:
  - External settings such as domains, creds, and certificates.
  - Plans to test against.
    For example, the [Redis for <%= vars.platform_name %>](https://github.com/pivotal-cf/cf-redis-smoke-tests) smoke tests use identical code
    for two different service plans, pre-provisioned and on-demand.
  - Timeouts, numbers of retries, and other things that must be adjusted for different environments.
  - Switches to include or exclude portions of the tests such as generating metrics or backups.

* Reuse tests that exist already, for example, in Concourse.

* Use an example Cloud Foundry app that uses your service. This app can serve for
testing, doing a demo of your tile capabilities, and as a code example. See the [MySQL Test App](https://github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests/tree/master/assets/sinatra_app) an example.

* When testing manually, using the UI is better than calling the underlying API directly.
  Use UIs and APIs the way your customer does.