[cli] Create test harness (#4268)

* create test harness

* add testing framework that's transferable to contribution

* use different versions of flow

* add basic documentation in readme

* write docs

* correct md copy

.gitignore

@@ -36,6 +36,9 @@ cli/.flow-bins-cache/*
 cli/package-lock.json
 cli/README.md
 
+# Test harness
+harness/
+
 # Editors
 .vscode
 .idea

CONTRIBUTING.md

@@ -88,6 +88,8 @@ and now you'd like to contribute it.
 >
 > This will cover steps 1 - 4, of course without real definitions or tests written for you.
 
+> Before you start, if you think you have a fairly complicated library definition or one that you have not yet written and tested with your personal project you may consider leveraging our **test harness** which allows you to write code immediately in a simplified clean environment. Learn more [here](https://flow-typed.github.io/flow-typed/#/harness) or just create the harness (`./test_harness.sh`) to get started.
+
 #### 1) Create a new directory called `definitions/npm/left-pad_v4.x.x/`.
 
 ***We call this the "package version directory".***

docs/_sidebar.md

@@ -13,5 +13,6 @@
   - [update-cache](cache.md)
   - [validate-defs](validate.md)
   - [version](version.md)
+- [Test Harness](harness.md)
 - [FAQ](faq.md)
 - [Changelog](changelog.md)

docs/harness.md

@@ -0,0 +1,23 @@
+# Test Harness
+
+A testing ground created to help you write and test flow definitions before contributing them back to our registry.
+
+After cloning the repository you can run `./test_harness.sh` in the root directory to create the `./harness` directory. Here you can write general flow code or definitions to verify functionality in real time with an IDE.
+
+``` bash
+# Sample usage
+./test_harness.sh
+./test_harness.sh 0.140.0 # creates the test harness with flow-bin set to `v0.140.0`
+```
+
+The directory is git ignored so please feel free to add any settings to `.flowconfig`, dependencies to package.json or other library definitions to achieve your needs.
+
+Once you're done with development you can delete the directory or leave it for next time, the choice is yours.
+
+## Contributing back
+
+If you wrote a definition you'd like to now contribute back you can read our [contributing](contributing.md) guide and copy the definition over. The test harness is originally setup with a `definition.js` and `test_definition.js` for definition creation and testing respectively for this purpose.
+
+Once you have them copied, you can run our full test suite that runs them against various flow versions to ensure they're up to the standards we need for contribution.
+
+**Contrary to the test harness code written in the `definitions` directory can't be checked by flow in real time**

test_harness.sh

@@ -0,0 +1,38 @@
+#!/bin/sh
+
+flowVersion=$1
+if [ -z "$flowVersion" ]
+then
+  lib="flow-bin@latest"
+else
+  lib="flow-bin@$flowVersion"
+fi
+
+if [ -d "./harness" ]
+then
+  cd harness
+  npm i --save-dev $lib
+else
+  mkdir harness
+  cd harness
+  npm init -y
+  npm i --save-dev $lib
+  npx flow-setup flow-typed-harness
+  cp ../docs/harness.md README.md
+  cat <<EOF > definition.js
+declare module "" {
+
+}
+EOF
+  cat <<EOF > test_definition.js
+// @flow
+import { describe, it } from 'flow-typed-test';
+// import lib from 'test';
+
+describe('', () => {
+  it('', () => {
+
+  });
+});
+EOF
+fi