Supermarket Quality Review

abstract release requirements

• software packages, all platforms
• documentation: docs.chef.io readme
• communication: emails, blogposts, marketing
• user feedback loops : bugs, questions, feedback

installation validation:

• RHEL
• Ubuntu

holistic

1. supermarket-ctl testing
2. service validation
3. logging validation
4. soak test for memory leaks
5. functional verification matrix, see work item below

WORK ITEM: Need to understand which of the functional verifications need to happen on:

• all platform versions
• one version of each platform
• one test, doesn't matter where

functional verification

1. matrix for: https://github.com/opscode/omnibus-supermarket/blob/master/cookbooks/omnibus-supermarket/attributes/default.rb
2. cookbook upload/download (sanity?)
3. oc-id integration
4. s3 integration
5. special case: no s3, multiple app servers (because w/o s3, it saves it locally)
6. supermarket.json loading after default generated config
7. filepath for cookbook storage
8. search

docs validation

1. explicit golden path for installations
2. configuration tunables, both .rb and .json methods
3. s3 vs. local filesystem caveats
4. configuration methods across supported topologies

cross-browser functional testing

• browserstack
• sauce
• initial selenium tests written and in repo needs definition. we at least need a tooling run for low hanging (and embarrassing) fruit (xenu, fiddler, devtools, yslow)