QA Engineer Assessment Solution

Introduction

GitHub

LinkedIn

Hello, I'm Ricardo and this is my solution for the challenge. Spoke to Arpan about the possibility of building a solution using webdriverIO instead of protractor and after the go-ahead, came up with this. I based this solution on a boilerplate I've been working for some time.

Folder structure

• /
  • .idea - Run configuration for your convenience
  • .run - The folder aggregating the latest test run artifacts
    • features - Running in parallel mode will split scenarios into feature files, they are store here
    • logs
      • wdio.log - Main process log file
    • reports
      • allure - Historical Allure report
      • cucumber - Latest run Multiple Cucumber test report
    • results
      • allure - Allure XML results
      • json - Cucumber JSON results
      • visual - Visual regression test results
    • shots - Screenshots are stored here
  • src - Contains the app
  • tests - Base tests folder, contains E2E and visual tests
    • e2e
      • components
      • configs
      • features
      • pages
      • services
      • steps
      • utils
    • visual
      • configs
      • snapshots
    • base.conf.js - Base configuration for tests E2E or not
  • docker-compose.yml - A Docker compose file to bring up a local Zalenium grid

Bootstrapping

As per assessment, use Node 10 for the app. My custom webdriverIO-based project however needs at least Node 12 LTS. The app also does not seem to mind it. So my recommendation is to go with Node 12 LTS. If you're running nvm, you can simply run:

# nvm install 12
nvm use 12

Install the dependencies

npm install

There are several options to run the tests as highlighted by this table:

Description	Automation Protocol	npm script	Configuration file
On the local machine with CDP / Pupeteer	Chrome Devtools Protocol	test:e2e/test:e2e:local	local.conf.js
On the local machine with Selenium Grid	webdriver	test:e2e:grid	grid.conf.js
Against a local (or remote) Zalenium Grid	webdriver	test:e2e:zal	zalenium.conf.js
Against BrowserStack	webdriver	test:e2e:bs	bs.conf.js

To run a specific feature file pass --spec flag to the command with the path, something like:

npm run test:e2e -- --browserName chrome --spec tests/e2e/features/SearchPlanet.feature

For specfic Scenarios, in the --spec file passed, add the line number of the Scenario

npm run test:e2e -- --browserName chrome --spec tests/e2e/features/SearchCharacter.feature:11

For convenience, there are 3 suites configured, one for general Search, one for Character and another for Planet. Simply pass --suite flag to the command with one of the suite's names:

npm run test:e2e -- --browserName chrome --suite planet

You can combine these options with parallel mode which will spin up a browser instance per Scenario instead of per feature file (capped at a maximum of 4 as per tests/base.conf.js). Simply add --parallel to any command like so:

npm run test:e2e:local --parallel`

By default, the tests will go against 2 browsers: Chrome and Firefox. If you want to specify which browser (and this is valid for all test run options) use --browserName as such:

npm run test:e2e:local -- --browserName chrome

To run Firefox via Chrome Devtools Protocol you need to:

• Firefox Nightly
• specify the binary path in local.conf.js
• have "nightly" in the that path

npm run test:e2e:local -- --browserName firefox

	Too many options? Use an IntelliJ IDE and pick a run configuration. For Scenarios, the cursor should be on their name / line number in order to work.

Problems faced

• In the provided protactor-based boilerplate, the globing pattern for the .feature files was missing. Tests obviously would not run. Fixed by editing protactor.conf.js, replacing:

  ./e2e/features/*/*.feature

  with

  ./e2e/features/**/*.feature

• In the provided protactor-based boilerplate, I could not run the tests without first correcting protactor's Cucumber framework configuration. I've tried simply changing the framework from custom to cucumber but more changes were necessary as per the documentation:

  Note: Cucumber is no longer included by default as of version 3.0. You can integrate Cucumber with Protractor with the custom framework option. For more information, see the Protractor Cucumber Framework site or the Cucumber GitHub site. If you would like to use the Cucumber test framework, download the dependencies with npm. Cucumber should be installed in the same place as Protractor - so if protractor was installed globally, install Cucumber with -g.

  framework: 'custom',
  frameworkPath: require.resolve('protractor-cucumber-framework')

• Running the tests was failing due to a missing tests-reports folder in the qa-tests-assessment-master base folder. Creating this folder solves the problem, perhaps this step should be part of the setup.

Approach

data-test attributes

Since it was specified modifying the app was an option, I've added data-test attributes to some important components and static elements. My reasoning for this is the based on the common justification for using this convention:

• It anchors important elements and removes this burden from other properties like the id attributes and CSS classes which are heavily linked to functionality and style and are therefore prone to change.
• Allows for a more readable and overall more (application) conforming Page Object design.

For the App component:

• app-loading - For the loading element, important to assess whether the app is in an expectation-ready state
• app-notfound- For the Not Found message

For the Planet component:

• planet - For the root element
• planet-name
• planet-population
• planet-climate
• planet-gravity

For the Character component:

• character - For the root element
• character-name
• character-gender
• character-year
• character-eye
• character-skin

For the SearchForm component, search-form at the root.

Reporting

There are two reports you can browse.

Allure

If it was not done automatically, you can generate the report after a test run with

npm run report:generate

You can then serve and open it by

npm run report:open

Multiple Cucumber

You should be provided a link to the automatically generated HTML file after the test run. Either way, the last run report always kept in here.

Remarks

• There is a bit of code being reused among the Planet-focused and Characters-focused step implementations. Figured it was more important to show proper separation of concerns and not have big, do-it-all Frankestein-monster steps :)

  As a practical example of this design choice, in search.steps.js there is a generic step to assess if any search result is present:

  Then(/^I should( not)? see any search results$/i, shouldNot => {
  
     homePage.loadingMsg.waitForDisplayed({ reverse : true });
  
     const { Characters, Planets } = homePage;
  
     const displayedResults = [Planets, Characters]
         .flat()
         .filter(c => c.container.isDisplayed())
         // Better assertion output on failure
         .map(r => r.asModel);
  
     shouldNot ?
         expect(displayedResults).toHaveLength(0) :
         expect(displayedResults.length).toBeGreaterThan(0);
  });

  In which the keyword's regular expression, logic and assertion could be changed to accommodate not only generic search results but also only Characters and only Planets. Would increase a bit the complexity of the step, making it more difficult to maintain and read while also breaking this separation of concerns principle. In this step implementation, we could easily iterate on it to add other possible categories easily like Starships by adding a Startships page component to the array of component lists we consider search results, something like

      const displayedResults = [Planets, Characters, Starships]

  Because of these reasons, I'd rather reuse a bit of code/logic in these two extra steps:

  • people.steps.js:

    Then(/^I should( not)? see a list of (?:people|characters)$/i, notSee => {
       homePage.loadingMsg.waitForDisplayed({ reverse : true });
       const displayedCharacters = homePage.Characters.filter(p => p.container.isDisplayed().map(r => r.asModel));
       notSee ?
           expect(displayedCharacters).toHaveLength(0) :
           expect(displayedCharacters.length).toBeGreaterThan(0);
    
    });

  • planets.steps.js:

    Then(/^I should( not)? see a list of [pP]lanets$/i, notSee => {
        homePage.loadingMsg.waitForDisplayed({ reverse : true });
        const displayedPlanets = homePage.Planets.filter(p => p.container.isDisplayed().map(p => p.asModel));
        notSee ?
            expect(displayedPlanets).toHaveLength(0) :
            expect(displayedPlanets.length).toBeGreaterThan(0);
    });

• Fixed a problem I created, made the mistake of writing some tests where I was destructuring the page and therefore querying the DOM before making the wait for the loading message to disappear

// Mistakenly evaluating the Characters before waiting for loadingMsg to disappear in next line
const { loadingMsg, Characters } = homePage;

homePage.loadingMsg.waitForDisplayed({ reverse: true });

// App has characters but assertion fails due to Characters being initialized too early and therefore being empty
expect(Characters.filter(c => c.container.isDisplayed())).toBeGreaterThan(0)

• I realize more coverage could be had by building more scenarios around the "When I press Enter" step
• A lot of effort went into holding myself back from including any /r/prequelmemes :)