Research Strategy
User research will be a core aspect of the U.S. Web Design Standards project and each form of research should occur at a different frequency. Below are the recommended activities that the design team should regularly be performing.
Frequency - Weekly
Once web analytics are set up, every week a member of the team should provide a read out of the analytics that are tracking user activity on U.S. Web Design Standards. This read out should attempt to point out areas of concern and pain points that are being communicated by metrics like form abandonment and the time it takes a user to complete specific form fields.
Dependencies
- The team has defined the metrics and funnels they want to track and report out on
- A team member must be familiar enough with web analytics to set up the tracking and reporting
References
- https://www.nngroup.com/articles/analytics-reports-ux-strategists/
- http://www.uxbooth.com/articles/an-analytics-first-approach-to-ux-part-1/
Frequency - Once a month
The work of the team should be tested early and often. To meet this goal, once a month the team should take what they've been working on and put it in front of end users. Using live recruiting methods or recruiting via the micropurchase program, the team should have easy access to potential participants. These tactical usability test sessions should last between 15 and 30 minutes and focus specifically on the problems the team has been recently trying to solve.
Incentives
If possible, reimburse participants for their time with an honorarium of $25 to $50, depending on the length of the sessions.
References
- https://www.nngroup.com/articles/feature-user-test/
- https://www.nngroup.com/articles/task-scenarios-usability-testing/
- http://www.uxbooth.com/articles/guerrilla-research-tactics-tools/
- https://www.interaction-design.org/literature/article/10-hints-for-carrying-out-better-guerrilla-usability-testing
- https://www.gov.uk/service-manual/user-centred-design/user-research/guerrilla-testing.html
Frequency - Once a quarter
Once a quarter, it's recommended that the design team perform an end-to-end evaluation of the entire site. This should be a focused effort where others work efforts are limited or paused, and it should involve the entire team in some capacity. The team should use the live recruiting techniques and the micropurchase program to recruit participants. These participants will go through the website from start to finish. Each session should last between 45 and 60 minutes.
Incentives
If possible, reimburse participants for their time with an honorarium of $50 to $75 dollars, depending on the length of the sessions.
Dependencies
- 2 weeks prep time for research plan and recruiting
- Recruiting method to contact potential participants who need to register to vote
- Remote viewing technology to allow the wider team to observe testing sessions
References
- https://www.nngroup.com/articles/usability-101-introduction-to-usability/
- https://www.nngroup.com/articles/when-high-cost-usability-makes-sense/
- http://www.amazon.com/Handbook-Usability-Testing-Conduct-Effective/dp/0470185481
- http://www.usability.gov/how-to-and-tools/methods/usability-testing.html
Frequency - Every six months
While people's end goals hardly ever change, the ways people go about accomplishing their goals do. Twice a year, a subset of the design team should spend a few weeks meeting with potential users and exploring their world and mindset as they apply to voter registration. The participant pool should be a mix of people that represent the user profiles.
Incentives
If possible, reimburse participants for their time with an honorarium of $50 to $75 dollars, depending on the length of the sessions.
References
- http://www.userfocus.co.uk/articles/ethnography.html
- http://www.uxmatters.com/mt/archives/2014/09/conducting-large-scale-user-research.php
- https://uxmag.com/articles/making-the-most-of-ethnographic-research
- https://articles.uie.com/field_studies/
Each form of research will be reported differently, which will allow the team and stakeholders can understand what was done and what the team learned. The importance of reporting out the research findings is to remind everyone not only the value of research, but the value of including users in the overall design process.
The weekly report of the web analytics should take two forms. The first is an email sent out to the team highlighting any points of interest and areas that might need to be considered for further research. Since email can easily be missed, analytics-related findings will be reported in a second form: a quick, five-minute report out given to the team during one of the daily stand ups early in the week (e.g., Create report on Monday, report to the team on Tuesday).
References
- http://www.digitalgov.gov/2014/03/04/creating-awesome-web-analytics-reports-and-presentations/
- https://megalytic.com/blog/creating-a-web-analytics-report-for-the-big-picture-executive
- https://megalytic.com/blog/elements-of-an-effective-web-analytics-report
The report for the usability tests the team will conduct depends on the focus of the study.
Since prototype usability tests focus on a small set of tasks or UI components, the report should reflect this focus. Each task or UI component of focus should be reported on based on user interactions and comments. A severity rating, based on NNGroup's rating scale (https://www.nngroup.com/articles/how-to-rate-the-severity-of-usability-problems/), should be used to help the team prioritize the work necessary to correct the issues.
End-to-end testing differs from tactical testing since the team is trying to assess the entire experience of a product. The report for this form of testing should reflect this difference by focusing on the overall themes that come out of the testing. Each theme should be given a title and the evidence for why that theme is present should be detailed out. A recommendation for how to address the problems that theme represents should be made with the same severity scale from NNGroup.
References
- https://www.nngroup.com/articles/actionable-usability-findings/
- http://www.usability.gov/how-to-and-tools/methods/reporting-usability-test-results.html
- https://www.nngroup.com/articles/formal-vs-quick-usability-reports/
- http://www.uxmatters.com/mt/archives/2012/02/communicating-user-research-findings.php
The findings coming out of user research should follow the same manner as the Full Experience Test Reporting. When the team has collected all the data, look for the patterns that stand out around user goals, needs, pains, joys, and missed opportunities. These patterns will turn into the themes that should guide the next phase of design efforts.
References