2021 07 29 External Link Indicator Research Findings

Abstract

Our research with members of the public revealed some key information about the USWDS link component. Although users didn't consistently understand both external link indicator variants, they did understand a link's destination, when that destination was included either in the link text or in the contextual language around the link. Users found agency names or branding in the top left corner of a webpage to be a critical "anchor" for orientation. Finally, users found long links hard to read and understand, and if acronyms and "government-speak" were included, that only made the experience more confusing.

Background

We updated link guidance during the USWDS Accelerator Emergency Response effort in 2021. The link component brief describes the work that went into building these updates to the link guidance. The Federal Web Council reviewed the updated guidance and gave us important feedback to help improve and clarify it.

From May to June 2021, the design system team conducted research around the consistent use of an indicator for text links to external websites. We first performed a landscape analysis of commonly used external link icons and designed an alternative variant of the external link indicator. Then, we conducted validation research detailed in this Usability Test Plan for External Link Indicators. We tested two variants: (A) an existing external link icon; and (B) a newly designed "Exit" badge to determine which visual option was best understood by users as an indication of an external link. Based on the findings outlined below, we updated the link component, design elements, and corresponding component implementation guidance.

Method

Research date: May–June 2021

Research type: Qualitative comparison A/B usability testing

Participants: 6 participants with no domain-specific technical experience

Summary

To determine which visual option was best understood by users as an indication of an external link, we conducted a qualitative comparison A/B usability testing of two variants. We followed the updated USWDS link guidance and, using a real web page of the stopbullying.gov website that contains both internal and external links, we designed two versions of a clickable inVision prototype. The body copy on each prototyped page was exactly the same---the only difference was the visual treatment of external links on the page.

We tested version A of the prototype with one set of users and version B prototype with another set of users to see which option performed better. We didn't show both versions to the same set of users to avoid skewing test results.

Variant A features an existing indicator icon:
Variant B has a newly designed "EXIT" badge indicator:

For this study, we prioritized recruiting members of the general public with low levels of technology comfort, such as senior citizens and people who don't work on a computer daily, but who may use one for their personal needs. We thought that if we could get it right for the people with technology challenges, we would get it right for the majority of people.

Findings

The following insights helped determine which design creates an easier, more intuitive reading experience.

Universal online behavior insights

Links, themselves, can be a distraction. If users have a specific goal to accomplish on a web page, they won't leave that page and follow a link elsewhere until they complete their task on the current page. Therefore, links can be a distraction when users read blocks of text. To the average user, it is inconsequential whether the link is internal or external to a particular website. They will click through links and other navigational elements to find what they want.

Quote: "I would read whatever is on this page first, depending on why I'm on this site. If I was sure my kid was being bullied, I'd go to the action-related sections. It depends on what I'm looking for." P9

Users are accustomed to skimming a web page. They look for visually different text that "calls attention" to get a general understanding of the page's overall content. However, government websites are often text-heavy and complex to navigate, requiring a lot of clicking, scrolling, and careful reading to find information. Additionally, paragraphs with several links can be hard to read or understand. If site content has poor visual hierarchy, people often miss important information on the page and turn to popular search engines like Google for more pointed search results.

Quotes:

"I'm not very sequential in how I read websites." P8

"I'm skimming it rather than reading it carefully." P7

"It's a lot of information here on one page. A lot for one page, I think. I don't know if I would necessarily scroll all the way down. Because it's a lot. I guess, nowadays, people are used to very short paragraphs with very basic information and not scrolling down and reading through a lot on one page." - P9

"[When one link didn't work] I would just pursue some other link." P8

"I think that blue text is a good idea - it calls attention to it, what it's about." P7

Users find the agency name and branding in the top left corner of a web page critical for orientation. Users look to the biggest name and branding in the top-left corner for clues like "gov" or "government" when determining if a site is a U.S. government site. Users tend to miss the .gov extension of the URL in the browser window, and they also tend to miss the gov banner.

Quotes:

"It says CDC so I don't think I'm on a government site. It doesn't say .gov." P6

"This is a government site because it has the icons on the top and government language." P6

"Stopbullying.gov is different from CDC at the top, plus they just have a different feel to them." P8

The gov banner component is not well-noticed. It's also often not present on government sites.

Quote: [after prompting] "At the top, I now see the United States flag and official website of the United States government. Here's how you know [that you are on the government website], to validate the... Oh, beautiful! To validate the article." P11

People with low technical proficiency have difficulty keeping track of where they are on the internet or in their device. More so, they are reluctant to trust technology because they don't know how to use it to the full extent.

Quotes:

"I don't know what happened — to be honest I'm afraid to hit stuff, I'm afraid I'm going to lose stuff." U10

"If I click on it [link], I'll probably lose the definition of bullying." P7

"When you talk on the phone and they [a business] want your email, I won't even give my email out because I don't want all this stuff on my phone. I don't order anything online because I don't know how. I get the phone number from online, and then I'll call them [to place the order]." P10

Multitasking users are distracted and have a short attention span. While multitasking, people keep multiple tabs open across a browser, so they often become easily lost between tabs.

Quote: "Where's my thing? Where's this website? See how much I have opened here? My gosh, I need to go back. Do I need to click back [to zoom call] to get the link?" P9

Link-specific insights

Users understand a link's destination when that destination is included in the link text or the context of the link. If an external agency was mentioned in the text adjacent to the link, people were not surprised when they clicked the link and landed on a different website. The external link indicators were not a critical part of how people understood that they were leaving the current website.

Quotes:

"Maybe people would want to go to that - cdc.gov. I assume I can go there, I would just click on this text [on the cdc.gov]." P7

"These two links, they're pretty long. But the fact that I see here is that this comes from CDC. Again, it's official statistics from a reliable source." P9

Users are more likely to ignore link icons and badges than think about their meaning. Majority of the participants didn't mention the external link indicators when asked to read aloud the paragraph that contained external links.

They read the link copy first and tried to understand from the link text and sentence context what kind of resources the links provided.

Quotes:

"I don't care about the stuff at the end [of the link]." P6

[Variant A: Icon]: "I don't know why. I kind of noticed it, but I didn't even really think about it. I don't know. Maybe some documents there. Now, is it the same thing? It seems like the same thing, right?" U9

Users didn't consistently understand the external link icon or the "Exit" badge.

No one confused the current indicator icon (A) with a "share" button. They simply didn't know what it meant.

Quote: "I have no idea [what this icon means]. But I would assume I mean, I would just hit on that link and see what it does. But it probably takes you somewhere. To me, it kind of indicates that it's going somewhere." U6
Users understood the word "EXIT," but did not know just what they were exiting. One user explicitly said that it would take her "to a different website." But another user said that she didn't know how to use the "EXIT."

Quote: "If I hit the PDF 249 pages, the article would appear. I would think the EXIT would be to get me out [of the article]... Exit tells me that I am going to a new place and it's a button to hit for me to come back here... but if I leave how do I click this button? I am not going to see it... ." P10

Users find long link text hard to read and understand.

Quote: "I had to read it again because I didn't quite get what it is." U9

Government-speak and acronyms make links more confusing. As a result, half of the participants didn't understand what kind of resources the link was going to take them to. People didn't immediately know from the acronym URL what their site destination would be. However, if the URL in square brackets was familiar, it was understood as an indicator of an external website.

Quotes:

[before clicking the link] "Well, if you knew what U-N-E-S-C-O Institute of Statistics was, you would know but I don't know what that is." U6

"I'm not sure [where the link would take me] because of the way it's worded. It says 'indicators of school crime and safety.' I'm not sure what these indicators are? Is it going to be definitions of crime? Or what kind of crimes? Are they in school?" U9

Users interpreted both A (icon) and B (EXIT) indicators as clickable elements, but no one found those indicators to be common or familiar.

Quotes:

[Variant B: EXIT] "I don't know if I've ever seen that before." P8

[Variant A: Icon] "The little square with the arrow? Not that I remember, no. Because I don't know what it is." P6

[Variant A: Icon] "I've never seen that kind of a box." P10

[Variant A: Icon] "Is that necessary? That little square with the arrow? Because if you hit on that link, it would go anyway. Right?" P6

Users expect to find reference links at the bottom of the page.

Quote: "It took me to like an addendum clarifying information that are the effects of bullying. And it gives further examples, who it is by — I'd have to look down at the bottom again, to see, you know what your source is." P11

Recommendations

As a result of this study, we propose the following improvements to the USWDS link component guidance, which should be considered by content and design teams across agencies.

What USWDS should do

Improve our knowledge and guidance around what makes a good link.
How long should link text be? The entire phrasing captured as a blue hyperlink only makes it look longer and less comprehensible.
What metadata should be part of the link? Should data like file type and size be a part of the link itself. It's useful information, but can make the link harder to read.
How best to incorporate plain language guidance?

Guidance updates

Incorporate plain language guidance into your site where applicable. The link should be written in plain language to convey the meaning of the resource but not be so long that the link's text becomes hard to read.
Acknowledge the source of the link in the link text or sentence that precedes the link. Example: "This resource is located on a [agency's name] website"; or "The Center for Disease Control says... ."
Minimize disruptions in the reading flow. Be cautious about adding links to body text. Do they need to be there? Do they help users complete page tasks?
Use the external link icon, but not as the only way of indicating an external link. The external link icon is a consistent, if largely ineffective, external link indicator. But it does not adversely affect usability, and it may be a more effective multilingual solution than a text-based badge. Since it does little harm and can provide modest functionality, we see it as a good way to strengthen a link approach that addresses M-17-06. The link icon is not essential but does compliment the external link indicators alongside contextual sourcing and plain language.
Use plain-language metadata. It is more meaningful to indicate the size of a non-HTML document (like PDF) in number of pages instead of kilobytes or megabytes.
Be cautious adding links within a flow of a task that users are trying to complete. Links can be distractions, and you should assume a distractible user. Consider adding page links to a dedicated section on the page instead of in the middle of body text or where it might disrupt a user's task.

If we had more time:

We would have added to our guidance the consideration to collect related resources in a single place. Adding links in one resource section aligns with people's expectations of finding reference links in one place — at the bottom of the page, for example. This could potentially become a new USWDS component called "Resources."
When using deep linking, consider what part of the page you are linking to and whether it will make someone miss the federal website banner at the top, which plays a critical role in the website identity. Anchor the federal website banner on the top of the page, so it remains visible even when users scroll down the page.

Opportunities for further research

Conduct additional rounds of testing with tech-savvy participants.
Understand how we might improve the effectiveness of the gov banner.
Dig into how link text length affects readability
Better understand whether link metadata like filetype and size should be included in the link text.
Test the effectiveness of separating links from body text, like in a links section.
Test a combination of icon and tooltip.
Test the external link indicator with non-English speakers.
Test how deep-linking affects contextual awareness.

Next steps

Determine what the screen readers will say when verbalizing the external link icon.
Apply findings and new knowledge to adjust the link component guidance as needed.
Apply the link guidance to the USWDS website.