Improvements to positioning on mobile for better text scanning ability

[StarScape]

Opening this discussion to expound on something I previously touched on with Brian in #1737.

Motivation:

Currently, scanning through text is one of my major UX pain points on mobile. With the current behavior, scanning can be difficult because the popup positions itself unpredictably.

Say I have hovered over a word, like this one here:

If I then want to hover over some word above the current line, I have a few options. If I simply go up (as I might do on desktop, while still holding Ctrl or Alt), the puck moves into the popup area, but the popup does not disappear.

So that’s a no-go. I can also hit the ‘X’ button to close the popup and then go up, but then I run the risk of accidentally hovering over another word on the way to the one I want. In fact, it will probably just trigger the same word again since that’s where the puck’s cursor left off.

So I’m left with awkwardly trying to weave my puck off to one side and then up the margin of the page in order to get to the word I want without accidentally triggering something.

Luckily, scanning left or right isn’t an issue, so this does work. But it’s awkward and a bit frustrating. I’ve also found myself needing to do this on pages without much of a margin, which makes it harder.

You’ll notice I’ve only discussed scanning up, rather than down. Of course, scanning down into subsequent lines is the more common use case. You’ll mostly only find yourself scanning upwards when doubling back (which, as a not-too-advanced learner, I do quite a bit) or when skimming a page non-linearly.

Going downwards also isn’t an issue because, whenever the popup is positioned below, there seems to always be a significant margin between the word highlighted and the top of the popup:

As far as I can tell, this margin only appears whenever the lookup puck is active. I assume the original intent of this was to prevent the puck itself from intersecting with the popup, but it has the added benefit of helping with scanning downwards.

But as soon as you get maybe 1/3 of the way down the page, the popup switches to positioning above the hovered word ¹. But there’s no margin between the word and popup. And at least at the y-offset where it first starts positioning above, there’s no room for one:

Another factor is, since the popup appears immediately above the line I’m reading, it blocks most of the sentence the target word is in. If a user is looking up a word they don’t know, there’s a high probability they’ll want to read back over what they just read—or at least, I often do this.

Therefore, if at all possible, I think it would be desirable to keep as much of the text around the target word visible. This is particularly relevant on mobile, where dismissing the popup involves a bit more friction than on desktop.

Potential Solutions

Option 1: Add option to default popup position to top/bottom on mobile

Proposed change: Add two options to the settings menu, one for “Default popup position to top” and another for “Default popup position to bottom.” Enabling one of these would make the popup position you get by pressing j/k the default. The two options are mutually exclusive, so would likely appear as a two radio buttons hidden behind a “Change popup positioning” checkbox, greyed out if the checkbox is unchecked.

Option 2: Add margin between hovered text and popup when popup is above

Proposed change: Just like there is a margin between the hovered text and popup when the popup is below the puck, add a margin when the popup is above the puck. I’m not sure how this would work when there is only just enough room to fit the popup, e.g. in the situation in the 6th screenshot above.

Option 3: Attempt “smart positioning”

Proposed change: Do a sort of hybrid between the two options above. Position the popup at the top of the screen (i.e. where it would be after using the j shortcut) if there is enough room for the popup, plus a reasonable margin, between the hovered word and the top of the viewport. This margin should be at least large enough to see the line above, so the user could then scan up into that line, and from there into the next line above, and so on.

If there is enough for the popup plus margin at the bottom of the screen, position the popup there (i.e. where it would be after using the k shortcut).

If there’s no room for the popup plus margin on either above or below the hovered text (which I think could happen with a particularly large entry on a small phone screen) I think it would be best to default to positioning on top, even if that means it immediately abuts the word. Scanning downward is still a more common use case than scanning upward.

Option 4: Popup disappears when puck intersects it

Proposed change: When the puck is moved up into the popup window itself, as in the 2nd screenshot above, the popup will disappear. The page text behind the popup can then be looked up as normal. Essentially, don't treat the puck like a mouse pointer, and don't have it hold the popup open.

…And possibly others?

If you have them, please share them 🙂!

Thoughts on potential solutions

Option 1, since there’s already code for this in the j/k shortcuts, would mostly be a matter of adding some option switches. This might also be worth doing regardless.

Option 2, if it works, would also be fairly simple, and comes with the added benefit of not adding another setting.

Option 3 is intriguing to me, because I think it has the potential to provide the best mobile UX. What I like about it is that it would leave the most text visible on either side of the hovered word, which I think would make for a better reading experience. And on mobile, I think that the benefit of displaying the popup inline is mostly negated. There’s no need for it to be nearby for mouse interaction since there is no mouse. Maybe using 10ten on a large tablet screen would change my mind, though?

Option 4 is the simplest, but I could be overlooking something that makes it non-viable. Is there any reason hovering the puck over the popup should make it remain visible? Of course it makes sense with a mouse, but I’m not sure there’s any need for it on a touch screen.

Regardless of the solution, the goal is the same: for any word on any given line, ensure that when a word is hovered, you can still move the puck cursor into the lines both above and below.

Conclusion

If you’ve read this far, thank you! I did not intend to spill a gallon of ink over a relatively minor UX nit. But please do share your thoughts on the above whenever you have time.

Footnotes

1. I haven’t looked at the positioning code and and admittedly I’m not sure what the actual metric for when it switches to displaying the popup above vs below is. ↩

[birtles]

Thanks for the detailed analysis!

For what it's worth, I found it quite hard to reproduce this issue on my Android phone but that might be a result of the particular content I was interacting with.

I assume the original intent of this was to prevent the puck itself from intersecting with the popup, but it has the added benefit of helping with scanning downwards.

I think this actually came about because otherwise your finger would obscure the popup:

10ten-ja-reader/src/content/content.ts

Lines 2287 to 2297 in 4229623

	// Although we can't tell whether the left or right thumb is in use
	// (so we don't make corresponding adjustments to left/right), we can at
	// least be reasonably sure that the thumb extends downwards!
	const extraMarginToClearThumb =
	this.puck.getTargetOrientation() === 'above' ? 100 : 0;
	cursorClearance = {
	top,
	right,
	bottom: bottom + extraMarginToClearThumb,
	left,
	};

Therefore, if at all possible, I think it would be desirable to keep as much of the text around the target word visible. This is particularly relevant on mobile, where dismissing the popup involves a bit more friction than on desktop.

Good suggestion. I can see being able to read the prior line of text as being useful.

Option 3: Attempt “smart positioning”

Proposed change: Do a sort of hybrid between the two options above. Position the popup at the top of the screen (i.e. where it would be after using the j shortcut) if there is enough room for the popup, plus a reasonable margin, between the hovered word and the top of the viewport.

I think we already do this minus the extra margin? i.e. when using the puck we prefer the 'before' side:

10ten-ja-reader/src/content/popup-position.ts

Lines 411 to 431 in 4229623

	// Generate the possible position sizes in order of preference.
	//
	// We prefer positions in the block direction on the 'after' side unless we
	// are looking up horizontal text with the puck, in which case we prefer the
	// 'before' side (i.e. above the target text).
	// Prefer the block direction
	const axisOrder = isVerticalText
	? (['horizontal', 'vertical'] as const)
	: (['vertical', 'horizontal'] as const);
	// Prefer the 'after' side
	const sides = ['after', 'before'] as const;
	// Store the possible layouts
	const candidates: Array<ExtendedPopupPosition | undefined> = [];
	for (const axis of axisOrder) {
	// Prefer the 'before' side when we are looking up horizontal text with the
	// puck.
	const swapSides = pointerType === 'puck' && axis === 'vertical';

If the popup doesn't fit on either side, we use the side that is least constrained.

Option 4: Popup disappears when puck intersects it

Proposed change: When the puck is moved up into the popup window itself, as in the 2nd screenshot above, the popup will disappear. The page text behind the popup can then be looked up as normal. Essentially, don't treat the puck like a mouse pointer, and don't have it hold the popup open.

That certainly makes sense. I'm surprised we don't already do that!

My current take of this is:

1. We should make the popup disappear when the puck is over it (i.e. option 4). I can't think of any reason not to do that.
2. It might be useful to also add more vertical margin when the popup is positioned on top so you can read the line prior to the target word. Ideally we'd only do that when there's enough space but it might complicate the positioning code too much to include that negotiation.

(Also, just in case you haven't discovered it, you can disable the puck with a single tap which will also hide the popup which might help in some cases until this issue is fixed.)

[StarScape]

For what it's worth, I found it quite hard to reproduce this issue on my Android phone but that might be a result of the particular content I was interacting with.

It could be more of an issue for me because of screen size. I use an iPhone Mini, which is quite small. I tried on an iPad just now, and while I could still reproduce the issue, it wasn't nearly so annoying since the popup took up a much smaller segment of the screen.

I think we already do this minus the extra margin? i.e. when using the puck we prefer the 'before' side:

Almost. What I was talking about here was always positioning the popup on either the top-left or bottom-right, never anywhere in the middle of the screen nearer the next.

So for example, not here:

But here instead:

And not here:

But here instead:

So basically, there'd be a space in the middle of the screen that is never covered by the popup:

I think this could make for a smoother reading experience on a smaller screen, but it's definitely the kind of thing I would need to test to see if it works as well in practice as it does in my head.

My current take of this is:

1. We should make the popup disappear when the puck is over it (i.e. option 4). I can't think of any reason not to do that.
2. It might be useful to also add more vertical margin when the popup is positioned on top so you can read the line prior to the target word. Ideally we'd only do that when there's enough space but it might complicate the positioning code too much to include that negotiation.

Yeah, agreed. I think implementing #1 will probably solve 90% of this annoyance. It's also a low-risk change that shouldn't involve modifying the positioning code much.

So, as long as you're cool with it, I think I'll open a PR for that soon, and then we can see from there if more vertical margin is still worth adding.

[birtles]

Almost. What I was talking about here was always positioning the popup on either the top-left or bottom-right, never anywhere in the middle of the screen nearer the next.

Ah, I see. That makes sense. Yes, I think that would be do-able but, like you say, it's probably worth fixing the lower-hanging fruit first and then seeing if it's still necessary.

My current take of this is:

1. We should make the popup disappear when the puck is over it (i.e. option 4). I can't think of any reason not to do that.
2. It might be useful to also add more vertical margin when the popup is positioned on top so you can read the line prior to the target word. Ideally we'd only do that when there's enough space but it might complicate the positioning code too much to include that negotiation.

Yeah, agreed. I think implementing #1 will probably solve 90% of this annoyance. It's also a low-risk change that shouldn't involve modifying the positioning code much.

So, as long as you're cool with it, I think I'll open a PR for that soon, and then we can see from there if more vertical margin is still worth adding.

By all means, go for it! I guess it would involve adding code somewhere in the puck code to check if the target element is part of the popup. e.g. here:

10ten-ja-reader/src/content/puck.ts

Line 443 in 4229623

// Check if we need to adjust the content to look it up.