RFC: TextInput selection(cursor) coordinates

[perunt]

This RFC aims to extend the TextInput component by introducing tracking of text selection positions.

This proposal introduces an enhancement to the React Native TextInput component, focusing on advanced tracking of text selection positions. Specifically, it aims to provide developers with screen coordinates (x and y positions) for both the start and end of text selection and the cursor within a TextInput.

To clarify, these coordinates are TextInput-relative, meaning they indicate the position of the text selection or cursor relative to the entire TextInput.

The primary application of this feature is to enable functionalities like inline suggestions (such as emojis or text completions) that appear right where the text is being selected or where the cursor is located. This enhancement significantly improves interactivity and user experience in text input fields by allowing context-aware and location-specific interfaces.

Imagine a user selecting a portion of text in a TextInput field. With this feature, the app could display a pop-up suggestion box right above or below the selected text, offering context-relevant options like synonyms, emoji replacements, or grammar corrections. The suggestion box's position on the screen would dynamically adjust based on the text selection's start and end coordinates, ensuring a seamless and intuitive user interaction.

In summary, this enhancement makes text interaction within apps more dynamic and responsive, catering to a wide range of applications from simple text suggestions to complex rich text editing tools, all by leveraging precise screen-relative coordinates of text selection and cursor positions.

type SelectionType = {
        start: number,
        end: number,
        selectionPosition: {
            start: {x: number, y: number},
            end: {x: number, y: number}
        }
      }

      <TextInput
        onSelectionChange={(e) => {
          const selection: SelectionType = e.nativeEvent.selection;
        }}
      />

250882266-b010d4e7-3777-43fb-956a-673b725c15e7.mp4

The formatted RFC is here
Discussion: #736

[TheSavior]

Some drive by thoughts here, although I might not have the full context or right answers

1. How would you build this on web? I don't think this matches a web API. Why does the way you'd build it on web not work in RN? Should it / could it?
2. idk about it specifically being the screen coords, as I could imagine someone wanting the offset from its parent. Primarily I'd defer to how people build it on web though per Changed "core contributors" to "top contributors" #1. And maybe this has some relation to the pointerEvents APIs that now exist in RN to match web, where the shape you'd get back looks pretty different I assume.

[perunt]

@TheSavior, thank you for your insights.
In web development, tracking cursor or selection positions often require hukky complex approaches, such as using shadow text components, which can be seen in the Expo Snack example.
In React Native, I'm suggesting using native capabilities for better efficiency, which is different from web approaches.
Of course, we can apply the web approaches to mobile, but recalculating the coordinates for low-end Android devices becomes challenging in that case.
Unfortunately, the pointerEvents can't help with it as it mostly represents mouse and touch events which can't be correlated with caret position (except case when we directly tap on it)

I agree with your point on coordinates. In the RFC, I meant to use coordinates relative to the TextInput itself, not screen relative. The rest of the text in RFC PR should be good. I'll fix this in this description
Also, I will attach a potential example of how it could look like:

250882266-b010d4e7-3777-43fb-956a-673b725c15e7.mp4

[rozele]

The onSelectionChange event is something that fires for every keystroke - perhaps not on iOS if the event isn't subscribed, but on Android, many events fire unconditionally (I haven't actually checked this one). I don't know what the actual performance characteristics of the implementations you propose for iOS and Android would be, but I can say on the current OSS react-native-windows TextInput implementation, the perf would not be ideal, and it would happen on every keystroke.

This proposal is missing a way to address how this could be done without impacting the perf of components that don't need the cursor position feature, ideally without also introducing a bespoke prop that tells us whether the feature should be enabled or not.

In general, my opinion is that the main missing feature here is the ability to easily "extend" core components like TextInput for app specific behaviors. Lots of apps need lots of bespoke features, perhaps they need undo/redo events, copy/paste events, context menu behaviors, etc. If we keep piling on APIs to core components like TextInput, without letting apps inject these behaviors easily, the public surface of TextInput becomes untenable, and folks building out of tree platforms, like react-native-windows or react-native-macos, are forced to catch up.