Keyboard.addListener('keyboardDidShow', ...) callback is invoked with inaccurate measures in Android devices with notch

[jsamr]

Incriminated API

// In Android
Keyboard.addListener('keyboardDidShow', e => {
   // On OnePlus 7, and Pixel 3XL (simulator) this value is underestimated by exactly StatusBar.currentHeight.
    e.endCoordinates.height
})

Prerequisites

In AndroidManifest.xml, set android:windowSoftInputMode="adjustPan".

Visual tests

The black horizontal bar should be the same width (40dp) as the red bar, in a container with a bottom padding of the keyboard's height. This component mimics KeyboardAvoidingView with behavior padding set. The height of keyboard should be the same as the height of the red bar. The issue has been reproduced with GBoard, SwiftKey...

Remarks: I would have imagined that the view height would equal keyboard height + keyboard Y. But that is only true ... on OnePlus7, which is the problematic device!

Emulator, Pixel 2 XL, Android 10	OnePlus 7, Android 10, OxygenOS 10.0.1

React Native version:

System:
    OS: Linux 4.19 Manjaro Linux undefined
    CPU: (8) x64 Intel(R) Core(TM) i7-8809G CPU @ 3.10GHz
    Memory: 565.35 MB / 31.33 GB
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 12.13.0 - /usr/bin/node
    npm: 6.12.0 - /usr/bin/npm
  IDEs:
    Android Studio: 3.5 AI-191.8026.42.35.5900203
  npmPackages:
    react: 16.8.1 => 16.8.1 
    react-native: 0.61.3 => 0.61.3

Steps To Reproduce

1. Open the reproduction demo in a Pixel 3XL emulator with Android 10+.
2. Observe that KeyboardAvoidingView with behavior padding doesn't add enough padding.

Describe what you expected to happen: The padding should match keyboard's real height.

Full reproduction : react-native-oneplus7-keyboard-height-discrepancy

Observations

• StatusBar.currentHeight is substituted to the keyboard layout end frame height.
• StatusBar.currentHeight is also substituted from keyboard layout end frame screenY

[cgsdev0]

I've also observed a similar issue between devices (in this case, Google Pixel 3 and Essential Phone)

[TuurDutoit]

Do these devices have notches? I think this might be the same as this: #26296

[stale]

Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? This issue may be closed if no further activity occurs. You may also label this issue as a "Discussion" or add it to the "Backlog" and I will leave it open. Thank you for your contributions.

[jsamr]

Has anyone been able to reproduce the issue with the latest RN release?

[TuurDutoit]

We haven't upgraded yet, so I haven't been able to test. Your demo repo should be easy to upgrade and test though.

[fabOnReact]

we need OnePlus 7 so this issue will not be solved... I don't have such device OnePlus 7

[jsamr]

OK I am gooing to test soon.

[fabOnReact]

@jsamr I wrote a pull request #29292 that fixes the same issue with event keyboardDidHide. I can not reproduce your issue as I don't have a OnePlus device. and I can not fix it, but you can try applying a similar fix

react-native/ReactAndroid/src/main/java/com/facebook/react/ReactRootView.java

Lines 671 to 677 in cfa4260

	sendEvent(
	"keyboardDidShow",
	createKeyboardEventPayload(
	PixelUtil.toDIPFromPixel(mVisibleViewArea.bottom),
	PixelUtil.toDIPFromPixel(mVisibleViewArea.left),
	PixelUtil.toDIPFromPixel(mVisibleViewArea.width()),
	PixelUtil.toDIPFromPixel(mKeyboardHeight)));

you can build from source, change the above source code with either numbers like 10 or 900 and see the effect to understand how the function works.
https://github.com/facebook/react-native/wiki/Building-from-source#publish-your-own-version-of-react-native

This is the definition of createKeyboardEventPayload

react-native/ReactAndroid/src/main/java/com/facebook/react/ReactRootView.java

Lines 755 to 770 in cfa4260

	private WritableMap createKeyboardEventPayload(
	double screenY, double screenX, double width, double height) {
	WritableMap keyboardEventParams = Arguments.createMap();
	WritableMap endCoordinates = Arguments.createMap();
	endCoordinates.putDouble("height", height);
	endCoordinates.putDouble("screenX", screenX);
	endCoordinates.putDouble("width", width);
	endCoordinates.putDouble("screenY", screenY);
	keyboardEventParams.putMap("endCoordinates", endCoordinates);
	keyboardEventParams.putString("easing", "keyboard");
	keyboardEventParams.putDouble("duration", 0);
	return keyboardEventParams;
	}
	}

Those values are returned to the react-native javascript library which is responsible for displaying the Black Area

additionally there is no OxygenOS emulator https://stackoverflow.com/a/43549663/11452286

[lightrow]

i have this issue on oneplus 6, i need to add exactly 30dp for it to be perfect. On the iOS simulator it's the other way around, i need to subtract some value, and the value is not always the same ( looks like it's -35 for iphones without home button and 0 for the others - maybe the reported keyboard height doesn't take into account the gesture bar which expands slightly when keyboard is opened? )

[jsamr]

I have updated the reproduction to React Native 0.63.3 and it consistently fails on any device with a notch, including the Pixel 3XL in emulator. I noticed that the underestimated height consistently corresponds to the height of StatusBar.currentHeight.

[fabOnReact]

not releated to my commit which for my understanding is still in master. Hopefully gets fixed by https://github.com//pull/29292. Thanks

[jsamr]

@fabriziobertoglio1987 I thought your PR was only about keyboardDidHide?

[fabOnReact]

react-native/ReactAndroid/src/main/java/com/facebook/react/ReactRootView.java

Lines 757 to 758 in d792121

	final int heightDiff =
	DisplayMetricsHolder.getWindowDisplayMetrics().heightPixels - mVisibleViewArea.bottom;

Without Notch
DisplayMetricsHolder.getWindowDisplayMetrics().heightPixels returns 1794

With Notch
DisplayMetricsHolder.getWindowDisplayMetrics().heightPixels returns 1668

CLICK TO OPEN TESTS RESULTS

Without Notch
DisplayMetricsHolder.getWindowDisplayMetrics().heightPixels returns 1794

BEFORE	AFTER

With Notch
DisplayMetricsHolder.getWindowDisplayMetrics().heightPixels returns 1668

BEFORE	AFTER

My conclusions are:

• mVisibleViewArea.bottom value 1053 is correct. The keyboard height does not change in the above tests examples
• DisplayMetricsHolder.getWindowDisplayMetrics().heightPixels values (1668 and 1794) are correct (the change in height is caused by the notch)

I believe this is not a bug in react-native.
KeyboardDidHide will use the above android APIs to calculate the available height and subtract to this height the keyboard to return the available view, there is no other scenarios for this functionality.

[jsamr]

@fabriziobertoglio1987 Which version / branch are you using to perform those tests?

[jsamr]

@fabianeichinger I've updated the example here: https://github.com/jsamr/react-native-oneplus7-keyboard-height-discrepancy
It is much more straightforward. With react-native@0.63.4, you can very well see the keybord height discrepancy in the below screenshots. The height of the red bar is assigned to ev.endCoordinates.height (keyboardDidShow). The red bar has been duplicated to bottom for height comparison with Keyboard:

Pixel 3 XL (with notch), Android 11 (emulator)	Nexus 5X, Android 11 (emulator)

I haven't dig into the native java code, and I am not familiar with Android UI APIs. But perhaps what you're showing is actually the fixed version on master.

[fabOnReact]

Thanks for your feedback @jsamr I allways work with the updated react-native master branch

My main point in message #27089 (comment) was that the keyboard height was not causing this issue, but the presence of a Notch or No Notch

Now you include different scenarios with different status bars.. I can not say about that, but I can say that DisplayMetricsHolder.getWindowDisplayMetrics().heightPixels values seems to me correct taking in consideration we added the notch to the device.

The screenshots you include are:

• from two different devices
• with 2 different StatusBar heights
• with 2 different keyboards

I would avoid to add more troubleshooting scenarios, but actually to limit the number of scenarios and identify the bug

What is causing your issue?
The Notch? The StatusBar? The Keyboard?

Maybe you think the computation of the Keyboard Height is wrong?

This does not make sense, because if I remove the Notch it works perfectly, while the Keyboard Height computation does not change.

CLICK TO OPEN TESTS RESULTS

Without Notch
DisplayMetricsHolder.getWindowDisplayMetrics().heightPixels returns 1794

BEFORE	AFTER

With Notch
DisplayMetricsHolder.getWindowDisplayMetrics().heightPixels returns 1668

BEFORE	AFTER

[TuurDutoit]

As I understand it, the height of the keyboard, as reported in the event object, changes based on the presence of a notch. In fact, looking at your test screenshots, you can see this exact issue: in the case with no notch, the height is reported as 282dp; in the case with a notch, it reports 234dp! That would mean the height of the keyboard changed, but it doesn't look like that's the case.
The discrepancy seems to be caused by the change in height of the view: the keyboard's y coordinate doesn't change, but the view's height does.

[jsamr]

@fabianeichinger The goal of the screenshots is to compare with notch (Pixel 3XL) and without notch (Nexus 5X). With notch, the red vertical bar height doesn't match the keyboard's height. Without, it does.

with 2 different keyboards

That was because I used android 10 in the Pixel and android 11 in the nexus. I have fixed that issue and updated the screenshots. But anyway, the relevant part is that it seems that having a notch causes ev.endCoordinates.height to be miscalculated. Compare the vertical red bar with keyboard's height on the Pixel, you'll notice a mismatch.

[fabOnReact]

Thanks, you are both right. I believe the best solution is using the Android API to get the Keyboard height instead of calculating it as the difference

react-native/ReactAndroid/src/main/java/com/facebook/react/ReactRootView.java

Lines 757 to 758 in d792121

	final int heightDiff =
	DisplayMetricsHolder.getWindowDisplayMetrics().heightPixels - mVisibleViewArea.bottom;

I'll work on this tomorrow 🙏

[jsamr]

@fabianeichinger Thank you so much for your great involvement in this open source project, looking forwards for your take on this!

[fabOnReact]

Thanks a lot @jsamr My ambition is to really get to know the reactnative sourcecode and hopefully one day give relevant contributions to the community

The issue is caused by getWindowVisibleDisplayFrame which sets the mVisibleViewArea.bottom used to compute the keyboard height.

react-native/ReactAndroid/src/main/java/com/facebook/react/ReactRootView.java

Line 756 in 1049835

getRootView().getWindowVisibleDisplayFrame(mVisibleViewArea);

• mVisibleViewArea.bottom is 1794 when the keyboard is hidden and covers the entire screen from top to bottom
• mVisibleViewArea.bottom is 1053 when the keyboard is open and cover from the top till the top margin of the keyboard

This computation is wrong and does not take in consideration the Notch or other screen decorations, as you can see below does not change the total height when using the notch.

I found a workaround to fix this by calculating the notch height and substracting it from the above calculation. I will publish my pr soon. I researched and it is common practice on Android Sdk to calculate keyboard height by deducting it from other parameters

TEST CASE - BEFORE FIX

Full Screen	Keyboard Did Show

TEST CASE - AFTER FIX

Full Screen	Keyboard Did Show

MORE DETAILS

Without Notch

• DisplayMetricsHolder.getWindowDisplayMetrics().heightPixels is 1794
• mVisibleViewArea.bottom is 1794

With Notch

• DisplayMetricsHolder.getWindowDisplayMetrics().heightPixels is 1668
• mVisibleViewArea.bottom is 1794
  The android View sourcecode is available here

    /**
     * Retrieve the overall visible display size in which the window this view is
     * attached to has been positioned in.  This takes into account screen
     * decorations above the window, for both cases where the window itself
     * is being position inside of them or the window is being placed under
     * then and covered insets are used for the window to position its content
     * inside.  In effect, this tells you the available area where content can
     * be placed and remain visible to users.
     *
     * <p>This function requires an IPC back to the window manager to retrieve
     * the requested information, so should not be used in performance critical
     * code like drawing.
     *
     * @param outRect Filled in with the visible display frame.  If the view
     * is not attached to a window, this is simply the raw display size.
     */
    public void getWindowVisibleDisplayFrame(Rect outRect) {
        if (mAttachInfo != null) {
            try {
                mAttachInfo.mSession.getDisplayFrame(mAttachInfo.mWindow, outRect);
            } catch (RemoteException e) {
                return;
            }
            // XXX This is really broken, and probably all needs to be done
            // in the window manager, and we need to know more about whether
            // we want the area behind or in front of the IME.
            final Rect insets = mAttachInfo.mVisibleInsets;
            outRect.left += insets.left;
            outRect.top += insets.top;
            outRect.right -= insets.right;
            outRect.bottom -= insets.bottom;
            return;
        }
        // The view is not attached to a display so we don't have a context.
        // Make a best guess about the display size.
        Display d = DisplayManagerGlobal.getInstance().getRealDisplay(Display.DEFAULT_DISPLAY);
        d.getRectSize(outRect);
    }