[HOLD for payment 2024-04-15] [$500] Characters missing when typing message in compose box

[mallenexpensify]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Coming from here where we were able to document reliable reproduction steps.

Version Number:
Reproducible in staging?: Yes
Reproducible in production?: Yes
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: @mallenexpensify
Slack conversation: here

Reproduction Steps

1. Open Script Editor (Mac)
2. New Document > Paste the following script

on run
	set textToType to "JUMP "
	repeat
		emulateTyping(textToType)
		delay 1
	end repeat
end run

on emulateTyping(textToType)
	tell application "System Events"
		repeat with i from 1 to count of characters of textToType
			set currentChar to character i of textToType
			keystroke currentChar
			delay 1
		end repeat
	end tell
end emulateTyping

3. Press the play button (run the script)
4. Focus the desktop newDot app composer
5. Wait until you observe that a letter from the word 'JUMP' is being omitted after being typed.

Expected result:
The word 'JUMP' needs to be typed with consistency in the composer. ignore the first word it is because we had a delay when focusing the input.

Actual result:
After a few tries character gets omitted and the word becomes inconsistent. i.e. JUMP JUMP JMP

Screen.Recording.2024-02-22.at.11.35.41.AM.mov

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android: Native
• Android: mWeb Chrome
• iOS: Native
• iOS: mWeb Safari
• MacOS: Chrome / Safari
• MacOS: Desktop

Screenshots/Videos

View the other issue.

Additional Deliverable

From here

FWIW the issue also occurred when editing the message, I think that should also be fixed in this post.

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~0104493dfd4225a65e
• Upwork Job ID: 1763648489851805696
• Last Price Increase: 2024-03-13
• Automatic offers:
• getusha | Reviewer | 0
• tienifr | Contributor | 0

[melvin-bot]

Triggered auto assignment to @johncschuster (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[mallenexpensify]

@johncschuster, I'll take this, I have context from the issue this came form, and the bug.

[wildan-m]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Characters missing when typing message in compose box

What is the root cause of that problem?

Race conditions occur when updating the composer value by typing and updating the value from the Onyx draft.

In the main composer, the SilentCommentUpdater update can be triggered when the user types at a debounce interval.

App/src/pages/home/report/ReportActionCompose/SilentCommentUpdater/index.tsx

Lines 34 to 36 in b7a230e

	updateComment(comment ?? '');
	}, [prevCommentProp, prevPreferredLocale, prevReportId, comment, preferredLocale, reportID, updateComment, value, commentRef]);

When editing chat, this is the culprit that causes it to behave similarly to the main composer.

App/src/pages/home/report/ReportActionItemMessageEdit.tsx

Lines 129 to 131 in b7a230e

	}
	setDraft(Str.htmlDecode(draftMessage));
	}, [draftMessage, action]);

The problem is that when typing a comment, the value from the composer will be more recent than the updates above, which means the above updates might use outdated value.

What changes do you think we should make in order to solve the problem?

Ensure the updated value is not outdated by using Composer's most recent value when the user is actively typing.
Track the current user's typing status with the useTypingStatus hook:

const useTypingStatus = ({delay = 1000}: UseTypingStatusProps = {}): UseTypingStatusReturn => {
    const isTypingRef = useRef(false);

    const setTypingFalse = useCallback(() => {
        const debouncedFunc = lodashDebounce(() => {
            isTypingRef.current = false;
            /* The +500 difference in the setTypingFalse debounce function is added
            to ensure that the typing status is not set to false immediately right after 
            the user starts typing delay ended. It allows a small buffer period after the delay 
            before considering the user as not typing anymore. */
        }, delay + 500);
        debouncedFunc();
    }, [delay]);

    const handleUserTyping = useCallback(() => {
        const throttledFunc = lodashThrottle(() => {
            if (!isTypingRef.current) {
                isTypingRef.current = true;
            }
            setTypingFalse();
        }, delay);
        throttledFunc();
    }, [delay, setTypingFalse]);

    return {isTypingRef, handleUserTyping};
};

In ComposerWithSuggestion, we can initiate track typing within onChangeText

   const onChangeText = useCallback(
        (commentValue: string) => {
            handleUserTyping();
            updateComment(commentValue, true);
.............
        },
        [updateComment, handleUserTyping],
    );

Pass the isTypingRef to the SilentCommentUpdater.

                <SilentCommentUpdater
                    reportID={reportID}
                    value={value}
                    updateComment={updateComment}
                    commentRef={commentRef}
                    isTypingRef={isTypingRef}
                />

and use commentRef.current when the user is typing

        const newComment = isTypingRef.current ? commentRef.current : comment;
        updateComment(newComment ?? '');
    }, [prevCommentProp, prevPreferredLocale, prevReportId, comment, preferredLocale, reportID, updateComment, value, commentRef, isTypingRef]);

When editing ReportActionItemMessageEdit.tsx, we can update this code to utilize textInputRef.current?.value during active typing.

    useEffect(() => {
        if (ReportActionsUtils.isDeletedAction(action) || (action.message && draftMessage === action.message[0].html)) {
            return;
        }

        const newDraft = (isTypingRef.current ? textInputRef.current?.value : draftMessage) ?? '';

        setDraft(Str.htmlDecode(newDraft));
    }, [draftMessage, action, isTypingRef]);

Trigger the typing tracking here.

                            onChangeText={(newText) => {
                                handleUserTyping();
                                updateDraft(newText);
                            }} // Debounced saveDraftComment

Or perform early return when isTypingRef.current === true

Branch to test

What alternative solutions did you explore? (Optional)

Alternative 1

We can remove the shouldDebounceSaveComment feature and its function debouncedSaveReportComment. Though it functions, it will revert us to the pre-optimized state, resulting in noticeable lag as the method will be executed for each character.

GMT20240229-024116_Clip_Wildan.M.s.Clip.02_29_2024.mp4

Alternative 2

What if we set the trailing to false and leading to true?

It means the debounced function will be invoked immediately when the event occurs, and then it will be debounced for subsequent calls.

It will not miss a keystroke, but the onyx saveReportComment data will not be properly updated, as it will not call the method at the end of the debounce trailing phase.

The issue illustrated in this video

GMT20240229-031351_Clip_Wildan.M.s.Clip.02_29_2024.mp4

Better solution

We can trigger the saveReportComment function in both the leading and trailing phases to ensure setState doesn't missed important invocation.

Change this lines:

App/src/pages/home/report/ReportActionCompose/ComposerWithSuggestions/ComposerWithSuggestions.tsx

Lines 322 to 328 in 3e74ddf

	const debouncedSaveReportComment = useMemo(
	() =>
	lodashDebounce((selectedReportID, newComment) => {
	Report.saveReportComment(selectedReportID, newComment || '');
	}, 1000),
	[],
	);

To

....

    const debouncedSaveReportComment = useMemo(
        () =>
            lodashDebounce(
                (selectedReportID, newComment) => {
                    Report.saveReportComment(selectedReportID, newComment || '');
                }
                , 1000, { leading: true, trailing: true }),
        [],
    );

Also implement this modification in src/pages/home/report/ReportActionItemMessageEdit.tsx for debouncedSaveDraft

Test with 'JUMP' at 1-second typing interval.

GMT20240229-023310_Clip_Wildan.M.s.Clip.02_29_2024.mp4

Test with paragraph at 100 millisecond typing interval

GMT20240229-022734_Clip_Wildan.M.s.Clip.02_29_2024.mp4

[jeremy-croff]

Proposal

GMT20240229-024116_Clip_Wildan.M.s.Clip.02_29_2024.mp4

I would like to explain why your solution seems fail to reproduce, but not mitigate the bug:

Specifically note from the docs the following:
If leading and trailing options are true, func is invoked on the trailing edge of the timeout only if the debounced function is invoked more than once during the wait timeout.

If you type 1 character per second, and the wait is 1 second, it will now never call the debounce. I think we see a few of those occurrences in your video.

If we review this report https://ellenaua.medium.com/throttle-debounce-behavior-lodash-6bcae1494e03 it shows how the debounce also now no longer reliable runs at the 1 second interval as is required to actually reproduce the bug with the script.

[wildan-m]

@jeremy-croff Thanks for your comment. To help readers stay focused, I filter my console log with the 'wildebug' keyword. This is my personal debug code.

[jeremy-croff]

@jeremy-croff Thanks for your comment. To help readers stay focused, I filter my console log with the 'wildebug' keyword. This is my personal debug code.

Thank you and my apologies, I edited my wording a bit in the same train of thought. I also deleted my original proposal and have a new one to consider:

Proposal

Please re-state the problem that we are trying to solve in this issue.

We are missing characters when typing

What is the root cause of that problem?

1. We have an async report draft saving lifecycle to the onyx store, that will rerender the component with what's in the store.
2. At the same time we have an async setTimeout, with the debounce which can reset the draft text in the store
3. We have a state variable in the Composer component with the current text value that we set here

These three compete with what's on the screen, what should get debounced, and what's in the store.

To reproduce the bug, we we maximize the draft saving lifecycles, to trigger rerenders, which occur each time you wait >1 sec because the debounce, Let's say you type a character before that change is reflected on the input 1. a debounce will save it, but it gets wiped while it repaints from onyx. And after the repaint 2. changeText is detected and now the debounce will just simply save the text with the missing character.

I was able to isolate the related code early on here in the original thread.

What changes do you think we should make in order to solve the problem?

When reviewing a solution, I wanted to decouples these 3 factors,
I came up with the following:

1. one source of truth for what we try to save as a draft, which is what's in the components state variable, not in a debounced memory
2. Example: useEffect(() => _.debounce((selectedReportID) => Report.saveReportComment(selectedReportID, value || ''), 1000), [value]);
   note: value is the existing state variable that we set on text update here or directly from the onyx store here

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~0104493dfd4225a65e

[melvin-bot]

Triggered auto assignment to Contributor Plus for review of internal employee PR - @sobitneupane (Internal)

[mallenexpensify]

@getusha , can you please review the above proposals? Assigning to you since you're also on the issue for documenting reproducible steps.

@sobitneupane , sorry about adding you by mistake via the Internal label.

[getusha]

Thanks @mallenexpensify,
Testing proposals i will update soon.

[getusha]

@jeremy-croff thanks for the proposal, could you add a little more details to the solution you're proposing?

[jeremy-croff]

@jeremy-croff thanks for the proposal, could you add a little more details to the solution you're proposing?

I would add useEffect(() => _.debounce((selectedReportID) => Report.saveReportComment(selectedReportID, value || ''), 1000), [value]);

and delete this line:

App/src/pages/home/report/ReportActionCompose/ComposerWithSuggestions/ComposerWithSuggestions.tsx

Line 429 in 780e2ac

debouncedSaveReportComment(reportID, newCommentConverted);

And the existing useMemo callback: here

App/src/pages/home/report/ReportActionCompose/ComposerWithSuggestions/ComposerWithSuggestions.tsx

Line 322 in 780e2ac

const debouncedSaveReportComment = useMemo(

We already are saving the text input in this line here to the state:

App/src/pages/home/report/ReportActionCompose/ComposerWithSuggestions/ComposerWithSuggestions.tsx

Line 403 in 780e2ac

setValue(newCommentConverted);

So we would be responding to the state variable changing, not the callback from user typing.

Is that enough detail?

[mallenexpensify]

@getusha can you please respond to @jeremy-croff 's post above? Thx

[getusha]

@jeremy-croff it is not clear how you plan to pass selectedReportID to the useEffect?

[jeremy-croff]

@jeremy-croff it is not clear how you plan to pass selectedReportID to the useEffect?

ReportId is available in the scope as a component prop

[jeremy-croff]

useEffect(() => _.debounce(() => Report.saveReportComment(reportID, value || ''), 1000), [value]); is the valid implementation with the right variable name

[mallenexpensify]

Added to #vip-vsb room.

[mallenexpensify]

@getusha , 👀 plz on @jeremy-croff 's comment above. I'd love to get this resolved soon, if only cuz I personally run in it semi-frequently.

[tienifr]

@getusha PR is ready to review

[melvin-bot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.4.60-13 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• fix: draft message sync issue #38912

If no regressions arise, payment will be issued on 2024-04-15. 🎊

For reference, here are some details about the assignees on this issue:

• @getusha requires payment automatic offer (Reviewer)
• @tienifr requires payment automatic offer (Contributor)

[melvin-bot]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

• [@getusha] The PR that introduced the bug has been identified. Link to the PR:
• [@getusha] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
• [@getusha] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
• [@getusha] Determine if we should create a regression test for this bug.
• [@getusha] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
• [@mallenexpensify] Link the GH issue for creating/updating the regression test once above steps have been agreed upon: https://github.com/Expensify/Expensify/issues/388366

[mallenexpensify]

Contributor: @tienifr paid $500 via Upwork
Contributor+: @getusha paid $500 via Upwork

I created the TestRail GH with the test steps below from the PR, comment if there are better steps.

@getusha, please complete the rest of the BZ checklist above. thx

Tests

1. Open Script Editor (Mac)
2. New Document > Paste the following script

on run
	set textToType to "JUMP "
	repeat
		emulateTyping(textToType)
		delay 1
	end repeat
end run

on emulateTyping(textToType)
	tell application "System Events"
		repeat with i from 1 to count of characters of textToType
			set currentChar to character i of textToType
			keystroke currentChar
			delay 1
		end repeat
	end tell
end emulateTyping

3. Press the play button (run the script)
4. Focus the desktop newDot app composer
5. Wait until you observe that a letter from the word 'JUMP' is being omitted after being typed.
6. The word 'JUMP' needs to be typed with consistency in the composer. ignore the first word it is because we had a delay when focusing the input.

[getusha]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

[@getusha] The PR that introduced the bug has been identified. Link to the PR: #25758
[@getusha] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment: https://github.com/Expensify/App/pull/25758/files#r1573699887
[@getusha] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion: N/a, this is hard to reproduce and catch. i don't think we need other actions to be taken other than adding the tests steps.