[GOBBLIN-1920] Use db laundered timestamp for reminder event

[umustafi]

Dear Gobblin maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

• My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
  • https://issues.apache.org/jira/browse/GOBBLIN-1920

Description

• Here are some details about my PR, including screenshots (if applicable):
  When setting the reminder event we use the trigger time local to each host. Instead we want to use the timestamp from database to see consistency between hosts in case their clock drift is significant. 

Tests

• My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

• My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

[phet]

good catch!

only improvement to recommend is a comment explaining why this is important

[prasadkarkera]

We can probably remove this from function parameter as well?

[codecov-commenter]

Codecov Report

Merging #3788 (93cfff1) into master (c98ef9d) will decrease coverage by 1.63%.
Report is 1 commits behind head on master.
The diff coverage is 33.33%.

@@             Coverage Diff              @@
##             master    #3788      +/-   ##
============================================
- Coverage     48.96%   47.33%   -1.63%     
- Complexity     3577    10955    +7378     
============================================
  Files           697     2152    +1455     
  Lines         28234    85089   +56855     
  Branches       3292     9452    +6160     
============================================
+ Hits          13824    40278   +26454     
- Misses        13007    41158   +28151     
- Partials       1403     3653    +2250     

Files	Coverage Δ
...vice/modules/orchestration/FlowTriggerHandler.java	21.35% <33.33%> (ø)

... and 1463 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[meethngala]

nice catch! I was wondering if it even makes sense for us to provide eventTimeMillis to the method signature of scheduleReminderForEvent. Earlier we were using it to create JobDetail, but now the only place we use it is for logging and the info that it conveys is the clock drift (which we might be okay not logging)... let me know what's your take?

[umustafi]

nice catch! I was wondering if it even makes sense for us to provide eventTimeMillis to the method signature of scheduleReminderForEvent. Earlier we were using it to create JobDetail, but now the only place we use it is for logging and the info that it conveys is the clock drift (which we might be okay not logging)... let me know what's your take?

I am leaving it in to catch that clock drift in time being. I suspect it's not the root of our problem but until we are reliably bug free keeping logs in.