🪟 🐛 Fix direct job linking to work with pagination #16517

lmossman · 2022-09-09T00:54:07Z

What

Resolves #16369

See above linked issue for context.

How

This PR solves the problem by utilizing the list job API improvements made in #16415 to directly fetch the desired list of jobs when a job ID is linked to in the URL. This means that if someone links to a job outside of the first page of jobs, this view will immediately load the necessary number of pages required to include the linked job, as demonstrated in this screen recording:

Screen.Recording.2022-09-08.at.5.50.06.PM.mov

This PR also adds a fallback view for the edge case where someone puts an invalid job ID into the URL, with a link back to the normal sync history:

Screen.Recording.2022-09-08.at.5.52.42.PM.mov

Lastly, this PR also contains a small fix to utilize the new totalJobCount property in the list jobs response, to decide if there are more jobs to load and therefore whether the Load more button should be shown.

🚨 User Impact 🚨

Direct job linking should now work as expected again.

lmossman · 2022-09-09T00:55:37Z

airbyte-webapp/src/components/JobItem/JobItem.tsx

@@ -48,7 +48,7 @@ export const JobItem: React.FC<JobItemProps> = ({ job }) => {
  const didSucceed = didJobSucceed(job);

  const onExpand = () => {
-    setIsOpen(!isOpen);
+    setIsOpen((prevIsOpen) => !prevIsOpen);


Just updating this to use the best practice of using the arrow function for a change to a state value that depends on itself

… be found

… to include job

edmundito

Tested locally. The code looks good but had a few comments.

edmundito · 2022-09-09T19:24:31Z

airbyte-webapp/src/locales/en.json

@@ -361,6 +361,8 @@
  "connection.cancelSync": "Cancel Sync",
  "connection.cancelReset": "Cancel Reset",
  "connection.canceling": "Canceling...",
+  "connection.linkedJobNotFound": "The linked job could not be found",


Could also be "Job not found"

airbyte-webapp/src/services/job/JobService.tsx

edmundito · 2022-09-12T16:10:34Z

airbyte-webapp/src/pages/ConnectionPage/pages/ConnectionItemPage/components/StatusView.tsx

+        ) : linkedJobNotFound ? (
+          <EmptyResource
+            text={<FormattedMessage id="connection.linkedJobNotFound" />}
+            description={
+              <Link to={pathname}>
+                <FormattedMessage id="connection.returnToSyncHistory" />
+              </Link>
+            }
+          />


I wonder if instead of showing this, what if it showed an error notification at the bottom of the screen but still showed the jobs list? I suppose that in this case, the API would need to still return the jobs as usual.

Yeah that was how I had originally implemented the backend API, but the backend team pushed back on that a bit saying it made the endpoint somewhat confusing to return results even if the specific job ID could not be found (it was basically behaving in two different ways depending on that), which I kind of agreed with.

I don't expect users to run into this situation very often, since it would require them to manually edit the job ID in the URL, so hopefully having this separate view with a single click to get back to the main job view is not too cumbersome

Co-authored-by: Edmundo Ruiz Ghanem <168664+edmundito@users.noreply.github.com>

* master: (200 commits) 🪟 🧹 Display returned error messages on replication view (#16280) 🎉 Source mixpanel: Use "Retry-After" header for backoff (#16770) 🐛 Source google ads: mark custom query fields required (#15858) 🪟 🔧Remove useRouter hook (#16598) CDK: improve TypeTransformer to convert simple types to array of simple types (#16636) CDK: TypeTransformer - warning message more informative (#16695) Source MySQL: Add Python SAT to detect backwards breaking changes (#16445) remove eager (#16756) bump com.networknt:json-schema-validator to latest version (#16619) Remove Cloud from Kafka docs (#16753) Normalization Summaries table and read/write methods (#16655) comment out flaky test suite while it is being investigated (#16752) Update ConfigRepository to read protocol version (#16670) Use LOG4J2 to wrap connectors logs to JSON format (#15668) Update connector catalog (#16749) 🪟 🎨 Remove feedback modal from UI (#16548) Add missing env var for Kube overlays (#16747) Prepare for React v18 upgrade (#16694) 🪟 🐛 Fix direct job linking to work with pagination (#16517) Fix formatting (#16743) ...

* start implementation of new persistence method * add includingJobId and totalJobCount to job list request * format * update local openapi as well * refactor queries into JOOQ and return empty list if target job cannot be found * fix descriptions and undo changes from other branch * switch including job to starting job * fix job history handler tests * rewrite jobs subqueries in jooq * fix multiple config type querying * remove unnecessary casts * switch back to 'including' and return multiple of page size necessary to include job * undo webapp changes * fix test description * format * save progress * use list job api changes to retrieve linked job if linked * change job not found wording * Update airbyte-webapp/src/services/job/JobService.tsx Co-authored-by: Edmundo Ruiz Ghanem <168664+edmundito@users.noreply.github.com> Co-authored-by: Edmundo Ruiz Ghanem <168664+edmundito@users.noreply.github.com>

github-actions bot added area/platform issues related to the platform area/frontend Related to the Airbyte webapp labels Sep 9, 2022

lmossman commented Sep 9, 2022

View reviewed changes

lmossman requested a review from timroes September 9, 2022 01:01

lmossman marked this pull request as ready for review September 9, 2022 01:02

lmossman requested a review from a team as a code owner September 9, 2022 01:02

lmossman mentioned this pull request Sep 9, 2022

Improve job list API with more fetching capabilities #16415

Merged

lmossman added 14 commits September 9, 2022 10:56

start implementation of new persistence method

8eb89b8

add includingJobId and totalJobCount to job list request

88a99d8

format

9002461

update local openapi as well

80c213e

refactor queries into JOOQ and return empty list if target job cannot…

aeaffad

… be found

fix descriptions and undo changes from other branch

df71109

switch including job to starting job

d68fc5a

fix job history handler tests

ac750d9

rewrite jobs subqueries in jooq

ef2d6dc

fix multiple config type querying

6419233

remove unnecessary casts

84b8dbe

switch back to 'including' and return multiple of page size necessary…

5d53ab7

… to include job

undo webapp changes

5a5adea

fix test description

ed71403

lmossman force-pushed the lmossman/add-including-job-to-list-jobs-api branch from c57a400 to ed71403 Compare September 9, 2022 17:57

format

c991275

edmundito reviewed Sep 12, 2022

View reviewed changes

Merge branch 'master' into lmossman/add-including-job-to-list-jobs-api

09d8b92

github-actions bot added the area/scheduler label Sep 12, 2022

lmossman temporarily deployed to more-secrets September 12, 2022 21:19 Inactive

lmossman added 2 commits September 12, 2022 14:23

save progress

be27e62

use list job api changes to retrieve linked job if linked

ac7e103

lmossman force-pushed the lmossman/fix-direct-job-linking branch from 4878e2c to ac7e103 Compare September 12, 2022 21:23

github-actions bot removed the area/scheduler label Sep 12, 2022

lmossman temporarily deployed to more-secrets September 12, 2022 21:25 Inactive

lmossman and others added 2 commits September 12, 2022 14:26

change job not found wording

97958e1

Update airbyte-webapp/src/services/job/JobService.tsx

93c4d73

Co-authored-by: Edmundo Ruiz Ghanem <168664+edmundito@users.noreply.github.com>

lmossman requested a review from edmundito September 12, 2022 21:28

lmossman temporarily deployed to more-secrets September 12, 2022 21:29 Inactive

Base automatically changed from lmossman/add-including-job-to-list-jobs-api to master September 12, 2022 23:32

lmossman added 3 commits September 12, 2022 16:33

Merge branch 'master' into lmossman/fix-direct-job-linking

a787b05

merge with master

cdf4a59

Merge branch 'master' into lmossman/fix-direct-job-linking

9264180

edmundito approved these changes Sep 14, 2022

View reviewed changes

Merge branch 'master' into lmossman/fix-direct-job-linking

85e31da

lmossman merged commit 638a88c into master Sep 14, 2022

lmossman deleted the lmossman/fix-direct-job-linking branch September 14, 2022 17:37

octavia-squidington-iii mentioned this pull request Sep 14, 2022

Bump Airbyte version from 0.40.6 to 0.40.7 #16754

Closed

octavia-squidington-iii mentioned this pull request Sep 15, 2022

Bump Airbyte version from 0.40.6 to 0.40.7 #16785

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🪟 🐛 Fix direct job linking to work with pagination #16517

🪟 🐛 Fix direct job linking to work with pagination #16517

lmossman commented Sep 9, 2022 •

edited

Loading

lmossman Sep 9, 2022

edmundito left a comment

edmundito Sep 9, 2022

edmundito Sep 12, 2022

lmossman Sep 12, 2022

🪟 🐛 Fix direct job linking to work with pagination #16517

🪟 🐛 Fix direct job linking to work with pagination #16517

Conversation

lmossman commented Sep 9, 2022 • edited Loading

What

How

Recommended reading order

🚨 User Impact 🚨

lmossman Sep 9, 2022

Choose a reason for hiding this comment

edmundito left a comment

Choose a reason for hiding this comment

edmundito Sep 9, 2022

Choose a reason for hiding this comment

edmundito Sep 12, 2022

Choose a reason for hiding this comment

lmossman Sep 12, 2022

Choose a reason for hiding this comment

lmossman commented Sep 9, 2022 •

edited

Loading