Proposal: Add paging support to clients

[wslulciuc]

We have recently added offset-based pagination support in #1655 to Marquez. Next, we'll need to define a API contract so that our clients can support pagination when calling list APIs. Both the java and python clients support listing datasets and jobs (as well as runs, etc).

How are datasets and jobs listed without paging?

python

The list methods return a dict containing a list of datasets and jobs:

MarquezClient.list_datasets()
MarquezClient.list_jobs()

java

The list methods return a List<T> containing a list of datasets and jobs:

List<Dataset> MarquezClient.listDatasets()
List<Job> MarquezClient.listJobs()

How will datasets and jobs listed with paging?

To avoid any breaking changes and allow the deprecation of the methods above, I proposal we overload the list methods by accepting an Option class:

python

MarquezClient.list_datasets(ListOption)  -> ResultsPage[Dataset]
MarquezClient.list_jobs(ListOption)  -> ResultsPage[Job]

java

ResultsPage<Dataset> MarquezClient.listDatasets(ListOption)
ResultsPage<Job> MarquezClient.listJobs(ListOption)

where ListOption is defined as:

/** Class for specifying list options. */
class ListOption {
  int limit;
  int offset
  // more options ...
}

and the resulting paging class ResultsPage<T> defined as:

/** A Page object wraps an API list method response. */
class ResultsPage<T> {
  int limit;
  int offset;
  int totalCount;
  /** Returns true if there are more pages that can be retrieved. */
  boolean hasNextPage();
  /** Retrieves the next Page object using the limit and offset. */
  Page<T> getNextPage();
  /** Returns elements in this page. */
  List<T> getValues();
}