Long Running Operation APIs don't have a way to determine when the operation is created

[dpcollins-google]

Does this issue affect the google-cloud-cpp project?
Yes

What component of google-cloud-cpp is this related to?
Any that use google.longrunning.Operation in their APIs.

Describe the bug
The purpose of long running operations is that they are long running and have two phases: submission, and completion. There is no way with the current api, which returns a future with the response_type, to determine when submission has succeeded. This means it is impossible to write a short running command line tool that would create the operation and print its identifier for later use without blocking on the result.

[dpcollins-google]

The specific issue for me is with pubsublite, where the operation for SeekSubscription will never complete until the user connects a subscriber client for the first time or seeks again. This means it is impossible to write an infrastructure tool with this API that creates a subscription seeked to a predetermined location.

[coryan]

It should be possible to split the LRO into a *Client::SubmitFoo() function, returning a future<StatusOr<Operation>> and a AwaitFoo() overload set, returning future<StatusOr<ReturnType>> and consuming either StatusOr<Operation> or future<StatusOr<Operation>>.

[coryan]

Some ideas on this. Where today we have something like:

  future<StatusOr<google::spanner::admin::database::v1::Backup>> CreateBackup(
      google::spanner::admin::database::v1::CreateBackupRequest const& request,
      Options opts = {});

In the future we would have two new functions:

  future<StatusOr<google::longrunning::Operation>> StartCreateBackup(
      google::spanner::admin::database::v1::CreateBackupRequest const& request,
      Options opts = {});
  future<StatusOr<google::spanner::admin::database::v1::Backup>> AwaitCreateBackup(
      google::longrunning::Operation op,
      Options opts = {});

And the old function would be implemented using:

  future<StatusOr<google::spanner::admin::database::v1::Backup>> CreateBackup(
      google::spanner::admin::database::v1::CreateBackupRequest const& request,
      Options opts = {}) {
    return StartCreateBackup(request, opts).then([options = std::move(opts)](auto f) mutable {
      auto op = f.get();
      if (op) return AwaitCreateBackup(*std::move(op), std::move(options));
      return make_ready_future(StatusOr<google::spanner::admin::database::v1::Backup>>(std::move(op).status()));
    });
  }

[devbww]

Presumably that StartCreateBackup() returns something more like future<Operation> than future<Backup>.

[coryan]

Presumably that StartCreateBackup() returns something more like future<Operation> than future<Backup>.

Doh! Fixed.

[devbww]

I think that still begs the question of how CreateBackup() should behave when AwaitCreateBackup() exhausts its polling policy.

[coryan]

I think that still begs the question of how CreateBackup() should behave when AwaitCreateBackup() exhausts its polling policy.

It does. One answer is "The same way it does right now". Of course that has the problem of abandoning the operation. For applications where that is a bad idea, at least you can implement your own thing to cancel the operation:

auto op = client.StartCreateBackup(request);
if (!op) return std::move(op).status();
auto operation = *op;
return AwaitCreateBackup(operation).then([o = *std::move(operation)](auto f) {
  auto backup = f.get();
  if (backup) return make_ready_future(std::move(backup));
  CancelOperation(o);
  return AwaitCreateBackup(o);
});

Alternatively, we could add a CancelStalledLongrunningOperationsOption{} that let's one toggle between the current behavior and the one you implemented in #8695

PS: beware ownership of the background threads if implementing any of this.

[devbww]

One answer is "The same way it does right now". Of course that has the problem of abandoning the operation. For applications where that is a bad idea, at least you can implement your own thing to cancel the operation:

Right. I still think non-abandonment should be the default behavior of a CreateBackup(), but I concede that does run the risk of the cancellation request being ignored and the operation taking a long time to succeed/fail.

Alternatively, we could add a CancelStalledLongrunningOperationsOption{} that let's one toggle between the current behavior and the one you implemented in #8695

Yes, perhaps.

[coryan]

I will try to put this in the schedule for 2022-Q4

[coryan]

We do not have time to work on this, closing.

[scotthart]

This functionality is now present for all LRO methods.