Improve callable function for long running requests.

[taeold]

In #1634, we introduced new API to return response chunks for callable functions. This means that callable function response my be long-running.

To ensure that long-running requests are handled well, we are adding 2 new features:

1. heartbeat: Some network proxies may close request connection when no response data is sent for some time interval. By default, callable function will send a heartbeat ping every 30 seconds. Our client SDKs will ignore these pings. Users can disable heartbeat for streaming response by setting heartbeatSeconds configuration to null.

2. AbortSignal: response object of the callable handler will now include signal property which the developer can use to check and see if a client connection has been dropped. They can forward this signal property to fetch-like APIs so that underlying calls would be terminated properly.

e.g.

exports.generateText = onCall(
  { 
    heartbeatSeconds: 10 // (Default: 30. Set to 'null' to disable heartbeat. )
  },
  async (request, response) => {
    const prompt = request.data.text;
    const result = await model.generateContentStream(
      prompt,
      { signal: response.signal }
    );

    for await (const chunk of result.stream) {
      if (response.acceptsStreaming) {
        // if the client did not request a streaming response,
        // response.write() is a no-op.
        response.write(chunk.text())
      }
    }
    return (await result.data()).text()
  }

[blidd-google]

I'm confused by the example you provided. Why are we awaiting response() for the final return value?

[blidd-google]

Why not use setInterval()?

[taeold]

Using setTimeout instead since I'm trying ton control more precisely whether next "tick" should happen or not. That's why this call is recursive.