Spanner Beta

[lukesneeringer]

This shall be the "root" PR for getting Spanner to beta. Multiple PRs are going to be made using it as a base.

The remaining to-do items are:

• Re-generate the autogen libraries for the data and admin APIs.
  • Spanner Admin Database (Move the admin database GAPIC to source delivery. #4029)
  • Spanner Admin Instance (Move the Spanner Admin Instance GAPIC. #4030)
  • Regular ol' Spanner, yo (Update Spanner auto-gen layer. #4033)
• Update documentation to include autogen API reference. (superceded by Update Spanner auto-gen layer. #4033)
• Bit flip setup.py and appropriate documentation to claim beta status. (Bitflip to beta. #4034)
• Pin gRPC to < 1.6 (superceded by Move the admin database GAPIC to source delivery. #4029)

[tseaver]

From #4027, which this PR supersedes:

---

Draft release notes:

google-cloud-spanner 0.27.1

• Pin dependency google-auth >= 1.1.0 to fix use of google.cloud.spanner on GCE. (PR Pin google-auth >= 1.1.0. #3950, issue Cannot create a spanner session when running in compute engine #3933)
• Fix merge of partial timestamp and date field values. (PR Merge dates and timestamps properly. #4015, issue Spanner should know how to merge chunk data of TIMESTAMP and DATE type #3981, Google cloud spanner client fails when reading large columns #3998)
• Restart streaming iterators from read / execute_sql operations after resumable error (PR Retry streaming exceptions (this time for sure, Rocky!) #4016, issue Cloud spanner client does not seem to be retrying broken streams #3775).

---

Internal housekeeping not included in release notes:

• Suppress instance creation tests by default on CI (PR Suppress instance creation tests by default on CI #3951)

[bjwatson]

@lukesneeringer @jonparrott We also want a quick review of the API Reference docs to make sure it's reasonably in sync with the client library. This is just for the manual docs; the autogen docs should be fine.

[theacodes]

@lukesneeringer is this ready for me to review the docs?

[lukesneeringer]

@jonparrott Yes.

[bjwatson]

Thanks @lukesneeringer, @tseaver, and @jonparrott.

[theacodes]

Non-documentation issues/concerns

• google.cloud.spanner is not just an alias to google.cloud.spanner_v1. This is inconsistent with the approach we took with pubsub.

Documentation issues (these may be deferred by moving them into their own bugs):

• The landing page is... not useful
  • Improve the Spanner landing page. #4060
• Instance usage: says that list_instance_configs and list_instances returns a token, but these return iterators.
  • Correct documentation about instance iterators. #4044
  • Also see Correctly iterate over list_databases. #4047
• Instance usage: references the old operation interface, there is no finished() in the new one. It should just reference result().
  • Provide correct API for monitoring an LRO (instances). #4045
• Database usage: Spanner does not have its own Operation class.
  • Just refer to operations as being like Future. #4046
• Database usage: has an issue similar to (2)
  • Provide correct API for monitoring an LRO (database). #4048
• Database usage: the sample here has a typo (and seems to be formatted strangely)
  • Improve the session pool documentation. #4049
• The sidebar for spanner is totally whack. The low-level gapic stuff is mixed in with the high-level wrappers, and one page's sub-titles are treated as separate pages.
  • Fix up the documentation nav for Spanner. #4054
• The page titles mostly end with "API" which is totally weird and overloaded in this context.
  • @lukesneeringer: I am happier with this one in a post-Fix up the documentation nav for Spanner. #4054 world.

[bjwatson]

Thanks for your detailed review @jonparrott!

@lukesneeringer is the "Non-documentation issues/concern" a breaking change to fix later. I'm assuming consistency is important here; what do you think?

@tseaver It seems like documentation items 2-6 (shown as 1-5 due to the graphic) should be fixed before the Beta release. The other presentation and organizational concerns can be fixed before GA. Do you agree @jonparrott and @lukesneeringer?

FYI @vkedia

[theacodes]

Item 6 should really be fixed ASAP.

[bjwatson]

@jonparrott I see what you're saying with that. The Python API Reference combines manual and GAPIC at the same level, whereas the Ruby docs (and those for other languages) puts it another level below under v1.

@lukesneeringer Is this particularly difficult to fix with the current docs setup? I don't think we want to create a lot of work to finalize docs for Beta, but I do see the usability concern that Jon is raising.

[lukesneeringer]

Putting in PRs for these.

[lukesneeringer]

Okay, I fixed all of these except the spanner alias (the item without a PR number in the comment above is waiting on #4054 to be approved, because it needs it as a baseline).

@jonparrott, I am having trouble deciding what to do about the spanner_v1 thing. I actually considered moving it, and decided against it because doing so would break people in a few cases (e.g. anyone who typed from google.cloud.spanner.client import Client, which the documentation previously recommended in one spot, now corrected).

With Pub/Sub, we were breaking everyone anyway, so it was an obvious decision. Additionally, since Pub/Sub was a thin client, it was quite necessary to do this.

I am willing to do it here also, but due to the potential customer randomization, as well as the thicker client, it is less clear to me that it is the correct path. Please advise. :-)

[theacodes]

I think we need to make a firm decision on the api -> api_{version} going forwards - either it's a direct alias or it adds functionality. Either spanner or pubsub will need to change. I strongly think we should follow what pubsub did - spanner is transitioning into beta so breaking is allowed (although less than ideal, sure).

[theacodes]

fwiw, the pubsub samples use google.cloud.pubsub_v1 explicitly.

[lukesneeringer]

Okay, I am happy to change it then. :-)

[bjwatson]

Thanks, @lukesneeringer for fixing just about everything. I agree that I'm happier with all the API suffixes in a post-4054 world, too.

What's the PR for the alias fix?

[lukesneeringer]

What's the PR for the alias fix?

There is not one yet. I had wall-to-wall meetings followed by lunch. :-)

[bjwatson]

Understood. Must be Monday. Lots of meetings.

[lukesneeringer]

@jonparrott @bjwatson Okay, #4064 exists.

This is probably a breaking change at the margins. In other words, most users will not be affected, but some might be. In particular, we did have at least one doc that told users to from google.cloud.spanner.client import Client, and that will break.

Do we need any kind of migration guide?

[bjwatson]

@lukesneeringer We should at least have release notes that explain this breaking change.

[lukesneeringer]

@bjwatson We will have release notes either way. :-)

[bjwatson]

Woo hoo! Great job @lukesneeringer, @tseaver, and @jonparrott!