Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: Add migration guide from version 2.x to 3.x #1027
docs: Add migration guide from version 2.x to 3.x #1027
Changes from all commits
28d7f28
babf0d1
c29c850
691409d
47f8e01
bdb205a
760fd99
cc8c9c0
9cc3c86
d432e6e
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
3.0.0 Migration Guide
TODO
New Required Dependencies
Some of the previously optional dependencies are now required in
3.x
versions of the library, namely google-cloud-bigquery-storage (minimum version2.0.0
) and pyarrow (minimum version3.0.0
).The behavior of some of the package "extras" has thus also changed:
The
pandas
extra now requires the db-types package.The
bqstorage
extra has been preserved for comaptibility reasons, but it is now a no-op and should be omitted when installing the BigQuery client library.Before:
After:
The
bignumeric_type
extra has been removed, asBIGNUMERIC
type is now automatically supported. That extra should thus not be used.Before:
After:
Type Annotations
The library is now type-annotated and declares itself as such. If you use a static type checker such as
mypy
, you might start getting errors in places wheregoogle-cloud-bigquery
package is used.It is recommended to update your code and/or type annotations to fix these errors, but if this is not feasible in the short term, you can temporarily ignore type annotations in
google-cloud-bigquery
, for example by using a special# type: ignore
comment:But again, this is only recommended as a possible short-term workaround if immediately fixing the type check errors in your project is not feasible.
Re-organized Types
The auto-generated parts of the library has been removed, and proto-based types formerly found in
google.cloud.bigquery_v2
have been replaced by the new implementation (but see the section below).For example, the standard SQL data types should new be imported from a new location:
Before:
After:
The
TypeKind
enum defining all possible SQL types for schema fields has been renamed and is not nested anymore underStandardSqlDataType
:Before:
After:
Issuing queries with
Client.create_job
preserves destination tableThe
Client.create_job
method no longer removes the destination table from a query job's configuration. Destination table for the query can thus be explicitly defined by the user.Changes to data types when reading a pandas DataFrame
The default dtypes returned by the
to_dataframe
method have changed.BOOLEAN
data type maps to the pandasboolean
dtype. Previously, this mapped to the pandasbool
dtype when the column did not containNULL
values and the pandasobject
dtype whenNULL
values are present.INT64
data type maps to the pandasInt64
dtype. Previously, this mapped to the pandasint64
dtype when the column did not containNULL
values and the pandasfloat64
dtype whenNULL
values are present.DATE
data type maps to the pandasdbdate
dtype, which is provided by the db-dtypes package. If any date value is outside of the range of pandas.Timestamp.min (1677-09-22) and pandas.Timestamp.max (2262-04-11), the data type maps to the pandasobject
dtype. Thedate_as_object
parameter has been removed.TIME
data type maps to the pandasdbtime
dtype, which is provided by the db-dtypes package.Changes to data types loading a pandas DataFrame
In the absence of schema information, pandas columns with naive
datetime64[ns]
values, i.e. without timezone information, are recognized and loaded using theDATETIME
type. On the other hand, for columns with timezone-awaredatetime64[ns, UTC]
values, theTIMESTAMP
type is continued to be used.Changes to
Model
,Client.get_model
,Client.update_model
, andClient.list_models
The types of several
Model
properties have been changed.Model.feature_columns
now returns a sequence ofgoogle.cloud.bigquery.standard_sql.StandardSqlField
.Model.label_columns
now returns a sequence ofgoogle.cloud.bigquery.standard_sql.StandardSqlField
.Model.model_type
now returns a string.Model.training_runs
now returns a sequence of dictionaries, as recieved from the BigQuery REST API.Legacy Protocol Buffers Types
For compatibility reasons, the legacy proto-based types still exists as static code and can be imported:
Mind, however, that importing them will issue a warning, because aside from being importable, these types are not maintained anymore. They may differ both from the types in
google.cloud.bigquery
, and from the types supported on the backend.Maintaining compatibility with
google-cloud-bigquery
version 2.0If you maintain a library or system that needs to support both
google-cloud-bigquery
version 2.x and 3.x, it is recommended that you detect when version 2.x is in use and convert properties that use the legacy protocol buffer types, such asModel.training_runs
, into the types used in 3.x.Call the
to_dict
method on the protocol buffers objects to get a JSON-compatible dictionary.2.0.0 Migration Guide
The 2.0 release of the
google-cloud-bigquery
client drops support for Python versions below 3.6. The client surface itself has not changed, but the 1.x series will not be receiving any more feature updates or bug fixes. You are thus encouraged to upgrade to the 2.x series.If you experience issues or have questions, please file an issue.
Supported Python Versions
The 2.0.0 release requires Python 3.6+.
Supported BigQuery Storage Clients
The 2.0.0 release requires BigQuery Storage
>= 2.0.0
, which dropped support forv1beta1
andv1beta2
versions of the BigQuery Storage API. If you want to use a BigQuery Storage client, it must be the one supporting thev1
API version.Changed GAPIC Enums Path
Generated GAPIC enum types have been moved under
types
. Import paths need to be adjusted.Before:
After: