Fix segmentby typmod and collation of compressed chunks #6332

svenklemm · 2023-11-21T08:26:12Z

We added a workaround for segmentby columns with incorrect typmod
and collation in c73c5a7 but did not adjust pre-existing relations.
This patch will fix any existing relations where the segmentby columns
of compressed chunks have incorrect typmod and collation and remove
the code workaround.

github-actions · 2023-11-21T08:26:37Z

@mkindahl, @antekresic: please review this pull request.

Powered by pull-review

codecov · 2023-11-21T08:36:44Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (747f4e2) 65.48% compared to head (39204db) 65.44%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6332      +/-   ##
==========================================
- Coverage   65.48%   65.44%   -0.05%     
==========================================
  Files         249      249              
  Lines       57995    57940      -55     
  Branches    12920    12899      -21     
==========================================
- Hits        37980    37919      -61     
- Misses      18157    18176      +19     
+ Partials     1858     1845      -13

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

We added a workaround for segmentby columns with incorrect typmod and collation in c73c5a7 but did not adjust pre-existing relations. This patch will fix any existing relations where the segmentby columns of compressed chunks have incorrect typmod and collation and remove the code workaround.

jnidzwetzki · 2023-11-21T13:04:37Z

tsl/src/nodes/decompress_chunk/planner.c

@@ -177,30 +177,9 @@ build_decompression_map(PlannerInfo *root, DecompressChunkPath *path, List *scan
 			/*
 			 * Normal column, not a metadata column.
 			 */
-			AttrNumber hypertable_attno = get_attnum(path->info->ht_rte->relid, column_name);
 			AttrNumber chunk_attno = get_attnum(path->info->chunk_rte->relid, column_name);


Could we add an Assert(compression_info->algo_id != _INVALID_COMPRESSION_ALGORITHM) here?

I dont think that would be very useful, since that column will completely vanish very soon. For the current implementation of compression the source of truth for the compression algorithm is the batch header and not algo_id in Hypertable_compression

Oh after rereading your suggestion, this will not work all segmentby columns would fail that check since they have invalid compression algorihtm since they are not stored compressed.

timescale-automation · 2023-11-21T14:04:18Z

Automated backport to 2.12.x not done: cherry-pick failed.

Git status

HEAD detached at origin/2.12.x
You are currently cherry-picking commit d1c8cb067.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   .unreleased/pr_6332
	modified:   tsl/src/nodes/decompress_chunk/planner.c

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   sql/updates/latest-dev.sql

Job log

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * #5575 Add chunk-wise sorted paths for compressed chunks * #5761 Simplify hypertable DDL API * #5890 Reduce WAL activity by freezing compressed tuples immediately * #6050 Vectorized aggregation execution for sum() * #6062 Add metadata for chunk creation time * #6077 Make Continous Aggregates materialized only (non-realtime) by default * #6177 Change show_chunks/drop_chunks using chunk creation time * #6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * #6185 Keep track of catalog version * #6227 Use creation time in retention/compression policy * #6307 Add SQL function cagg_validate_query **Bugfixes** * #6188 Add GUC for setting background worker log level * #6222 Allow enabling compression on hypertable with unique expression index * #6240 Check if worker registration succeeded * #6254 Fix exception detail passing in compression_policy_execute * #6264 Fix missing bms_del_member result assignment * #6275 Fix negative bitmapset member not allowed in compression * #6280 Potential data loss when compressing a table with a partial index that matches compression order. * #6289 Add support for startup chunk exclusion with aggs * #6290 Repair relacl on upgrade * #6297 Fix segfault when creating a cagg using a NULL width in time bucket function * #6305 Make timescaledb_functions.makeaclitem strict * #6332 Fix typmod and collation for segmentby columns * #6339 Fix tablespace with constraints * #6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * #5575 Add chunk-wise sorted paths for compressed chunks * #5761 Simplify hypertable DDL API * #5890 Reduce WAL activity by freezing compressed tuples immediately * #6050 Vectorized aggregation execution for sum() * #6062 Add metadata for chunk creation time * #6077 Make Continous Aggregates materialized only (non-realtime) by default * #6177 Change show_chunks/drop_chunks using chunk creation time * #6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * #6185 Keep track of catalog version * #6227 Use creation time in retention/compression policy * #6307 Add SQL function cagg_validate_query **Bugfixes** * #6188 Add GUC for setting background worker log level * #6222 Allow enabling compression on hypertable with unique expression index * #6240 Check if worker registration succeeded * #6254 Fix exception detail passing in compression_policy_execute * #6264 Fix missing bms_del_member result assignment * #6275 Fix negative bitmapset member not allowed in compression * #6280 Potential data loss when compressing a table with a partial index that matches compression order. * #6289 Add support for startup chunk exclusion with aggs * #6290 Repair relacl on upgrade * #6297 Fix segfault when creating a cagg using a NULL width in time bucket function * #6305 Make timescaledb_functions.makeaclitem strict * #6332 Fix typmod and collation for segmentby columns * #6339 Fix tablespace with constraints * #6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

github-actions bot assigned svenklemm Nov 21, 2023

github-actions bot requested a review from antekresic November 21, 2023 08:26

github-actions bot requested a review from mkindahl November 21, 2023 08:26

svenklemm force-pushed the comp_collation branch from 3fa8052 to fc147dd Compare November 21, 2023 08:31

svenklemm requested a review from akuzm November 21, 2023 08:32

svenklemm added the bug label Nov 21, 2023

svenklemm added this to the TimescaleDB 2.13 milestone Nov 21, 2023

svenklemm requested a review from jnidzwetzki November 21, 2023 09:47

svenklemm force-pushed the comp_collation branch from fc147dd to 7590d16 Compare November 21, 2023 11:21

svenklemm changed the title ~~Fix segmentby atttypmod and attcollation~~ Fix segmentby typmod and collation of compressed chunks Nov 21, 2023

svenklemm force-pushed the comp_collation branch from 7590d16 to 8280ee6 Compare November 21, 2023 11:36

svenklemm force-pushed the comp_collation branch from 8280ee6 to 39204db Compare November 21, 2023 11:38

fabriziomello approved these changes Nov 21, 2023

View reviewed changes

jnidzwetzki reviewed Nov 21, 2023

View reviewed changes

jnidzwetzki approved these changes Nov 21, 2023

View reviewed changes

svenklemm enabled auto-merge (rebase) November 21, 2023 13:35

svenklemm merged commit d1c8cb0 into timescale:main Nov 21, 2023
40 checks passed

timescale-automation added the auto-backport-not-done Automated backport of this PR has failed non-retriably (e.g. conflicts) label Nov 21, 2023

jnidzwetzki mentioned this pull request Nov 23, 2023

Release 2.13.0 #6336

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix segmentby typmod and collation of compressed chunks #6332

Fix segmentby typmod and collation of compressed chunks #6332

svenklemm commented Nov 21, 2023 •

edited

Loading

github-actions bot commented Nov 21, 2023

codecov bot commented Nov 21, 2023 •

edited

Loading

jnidzwetzki Nov 21, 2023

svenklemm Nov 21, 2023

svenklemm Nov 21, 2023

timescale-automation commented Nov 21, 2023

Fix segmentby typmod and collation of compressed chunks #6332

Fix segmentby typmod and collation of compressed chunks #6332

Conversation

svenklemm commented Nov 21, 2023 • edited Loading

github-actions bot commented Nov 21, 2023

codecov bot commented Nov 21, 2023 • edited Loading

Codecov Report

jnidzwetzki Nov 21, 2023

Choose a reason for hiding this comment

svenklemm Nov 21, 2023

Choose a reason for hiding this comment

svenklemm Nov 21, 2023

Choose a reason for hiding this comment

timescale-automation commented Nov 21, 2023

Git status

svenklemm commented Nov 21, 2023 •

edited

Loading

codecov bot commented Nov 21, 2023 •

edited

Loading