Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix segmentby typmod and collation of compressed chunks #6332

Merged
merged 1 commit into from
Nov 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .unreleased/pr_6332
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fixes: #6332 Fix typmod and collation for segmentby columns
39 changes: 39 additions & 0 deletions sql/updates/latest-dev.sql
Original file line number Diff line number Diff line change
Expand Up @@ -403,3 +403,42 @@ BEGIN
END LOOP;
END;
$$ LANGUAGE PLPGSQL;

-- fix atttypmod and attcollation for segmentby columns
DO $$
DECLARE
htc_id INTEGER;
htc REGCLASS;
_attname NAME;
_atttypmod INTEGER;
_attcollation OID;
BEGIN
-- find any segmentby columns where typmod and collation in
-- the compressed hypertable does not match the uncompressed
-- hypertable values
FOR htc_id, htc, _attname, _atttypmod, _attcollation IN
SELECT cat.htc_id, cat.htc, pga.attname, ht_mod, ht_coll
FROM pg_attribute pga
INNER JOIN
(
SELECT
htc.id AS htc_id,
format('%I.%I',htc.schema_name,htc.table_name) AS htc,
att_ht.atttypmod AS ht_mod,
att_ht.attcollation AS ht_coll,
c.attname
FROM _timescaledb_catalog.hypertable_compression c
INNER JOIN _timescaledb_catalog.hypertable ht ON ht.id=c.hypertable_id
INNER JOIN pg_attribute att_ht ON att_ht.attname = c.attname AND att_ht.attrelid = format('%I.%I',ht.schema_name,ht.table_name)::regclass
INNER JOIN _timescaledb_catalog.hypertable htc ON htc.id=ht.compressed_hypertable_id
WHERE c.segmentby_column_index > 0
) cat ON cat.htc::regclass = pga.attrelid AND cat.attname = pga.attname
WHERE pga.atttypmod <> ht_mod OR pga.attcollation <> ht_coll
LOOP
-- fix typmod and collation for the compressed hypertable and all compressed chunks
UPDATE pg_attribute SET atttypmod = _atttypmod, attcollation = _attcollation WHERE attname = _attname AND attrelid IN (
SELECT format('%I.%I',schema_name,table_name)::regclass from _timescaledb_catalog.chunk WHERE hypertable_id = htc_id AND NOT dropped UNION ALL SELECT htc
);
END LOOP;
END
$$;
21 changes: 0 additions & 21 deletions tsl/src/nodes/decompress_chunk/planner.c
Original file line number Diff line number Diff line change
Expand Up @@ -177,30 +177,9 @@ build_decompression_map(PlannerInfo *root, DecompressChunkPath *path, List *scan
/*
* Normal column, not a metadata column.
*/
AttrNumber hypertable_attno = get_attnum(path->info->ht_rte->relid, column_name);
AttrNumber chunk_attno = get_attnum(path->info->chunk_rte->relid, column_name);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add an Assert(compression_info->algo_id != _INVALID_COMPRESSION_ALGORITHM) here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont think that would be very useful, since that column will completely vanish very soon. For the current implementation of compression the source of truth for the compression algorithm is the batch header and not algo_id in Hypertable_compression

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh after rereading your suggestion, this will not work all segmentby columns would fail that check since they have invalid compression algorihtm since they are not stored compressed.

Assert(hypertable_attno != InvalidAttrNumber);
Assert(chunk_attno != InvalidAttrNumber);

/*
* The versions older than this commit didn't set up the proper
* collation and typmod for segmentby columns in compressed chunks,
* so we have to determine them from the main hypertable.
* Additionally, we have to set the proper type for the compressed
* columns. It would be cool to get rid of this code someday and
* just use the types from the compressed chunk, but the problem is
* that we have to support the chunks created by the older versions
* of TimescaleDB.
*/
if (compression_info->algo_id == _INVALID_COMPRESSION_ALGORITHM)
{
get_atttypetypmodcoll(path->info->ht_rte->relid,
hypertable_attno,
&var->vartype,
&var->vartypmod,
&var->varcollid);
}

if (bms_is_member(0 - FirstLowInvalidHeapAttributeNumber, chunk_attrs_needed))
{
/*
Expand Down
Loading