Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split tables not working for tables sorrounded by double quotes #463

Closed
shubhamdhama opened this issue Sep 12, 2023 · 2 comments · Fixed by #464
Closed

Split tables not working for tables sorrounded by double quotes #463

shubhamdhama opened this issue Sep 12, 2023 · 2 comments · Fixed by #464

Comments

@shubhamdhama
Copy link
Contributor

Schema

CREATE SCHEMA IF NOT EXISTS "Sp1eCial .Char";

CREATE TABLE "Sp1eCial .Char"."source1testing"("s0" int PRIMARY KEY, "s1" int NOT NULL
);

INSERT INTO "Sp1eCial .Char"."source1testing"("s0", "s1")
SELECT
    x,
(x * 2) % 100000
FROM
    generate_series(1, 10000) AS t(x);
CREATE TABLE "Sp1eCial .Char"."Tabl e.1testing"("iD" int PRIMARY KEY, "regId" int, "status" int, "nA M.e" character varying(20) NOT NULL, CONSTRAINT "Tabl e_fk_1_testing" FOREIGN KEY ("iD") REFERENCES "Sp1eCial .Char"."source1testing"("s0")
);
INSERT INTO "Sp1eCial .Char"."Tabl e.1testing"("iD", "regId", "status", "nA M.e")
SELECT
    "s0",
    "s0",
    random() * 100,
    'Name ' || "s0"
FROM
    "Sp1eCial .Char"."source1testing";

Error

2023-09-12 13:12:03 1766 INFO   copydb_schema.c:270       Splitting source candidate tables larger than 47 MB
2023-09-12 13:12:03 1766 SQL    pgsql.c:1493              [SOURCE 1767]  with  key_bounds (min, max) as  (    select min("iD"), max("iD")      from ""Sp1eCial .Char"".""Tabl e.1testing""  ),  t (parts) as  (    select ceil(bytes::float / $1) as parts      from pgcopydb_table_size      where oid = $2 \tunion all \tselect 1 as parts \torder by parts desc \tlimit 1  ),  ranges(n, parts, a, b) as  (    select n,           parts,           x as a,           coalesce((lead(x, 1) over(order by n)) - 1, max) as b      from key_bounds, t,           generate_series(min, max, ((max-min+1)/parts)::bigint + 1)           with ordinality as s(x, n)  )    select n, parts, a, b, b-a+1 as count     from ranges order by n;
2023-09-12 13:12:03 1766 SQL    pgsql.c:1506              [SOURCE 1767] '50000000', '378873'
2023-09-12 13:12:03 1766 DEBUG  pgsql.c:2033              Postgres result status is PGRES_FATAL_ERROR
2023-09-12 13:12:03 1766 ERROR  pgsql.c:1920              [SOURCE 1767] ERROR:  zero-length delimited identifier at or near """"
2023-09-12 13:12:03 1766 ERROR  pgsql.c:1920              [SOURCE 1767] LINE 1: ...x) as  (    select min("iD"), max("iD")      from ""Sp1eCial...
2023-09-12 13:12:03 1766 ERROR  pgsql.c:1920              [SOURCE 1767]                                                              ^
2023-09-12 13:12:03 1766 ERROR  pgsql.c:1931              [SOURCE 1767] SQL query:  with  key_bounds (min, max) as  (    select min("iD"), max("iD")      from ""Sp1eCial .Char"".""Tabl e.1testing""  ),  t (parts) as  (    select ceil(bytes::float / $1) as parts      from pgcopydb_table_size      where oid = $2 \tunion all \tselect 1 as parts \torder by parts desc \tlimit 1  ),  ranges(n, parts, a, b) as  (    select n,           parts,           x as a,           coalesce((lead(x, 1) over(order by n)) - 1, max) as b      from key_bounds, t,           generate_series(min, max, ((max-min+1)/parts)::bigint + 1)           with ordinality as s(x, n)  )    select n, parts, a, b, b-a+1 as count     from ranges order by n
2023-09-12 13:12:03 1766 ERROR  pgsql.c:1939              [SOURCE 1767] SQL params: '50000000', '378873'
2023-09-12 13:12:03 1766 ERROR  schema.c:3155             Failed to compute partition list for table ""Sp1eCial .Char"".""Tabl e.1testing""
2023-09-12 13:12:03 1766 ERROR  cli_clone_follow.c:460    Failed to clone source database, see above for details
@shubhamdhama
Copy link
Contributor Author

shubhamdhama commented Sep 12, 2023

I tried changing this

			" ( "
			"   select min(%s), max(%s) "
			"     from %s.%s "
			" ), "

in schema_list_partitions but that wouldn't solve the issue because we didn't do the format %I for partKey. While I can fix it for partkey, we might face the original issue for column in COPY statements too.

@shubhamdhama
Copy link
Contributor Author

shubhamdhama commented Sep 12, 2023

I think we can fix this issue first, later we can take care of attributes. On quick search for other locations where we use \"%s\".\"%s\", src/bin/pgcopydb/indexes.c is the only other place. Regex for searching ^(?!\s*log).*\\"%s\\".*\\"%s\\".*

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant