Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat migrations #437

Merged
merged 110 commits into from
Nov 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
a7d7aee
Add mirroring filter
abnegate Jun 5, 2024
61f066f
Add mirror database
abnegate Jun 5, 2024
d2dc913
Add source property so filters call Database methods on source db ins…
abnegate Jun 6, 2024
2e02f0b
Use source instead of parent so changes applied via getSource are used
abnegate Jun 6, 2024
ba9fc4e
Add back nullable destination
abnegate Jun 6, 2024
c409fc5
Add base tests
abnegate Jun 6, 2024
2dd8891
Merge remote-tracking branch 'origin/main' into feat-migrations
abnegate Jun 6, 2024
712f3e8
Add xdebug
abnegate Jun 7, 2024
f812b84
Fix getting upgrade status
abnegate Jun 7, 2024
757fa91
Add missing create/update batch methods
abnegate Jun 7, 2024
5af4e8e
Fix upgrade checks
abnegate Jun 7, 2024
995be2e
Delegate more setters and delete
abnegate Jun 7, 2024
b5fae84
Static returns
abnegate Jun 7, 2024
81516d5
Add mirror containers to compose stack
abnegate Jun 7, 2024
f181c7e
Fix return type for setters
abnegate Jun 7, 2024
a4c4109
Fix events firing for destination db
abnegate Jun 7, 2024
944aed6
Fix timestamp conflicts
abnegate Jun 7, 2024
fd22e9a
Fix setters on parents
abnegate Jun 7, 2024
d6c7adf
Fix timestamps
abnegate Jun 7, 2024
ba308bd
Add document crud tests
abnegate Jun 7, 2024
e1c484d
Revert "Fix timestamps"
abnegate Jun 10, 2024
de90a3f
Add mirror to tests
abnegate Jun 10, 2024
5da0ff6
Add for each filter function
abnegate Jun 12, 2024
01468e5
Expose filters directly
abnegate Jun 12, 2024
f99148a
Add default method stubs
abnegate Jun 12, 2024
238e4a0
Allow hooking after even as well as before
abnegate Jun 12, 2024
a358f8d
Hook before delete
abnegate Jun 12, 2024
60b9f10
Add init and shutdown hooks to filters
abnegate Jun 12, 2024
1ffa285
Ensure destination is nullable
abnegate Jul 2, 2024
28e4132
Nullable dest only for init and shutdown
abnegate Jul 2, 2024
9801cf9
Don't init filters on construction to allow for pre-config
abnegate Jul 10, 2024
ec87dfb
Try read instead of check duplicate upgrades collection
abnegate Jul 11, 2024
6c86f49
Fetch internal ID for documents not created with it
abnegate Jul 11, 2024
89a0079
Ignore relationships that are virtual attributes
abnegate Jul 22, 2024
68c9881
Nullable filter params for noop hooks
abnegate Jul 23, 2024
6b0defb
Foce require document for document hooks
abnegate Jul 23, 2024
397aaa6
Allow null tenant to allow global tables with single metadata doc
abnegate Aug 12, 2024
fdc9da7
Don't require tenant for create database
abnegate Aug 12, 2024
8b6e992
Dont require tenant on get metadata collection
abnegate Aug 12, 2024
bcbee88
Fix check order for getDocument
abnegate Aug 12, 2024
0b2cda2
Remove exists tenant check
abnegate Aug 12, 2024
15dc0ae
Remove redundant missing tenant checks
abnegate Aug 12, 2024
4f91b4a
Remove get collection check
abnegate Aug 12, 2024
7bc4146
Throw on duplicate collection
abnegate Aug 12, 2024
9a476e8
Throw duplicate properly, create collections in transaction
abnegate Aug 12, 2024
4dc980c
Use getPDO
abnegate Aug 12, 2024
88570af
Fix check
abnegate Aug 12, 2024
df9ef17
Fix impicit commit
abnegate Aug 12, 2024
27444b1
Update null checks
abnegate Aug 12, 2024
ac68fa4
Don't drop collection on duplicate exception
abnegate Aug 13, 2024
d7ae07e
Fix precedence
abnegate Aug 13, 2024
09da873
Merge remote-tracking branch 'origin/feat-single-shared-metadata-doc'…
abnegate Aug 13, 2024
04ffb86
Allow getTenant
abnegate Aug 13, 2024
d02d44d
Merge remote-tracking branch 'origin/main' into feat-migrations
abnegate Aug 14, 2024
e37b23f
Re-order properties
abnegate Aug 14, 2024
545192b
Add withTenant, withPreservedDates
abnegate Aug 15, 2024
0682d0b
Always preverse in withPreserveDates
abnegate Aug 15, 2024
6662a2c
Fix tests
abnegate Aug 16, 2024
ddf94c9
Merge remote-tracking branch 'origin/main' into feat-migrations
abnegate Aug 19, 2024
da8113e
Fix in cases
abnegate Aug 19, 2024
9f76589
Update timeout test
abnegate Aug 19, 2024
0ba61b1
Fix dump output truncation
abnegate Aug 19, 2024
ab549c1
Fix timeouts not thrown
abnegate Aug 19, 2024
9fff328
Fix SQLite tests
abnegate Aug 19, 2024
fd1bc7f
Fix Mirror tests
abnegate Aug 19, 2024
8b169a4
Fix SQLite tests
abnegate Aug 19, 2024
088d549
Fix missing silent calls in mirror
abnegate Aug 19, 2024
bad0ea5
Lint
abnegate Aug 19, 2024
67bf1fe
Merge pull request #439 from utopia-php/feat-atomic-updates
abnegate Aug 21, 2024
6eba4de
Merge pull request #442 from utopia-php/feat-resizing-string-attributes
abnegate Aug 30, 2024
46fa933
Merge pull request #444 from utopia-php/feat-resizing-string-attributes
abnegate Aug 30, 2024
6e2547a
Merge pull request #445 from utopia-php/feat-resizing-string-attributes
abnegate Aug 30, 2024
4be46b9
Merge pull request #446 from utopia-php/fix-deadlock
abnegate Sep 2, 2024
b6b7b54
Merge pull request #448 from utopia-php/fix-add-escapes-for-rename
abnegate Sep 10, 2024
32ab2f7
Lint
abnegate Sep 12, 2024
b3cf767
Add rollbacks on bad state
abnegate Sep 13, 2024
c483ca3
Merge remote-tracking branch 'origin/0.53.x' into feat-migrations
abnegate Oct 14, 2024
238883a
Exception simplification
abnegate Oct 14, 2024
9105080
Remove redundant get document
abnegate Oct 14, 2024
fde3478
Lint/check fixes
abnegate Oct 14, 2024
790b870
Fix merge
abnegate Oct 14, 2024
68f5bf6
Fix tests
abnegate Oct 14, 2024
74d7f8d
Fix cache not flushed on db delete
abnegate Oct 14, 2024
e06001b
Fix upgrade read on source/dest mirror
abnegate Oct 14, 2024
ec47a5e
Fix max allowed packet on mirror
abnegate Oct 14, 2024
58c7c68
Improve delegation
abnegate Oct 15, 2024
6639898
Fix spelling
abnegate Oct 15, 2024
a9de51b
Add redis mirror
abnegate Oct 15, 2024
30e470f
Use mirror cache for mirror db
abnegate Oct 15, 2024
781b421
Lint
abnegate Oct 15, 2024
651cb4f
Fix SQLite tests
abnegate Oct 15, 2024
b63c999
Fix order of operations issues
abnegate Oct 15, 2024
8bb5793
Fix mongo tests
abnegate Oct 15, 2024
78f0e9d
Bail out of commit if no active transaction
abnegate Oct 15, 2024
efd0f4e
Fix stan
abnegate Oct 15, 2024
b462b31
Merge remote-tracking branch 'origin/0.53.x' into feat-migrations
abnegate Oct 22, 2024
5b9462f
Add log dumps
abnegate Oct 22, 2024
7bd224a
Debug
abnegate Oct 22, 2024
df57463
Question?
fogelito Oct 31, 2024
3357657
Revert "Question?"
abnegate Nov 6, 2024
4688aef
Merge remote-tracking branch 'origin/0.53.x' into feat-migrations
abnegate Nov 6, 2024
a23aec2
Update index order
abnegate Nov 6, 2024
ad8f67a
Merge remote-tracking branch 'origin/0.53.x' into feat-migrations
abnegate Nov 6, 2024
6dde7ef
composer
shimonewman Nov 7, 2024
b593787
Merge remote-tracking branch 'origin/feat-migrations' into feat-migra…
shimonewman Nov 7, 2024
5af02bf
composer
shimonewman Nov 7, 2024
88b1975
composer
shimonewman Nov 7, 2024
9d5faac
composer
shimonewman Nov 7, 2024
31571bd
composer
shimonewman Nov 7, 2024
439655a
Merge branch '0.53.x' into feat-migrations
abnegate Nov 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ jobs:
Postgres,
SQLite,
MongoDB,
Mirror,
SharedTables/MariaDB,
SharedTables/MySQL,
SharedTables/Postgres,
Expand Down
42 changes: 28 additions & 14 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,28 +1,42 @@
FROM composer:2.0 AS composer

ARG TESTING=false
ENV TESTING=$TESTING
ARG DEBUG=false
ENV DEBUG=$DEBUG

WORKDIR /usr/local/src/

COPY composer.lock /usr/local/src/
COPY composer.json /usr/local/src/

RUN composer install --ignore-platform-reqs --optimize-autoloader \
--no-plugins --no-scripts --prefer-dist

FROM php:8.3.3-cli-alpine3.19 AS compile
RUN composer install \
--ignore-platform-reqs \
--optimize-autoloader \
--no-plugins \
--no-scripts \
--prefer-dist

FROM php:8.3.10-cli-alpine3.20 AS compile

ENV PHP_REDIS_VERSION=6.0.2 \
PHP_SWOOLE_VERSION=v5.1.2 \
PHP_MONGO_VERSION=1.16.1 \
PHP_XDEBUG_VERSION=3.3.2
ENV PHP_REDIS_VERSION="6.0.2" \
PHP_SWOOLE_VERSION="v5.1.3" \
PHP_MONGO_VERSION="1.16.1" \
PHP_XDEBUG_VERSION="3.3.2"

RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

RUN \
apk update \
&& apk add --no-cache postgresql-libs postgresql-dev make automake autoconf gcc g++ git brotli-dev linux-headers \
&& apk add --no-cache \
postgresql-libs \
postgresql-dev \
make \
automake \
autoconf \
gcc \
g++ \
git \
brotli-dev \
linux-headers \
&& docker-php-ext-install opcache pgsql pdo_mysql pdo_pgsql \
&& apk del postgresql-dev \
&& rm -rf /var/cache/apk/*
Expand Down Expand Up @@ -86,6 +100,7 @@ RUN echo extension=redis.so >> /usr/local/etc/php/conf.d/redis.ini
RUN echo extension=swoole.so >> /usr/local/etc/php/conf.d/swoole.ini
RUN echo extension=mongodb.so >> /usr/local/etc/php/conf.d/mongodb.ini
RUN echo extension=pcov.so >> /usr/local/etc/php/conf.d/pcov.ini
RUN echo extension=xdebug.so >> /usr/local/etc/php/conf.d/xdebug.ini

RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini"

Expand All @@ -100,14 +115,13 @@ COPY --from=mongodb /usr/local/lib/php/extensions/no-debug-non-zts-20230831/mong
COPY --from=pcov /usr/local/lib/php/extensions/no-debug-non-zts-20230831/pcov.so /usr/local/lib/php/extensions/no-debug-non-zts-20230831/
COPY --from=xdebug /usr/local/lib/php/extensions/no-debug-non-zts-20230831/xdebug.so /usr/local/lib/php/extensions/no-debug-non-zts-20230831/

# Add Source Code
COPY ./bin /usr/src/code/bin
COPY ./src /usr/src/code/src
COPY ./dev /usr/src/code/dev

# Add Debug Configs
RUN if [ "$DEBUG" == "true" ]; then cp /usr/src/code/dev/xdebug.ini /usr/local/etc/php/conf.d/xdebug.ini; fi
RUN if [ "$DEBUG" == "true" ]; then mkdir -p /tmp/xdebug; fi
RUN if [ "$DEBUG" = "true" ]; then cp /usr/src/code/dev/xdebug.ini /usr/local/etc/php/conf.d/xdebug.ini; fi
RUN if [ "$DEBUG" = "true" ]; then mkdir -p /tmp/xdebug; fi
RUN if [ "$DEBUG" = "false" ]; then rm -rf /usr/src/code/dev; fi
RUN if [ "$DEBUG" = "false" ]; then rm -f /usr/local/lib/php/extensions/no-debug-non-zts-20220829/xdebug.so; fi

Expand Down
10 changes: 5 additions & 5 deletions composer.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 10 additions & 5 deletions dev/xdebug.ini
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
zend_extension=xdebug
zend_extension = xdebug.so

[xdebug]
xdebug.mode=develop,debug,profile
xdebug.mode = develop,debug,profile
xdebug.start_with_request = yes
xdebug.use_compression=false
xdebug.client_host=host.docker.internal
xdebug.start_with_request=yes
xdebug.output_dir=/tmp/xdebug
xdebug.use_compression=false
xdebug.client_port = 9003
xdebug.log = /tmp/xdebug.log

xdebug.var_display_max_depth = 10
xdebug.var_display_max_children = 256
xdebug.var_display_max_data = 4096
65 changes: 55 additions & 10 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,25 +14,35 @@ services:
- ./tests:/usr/src/code/tests
- ./dev:/usr/src/code/dev
- ./phpunit.xml:/usr/src/code/phpunit.xml
ports:
- "8708:8708"
- ./dev/xdebug.ini:/usr/local/etc/php/conf.d/xdebug.ini

adminer:
image: adminer
container_name: utopia-adminer
restart: always
ports:
- "8760:8080"
- "8700:8080"
networks:
- database

postgres:
image: postgres:13
image: postgres:16.4
container_name: utopia-postgres
networks:
- database
ports:
- "8700:5432"
- "8701:5432"
environment:
POSTGRES_USER: root
POSTGRES_PASSWORD: password

postgres-mirror:
image: postgres:16.4
container_name: utopia-postgres-mirror
networks:
- database
ports:
- "8702:5432"
environment:
POSTGRES_USER: root
POSTGRES_PASSWORD: password
Expand All @@ -44,7 +54,18 @@ services:
networks:
- database
ports:
- "8701:3306"
- "8703:3306"
environment:
- MYSQL_ROOT_PASSWORD=password

mariadb-mirror:
image: mariadb:10.11
container_name: utopia-mariadb-mirror
command: mariadbd --max_allowed_packet=1G
networks:
- database
ports:
- "8704:3306"
environment:
- MYSQL_ROOT_PASSWORD=password

Expand All @@ -54,7 +75,7 @@ services:
networks:
- database
ports:
- "8702:27017"
- "8705:27017"
environment:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: example
Expand All @@ -65,7 +86,23 @@ services:
networks:
- database
ports:
- "8703:3307"
- "8706:3307"
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: default
MYSQL_USER: user
MYSQL_PASSWORD: password
MYSQL_TCP_PORT: 3307
cap_add:
- SYS_NICE

mysql-mirror:
image: mysql:8.0.33
container_name: utopia-mysql-mirror
networks:
- database
ports:
- "8707:3307"
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: default
Expand All @@ -76,10 +113,18 @@ services:
- SYS_NICE

redis:
image: redis:6.0-alpine
image: redis:7.4.1-alpine3.20
container_name: utopia-redis
ports:
- "8706:6379"
- "8708:6379"
networks:
- database

redis-mirror:
image: redis:7.4.1-alpine3.20
container_name: utopia-redis-mirror
ports:
- "8709:6379"
networks:
- database

Expand Down
14 changes: 7 additions & 7 deletions src/Database/Adapter.php
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ abstract class Adapter
*
* @return $this
*/
public function setDebug(string $key, mixed $value): self
public function setDebug(string $key, mixed $value): static
{
$this->debug[$key] = $value;

Expand All @@ -59,9 +59,9 @@ public function getDebug(): array
}

/**
* @return self
* @return static
*/
public function resetDebug(): self
public function resetDebug(): static
{
$this->debug = [];

Expand Down Expand Up @@ -197,7 +197,7 @@ public function getTenant(): ?int
* @param mixed $value
* @return $this
*/
public function setMetadata(string $key, mixed $value): self
public function setMetadata(string $key, mixed $value): static
{
$this->metadata[$key] = $value;

Expand Down Expand Up @@ -228,7 +228,7 @@ public function getMetadata(): array
*
* @return $this
*/
public function resetMetadata(): self
public function resetMetadata(): static
{
$this->metadata = [];

Expand Down Expand Up @@ -354,9 +354,9 @@ public function withTransaction(callable $callback): mixed
* @param string $event
* @param string $name
* @param ?callable $callback
* @return self
* @return static
*/
public function before(string $event, string $name = '', ?callable $callback = null): self
public function before(string $event, string $name = '', ?callable $callback = null): static
{
if (!isset($this->transformations[$event])) {
$this->transformations[$event] = [];
Expand Down
Loading
Loading