Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: occ db:convert-type broken since PrimaryReadReplicaConnection support added #45257

Open
5 of 8 tasks
Tracked by #4564
ksmonkey123 opened this issue May 10, 2024 · 62 comments · May be fixed by #46908 or #48666
Open
5 of 8 tasks
Tracked by #4564

[Bug]: occ db:convert-type broken since PrimaryReadReplicaConnection support added #45257

ksmonkey123 opened this issue May 10, 2024 · 62 comments · May be fixed by #46908 or #48666

Comments

@ksmonkey123
Copy link

⚠️ This issue respects the following points: ⚠️

Bug description

The OCC-Command db:convert-type seems to ignore the DB properties (type, user, host, database, port, password) provided through the CLI and instead operates on the database configured in config.php

I believe this may be related to the commit 79c4986 as this reworked the parameter handling in ConnectionFactory.php.

If I read this correctly, the default parameters are now always generated based on the system config instead of the passed arguments.

Running db:convert-type --clean-schema drops the tables in the "old" database.

Steps to reproduce

  1. Have Nextcloud 29.0.0 running with a MySQL database, and an empty PostgreSQL database
  2. run php occ db:convert-type --password=<PASSWORD> --port=5432 pgsql nextcloud_app postgres nextcloud (note: the host postgres is a Kubernetes service name)
  3. The command executes no matter if the Postgres Database is running or not, the command throws Constraint Violation errors in the table oc_accounts, I assume because it tries to migrate the content of the old database into itself.

Expected behavior

db:convert-type

Installation method

Community Docker image

Nextcloud Server version

29

Operating system

Debian/Ubuntu

PHP engine version

None

Web server

None

Database engine version

MySQL

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other

Configuration report

{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "192.168.178.32",
            "cloud.awae.ch"
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "29.0.0.19",
        "overwrite.cli.url": "https:\/\/cloud.awae.ch",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "overwriteprotocol": "https",
        "theme": "",
        "loglevel": 2,
        "maintenance": false,
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_smtpauth": 1,
        "mail_smtpauthtype": "LOGIN",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpsecure": "tls",
        "default_language": "de",
        "default_phone_region": "CH",
        "app_install_overwrite": [
            "polls"
        ]
    }
}

List of activated Apps

Enabled:
  - activity: 2.21.1
  - circles: 29.0.0-dev
  - cloud_federation_api: 1.12.0
  - comments: 1.19.0
  - contactsinteraction: 1.10.0
  - dashboard: 7.9.0
  - dav: 1.30.1
  - federatedfilesharing: 1.19.0
  - federation: 1.19.0
  - files: 2.1.0
  - files_downloadlimit: 2.0.0
  - files_pdfviewer: 2.10.0
  - files_reminders: 1.2.0
  - files_sharing: 1.21.0
  - files_trashbin: 1.19.0
  - files_versions: 1.22.0
  - firstrunwizard: 2.18.0
  - logreader: 2.14.0
  - lookup_server_connector: 1.17.0
  - nextcloud_announcements: 1.18.0
  - notes: 4.10.0
  - notifications: 2.17.0
  - oauth2: 1.17.0
  - password_policy: 1.19.0
  - photos: 2.5.0
  - polls: 7.0.3
  - privacy: 1.13.0
  - provisioning_api: 1.19.0
  - recommendations: 2.1.0
  - related_resources: 1.4.0
  - serverinfo: 1.19.0
  - settings: 1.12.0
  - sharebymail: 1.19.0
  - support: 1.12.0
  - survey_client: 1.17.0
  - systemtags: 1.19.0
  - text: 3.10.0
  - theming: 2.4.0
  - twofactor_backupcodes: 1.18.0
  - updatenotification: 1.19.1
  - viewer: 2.3.0
  - workflowengine: 2.11.0
Disabled:
  - admin_audit: 1.19.0
  - bruteforcesettings: 2.9.0 (installed 2.4.0)
  - encryption: 2.17.0
  - extract: 1.3.6 (installed 1.3.6)
  - files_external: 1.21.0
  - files_rightclick: 0.15.1 (installed 1.6.0)
  - maps: 1.1.0 (installed 1.1.0)
  - suspicious_login: 7.0.0
  - twofactor_totp: 11.0.0-dev
  - user_ldap: 1.20.0
  - user_status: 1.9.0 (installed 1.4.0)
  - weather_status: 1.9.0 (installed 1.4.0)

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

No response

Additional info

  • Docker Image: nextcloud:29.0.0
  • MySQL Database: mysql:8.4.0
  • Postgres Databse: postgres:16.3
@ksmonkey123 ksmonkey123 added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels May 10, 2024
@ksmonkey123
Copy link
Author

Logging the Parameter $additionalConnectionParams in ConnectionFactory.php on Line 166 shows the following values for the "fromDB" and the "toDB":

FromDB

Array
(
    [adapter] => OC\DB\AdapterMySQL
    [charset] => utf8mb4
    [driver] => pdo_mysql
    [wrapperClass] => OC\DB\Connection
    [driverOptions] => Array
        (
            [1005] => 1
        )

    [user] => nextcloud
    [password] => *************
    [host] => nextcloud-db
    [dbname] => nextcloud
    [tablePrefix] => oc_
    [sqlite.journal_mode] => WAL
    [defaultTableOptions] => Array
        (
            [collate] => utf8mb4_bin
            [charset] => utf8mb4
            [tablePrefix] => oc_
        )

    [primary] => Array
        (
            [adapter] => OC\DB\AdapterMySQL
            [charset] => utf8mb4
            [driver] => pdo_mysql
            [wrapperClass] => OC\DB\Connection
            [driverOptions] => Array
                (
                    [1005] => 1
                )

            [user] => nextcloud
            [password] => ****************
            [host] => nextcloud-db
            [dbname] => nextcloud
            [tablePrefix] => oc_
            [sqlite.journal_mode] => WAL
            [defaultTableOptions] => Array
                (
                    [collate] => utf8mb4_bin
                    [charset] => utf8mb4
                    [tablePrefix] => oc_
                )

        )

    [replica] => Array
        (
            [0] => Array
                (
                    [adapter] => OC\DB\AdapterMySQL
                    [charset] => utf8mb4
                    [driver] => pdo_mysql
                    [wrapperClass] => OC\DB\Connection
                    [driverOptions] => Array
                        (
                            [1005] => 1
                        )

                    [user] => nextcloud
                    [password] => *************
                    [host] => nextcloud-db
                    [dbname] => nextcloud
                    [tablePrefix] => oc_
                    [sqlite.journal_mode] => WAL
                    [defaultTableOptions] => Array
                        (
                            [collate] => utf8mb4_bin
                            [charset] => utf8mb4
                            [tablePrefix] => oc_
                        )

                )

        )

)

ToDB

Array
(
    [adapter] => OC\DB\AdapterMySQL
    [charset] => utf8mb4
    [driver] => pdo_mysql
    [wrapperClass] => OC\DB\Connection
    [driverOptions] => Array
        (
            [1005] => 1
        )

    [user] => nextcloud_app
    [password] => ***************
    [host] => postgres
    [dbname] => nextcloud
    [tablePrefix] => oc_
    [sqlite.journal_mode] => WAL
    [defaultTableOptions] => Array
        (
            [collate] => utf8mb4_bin
            [charset] => utf8mb4
            [tablePrefix] => oc_
        )

    [primary] => Array
        (
            [adapter] => OC\DB\AdapterMySQL
            [charset] => utf8mb4
            [driver] => pdo_mysql
            [wrapperClass] => OC\DB\Connection
            [driverOptions] => Array
                (
                    [1005] => 1
                )

            [user] => nextcloud
            [password] => *************
            [host] => nextcloud-db
            [dbname] => nextcloud
            [tablePrefix] => oc_
            [sqlite.journal_mode] => WAL
            [defaultTableOptions] => Array
                (
                    [collate] => utf8mb4_bin
                    [charset] => utf8mb4
                    [tablePrefix] => oc_
                )

        )

    [replica] => Array
        (
            [0] => Array
                (
                    [adapter] => OC\DB\AdapterMySQL
                    [charset] => utf8mb4
                    [driver] => pdo_mysql
                    [wrapperClass] => OC\DB\Connection
                    [driverOptions] => Array
                        (
                            [1005] => 1
                        )

                    [user] => nextcloud
                    [password] => **********
                    [host] => nextcloud-db
                    [dbname] => nextcloud
                    [tablePrefix] => oc_
                    [sqlite.journal_mode] => WAL
                    [defaultTableOptions] => Array
                        (
                            [collate] => utf8mb4_bin
                            [charset] => utf8mb4
                            [tablePrefix] => oc_
                        )

                )

        )

    [port] => 5432
)

It is notable that in the second case (which should be the postgres-database configured via the CLI arguments), the host, dbname and port in the top-level array are set correctly from the CLI, Both the primary and the replica fields and even some top-level fields obviously use the DB-configs from config.php

@joshtrichards
Copy link
Member

joshtrichards commented May 11, 2024

Possibly related: #45097

I don't have time to look tonight, but your analysis seems sound (and, given the added evidence from the log output, even more so).

Cc: @juliushaertl

@intersectRaven
Copy link

I can verify but can't contribute logs. I used the --clear-schema option in my attempt and my original MariaDB DB got deleted. Good thing I have backups.

@joshtrichards joshtrichards added 1. to develop Accepted and waiting to be taken care of and removed 0. Needs triage Pending check for reproducibility or if it fits our roadmap labels May 12, 2024
@Delagen
Copy link

Delagen commented May 14, 2024

I modified https://github.com/nextcloud/server/blob/master/core/Command/Db/ConvertType.php#L247
to

$connectionParams = $this->connectionFactory->getDefaultConnectionParams($type);

and seems all converted

Also have to remove mysql.utf8mb4 parameter from config.php to perform conversion to remove client_encoding from params of DbFactory

@HammyHavoc
Copy link

Got hit by this a week ago. Is that particular Nextcloud instance completely screwed?

@joshtrichards
Copy link
Member

joshtrichards commented May 17, 2024

Yes, unfortunately. 😢 The end result is the parameters get swapped so it likely dropped the source database.

Fortunately, recovery should be a simple matter of restoring from backup (just the db portion): https://docs.nextcloud.com/server/latest/admin_manual/maintenance/restore.html#restore-database (or whatever other mechanism you utilize in your environment for database backup/recovery)

@joshtrichards
Copy link
Member

@Delagen Mind pushing that as PR? I know there may be additional changes needed, but it'll start things in motion.

joshtrichards added a commit that referenced this issue May 18, 2024
Needed until #45257 is addressed to prevent data loss

Signed-off-by: Josh <josh.t.richards@gmail.com>
@Delagen
Copy link

Delagen commented May 18, 2024

@joshtrichards I don't mind that my changes solve the problem correctly. It only makes possible for me successful conversion from MySQL to Postgres

@chimpboy
Copy link

I have just experienced this problem on a new installation.

@joshtrichards joshtrichards mentioned this issue May 22, 2024
24 tasks
backportbot bot pushed a commit that referenced this issue May 22, 2024
Needed until #45257 is addressed to prevent data loss

Signed-off-by: Josh <josh.t.richards@gmail.com>
@rotdrop
Copy link
Contributor

rotdrop commented May 24, 2024

Would not be it the simplest solution, if the ConnectionFactory::getConnection() method would first check if the $additionalConnectionParams argument contains explicit connection parameters and in that case simply skip that fancy primary / replica stuff? At least for the case were the explicitly specified connection differs from the "internal" database connection configured in config/config.php. In principle this should restore the previous behaviour when using the factory for alternate database connections.

SebastianKrupinski pushed a commit that referenced this issue May 27, 2024
Needed until #45257 is addressed to prevent data loss

Signed-off-by: Josh <josh.t.richards@gmail.com>
@szaimen
Copy link
Contributor

szaimen commented May 31, 2024

Hi, is this #45013 fixing the problem?

@CorneliousJD
Copy link

Did we really shift this to milestone 31...? After people were instructed to wait "until the next maintenance release" for several months? 😅

I continue to have issues w/ MariaDB/MySQL breaking and corrupting itself specifically just w/ the nextcloud database, I'd really like to move to Postgres but at this point I think I may just abandon nextcloud alltogether -- I've been waiting for 4 months for this to be corrected and it keeps being pushed back.

I realized the developers of Nextcloud owe me absolutely nothing, but this has still been a frustrating experience.

@joshtrichards
Copy link
Member

Did we really shift this to milestone 31...? After people were instructed to wait "until the next maintenance release" for several months? 😅

It'll be a bug fix. Any fix will be backported to whatever earlier majors need it. It can't have a v30 milestone since it's already released. Development occurs against the master branch (upcoming v31 at the moment). Bug fixes get backported after they're merged there (and then land in maintenance releases for earlier majors).

I realized the developers of Nextcloud owe me absolutely nothing, but this has still been a frustrating experience.

I'm sorry it has been a frustrating experience. I'm hopeful someone more familiar that area of code than me (or with more time on their hands to get familiar with it) will take a closer took soon.

If you are a customer of Nextcloud GmbH, you can tell your contacts there through the appropriate channels that this Issue is a priority for you as a customer and ask them to fix it on your behalf. Otherwise you're at the mercy of whenever someone gets around to it in the project.

@CorneliousJD I'm curious about your corruption situation, but this isn't the place to follow-up on it. Have you posted about your corruption issues somewhere (such as on the community help forum) where I can take a look and gain some further context?

@joshtrichards joshtrichards changed the title [Bug]: db:convert-type uses systemConfig instead of command line parameters for connection to new target database [Bug]: occ db:convert-type broken since PrimaryReadReplicaConnection support added Sep 19, 2024
@CorneliousJD
Copy link

@CorneliousJD I'm curious about your corruption situation, but this isn't the place to follow-up on it. Have you posted about your corruption issues somewhere (such as on the community help forum) where I can take a look and gain some further context?

I have not posted elsewhere about it in great detail, but it will randomly have corrupt tables and it only happens within the nextcloud database. I had previously run w/ postgres w/out these issues so I wanted to revert back to postgres and see how things go before abandoning Nextcloud, but setting up a whole new instance w/out doing a db convert just isn't an option for me right now.

Before I dive too deep into it I am just waiting for the Postgres conversion to be back in the cards so I Can test that and see how it goes with that setup.

@Delogon
Copy link

Delogon commented Sep 20, 2024

For me the combined solutions of @Queuecumber, @normen and @lightonflux worked, and I was able to convert my mariadb to postgresql.

So combined it would look like this:

In lib/private/DB/ConnectionFactory.php:

-               $connectionParams = $this->createConnectionParams('', $additionalConnectionParams);
+               $connectionParams = $this->createConnectionParams('', $additionalConnectionParams, $type);
-       public function createConnectionParams(string $configPrefix = '', array $additionalConnectionParams = []) {
-               $type = $this->config->getValue('dbtype', 'sqlite');
+       public function createConnectionParams(string $configPrefix = '', array $additionalConnectionParams = [], $type = '') {
+               if ($type === '') {
+                       $type = $this->config->getValue('dbtype', 'sqlite');
+               }
+               if ($type === '') {
+                       $type = $this->config->getValue('dbtype', 'sqlite');
+               }
-               if ($this->config->getValue('mysql.utf8mb4', false)) {
+               if ($this->config->getValue('mysql.utf8mb4', false) && $type==="mysql") {

In core/Command/Db/ConvertType.php

- $connectionParams = $this->connectionFactory->createConnectionParams();
+ $connectionParams = $this->connectionFactory->createConnectionParams('', [], $type);

Maybe someone else could also check this?

@cburlacu
Copy link

@Delogon It worked for me (v30). Finally I was able to migrate to pgsql. At the first glance, all seems to be OK. Thanks!

@Investigamer
Copy link

@Delogon this is the way, worked great sqlite -> psql. Cheers and good luck everyone.

@cektom
Copy link

cektom commented Sep 28, 2024

@Delogon Thanks for the summary! Looks good on my end. Migrated from sqlite to mysql (mariadb Ver 15.1 Distrib 10.6.18-MariaDB) with the latest docker image (Nextcloud 30.0.0)

@KoMa1012
Copy link

KoMa1012 commented Oct 9, 2024

Still getting the error message:

In ConvertType.php line 158:

  This command is temporarily disabled (until the next maintenance release).

with command php occ db:convert-type --all-apps --password mypassword pgsql ncadmin postgresql17 nextcloud_db Running 30.0.0
Image

My Error Log is showing this message quite often:

RuntimeException image not found: image:apps/whiteboard.svg webroot: serverroot:/var/www/html

Another Error Log I'm seeing 4 times (but never occured after again):

Undefined array key 1 at /var/www/html/custom_apps/mail/vendor/phpmailer/dkimvalidator/src/Validator.php#86

I'm running mariadb and want to switch to psql.

Any ideas if I'm doing something wrong or if this case has not been implemented yet?

@leon1995
Copy link

leon1995 commented Oct 9, 2024

@KoMa1012 the issue has not been fixed officially. you can try the workaround provided by @Delogon.
regarding the whiteboard.svg checkout #47985

@KoMa1012
Copy link

KoMa1012 commented Oct 9, 2024

@Delogon I'm getting an error message:

In TableDoesNotExist.php line 16:

  There is no table with name "ncadmin.oc_group_folders_acl" in the schema.

I'm not sure if relevant, but this table is empty for me.
SQL Commands to generate this table in mariadb:

SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
START TRANSACTION;
SET time_zone = "+00:00";

CREATE TABLE `oc_group_folders_acl` (
  `acl_id` bigint(20) NOT NULL,
  `fileid` bigint(20) NOT NULL,
  `mapping_type` varchar(16) NOT NULL,
  `mapping_id` varchar(64) NOT NULL,
  `mask` smallint(6) NOT NULL,
  `permissions` smallint(6) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
ALTER TABLE `oc_group_folders_acl`
  ADD PRIMARY KEY (`acl_id`),
  ADD UNIQUE KEY `groups_folder_acl_unique` (`fileid`,`mapping_type`,`mapping_id`),
  ADD KEY `groups_folder_acl_mapping` (`mapping_type`,`mapping_id`);
ALTER TABLE `oc_group_folders_acl`
  MODIFY `acl_id` bigint(20) NOT NULL AUTO_INCREMENT;
COMMIT;

removing the group folder plugin solves the issue... however, now the group folders are gone, but this I can survive.

@WhiskyDelta
Copy link

@Delogon Since your fix seems to work, do you want to make a pull request? If not, I might be able to do it.

@hagak
Copy link

hagak commented Oct 14, 2024

So I am also trying to move from mariadb to postgres and on nextcloud v30. I did the fix that @Delogon posted and that got me further. However i end up with this during the conversion.

- oc_properties
 10/10 [============================] 100% < 1 sec/< 1 sec
In ExceptionConverter.php line 87:

  An exception occurred while executing a query: SQLSTATE[22021]: Character not in repertoire: 7 ERROR:  invalid byte sequence for encoding "UTF8": 0x00
  CONTEXT:  unnamed portal parameter $5


In Exception.php line 28:

  SQLSTATE[22021]: Character not in repertoire: 7 ERROR:  invalid byte sequence for encoding "UTF8": 0x00
  CONTEXT:  unnamed portal parameter $5


In Statement.php line 130:

  SQLSTATE[22021]: Character not in repertoire: 7 ERROR:  invalid byte sequence for encoding "UTF8": 0x00
  CONTEXT:  unnamed portal parameter $5

Any ideas?

@hagak
Copy link

hagak commented Oct 14, 2024

So I am also trying to move from mariadb to postgres and on nextcloud v30. I did the fix that @Delogon posted and that got me further. However i end up with this during the conversion.

- oc_properties
 10/10 [============================] 100% < 1 sec/< 1 sec
In ExceptionConverter.php line 87:

  An exception occurred while executing a query: SQLSTATE[22021]: Character not in repertoire: 7 ERROR:  invalid byte sequence for encoding "UTF8": 0x00
  CONTEXT:  unnamed portal parameter $5


In Exception.php line 28:

  SQLSTATE[22021]: Character not in repertoire: 7 ERROR:  invalid byte sequence for encoding "UTF8": 0x00
  CONTEXT:  unnamed portal parameter $5


In Statement.php line 130:

  SQLSTATE[22021]: Character not in repertoire: 7 ERROR:  invalid byte sequence for encoding "UTF8": 0x00
  CONTEXT:  unnamed portal parameter $5

Any ideas?

Found this related bug and it seems to be the same. Looks like NC has a bit more work to do on the postgres side

#37754

@lightonflux
Copy link

@hagak what is your encoding in mysql?

@hagak
Copy link

hagak commented Oct 15, 2024

@hagak what is your encoding in mysql?

UTF8 the issue from what i gather in that other related bug that i apparently forgot to put the link to. Shows that it is due to postgres schema using a TEXT column instead of a Bytea . I believe there is a PR to resolve that as well

Particular data in the oc_properties table with a valuetype=3. I only had one entry that was causing this so I just manually deleted that row and the convert finished.

Add the link to the issue number #37754

@Delogon
Copy link

Delogon commented Oct 23, 2024

I created the pull request #48666 but tbh I don't know if I did that correct, since creation nothing has happened. @joshtrichards or @juliusknorr maybe you could tell me what to do, to get it merged.

@daks
Copy link

daks commented Oct 25, 2024

I want to migrate from a standalone nginx/php-fpm/mariadb Nextcloud installation to AIO "The official Nextcloud installation method" but I'm blocked by this issue since several weeks.

Is the patch provided by @Delogon here #45257 (comment) or in the PR #48666 safe to use?
Is there any chance to see the issue solved in the following releases?

@jiriks74
Copy link

Is the patch provided by @Delogon here #45257 (comment) or in the PR #48666 safe to use?

I cannot answer that but even if it is you should make a database backup anyway

@daks
Copy link

daks commented Oct 26, 2024

Is the patch provided by @Delogon here #45257 (comment) or in the PR #48666 safe to use?

I cannot answer that but even if it is you should make a database backup anyway

Of course. I'm just worried to use it, don't detect any problem, and start using my new postgres DB without detecting a possible data loss (or other problem) :-/

@thespielplatz
Copy link

Is the patch provided by @Delogon here #45257 (comment) or in the PR #48666 safe to use? Is there any chance to see the issue solved in the following releases?

I just applied the PR [#48666] manually on nextcloud 29.0.9-fpm-alpine and successfully converted from mysql to postgres. So far everyhting looks good.

@jiriks74
Copy link

I'll be, probably, trying to migrate this evening. I'll report how it goes.

@jiriks74
Copy link

Well it failed while creating schema for me:
Image

But that's probably some other issue.

@Delogon
Copy link

Delogon commented Nov 15, 2024

Well it failed while creating schema for me: Image

But that's probably some other issue.

That bug is already known. A potential solution would be to disable the group folder plugin, but I do not know how suitable that is for you.

@Delogon I'm getting an error message:

In TableDoesNotExist.php line 16:

  There is no table with name "ncadmin.oc_group_folders_acl" in the schema.

I'm not sure if relevant, but this table is empty for me. SQL Commands to generate this table in mariadb:

SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
START TRANSACTION;
SET time_zone = "+00:00";

CREATE TABLE `oc_group_folders_acl` (
  `acl_id` bigint(20) NOT NULL,
  `fileid` bigint(20) NOT NULL,
  `mapping_type` varchar(16) NOT NULL,
  `mapping_id` varchar(64) NOT NULL,
  `mask` smallint(6) NOT NULL,
  `permissions` smallint(6) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
ALTER TABLE `oc_group_folders_acl`
  ADD PRIMARY KEY (`acl_id`),
  ADD UNIQUE KEY `groups_folder_acl_unique` (`fileid`,`mapping_type`,`mapping_id`),
  ADD KEY `groups_folder_acl_mapping` (`mapping_type`,`mapping_id`);
ALTER TABLE `oc_group_folders_acl`
  MODIFY `acl_id` bigint(20) NOT NULL AUTO_INCREMENT;
COMMIT;

removing the group folder plugin solves the issue... however, now the group folders are gone, but this I can survive.

@jiriks74
Copy link

jiriks74 commented Nov 15, 2024

That bug is already known. A potential solution would be to disable the group folder plugin, but I do not know how suitable that is for you.

That crossed my mind but Nextcloud kinda doesn't want to do it. I don't use the plugin but it was disabled but installed.

While writing this I managed to removed it with occ app:remove and I started the conversion.

EDIT: God Talk is broken now...
EDIT2: Nuked spreed as I didn't use it anyway. The little history there was is basically useless to me.
EDIT3: Migration complete. I'll test the new database now.

@jiriks74
Copy link

jiriks74 commented Nov 16, 2024

Ok. The database migration itself went smoothly after disabling group folders. Importing data into AIO is the part that was full of pain.

It had issues with the database dump, then the data files, then 3 apps that I had installed, ... But it seems to be done now.

MAKE BACKUPS and keep them! Even after you migrate and everything seems well you might find something broken. I, for example, had all my calendar entries missing and I had to redump the PostgreSQL database, change the data directory within it and the owner of the database and then re-import it again to get it back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment