Gitea attempts to include current backup file IN the backup

[TheFriendlyCoder]

Description

Based on my ad-hoc testing of the gitea dump command, I think the expected behaviour is that the default dump file created by the tool is supposed to be put in the current working folder. One way I found this out was that, while I was testing various options trying to get the dump tool to work (see #27423) I had changed directories into the config folder for gitea before running the dump operation:

cd /var/snap/gitea/common/conf
gitea dump

After waiting for a few minutes for the dump to complete, which was much longer than any of my previous attempts, I started to notice the performance of my system was starting to slow down. Closer examination of the memory, cpu and disk space showed that the dump was consuming tremendous amounts of resources. So I killed the dump operation and looked in the current working folder - there was a zip file there generated by the dump but it was many GB in size ... many times larger than the entire working folder for gitea.

What I think happened in this case is that the dump operation started creating the zip file in the config folder, but then it indexed the contents of the folder, picked up the backup file itself, and then tried to re-compress the file into the zip file, in an infinite recursion sort of way.

If I am correct, this is a rather nasty bug that could lead the system into a crash state fairly easily. It may even been abused as a potential exploit for a DoS type attack. Either way it probably should get fixed.

Gitea Version

Gitea version 1.20.5 built with GNU Make 4.3, go1.21.1 : bindata, sqlite, sqlite_unlock_notify, pam, cert

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

git version 2.34.1

Operating System

Ubuntu 22.04.3 LTS

How are you running Gitea?

From the console

Database

SQLite

[lng2020]

The problem is when using snap package, the /var/snap/gitea/common is the CUSTOM_PATH. So when the gitea dumper scan the CUSTOM_PATH folder, it includes the gitea-dumpxxxx.zip file(itself) and somehow fails into infinite recursion.
I think this is where the error occurs(https://github.com/go-gitea/gitea/blob/f2b2608a86b78f50de5d7a62b07040da1fbcc4ef/cmd/dump.go#L481C12-L481C12)
Probably add the check like the code above will solve the problem.( haven't test it yet)

[wxiaoguang]

There is also a design bug. There are many issues reporting that the dumped "zip" would be included into the dump data again.

[Xulunix]

This also affects Gitea running inside kubernetes.
Depending on your exact configuration, the only location to store larger amounts of data inside a pod are the volumes mounted to it, which in case of Gitea is only the data volume itself.
With gitea trying to backup the backup results in this:

gitea-677689f796-8pzzb:/var/lib/gitea$ gitea dump -c /data/gitea/conf/app.ini -f /data/dump_20231017.zip
2023/10/17 11:50:54 ...les/setting/cache.go:75:loadCacheFrom() [I] Cache Service Enabled
2023/10/17 11:50:54 ...les/setting/cache.go:90:loadCacheFrom() [I] Last Commit Cache Service Enabled
2023/10/17 11:50:54 ...s/setting/session.go:74:loadSessionFrom() [I] Session Service Enabled
2023/10/17 11:50:54 ...es/setting/mailer.go:237:loadMailerFrom() [I] Mail Service Enabled
2023/10/17 11:50:54 ...es/setting/mailer.go:259:loadNotifyMailFrom() [I] Notify Mail Service Enabled
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/attachments
2023/10/17 11:50:54 ...s/storage/storage.go:166:initAvatars() [I] Initialising Avatar storage with type: local
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/avatars
2023/10/17 11:50:54 ...s/storage/storage.go:192:initRepoAvatars() [I] Initialising Repository Avatar storage with type: local
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/repo-avatars
2023/10/17 11:50:54 ...s/storage/storage.go:186:initLFS() [I] Initialising LFS storage with type: local
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/lfs/data
2023/10/17 11:50:54 ...s/storage/storage.go:198:initRepoArchives() [I] Initialising Repository Archive storage with type: local
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/repo-archive
2023/10/17 11:50:54 ...s/storage/storage.go:208:initPackages() [I] Initialising Packages storage with type: local
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/packages
2023/10/17 11:50:54 ...s/storage/storage.go:219:initActions() [I] Initialising Actions storage with type: local
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/actions_log
2023/10/17 11:50:54 ...s/storage/storage.go:223:initActions() [I] Initialising ActionsArtifacts storage with type: local
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/actions_artifacts
2023/10/17 11:50:54 cmd/dump.go:258:runDump() [I] Dumping local repositories... /data/git/gitea-repositories
2023/10/17 11:53:15 cmd/dump.go:299:runDump() [I] Dumping database...
2023/10/17 11:53:28 cmd/dump.go:311:runDump() [I] Adding custom configuration file from /data/gitea/conf/app.ini
2023/10/17 11:53:28 cmd/dump.go:327:runDump() [I] Custom dir /data/gitea is inside data dir /data, skipped
2023/10/17 11:53:28 cmd/dump.go:339:runDump() [I] Packing data directory.../data
Failed to include data directory: data/dump_20231017.zip: copying contents: write /data/dump_20231017.zip: no space left on device
2023/10/17 11:55:24 cmd/dump.go:166:fatal() [F] Failed to include data directory: data/dump_20231017.zip: copying contents: write /data/dump_20231017.zip: no space left on device```

[wxiaoguang]

It's likely the dump output file is in the Gitea's data directory, then it packs itself again and again.

Some cases could be fixed by #30240 , while there might be some still simliar cases.

So you could use --file option to set the output file to a non-Gitea directory