Image api rework #5260

Nutomic · 2024-12-13T10:54:02Z

Move all image related endpoints under /api/v4/image/
Image parameters and responses are now in crates/api_common/src/image.rs, no more exposing of pictrs data structures
Reorganize code in crates/routes/images
Only use adapt_request() for image upload, not for simple things like delete or health check
New endpoints POST /api/v4/account/avatar which directly takes image upload, instead of setting avatar url via /api/v4/account/settings/save
Also POST /api/v4/account/banner, POST /api/v4/community/icon, POST /api/v4/community/banner, POST /api/v4/site/icon and POST /api/v4/site/banner

Nutomic · 2024-12-17T10:47:08Z

src/api_routes_v3.rs

 use lemmy_utils::rate_limit::RateLimitCell;

 // Deprecated, use api v4 instead.
 // When removing api v3, we also need to rewrite all links in database with
-// `/api/v3/image_proxy` to use `/api/v4/image_proxy` instead.
+// `/api/v3/image_proxy` to use `/api/v4/image/proxy` instead.
 pub fn config(cfg: &mut ServiceConfig, rate_limit: &RateLimitCell) {


Indentation was changed in this file, use "hide whitespace" in github to review it.

Nutomic · 2024-12-18T10:59:42Z

Ready to review.

dessalines

Nice job, adding these as API actions is going to be much better.

We still need a way to clear images. I'd add some POST or DELETE endpoints, that just delete the images and return that item. IE POST /site/icon/delete, /community/banner/delete...
- Speaking of which, I see that this PR is the first one that adds HTTP DELETE. Maybe we should use that instead of POST for all our delete API actions.
Banners can't be square. So whether that means adding both a max height and max width, or just unfortunately not being able to resize them.
We're gonna need a DB migration to fix the history of all proxied images. So unfortunately that means doing a DB replace for comment.content, and post.body, where local = true. I believe the regular image endpoints stayed the same, so no need to correct those.

config/defaults.hjson

crates/routes/src/images/upload.rs

crates/api/src/local_user/save_settings.rs

crates/routes/src/images/utils.rs

crates/utils/src/settings/structs.rs

crates/routes/src/images/upload.rs

crates/utils/src/utils/markdown/image_links.rs

src/api_routes_v4.rs

Nothing4You · 2024-12-19T23:59:19Z

is there a reason to expose the pict-rs delete token to the user?

i guess this was historically exposed to the user during upload before it got used in the media listing, but there isn't really any need to keep that, now that we have the media listing.
instead, the deletion api should probably just accept the image url itself and fetch the delete token from the user upload table, which can be combined with verifying that either the user is an admin or the user is the uploader.
for older uploads, in the unlikely case that someone recorded deletion tokens prior to lemmy recording which user the upload is associated with, a compatibility api could exist that allows deletion when the token is provided and the upload does not exist in the table.

Nutomic · 2025-01-10T13:45:31Z

Addressed all the review comments. However the api tests are consistently failing and Im not sure why. Seems to be a problem with deleteAllImages() but I cant reproduce it locally.

dessalines

I think the deleteImage is working fine, but look at https://github.com/LemmyNet/lemmy/blob/image-api-rework/api_tests/src/image.spec.ts#L61 , and also L92 .

Its weird because everything's passing locally for me too when I run the federation tests, but in a dockerized environment, this fetch might be timing out.

dessalines · 2025-01-10T18:14:45Z

.woodpecker.yml

@@ -95,15 +95,6 @@ steps:
    when:
      - event: pull_request

-  cargo_clippy:


Once we get the federation tests figured out, these all need to get re-added back in.

dessalines · 2025-01-10T18:26:03Z

api_tests/run-federation-test.sh

@@ -11,7 +11,7 @@ killall -s1 lemmy_server || true
 popd

 pnpm i
-pnpm api-test || true
+pnpm api-test-image || true


Also will need to get changed back.

dessalines · 2025-01-10T18:32:00Z

crates/routes/src/images/delete.rs

Looks good.

crates/routes/src/images/mod.rs

Nutomic · 2025-01-13T13:37:17Z

Found the problem, purges would delete the image from pictrs but not the row from LocalImage table. So it would try to delete the image a second time in afterAll hook using deleteAllImages. Previously this worked fine, but now there is a stricter success check. Not sure why it was passing locally though.

Nutomic added 3 commits December 12, 2024 15:59

Split image endpoints into API v3 and v4

56ab67f

Move into subfolders

05843fb

Upload avatar endpoint and other changes

cfa866a

Nutomic mentioned this pull request Dec 13, 2024

Rework image api LemmyNet/lemmy-js-client#425

Merged

Various other changes

d252be2

fixes #1772 fixes #4001

Nutomic force-pushed the image-api-rework branch from b38a9ee to d252be2 Compare December 13, 2024 14:06

Nutomic added 2 commits December 13, 2024 15:13

clippy

e815778

config options

60ba7af

Nutomic force-pushed the image-api-rework branch from 09b06f0 to 60ba7af Compare December 13, 2024 14:26

fix ts bindings

7c771d2

Nutomic force-pushed the image-api-rework branch from f66e392 to 7c771d2 Compare December 13, 2024 14:32

fix api tests

8ae4b40

Nutomic commented Dec 17, 2024

View reviewed changes

Nutomic added 8 commits December 17, 2024 16:10

Add option to disable image upload (fixes #1118)

b0d4bdb

split files into upload, download

55b8ace

move sitemap to top level, not in api

460ccc8

simplify code

5f06195

add upload user banner

5f49b2a

community icon/banner

a6f7e76

site icon/banner

ada8f1b

update js client

4951aa0

Nutomic marked this pull request as ready for review December 18, 2024 10:58

Nutomic requested review from dessalines, phiresky, dullbananas and SleeplessOne1917 as code owners December 18, 2024 10:58

dessalines reviewed Dec 19, 2024

View reviewed changes

Nothing4You self-requested a review December 19, 2024 22:31

Nutomic added 4 commits January 10, 2025 09:51

dont use static client

7cbbb9a

fix api tests

83c3304

shear

f040f91

proxy pictrs in request.rs (fixes #5270)

6123659

Nutomic force-pushed the image-api-rework branch from 9fc02be to 6123659 Compare January 10, 2025 10:10

Nutomic added 4 commits January 10, 2025 11:24

clippy

ec74669

try to fix api tests

ee091c7

skip api tests

76f4508

create user

c9dfbfe

Nutomic force-pushed the image-api-rework branch from 81db6c9 to c9dfbfe Compare January 10, 2025 11:43

Nutomic added 2 commits January 10, 2025 12:58

debug

7f78275

dbg

2e7a961

dessalines reviewed Jan 10, 2025

View reviewed changes

dullbananas approved these changes Jan 12, 2025

View reviewed changes

Nutomic added 3 commits January 13, 2025 11:22

test

1c12993

image

e484489

run another

a45fcc5

Nutomic force-pushed the image-api-rework branch from 99e7352 to a45fcc5 Compare January 13, 2025 10:49

fixed?

4b3f2f5

Nutomic force-pushed the image-api-rework branch from 01b3e80 to 4b3f2f5 Compare January 13, 2025 11:18

Merge branch 'main' into image-api-rework

4b94043

Nutomic force-pushed the image-api-rework branch from 5ba5391 to 4b94043 Compare January 13, 2025 11:33

Nutomic added 2 commits January 13, 2025 12:41

clippy

79cdb83

fix

2be56b7

fix health check

47638cc

dessalines approved these changes Jan 13, 2025

View reviewed changes

dessalines merged commit a91a03a into main Jan 13, 2025
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Image api rework #5260

Image api rework #5260

Nutomic commented Dec 13, 2024 •

edited

Loading

Nutomic Dec 17, 2024

Nutomic commented Dec 18, 2024

dessalines left a comment •

edited

Loading

Nothing4You commented Dec 19, 2024

Nutomic commented Jan 10, 2025

dessalines left a comment

dessalines Jan 10, 2025

dessalines Jan 10, 2025

dessalines Jan 10, 2025

Nutomic commented Jan 13, 2025

Image api rework #5260

Image api rework #5260

Conversation

Nutomic commented Dec 13, 2024 • edited Loading

Nutomic Dec 17, 2024

Choose a reason for hiding this comment

Nutomic commented Dec 18, 2024

dessalines left a comment • edited Loading

Choose a reason for hiding this comment

Nothing4You commented Dec 19, 2024

Nutomic commented Jan 10, 2025

dessalines left a comment

Choose a reason for hiding this comment

dessalines Jan 10, 2025

Choose a reason for hiding this comment

dessalines Jan 10, 2025

Choose a reason for hiding this comment

dessalines Jan 10, 2025

Choose a reason for hiding this comment

Nutomic commented Jan 13, 2025

Nutomic commented Dec 13, 2024 •

edited

Loading

dessalines left a comment •

edited

Loading