Code

feat: integrate opendal storage#11508

Code

Merged

laipz8200 merged 30 commits intomainlanggenius/dify:mainfrom feat/integrate-opendal-storagelanggenius/dify:feat/integrate-opendal-storageCopy head branch name to clipboardDec 11, 2024

+798-597

Commits

Commits on Dec 9, 2024

• feat: add opendal as a dependency

  Show description for aeb7f0e
  laipz8200

  committed
  aeb7f0e

  Copy full SHA for aeb7f0e

• feat(storage): add OpenDALStorage

  Show description for 96b02a0
  laipz8200

  committed
  96b02a0

  Copy full SHA for 96b02a0

• refactor(ext_storage): remove initial of storage_runner to avoid None value checking

  Show description for b0ed867
  laipz8200

  committed
  b0ed867

  Copy full SHA for b0ed867

Commits on Dec 10, 2024

• fix: dependency

  Show description for 88ef25e
  laipz8200

  committed
  88ef25e

  Copy full SHA for 88ef25e

• feat(storage): use opendal instead of local fs storage

  Show description for 31eee9c
  laipz8200

  committed
  31eee9c

  Copy full SHA for 31eee9c

• test(storage): create dir before testing

  Show description for e6931b5
  laipz8200

  committed
  e6931b5

  Copy full SHA for e6931b5

• test(storage): use home dir for tests

  Show description for 02b9533
  laipz8200

  committed
  02b9533

  Copy full SHA for 02b9533

• test(storage): update test path to home dir

  Show description for 9745964
  laipz8200

  committed
  9745964

  Copy full SHA for 9745964

• test(storage): rename test case

  Show description for dd295a2
  laipz8200

  committed
  dd295a2

  Copy full SHA for dd295a2

• chore(.env.example): add opendal settings

  Show description for 92c57fc
  laipz8200

  committed
  92c57fc

  Copy full SHA for 92c57fc

• chore(config): update the default STORAGE TYPE

  Show description for a60469f
  laipz8200

  committed
  a60469f

  Copy full SHA for a60469f

• refactor(opendal_storage): update environment variables

  Show description for 2d632cd
  laipz8200

  committed
  2d632cd

  Copy full SHA for 2d632cd

• test(test_opendal): fix tests

  Show description for a1a3567
  laipz8200

  committed
  a1a3567

  Copy full SHA for a1a3567

• feat(opendal_storage): add basic s3 support

  Show description for 87a4a69
  laipz8200

  committed
  87a4a69

  Copy full SHA for 87a4a69

• chore(.env.example): add opendal s3 settings

  Show description for b06357f
  laipz8200

  committed
  b06357f

  Copy full SHA for b06357f

• feat(opendal_storage): add support for aws sse with aws managed key

  Show description for 21f50f3
  laipz8200

  committed
  21f50f3

  Copy full SHA for 21f50f3

• fix(storage): compatible with local

  Show description for af55eb7
  laipz8200

  committed
  af55eb7

  Copy full SHA for af55eb7

• feat(storage): use opendal instead of boto3

  Show description for 470fdda
  laipz8200

  committed
  470fdda

  Copy full SHA for 470fdda

• fix(storage): set default endpoint for s3

  Show description for d8d98d5
  laipz8200

  committed
  d8d98d5

  Copy full SHA for d8d98d5

• fix(storage): check if region exists

  Show description for c90dcd0
  laipz8200

  committed
  c90dcd0

  Copy full SHA for c90dcd0

• fix(storage): set default s3 region to us-east-1

  Show description for 36a10ae
  laipz8200

  committed
  36a10ae

  Copy full SHA for 36a10ae

• fix(configs): avoid to use enum in BaseSettings

  Show description for 3cd538c
  laipz8200

  committed
  3cd538c

  Copy full SHA for 3cd538c

• feat(storage): compatible for R2

  Show description for 4a5a9c7
  laipz8200

  committed
  4a5a9c7

  Copy full SHA for 4a5a9c7

• fix(storage): correct r2 url

  Show description for 6c7af8f
  laipz8200

  committed
  6c7af8f

  Copy full SHA for 6c7af8f

• chore(storage): record some log

  Show description for 7d9a76c
  laipz8200

  committed
  7d9a76c

  Copy full SHA for 7d9a76c

• fix(storage): convert bool to string

  Show description for 600cbbf
  laipz8200

  committed
  600cbbf

  Copy full SHA for 600cbbf

• chore(storage): move logs

  Show description for af42993
  laipz8200

  committed
  af42993

  Copy full SHA for af42993

• chore: disable preload in gunicorn.

  Show description for d027a8b
  laipz8200

  committed
  d027a8b

  Copy full SHA for d027a8b

Commits on Dec 11, 2024

• refactor(opendal_storage): improve r2 support

  Show description for 1fda7cb
  laipz8200

  committed
  1fda7cb

  Copy full SHA for 1fda7cb

• Merge remote-tracking branch 'origin/main' into feat/integrate-opendal-storage

  laipz8200

  committed
  d223116

  Copy full SHA for d223116