Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

STORAGE-3098: Catalog refactor #3136

Merged
merged 16 commits into from
Nov 29, 2021
Merged

Conversation

BohuTANG
Copy link
Member

@BohuTANG BohuTANG commented Nov 28, 2021

I hereby agree to the terms of the CLA available at: https://databend.rs/policies/cla/

Summary

This patch cannot be split into many small PRs, it is easy to cause conflicts:<
Summary about this PR:

  1. Rename datasources to storages
  2. Put all db/table schema operation to catalog trait

In storages dir, the table engine defines:
How and where data is stored, where to write it and where read from.

storages
├── csv
├── memory
├── null
├── parquet
├── github
├── system
└── fuse

TODO(in another PR):

  1. Add database init operation to database trait for github engine @Veeupup

Changelog

  • Improvement

Related Issues

Fixes #3098

Test Plan

Unit Tests

Stateless Tests

@databend-bot
Copy link
Member

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

@codecov-commenter
Copy link

codecov-commenter commented Nov 28, 2021

Codecov Report

Merging #3136 (5fc894c) into main (0508a66) will increase coverage by 0%.
The diff coverage is 74%.

❗ Current head 5fc894c differs from pull request most recent head e0e51f6. Consider uploading reports for the commit e0e51f6 to get more accurate results
Impacted file tree graph

@@           Coverage Diff           @@
##            main   #3136     +/-   ##
=======================================
  Coverage     66%     67%             
=======================================
  Files        676     750     +74     
  Lines      35146   38273   +3127     
=======================================
+ Hits       23426   25765   +2339     
- Misses     11720   12508    +788     
Impacted Files Coverage Δ
query/src/storages/fuse/operations/append.rs 0% <0%> (ø)
query/src/storages/fuse/operations/commit.rs 0% <0%> (ø)
...uery/src/storages/fuse/operations/operation_log.rs 0% <0%> (ø)
query/src/storages/fuse/operations/read.rs 0% <0%> (ø)
query/src/storages/fuse/operations/truncate.rs 0% <0%> (ø)
query/src/storages/fuse/table.rs 0% <0%> (ø)
query/src/storages/storage_context.rs 0% <0%> (ø)
query/src/storages/storage_factory.rs 0% <0%> (ø)
query/src/storages/system/database.rs 0% <0%> (ø)
query/src/storages/system/one_table.rs 0% <0%> (ø)
... and 68 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0508a66...e0e51f6. Read the comment docs.

@mergify
Copy link
Contributor

mergify bot commented Nov 29, 2021

This pull request has merge conflicts that must be resolved before it can be merged. @BohuTANG please rebase it 🙏

@drmingdrmer
Copy link
Member

Wow~

@drmingdrmer
Copy link
Member

/lgtm

Much more clear and obvious~

@Veeupup
Copy link
Contributor

Veeupup commented Nov 29, 2021

/LGTM
learn a lot from it ...

@dantengsky
Copy link
Member

/LGTM & salute

@BohuTANG BohuTANG merged commit 7dae4cb into databendlabs:main Nov 29, 2021
@BohuTANG BohuTANG deleted the dev-catalog-3098 branch November 29, 2021 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Consider refactory catalog trait
6 participants