feat(dataset): support dataset builf from audio/video/image folder

[tianweidut]

Description

• refactor swcli dataset build command structure, supports multi building source.
  

  • build from dataset.yaml

     swcli dataset build
     swcli dataset build --yaml /path/to/dataset.yaml
    

  • build from python handler

    swcli dataset build -w . --handler mnist.dataset:iter_mnist_item
    swcli dataset build --handler mnist.dataset:iter_mnist_item
    

  • build from image folder

    swcli dataset build --image image-metadata-jsonl
    

  • build from video folder

    swcli dataset build --video /path/to/video
    

  • build from audio folder

    swcli dataset build --audio /path/to/audio
    

• Dataset SDK supports from_folder method.

• image folder build example:

  • cloud datasets:
    • https://cloud.starwhale.cn/projects/23/datasets/66/versions/94/files
    • https://cloud.starwhale.cn/projects/23/datasets/63/versions/91/files
    • https://cloud.starwhale.cn/projects/23/datasets/62/versions/90/files
    • https://cloud.starwhale.cn/projects/23/datasets/62/versions/93/files
    • https://cloud.starwhale.cn/projects/23/datasets/62/versions/92/files
  • auto import all image files.
    
  • auto import all image files and label it.
    
  • auto import all images and caption txt files.
    
  • auto import all images and metadata.csv.
    
  • auto import all images and metadata.jsonl
    

Modules

• Client
• Python-SDK

Checklist

• run code format and lint check
• add unit test
• add necessary doc

[codecov]

Codecov Report

Merging #2371 (e26f2c8) into main (2bb9711) will increase coverage by 8.52%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main    #2371      +/-   ##
============================================
+ Coverage     82.57%   91.09%   +8.52%     
============================================
  Files           429       98     -331     
  Lines         22872    11146   -11726     
  Branches       1347        0    -1347     
============================================
- Hits          18886    10154    -8732     
+ Misses         3377      992    -2385     
+ Partials        609        0     -609     

Flag	Coverage Δ
console	?
controller	?
standalone	91.09% <100.00%> (+0.06%)	⬆️
unittests	91.09% <100.00%> (+0.14%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
client/starwhale/api/_impl/dataset/model.py	98.80% <100.00%> (+0.15%)	⬆️
client/starwhale/base/type.py	100.00% <100.00%> (ø)
client/starwhale/core/dataset/model.py	94.48% <100.00%> (+0.23%)	⬆️
client/starwhale/core/dataset/type.py	94.22% <100.00%> (+0.03%)	⬆️
client/starwhale/core/dataset/view.py	63.22% <100.00%> (+1.48%)	⬆️

... and 331 files with indirect coverage changes

[anda-ren]

LGTM