Code

[Feat] OP-wise Insight Mining#516

Code

Merged

yxdyc merged 24 commits intomainmodelscope/data-juicer:mainfrom feat/insight_miningmodelscope/data-juicer:feat/insight_miningCopy head branch name to clipboardDec 20, 2024

+680-244

Commits

Commits on Dec 12, 2024

• + add auto mode for analyzer: load all filters that produce stats to analyze the target dataset

  HYLcool

  committed
  083b665

  Copy full SHA for 083b665

Commits on Dec 13, 2024

• + add default mem_required for those model-based OPs

  HYLcool

  committed
  662df5e

  Copy full SHA for 662df5e

• - support wordcloud drawing for str or str list fields in stats

  Show description for 926c3da
  HYLcool

  committed
  926c3da

  Copy full SHA for 926c3da

• - take the minimum one of dataset length and auto num

  HYLcool

  committed
  27347c0

  Copy full SHA for 27347c0

• * update default export path

  HYLcool

  committed
  d19f92f

  Copy full SHA for d19f92f

• * set version limit for wandb to avoid exception

  HYLcool

  committed
  fbd6726

  Copy full SHA for fbd6726

• + add docs for auto mode

  HYLcool

  committed
  9f9f85b

  Copy full SHA for 9f9f85b

Commits on Dec 16, 2024

• + support t-test for Measure

  HYLcool

  committed
  566eb5b

  Copy full SHA for 566eb5b

• * fix some bugs

  HYLcool

  committed
  7b8ee5c

  Copy full SHA for 7b8ee5c

Commits on Dec 17, 2024

• - support analyze a dataset object

  Show description for 601d9a2
  HYLcool

  committed
  601d9a2

  Copy full SHA for 601d9a2

• - support analysis on tags in meta

  HYLcool

  committed
  34f2ab6

  Copy full SHA for 34f2ab6

• - support analysis with tagging OPs

  HYLcool

  committed
  8531a01

  Copy full SHA for 8531a01

Commits on Dec 18, 2024

• - move tags into the meta field

  HYLcool

  committed
  4d6b701

  Copy full SHA for 4d6b701

• - do not tell tags using their suffix

  Show description for 35aa6bd
  HYLcool

  committed
  35aa6bd

  Copy full SHA for 35aa6bd

• - add insight mining

  HYLcool

  committed
  85e1392

  Copy full SHA for 85e1392

Commits on Dec 19, 2024

• * resolve the bugs when running insight mining in multiprocessing mode

  HYLcool

  committed
  e3d7b8b

  Copy full SHA for e3d7b8b

• Merge branch 'main' into feat/insight_mining

  Show description for 3ca9994
  HYLcool

  committed
  3ca9994

  Copy full SHA for 3ca9994

Commits on Dec 20, 2024

• * update unittests

  HYLcool

  committed
  16ca358

  Copy full SHA for 16ca358

• * update unittests

  HYLcool

  committed
  dfb0bca

  Copy full SHA for dfb0bca

• * update unittests

  HYLcool

  committed
  f8b9539

  Copy full SHA for f8b9539

• * update readme for analyzer

  HYLcool

  committed
  45259e5

  Copy full SHA for 45259e5

• Merge branch 'main' into feat/insight_mining

  Show description for 174ee05
  HYLcool

  committed
  174ee05

  Copy full SHA for 174ee05

• * use more detailed key

  HYLcool

  committed
  51f53dc

  Copy full SHA for 51f53dc

• + add reference

  HYLcool

  committed
  58001ca

  Copy full SHA for 58001ca