0.3.1
What's Changed
- Add kfp constraint by @PhilippeMoussalli in #341
- [DataComp] Update pipeline name, remove DockerCompiler by @NielsRogge in #340
- Deactivate dask string conversion by @RobbeSneyders in #349
- [Commoncrawl pipeline] Add component download_commoncrawl_segments by @shayorshay in #273
- Add kfp compiler by @GeorgesLorre in #291
- Remove node_pool_name arguments in examples by @RobbeSneyders in #350
- Small improvements to tox configuration by @RobbeSneyders in #343
- [CommonCrawl pipeline] Improve html extraction by @mrchtr in #351
- [DataComp] Add download images component by @NielsRogge in #348
- Add AWS credential arguments to commoncrawl download components. by @mrchtr in #353
- [LLM pipeline] Update text normalization component by @mrchtr in #335
- Remove output_partition_size argument and logic by @PhilippeMoussalli in #355
- [Commoncrawl pipeline] Add metadata for target_language by @shayorshay in #357
- [Commoncrawl pipeline] Add offset to load component by @shayorshay in #358
- Remove obsolete args from ComponentOp by @PhilippeMoussalli in #356
- Implement kfp runner with tests by @GeorgesLorre in #359
- Make download_component concurrent by @RobbeSneyders in #354
- Define kfp as extra and update error messages by @GeorgesLorre in #361
- Expand cli to support kfp compiling and running by @GeorgesLorre in #366
- Update docs with the new CLI commands by @GeorgesLorre in #370
- Update test setup of text_normalization component by @RobbeSneyders in #369
Full Changelog: 0.3.0...0.3.1