Releases: daq-tools/skeem
Releases · daq-tools/skeem
v0.1.1
What's Changed
- Added support for Python 3.12 and 3.13
- Adjusted SQL DDL for sqlalchemy-cratedb 0.40.0
- Adjusted ddlgenerator wrapper for pandas 2
- Updated to frictionless >=5.16
Details
- Chore: Update dependencies and documentation, and satisfy linter by @amotl in #3
- Bump docker/metadata-action from 4 to 5 by @dependabot in #4
- Bump docker/setup-buildx-action from 2 to 3 by @dependabot in #5
- Bump actions/checkout from 3 to 4 by @dependabot in #7
- Bump docker/login-action from 2 to 3 by @dependabot in #8
- Bump docker/build-push-action from 4 to 5 by @dependabot in #9
- Update requests requirement from <2.30 to <2.32 by @dependabot in #6
- Update types-requests requirement from <2.30 to <2.32 by @dependabot in #11
- Update pyproject-fmt requirement from <1.2 to <1.3 by @dependabot in #14
- Bump fsspec[gcs,github,http,s3] from 2023.1 to 2023.9.2 by @dependabot in #15
- Bump ruff from 0.0.291 to 0.0.292 by @dependabot in #17
- Update validate-pyproject requirement from <0.15 to <0.16 by @dependabot in #19
- Bump mypy from 1.5.1 to 1.6.0 by @dependabot in #18
- Bump ruff from 0.0.292 to 0.1.1 by @dependabot in #20
- Bump fsspec[gcs,github,http,s3] from 2023.9.2 to 2023.10.0 by @dependabot in #21
- Bump mypy from 1.6.0 to 1.6.1 by @dependabot in #22
- Bump ruff from 0.1.1 to 0.1.3 by @dependabot in #23
- Bump docker/setup-qemu-action from 2 to 3 by @dependabot in #24
- Bump ruff from 0.1.3 to 0.1.4 by @dependabot in #25
- Update pyproject-fmt requirement from <1.3 to <1.5 by @dependabot in #26
- Update pyproject-fmt requirement from <1.5 to <1.6 by @dependabot in #27
- Bump ruff from 0.1.4 to 0.1.6 by @dependabot in #29
- Bump mypy from 1.6.1 to 1.7.0 by @dependabot in #30
- Bump mypy from 1.7.0 to 1.7.1 by @dependabot in #31
- Bump fsspec[gcs,github,http,s3] from 2023.10.0 to 2023.12.0 by @dependabot in #32
- Bump ruff from 0.1.6 to 0.1.8 by @dependabot in #35
- Bump ruff from 0.1.8 to 0.1.9 by @dependabot in #37
- Bump actions/upload-artifact from 3 to 4 by @dependabot in #38
- Bump actions/setup-python from 4 to 5 by @dependabot in #39
- Bump fsspec[gcs,github,http,s3] from 2023.12.0 to 2023.12.2 by @dependabot in #36
- Bump ruff from 0.1.9 to 0.1.11 by @dependabot in #40
- Bump mypy from 1.7.1 to 1.8.0 by @dependabot in #41
- Update pyproject-fmt requirement from <1.6 to <1.7 by @dependabot in #42
- Bump ruff from 0.1.11 to 0.1.13 by @dependabot in #43
- Update pyproject-fmt requirement from <1.7 to <1.8 by @dependabot in #44
- Bump ruff from 0.1.13 to 0.1.14 by @dependabot in #45
- Update xarray requirement from <2024 to <2025 by @dependabot in #48
- Update pytest requirement from <8 to <9 by @dependabot in #51
- Bump actions/cache from 3 to 4 by @dependabot in #50
- Update validate-pyproject requirement from <0.16 to <0.17 by @dependabot in #52
- Bump fsspec[gcs,github,http,s3] from 2023.12.2 to 2024.3.1 by @dependabot in #55
- Update poethepoet requirement from <0.25 to <0.27 by @dependabot in #57
- Regular maintenance 202405 by @amotl in #60
- Update pyproject-fmt requirement from <1.9 to <2.2 by @dependabot in #64
- Update pytest-cov requirement from <5 to <6 by @dependabot in #63
- Update types-requests requirement from <2.32 to <2.33 by @dependabot in #66
- Bump fsspec[gcs,github,http,s3] from 2024.3.1 to 2024.6.0 by @dependabot in #67
- Dependencies: Migrate from
crate[sqlalchemy]
tosqlalchemy-cratedb
by @amotl in #68 - Dependencies: Use
numpy<2
to resolve compatibility issues by @amotl in #72 - Chore(deps): Update requests requirement from <2.32 to <2.33 by @dependabot in #70
- Chore(deps): Bump docker/build-push-action from 5 to 6 by @dependabot in #71
- Chore(deps): Update twine requirement from <5 to <6 by @dependabot in #69
- Update urllib3 requirement from <2 to <3 by @dependabot in #12
- Chore(deps): Update poethepoet requirement from <0.27 to <0.28 by @dependabot in #73
- Chore(deps): Update ruff requirement from <0.5 to <0.6 by @dependabot in #75
- Chore(deps): Bump fsspec[gcs,github,http,s3] from 2024.6.0 to 2024.6.1 by @dependabot in #76
- Chore(deps): Update mypy requirement from <1.11 to <1.12 by @dependabot in #79
- Chore(deps): Update validate-pyproject requirement from <0.17 to <0.19 by @dependabot in #80
- CI: Update to Docker Compose V2 by @amotl in #82
- Chore(deps): Update pyproject-fmt requirement from <2.2 to <2.3 by @dependabot in #81
- Chore(deps): Update validate-pyproject requirement from <0.19 to <0.20 by @dependabot in #83
- Chore(deps): Update ruff requirement from <0.6 to <0.7 by @dependabot in #84
- Chore(deps): Bump fsspec[gcs,github,http,s3] from 2024.6.1 to 2024.9.0 by @dependabot in #86
- Chore(deps): Update validate-pyproject requirement from <0.20 to <0.21 by @dependabot in #89
- Add support for Python 3.12 by @amotl in #90
- Chore(deps): Bump sqlalchemy-cratedb from 0.37.0 to 0.40.0 by @dependabot in #88
- Update pandas requirement from <2 to <3 by @dependabot in #61
- Chore(deps): Update numpy requirement from <2 to <3 by @dependabot in #77
- Add support for Python 3.13 by @amotl in #91
- Chore(deps): Update frictionless[excel,json,ods,parquet,sql] requirement from <5.6 to <5.19 by @dependabot in #87
- Chore: Limit pandas dependency to <2.3 by @amotl in #97
- This and that by @amotl in #98
- Chore(deps): Update ruff requirement from <0.7 to <0.8 by @dependabot in #92
- Chore(deps): Update poethepoet requirement from <0.28 to <0.30 by @dependabot in #93
- Chore(deps): Update validate-pyproject requirement from <0.21 to <0.22 by @dependabot in #95
- Chore(deps): Update mypy requirement from <1.12 to <1.13 by @dependabot in #96
- Chore(deps): Update pyproject-fmt requirement from <2.3 to <2.5 by @dependabot in #100
- Chore(deps): Bump fsspec[gcs,github,http,s3] from 2024.9.0 to 2024.10.0 by @dependabot in #94
New Contributors
- @dependabot made their first contribution in #4
Full Changelog: v0.1.0...v0.1.1
v0.1.0
What's Changed
- Initial working version, supporting JSON
- Add CLI subsystem
- Add logging
- Add SQL pretty printing
- Improve software tests
- Source: Unlock reading from stdin
- Derive table name from input file name
- Primary key detection
- Format: Content type detection using file extension or
--content-type
option - Format: Add support for CSV input format
- Format: Add support for XLSX and ODS input formats
- Tests: Add test case for basic nested JSON document
- Backend: Add
frictionless
backend - Format: Add support for Google Sheets and Parquet input formats
- Performance: Improve sampling large files
- Source: Support reading data from HTTP
- Performance: Use the Hunter code tracing toolkit to trace execution path
- Performance: Don't open resource twice when using
frictionless
backend - Performance: Improve peeking into Parquet files
- Performance: Configure
PEEK_LINES = 100
instead of 1000 - Performance: Optimize reading from remote NDJSON files
- Refactoring: I/O related code goes into
skeem.io
- Format: Add support for InfluxDB line protocol input format
- Source: Unlock reading from public S3 buckets anonymously (
--no-sign-request
) - Source: Unlock and document reading from public Google Cloud Storage (GCS) buckets,
and files on GitHub. - Tests: Add "roadrunner" tests, using a bunch of external resources. The tests
will only check for successful invocation, and not verify the generated SQL. - Format: Add NetCDF input format
- AutoPK: Fix heuristics where the first column is a dictionary
- Format: Add GRIB2 input format
- Refactoring: Rework
skeem.type
- Refactoring: Add
skeem.io.open
as a wrapper aroundfsspec.open
- Refactoring: Add
skeem.io.to_dataframe
fromskeem.autopk
- Refactoring: Add
skeem.util
folder instead of singleutil.py
- UX: Add
skeem info
subcommand - UX: Improve CLI help
- Format: Add support for reading compressed files in Gzip format
- CI: Run software tests on GHA
- CI: Use
versioningit
for automatic package versioning - CI: Add OCI image builder to provide images for Podman, Docker, etc.
Full Changelog: https://github.com/daq-tools/skeem/commits/0.1.0