replace lakefs_client with lakefs-sdk in Spark integration test harness #13296

Sign in to view logs

This workflow is awaiting approval from a maintainer in #8403

Triggered via pull request November 29, 2024 12:53

opened #8403

mahdikhashan:fix/use-updated-python-client-in-spark-integration-tests-8391

Status Action required

Total duration –

Artifacts –

This workflow is awaiting approval from a maintainer in #8403

esti.yaml

on: pull_request

Check if secrets are available.

Generate code from latest lakeFS app

Build lakeFS HadoopFS

Prepare Spark Apps

Build and push rclone export Docker image

Build and push Docker image

Build metadata client for Spark 3.x

Test lakeFS rclone export functionality

E2E - DynamoDB Local - Local Block Adapter

Run latest lakeFS app on AWS S3 + Basic Auth

Run latest lakeFS app on AWS S3 DynamoDB KV

Run latest lakeFS app on Azure with Azure Data Lake Storage Gen2 and CosmosDB

Run latest lakeFS app on Azure with Azure blobstore

Run latest lakeFS app on Google Cloud Platform and Google Cloud Storage

Test lakeFS Hadoop FileSystem

Test lakeFS against the python wrapper client

Test lakeFS multipart upload with Hadoop S3A

Test metastore client commands using trino

Matrix: Run latest lakeFS app on AWS S3

Waiting for pending jobs

Matrix: spark

Waiting for pending jobs

Test lakeFS metadata client export with Spark 3.x

Test unified gc