cleanup_metadata doesn't remove .checkpoint.parquet files

[cmackenzie1]

Environment

Delta-rs version: v0.12.0

Binding: rust

Environment:

• Cloud provider:
• OS:
• Other:

---

Bug

What happened:

Running the following snippet does not remove .checkpoint.parquet files.

let n = cleanup_expired_logs_for(
          table.version() + 1,
          table.object_store().as_ref(),
          Utc::now().timestamp_millis(),
      )
      .await
      .unwrap();

What you expected to happen:

.checkpoint.parquet files older than log_retention_timestamp should be removed.

How to reproduce it:

1. Create table with checkpoint(s)
2. Run cleanup_expired_logs_for setting log_retention_timestamp to Utc::now()
3. Observe only .json files are removed

More details:

https://regex101.com/r/OmoZMz/1

[cmackenzie1]

I think it is just missing the . at the end of the expression so it should be r#"_delta_log/(\d{20})\.(json|checkpoint).*$"#

delta-rs/rust/src/checkpoints.rs

Line 208 in b17f286

Regex::new(r#"_delta_log/(\d{20})\.(json|checkpoint)*$"#).unwrap();

diff --git a/rust/src/checkpoints.rs b/rust/src/checkpoints.rs
index 98a4e79..41385bb 100644
--- a/rust/src/checkpoints.rs
+++ b/rust/src/checkpoints.rs
@@ -205,7 +205,7 @@ pub async fn cleanup_expired_logs_for(
 ) -> Result<i32, DeltaTableError> {
     lazy_static! {
         static ref DELTA_LOG_REGEX: Regex =
-            Regex::new(r#"_delta_log/(\d{20})\.(json|checkpoint)*$"#).unwrap();
+            Regex::new(r#"_delta_log/(\d{20})\.(json|checkpoint).*$"#).unwrap();
     }
 
     let mut deleted_log_num = 0;

[cmackenzie1]

Also, should this be advancing by 1? Wouldn't that also remove the latest checkpoint ?

delta-rs/rust/src/checkpoints.rs

Line 106 in b17f286

table.version() + 1,

Edit: Removing the latest checkpoint seems to be the case on the table I just tested with.