Skip to content

Commit

Permalink
feat: add list_with_offset to DeltaObjectStore (#1410)
Browse files Browse the repository at this point in the history
# Description
Adds the `list_with_offset` delegation method to `DeltaObjectStore`.

# Related Issue(s)
- closes #1252 

# Documentation

apache/arrow-rs#3970

Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>
  • Loading branch information
ognis1205 authored May 30, 2023
1 parent 3c629b0 commit df98587
Showing 1 changed file with 12 additions and 0 deletions.
12 changes: 12 additions & 0 deletions rust/src/storage/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,18 @@ impl ObjectStore for DeltaObjectStore {
self.storage.list(prefix).await
}

/// List all the objects with the given prefix and a location greater than `offset`
///
/// Some stores, such as S3 and GCS, may be able to push `offset` down to reduce
/// the number of network requests required
async fn list_with_offset(
&self,
prefix: Option<&Path>,
offset: &Path,
) -> ObjectStoreResult<BoxStream<'_, ObjectStoreResult<ObjectMeta>>> {
self.storage.list_with_offset(prefix, offset).await
}

/// List objects with the given prefix and an implementation specific
/// delimiter. Returns common prefixes (directories) in addition to object
/// metadata.
Expand Down

0 comments on commit df98587

Please sign in to comment.