Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking Issue for seekable io::Take #97227

Open
wangbj opened this issue May 20, 2022 · 0 comments
Open

Tracking Issue for seekable io::Take #97227

wangbj opened this issue May 20, 2022 · 0 comments
Labels
C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.

Comments

@wangbj
Copy link

wangbj commented May 20, 2022

Feature gate: #![feature(seek_io_take)]

(feature gate open to change)

This is a tracking issue for adding seek instance for io::Take, see issue #37214.

Why

Imagine there's a smaller components within a larger file, the component have predefined ranges, reading beyond the range should be a hard error. But some fields are variable size encoded, so it's hard to know how many bytes to read until we already decoded/read a large part of it. It is super cumbersome to check every read is within the limit (of the component); however, with io::Take, this is much easier: We only need to create a new io::Take by limiting the upper bound to the component's spec. and just let the user to read what ever needed -- any attempt to read beyond the limit would cause a hard io error.

This is all good, but sometimes we need more powerful io rather than read. For instance, we may need peek and restore stream_position() depends on what data was peek-ed, skip a large chunk of unwanted bytes, or maybe we simply wanted to keep track of the stream_position() for better error handling. This is not possible because io::Take is not Seek.

Public API

Added Seek instance for io::Take<T>, so that we can call io::Seek APIs on io::Take<T>.

pub struct Take<T> {
    inner: T,
    limit: u64,
    cursor: u64,          // <---- new member, current position within `Take`.
}

impl<T: Seek> Seek for Take<T> {
    fn seek(&mut self, pos: SeekFrom) -> Result<u64> {
        // snip
    }
}

This is possible by adding a cursor member to io::Take, together with limit, we have:

  • cursor + limit === original_limit (when Take is constructed)
  • Seek beyond original_limit => stays at original_limit
  • Seek beyond 0 (< 0) => ErrorKind::InvalidInput (EINVAL).
  • Read beyond original_limit => io error.
  • seek without position changes is honored (This is expected, see BufReader doc).

The proposed method has an advantage over approach in #37214, by not using an extra stream_position() when io::Take is constructed.

@wangbj wangbj added C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. labels May 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

1 participant