refactor: export chunks in std/io

[bartlomieju]

This PR addresses old TODO to export chunks utility method in std.

• move chunks from std/xeval/mod.ts to std/io/bufio.ts
• add lines helper which is chunks(reader, "\n")
• add simple test case

CC @kevinkassimo @David-Else

[nayeemrmn]

We talked about naming this in denoland/std#581. I like readDelim().

[bartlomieju]

@nayeemrmn thanks for pointing this out, I forgot about it. As for the naming I'm open to changes.

Reading my comment from that issue I had idea to expose chunks and lines on BufReader, but since BufReader is used to operate strictly on Uint8Array and chunks/lines convert to string I believe making them a separate functions is fine.

Furthermore, thinking about it now, it might make more sense to place those functions in strings module instead.

[nayeemrmn]

@bartlomieju Ah, right. I think we wanted to remove the tacked-on encoding/decoding from this and make it binary. Then call it readDelim() and implement a readStringDelim() on top of it. I don't know how that fits into BufReader... that's a stateful thing and maybe these generators should just stay separate from it.

Edit: BufReader has readSlice() and readString(). These would be analogous to those but as self-contained generators.

[bartlomieju]

@bartlomieju Ah, right. I think we wanted to remove the tacked-on encoding/decoding from this and make it binary. Then call it readDelim() and implement a readStringDelim() on top of it. I don't know how that fits into BufReader... that's a stateful thing and maybe these generators should just stay separate from it.

Edit: BufReader has readSlice() and readString(). These would be analogous to those but as self-contained generators.

Then maybe BufReader.readStringChunks and BufReader.readStringLines could work

[nayeemrmn]

yield* chunks(reader, "\n");

[nayeemrmn]

@bartlomieju How would you go about adding them on BufReader anyway? They would be generator methods. Those would return iterables. Advancing these iterables would affect the state of the BufReader, but that's not how generator methods should work.

Just add a top-level readDelim() (binary), readStringDelim() (strings) and readLines() (strings, newline delim) where each shells out to the last. Swap out *Delim for whatever suffix.

[kevinkassimo]

Not really related, but maybe a Boyer-Moore-Horspool implementation could work better than KMP (I realized that v8 uses this for indexOf), like this one: https://github.com/mscdex/streamsearch/blob/master/lib/sbmh.js