RFC: audit Core and Base exports

[JeffBezanson]

I took a pass through to find Core and Base exports that should be removed. This is based on frequency of use in Base and stdlib, and my own judgment about which things people should generally not use or rely on. Up for debate though.

Un-exported from Core:

TypeName, SimpleVector, CodeInfo, MethodTable, TypeMapEntry, TypeMapLevel,
GotoNode, LabelNode, NewvarNode, SSAValue, Slot, SlotNumber, TypedSlot

I kept QuoteNode and LineNumberNode since those appear in ASTs and can be passed to macros. GlobalRef could go either way. But I believe it does not appear in surface ASTs, only expanded macros and later. I added a module Core.IR that exports the types used in the IR.

Un-exported from Base:

BufferStream, EachLine, Enumerate, RangeIndex, StackTrace, StackFrame

Opinions welcome.

[ararslan]

Having to qualify the sorting algorithms I think will be mildly annoying, but +1 for everything else.

[StefanKarpinski]

I agree with @ararslan. I also don't think we're likely to delete or change the meaning of the sorting algorithms, so I don't much mind exporting them. The other potential approach would be to re-export them from SortingAlgorithms and make the official way to pass even the algorithms defined in Base to be via that package. That would potentially argue for its inclusion in the stdlib.

[iamed2]

You could still do using Base.Sort to get them

[StefanKarpinski]

You could still do using Base.Sort to get them

I think we're going to want to consider anything that involves reaching into Base to get some functionality to be potentially breaking. Now that almost all of the public modules that used to be part of Base are in stdlib, you know you can safely do using DelimitedFiles and as long as you lock down a version of DelimitedFiles in a Manifest.toml file for your project, you're going to get the same behavior. For sorting, I think we either need to export these from Base and consider the stable or expose them from somewhere outside of Base which can be versioned in a manifest. However, that does not necessarily need to happen right now, so we could unexport them and then sort out where they live "officially" in the future (e.g. export them from SortingAlgorithms moved to stdlib).

[iamed2]

I think we're going to want to consider anything that involves reaching into Base to get some functionality to be potentially breaking.

Base.Iterators is still a thing, and I don't see how that could be decoupled or the functions exported.

[StefanKarpinski]

Yes, that's true. We should probably consider moving Iterators into stdlib and adding more iterators from the IterTools package as they are made sufficiently efficient and robust.

[iamed2]

IIRC the iterators from Iterators are actually used fairly centrally in Base. Perhaps it's a good idea to have those ones in Base but unexported and export them from an Iterators stdlib package?

[samoconnor]

Why BufferStream?

It seems to me that an ::IO that can be written to from one task and read from another is a fairly generally useful thing. When I proposed adding that ability to IOBuffer(), the feeling was that BufferStream already provides that and that adding a Condition to IOBuffer() was too expensive: #23391 (comment)

HTTP.jl uses BufferStream to unit-test code that is intended work with a TCPSocket or SSLContext.

[JeffBezanson]

I agree that it seems useful in theory, but it doesn't seem to be widely used in practice. Do you have uses for it other than testing?

[EricForgy]

Could it be the lack of popularity has to do with lack of awareness / documentation? I never knew it existed until seeing this (although I don't have a use case in mind at the moment) :)

If HTTP.jl becomes the defacto for Julia web apps (like I think it will) and BufferStreams are used there, it might increase their profile. I don't have a strong feeling either way though :)

[samoconnor]

I suspect it isn't widely used because it is almost undocumented.
Some have rolled their own: https://github.com/JuliaWeb/HTTP.jl/blob/master/src/fifobuffer.jl (although in that case, it turns out that the extra layer of buffering can be avoided completely). Part of the perceived need for doing that are the ambiguities in the IO interface #24526.

Aside from testing, the only place I'm using it is in Base.parse(HTTP.Message, str) where the message to be parsed is written to BufferStream and then processed by the usual HTTP stack as if it were a TCPSocket. I can't use IOBuffer here because the edge cases discussed in #24526.

Perhaps the solution is to have a kw arg to IOBuffer() that causes it to return a BufferStream (i.e. an IO, backed by a local buffer, that supports tasks). That would allow BufferStream to be unexported.

[iamed2]

Sam is right, we've been looking for a buffer stream type for a while but haven't found anything documented.

[JeffBezanson]

Ok, I'm certainly willing to re-export it and document it. Or maybe putting it in its own stdlib package would help improve discoverability?