Update to Julia 0.6-1.0, drop support for 0.4-0.5

[galenlynch]

Sorry about the changes to white space, I didn't realize that some parts of the
code used tabs until it was too late.

[galenlynch]

Failures on 1.0 are from Zip.jl

[fhs]

Sorry about the tabs. I've changed them to spaces. Can you rebase? Thanks.

[galenlynch]

done

[fhs]

Thanks for the fixes!

[fhs]

add a space after # like all other comments

[fhs]

I don't think you should be assuming we're writing NPY header. In the future, we might be writing string data. Either change it back take a String input instead of AbstractString so that the input is UTF-8 encoded or handle byte order for any type of string (e.g. UTF-16).

In fact, I don't understand why all the String needs to be converted to AbstractString in the functions below. Since the header is ASCII, String works for us. There is no need to be more general, which has potential to slow down parsing.

[galenlynch]

Hmm my comment isn't quite accurate, it should work on any utf-8 string, which is the only string encoding that I've come across in Julia since Julia 0.5. My intent with using AbstractString type bound is to catch the many AbstractStrings in Julia that use utf-8. In Julia 0.7 more functions produce SubString etc, which in fact leads to breakage in NPZ.jl (see my comment in #23).

Excluding UTF-16, and other string types that I haven't seen in julia, there are still plenty of AbstractStrings that would work for these functions, and these functions need to handle at least SubString. The change regarding production of non-String AbstractStrings by base functions that bites here is JuliaLang/julia#22496, where all strip functions return SubStrings. As you can see below, all AbstractString subtypes in base are utf-8 encoded, but manipulate the order, extent, or content of an underlying String.

julia> subtypes(AbstractString)
4-element Array{Any,1}:
 String            
 SubString         
 SubstitutionString
 Test.GenericString

In fact, utf-8 is always a sequence of single bytes, even when codepoints are not themselves single bytes, so utf-8 strings are indifferent to endianness [#1]. Ascii is a subset of utf-8, and I think can be safely treated as always being utf-8.

However, you are right that someone may want to write other utf-encodings using strings generated from some package or other source. I'll change it so the codeunits are inspected before being written. This should allow it to work for all utf-8 AbstractStrings

[fhs]

This function was already a bit wrong in that it assumes c is an ASCII character. If it's not, it'll crash:

julia> s = "\u2200 x \u2203 y"
"∀ x ∃ y"

julia> typeof(s)
String

julia> s[1]
'∀': Unicode U+2200 (category Sm: Symbol, math)

julia> s[2:end]
ERROR: UnicodeError: invalid character index 2 (0x88 is a continuation byte)
Stacktrace:
 [1] macro expansion at ./REPL.jl:97 [inlined]
 [2] (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:73

Now that it's AbstractString, it may be UTF-16 and become always wrong.

[galenlynch]

Hmm, it was already broken for more than just the character argument. String can be (and is) utf8, and you just demonstrated that it breaks on utf8.

To demonstrate...

julia> foo(s::String) = s
foo (generic function with 1 method)
julia> bar(s::String) = s[2]
bar (generic function with 1 method)
julia> s = "♨★  𝕦ᑎ𝐢ς𝐎𝐝є!!  ☯✌"
"♨★  𝕦ᑎ𝐢ς𝐎𝐝є!!  ☯✌"
julia> foo(s)
"♨★  𝕦ᑎ𝐢ς𝐎𝐝є!!  ☯✌"
julia> bar(s)
ERROR: StringIndexError("♨★  𝕦ᑎ𝐢ς𝐎𝐝є!!  ☯✌", 2)
Stacktrace:
 [1] string_index_err(::String, ::Int64) at ./strings/string.jl:12
 [2] getindex_continued(::String, ::Int64, ::UInt32) at ./strings/string.jl:216
 [3] getindex at ./strings/string.jl:209 [inlined]
 [4] bar(::String) at ./REPL[31]:1
 [5] top-level scope at none:0

[galenlynch]

I fixed the unicode problems.

[fhs]

I can't say I'm happy with the new string indexing -- it's too verbose. But it's acceptable.

[fhs]

Thanks for all the fixes!

[galenlynch]

Seems like the price you have to pay if you want to play with unicode. Thanks again for maintaining such a useful package!