You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've been experimenting with the CSV package and reading through the documentation, and I was surprised to find no function that can read data directly into Julia arrays.
The CSV.File function reads data into a CSV.File data structure, which is a read-only data type not as flexible as arrays or even static arrays. For example:
julia> using StaticArrays, Tables
julia> x = SMatrix{3}(1, 2, 3) # returns a static array
julia> 2*x # returns a static array
julia> x = 2*x # executes normally
julia> CSV.write("data.csv", Tables.table(x, header=[:x]))
julia> f = CSV.File("data.csv")
julia> f.x # returns a CSV.column object
julia> 2 * f.x # returns an **array** object
julia> f.x = 2 * f.x # returns an error
The CSV.read function outputs a DataFrame object which can be pretty versatile, though only if the data is copied, which means more memory usage. Plus, sometimes one just needs to access and manipulate the data in form of simple arrays, without the need for something as sophisticated as a DataFrame.
Therefore, I suggest adding a new function to the package that can read data directly into arrays (and/or static arrays), so that users can have better versatility in controlling the data. I wrote a simple wrapper for CSV.read that converts the DataFrame into arrays, though I think this wrapper will still copy the data in the convert step, so it is not very ideal:
function readarray(source; unpack::Bool=false, kwargs...)
df = CSV.read(source; kwargs...)
if !unpack
return convert(Matrix, df) # returns 2-D array
else
return (convert(Vector, x) for x in eachcol(df)) # returns tuple of 1-D arrays
end
end
I hope the developers would respond positively to my request!
Thank you.
The text was updated successfully, but these errors were encountered:
HTofi
changed the title
Add a new function that reads data durectly into Julia arrays (or static arrays)
Add a new function that reads data directly into Julia arrays (or static arrays)
May 3, 2020
Hi,
I've been experimenting with the CSV package and reading through the documentation, and I was surprised to find no function that can read data directly into Julia arrays.
The
CSV.File
function reads data into aCSV.File
data structure, which is a read-only data type not as flexible as arrays or even static arrays. For example:The
CSV.read
function outputs aDataFrame
object which can be pretty versatile, though only if the data is copied, which means more memory usage. Plus, sometimes one just needs to access and manipulate the data in form of simple arrays, without the need for something as sophisticated as a DataFrame.Therefore, I suggest adding a new function to the package that can read data directly into arrays (and/or static arrays), so that users can have better versatility in controlling the data. I wrote a simple wrapper for
CSV.read
that converts theDataFrame
into arrays, though I think this wrapper will still copy the data in theconvert
step, so it is not very ideal:I hope the developers would respond positively to my request!
Thank you.
The text was updated successfully, but these errors were encountered: