read_delim() and C stack usage too close to limit

[benzipperer]

Hi, when I try to use read_delim to import a space delimited file, I receive the error

library(readr)
new_data <- read_delim('nlsy97_all_1997-2013.dat',delim=' ')
Error: C stack usage  13203400 is too close to the limit

You can download the data here. The .dat file above is about 6GB. Can anyone reproduce this error?

How can I help solve this problem?

[jimhester]

This dat file contains 230298 columns and 8983 rows. readr seems to have particularly bad performance with very wide, shallow data like this, and potentially a bug in this case as well. For this particular problem where all the values are integers you might want to just use base::scan().

new_data <- scan("nlsy97_all_1997-2013.dat", what = integer())