FlatFiles 0.2.0.0 and 0.3.0.0 Not Backward Compatible

I have created a couple brand new releases of FlatFiles. They come packed with new features and performance enhancements. If you've been using FlatFiles for a while, you might notice this release is not backwards compatible.

As of 0.2.0.0

I decided that Excel is not a true flat file and needing to support OleDbConnection (the underlying implementation) will make it harder to migrate FlatFiles to next generation platforms (.NET Core). If you were using these features, you can continue to reference the old NuGet package (0.1.17.0). Furthermore, I recommend you start looking at other options, such as EPPlus or ClosedXML for reading and writing Excel 2007+ workbooks. These projects are way more flexible.

The FlatFiles.Mvc project will no longer be maintained within the same solution. I pushed out a new release that only supports FlatFiles 0.1.14.0 up to 0.1.17.0. If you're using an older version of the FlatFiles.Mvc package, I'd recommend updating it before trying to upgrade FlatFiles. Going forward, I think it makes more sense the project be a separate GitHub project and it use a NuGet package reference to FlatFiles. This way they can be maintained separately.

FlatFiles no longer treats double and single quotes the same. Generally speaking, only double quotes should be used to support embedded commas and quotes in CSV files. That's why, going forward, only double quotes will be treated as quotes. I added the ability to specify a different quote character in SeparatedValueOptions, in case you need to support something different.

The new separated value parser should be significantly faster than before. I created a simple recursive descent parser to replace the Regex nastiness from before. Not only is it significantly faster, it supports more oddball CSV files. I also optimized the Type Mappers to read/write to object properties using run time-code generation, which is significantly faster than calling PropertyInfo.GetValue and PropertyInfo.SetValue. FlatFiles is now reading and writing 1,000,000 record CSV files in under 10 seconds.

As of 0.3.0.0

A lot of the code in FlatFiles and the unit tests dealt with encodings. That's because FlatFiles worked at the Stream-level. It took a file name or a Stream. These do not necessarily provide enough information to determine the encoding. You could override the encoding by passing along an "option" object. Part of supporting complex properties (a.k.a., nested schemas), I realized I was converting strings into Streams and then right back into strings -- wasting a lot of time.

StringReader and StringWriter allow me to expose strings using the TextReader and TextWriter interfaces. Under the hood, FlatFiles was wrapping a Stream in a StreamReader or StreamWriter, which also implements TextReader and TextWriter, respectively. I was able to switch the underlying code to use TextReader and TextWriter. Now I had a common interface for working with Streams or Strings.

Once I had the TextReader and TextWriter implementations in place, I realized there was another benefit to moving the responsibility for managing lifetimes outside of the readers and writers. My code no longer had to deal with IDisposable and I didn't have to worry about encodings either. This dramatically simplified the FlatFiles codebase. It also allows the Type Mapper's Read method to read values on-demand rather than all up-front in memory.

I removed the ISeparatedValueTypeWriter and IFixedLengthTypeWriter interfaces. They only served a single purpose: writing anonymous types. Otherwise, they were just thin wrappers around the type mappers.