release-0.12.md

.NET for Apache Spark 0.12 Release Notes

New Features/Improvements and Bug Fixes

• Expose DataStreamWriter.ForeachBatch API (#549)
• Support for dotnet-interactive (#515) (#517) (#554)
• Support for Hyperspace v0.1.0 APIs (#555)
• Support for Spark 2.4.6 (#547)
• Bug fixes:
  • Udf bug caused by BroadcastVariablesRegistry (#551)
  • Null checks for TimestampType and DateType (#530)
• Update Microsoft.Data.Analysis to v0.4.0 (#528)

Infrastructure / Documentation / Etc.

• Improve build pipeline (#510) (#511) (#512) (#513) (#524)
• Update AppName for the C# Spark Examples (#548)
• Update maven links in build documentation (#558) (#560)

Breaking Changes

• None

Known Issues

• Broadcast variables do not work with dotnet-interactive (#561)

Compatibility

Backward compatibility

The following table describes the oldest version of the worker that the current version is compatible with, along with new features that are incompatible with the worker.

Oldest compatible Microsoft.Spark.Worker version	Incompatible features
v0.9.0	DataFrame with Grouped Map UDF (#277)
	DataFrame with Vector UDF (#277)
	Support for Broadcast Variables (#414)
	Support for TimestampType (#428)

Forward compatibility

The following table describes the oldest version of .NET for Apache Spark release that the current worker is compatible with.

Oldest compatible .NET for Apache Spark release version
v0.9.0

Supported Spark Versions

The following table outlines the supported Spark versions along with the microsoft-spark JAR to use with:

Spark Version	microsoft-spark JAR
2.3.*	microsoft-spark-2.3.x-0.12.0.jar
2.4.0	microsoft-spark-2.4.x-0.12.0.jar
2.4.1
2.4.3
2.4.4
2.4.5
2.4.6
2.4.2	Not supported