Automatically record new snapshots on failures

[kiliankoe]

First of all thanks for the fantastic snapshotting package, it's a joy to use and work with.

At my place of work we've recently gotten used to using a local override of the assertSnapshot function that uses verifySnapshot to check if a snapshot has changed and if so, automatically copying the new snapshot to the correct directory to overwrite the old one and then fail the test. That way a developer doesn't need to manually rerecord updated snapshots but already sees a new one in their git diff which they can either accept as a known change or discard in case it actually is wrong and needs code changes to fix.

So far this feels like a nice improvement in the typical workflow. Is something like this already possible but we just missed it entirely? Or could this be an improvement to the existing API or swift-snapshot-testing?

[tdrhq]

If you're trying to avoid manually recording screenshots you could use a third-party service to notify you when screenshots change in CI. I've built screenshotbot.io (open source at https://github.com/screenshotbot/screenshotbot-oss), but there's also another similar open source tool built by Workday https://github.com/workday/vizzy. With these you'll just always be running your tests in record mode, add your snapshots to .gitignore, and then in CI, you'll just upload the most recently generated screenshots, and let the tool do all the comparisons required to notify you.

[balrok]

Hi @kiliankoe,
we had a similar issue in our project. We imlemented it by forking this library and add a autoRecord flag.
The solution is basically just this:

diff --git a/Sources/SnapshotTesting/AssertSnapshot.swift b/Sources/SnapshotTesting/AssertSnapshot.swift
index 6f70992..591f62c 100644
--- a/Sources/SnapshotTesting/AssertSnapshot.swift
+++ b/Sources/SnapshotTesting/AssertSnapshot.swift
@@ -24,6 +24,7 @@ public var record: Bool {
 ///   - name: An optional description of the snapshot.
 ///   - recording: Whether or not to record a new reference.
 ///   - timeout: The amount of time a snapshot must be generated in.
+///   - autoRecord: when the test fails it will override the original but still throw the error
 ///   - file: The file in which failure occurred. Defaults to the file name of the test case in which this function was called.
 ///   - testName: The name of the test in which failure occurred. Defaults to the function name of the test case in which this function was called.
 ///   - line: The line number on which failure occurred. Defaults to the line number on which this function was called.
@@ -33,6 +34,7 @@ public func assertSnapshot<Value, Format>(
   named name: String? = nil,
   record recording: Bool = false,
   timeout: TimeInterval = 5,
+  autoRecord: Bool = true,
   file: StaticString = #file,
   testName: String = #function,
   line: UInt = #line
@@ -43,6 +45,7 @@ public func assertSnapshot<Value, Format>(
     as: snapshotting,
     named: name,
     record: recording,
+    autoRecord: autoRecord,
     timeout: timeout,
     file: file,
     testName: testName,
@@ -59,6 +62,7 @@ public func assertSnapshot<Value, Format>(
 ///   - strategies: A dictionary of names and strategies for serializing, deserializing, and comparing values.
 ///   - recording: Whether or not to record a new reference.
 ///   - timeout: The amount of time a snapshot must be generated in.
+///   - autoRecord: when the test fails it will override the original but still throw the error
 ///   - file: The file in which failure occurred. Defaults to the file name of the test case in which this function was called.
 ///   - testName: The name of the test in which failure occurred. Defaults to the function name of the test case in which this function was called.
 ///   - line: The line number on which failure occurred. Defaults to the line number on which this function was called.
@@ -67,6 +71,7 @@ public func assertSnapshots<Value, Format>(
   as strategies: [String: Snapshotting<Value, Format>],
   record recording: Bool = false,
   timeout: TimeInterval = 5,
+  autoRecord: Bool = true,
   file: StaticString = #file,
   testName: String = #function,
   line: UInt = #line
@@ -79,6 +84,7 @@ public func assertSnapshots<Value, Format>(
       named: name,
       record: recording,
       timeout: timeout,
+      autoRecord: autoRecord,
       file: file,
       testName: testName,
       line: line
@@ -93,6 +99,7 @@ public func assertSnapshots<Value, Format>(
 ///   - strategies: An array of strategies for serializing, deserializing, and comparing values.
 ///   - recording: Whether or not to record a new reference.
 ///   - timeout: The amount of time a snapshot must be generated in.
+///   - autoRecord: when the test fails it will override the original but still throw the error
 ///   - file: The file in which failure occurred. Defaults to the file name of the test case in which this function was called.
 ///   - testName: The name of the test in which failure occurred. Defaults to the function name of the test case in which this function was called.
 ///   - line: The line number on which failure occurred. Defaults to the line number on which this function was called.
@@ -101,6 +108,7 @@ public func assertSnapshots<Value, Format>(
   as strategies: [Snapshotting<Value, Format>],
   record recording: Bool = false,
   timeout: TimeInterval = 5,
+  autoRecord: Bool = true,
   file: StaticString = #file,
   testName: String = #function,
   line: UInt = #line
@@ -112,6 +120,7 @@ public func assertSnapshots<Value, Format>(
       as: strategy,
       record: recording,
       timeout: timeout,
+      autoRecord: autoRecord,
       file: file,
       testName: testName,
       line: line
@@ -156,6 +165,7 @@ public func assertSnapshots<Value, Format>(
 ///   - recording: Whether or not to record a new reference.
 ///   - snapshotDirectory: Optional directory to save snapshots. By default snapshots will be saved in a directory with the same name as the test file, and that directory will sit inside a directory `__Snapshots__` that sits next to your test file.
 ///   - timeout: The amount of time a snapshot must be generated in.
+///   - autoRecord: when the test fails it will override the original but still throw the error
 ///   - file: The file in which failure occurred. Defaults to the file name of the test case in which this function was called.
 ///   - testName: The name of the test in which failure occurred. Defaults to the function name of the test case in which this function was called.
 ///   - line: The line number on which failure occurred. Defaults to the line number on which this function was called.
@@ -167,6 +177,7 @@ public func verifySnapshot<Value, Format>(
   record recording: Bool = false,
   snapshotDirectory: String? = nil,
   timeout: TimeInterval = 5,
+  autoRecord: Bool = true,
   file: StaticString = #file,
   testName: String = #function,
   line: UInt = #line
@@ -297,6 +308,9 @@ public func verifySnapshot<Value, Format>(
         }
         #endif
       }
+      if autoRecord {
+        try snapshotting.diffing.toData(diffable).write(to: snapshotFileUrl)
+      }

       let diffMessage = diffTool
         .map { "\($0) \"\(snapshotFileUrl.path)\" \"\(failedSnapshotFileUrl.path)\"" }

We initially tried to write it as an extension, but the snapshotFileUrl unfortunately is sometimes autogenerated and cannot be easily extracted.

[dankinsoid]

Sometimes I need to rewrite snapshots manually before pulling and now I have to delete all failed snapshot first and CI tools cannot help me here, so the autoRecord option would be really helpful.