Enable Single Print when Generating Based on Swifttemplate

[art-divin]

Brief

Enables a single point of output into STDOUT during codegen from swifttemplate

Context

This PR changes the way swifttemplate files are processed, and makes it possible to generate code in swifttemplate using async/await or other concurrency.

Technical Details

For a use case when there are a lot of types to be processed in a for-loop, it is important to allow use of concurrency upon processing each type. Currently, Sourcery only allows code generation using a single-threaded model, where the contents of swifttemplate are processed and print statements are inserted in every single point where Sourcery would generate output after template parsing.

Now, it would be possible to use async/await within swifttemplate for processing types and other facilities of processed swift code (AST), and only output the contents once in the very end.

Example

let's consider the following code in swifttemplate:

1 <%_ for type in types.all
2                 where (type is Struct || type is Enum)
3                && (type.implements["AutoDecodable"] != nil || type.implements["AutoEncodable"] != nil) { -%>
4    <%_ let codingKeys = codingKeysFor(type) -%>
5    <%_ if let codingKeysType = type.containedType["CodingKeys"] as? Enum, codingKeys.generated.count > 0 { -%>
6 // sourcery:inline:auto:<%= codingKeysType.name %>.AutoCodable
7        <%_ for key in codingKeys.generated { -%>
8        case <%= key %>
9        <%_ } -%>
10 // sourcery:end
11 <%_ } -%>

here, line 1 would iterate over all types, that is, it might be a collection of 100000 types, and sequentially process each one of them.
But what if we do not need this sequential processing, and instead could leverage all of the M*-SoC CPUs, to process types in the codebase?
What if sourcery is run on a Linux machine with 128 CPUs? No possibility to leverage all those CPUs at once.

Now, with this PR, the following is possible, but let's first consider how the given example emits Swift code exactly:

• line 6 is printed as // sourcery:inline:auto:TypeName.name.AutoCodable as a comment in the generated source file
• line 10 is printed as // sourcery:end as a comment in the generated source file

This is happening using print statement emitted into the intermittent source file that SwiftTemplate binary uses to process the input.
Now, using print statement is inefficient on its own:

1. swifttemplate file might include dozens of these statements, all put throughtout the generated intermittent file
2. print is expensive in terms of File I/O compared to a in-memory buffer which is printed just once

This PR introduces that buffer named sourceryBuffer, and the following code is now possible:

1 <%
2 func generateAsync(_ type: Type) async -> String {
3  var sourceryBuffer = ""
4  if (type is Struct || type is Enum),
5    (type.implements["AutoDecodable"] != nil || type.implements["AutoEncodable"] != nil) {
6    let codingKeys = codingKeysFor(type)
7    if let codingKeysType = type.containedType["CodingKeys"] as? Enum, codingKeys.generated.count > 0 { -%>
8 <%_ // sourcery:inline:auto:<%= codingKeysType.name %>.AutoCodable -%>
9    <%_ for key in codingKeys.generated { -%>
10    <%_ case <%= key %> -%>
11    <%_ } -%>
12 <%_ // sourcery:end -%>
13    <%_ } -%>
14  }
15  return sourceryBuffer
16 }
17 -%>
18
19 let content = try await withThrowingTaskGroup(of: String.self) { group in
20     for type in all {
21         group.addTask {
22            await generateAsync(type)
23         }
24     }
25     var fullContent = ""
26     for try await content in group {
27         fullContent.append(content)
28     }
29     return fullContent
30 }
31 <%= content -%>