Streamline high-arity `combineLatest` and `zip`. #345

andersio · 2017-04-23T10:42:15Z

Preceded by ReactiveCocoa/ReactiveCocoa#2521.
Fixes #156.

One lock and one Signal regardless of the arity or the number of signals.
Stack friendly.
Human maintainable IMO.

Test item, 50000 iterations	`master`	This PR	Delta	Speedup
Tenary `combineLatest`, `Signal`	0.370 sec	0.205 sec	-18%	1.80x
Ternary `combineLatest`, `SignalProducer`	0.589 sec	0.313 sec	-47%	1.88x
6-ary `combineLatest`, `Signal`	1.733 sec	0.440 sec	-75%	3.94x
6-ary `combineLatest`, `SignalProducer`	2.541 sec	0.651 sec	-74%	3.90x
Blowing Main Stack (1 iteration)	4950	7450	N/A	N/A

Note

combineLatest and zip adds signals in a chain to a Singal.AggregateBuilder, which constructs a Signal when the chain ends with make(). Singal.AggregateBuilder encapsulates all the type erasure & recovery.

In ReactiveCocoa/ReactiveCocoa#2521, a primary dissent was that it introduced risks that the compiler cannot statically validate. This PR does not, and cannot mitigate, such risk with the performance goal due to the lack of variadic generics.

Note: Xcode 8.3.2, i5 6360U, Whole Module Optimisation. Benchmark Source (Gist).

mdiep

I love that this is getting addressed. 🤘

mdiep · 2017-05-02T12:48:45Z

Sources/Signal.swift

+				_isInitialized = values.reduce(true, { $0 && !($1 is Placeholder) })
+				return _isInitialized
+			}
+		}


I don't love using the term initialized. How about flipping the meaning and naming it hasEmptySignal?

mdiep · 2017-05-02T12:52:24Z

Sources/Signal.swift

+
+		private var values: ContiguousArray<Any>
+		private var completionCount: Int
+		private var _isInitialized: Bool


It's very minor, but in cases like this, I'd put the cached property immediately before the uncached version.

private let action: (ContiguousArray<Any>) -> Void private var _isInitialized: Bool private var isInitialized: Bool { … }

mdiep · 2017-05-02T12:53:29Z

Sources/Signal.swift

+					return true
+				}
+
+				_isInitialized = values.reduce(true, { $0 && !($1 is Placeholder) })


Trailing closure syntax?

mdiep · 2017-05-02T12:56:14Z

Sources/Signal.swift

+	private struct CombineLatestStrategy: SignalAggregateStrategy {
+		private struct Placeholder {}
+
+		private var values: ContiguousArray<Any>


Would it be better to use an Optional?

private var values: ContiguousArray<Any?>

The shared aggregator interface uses (ContiguousArray<Any>) -> Void, since zip does not need the optionality.

mdiep · 2017-05-02T12:57:30Z

Sources/Signal.swift

+		mutating func update(_ value: Any, at position: Int) -> Bool {
+			values[position].append(value)
+
+			if values.reduce(true, { $0 && !$1.isEmpty }) {


Maybe add a computed property for this to aid readability? Or add a mutating func that returns a ContiguousArray<Any>? of the values to send?

mdiep · 2017-05-02T12:58:30Z

Sources/Signal.swift

+				var buffer = ContiguousArray<Any>()
+				buffer.reserveCapacity(values.count)
+
+				for index in values.startIndex ..< values.endIndex {


values.indices?

Collection.indices holds a strong reference to the collection, and is not recommended if you gonna do in-place mutation.

mdiep · 2017-05-02T13:00:51Z

Sources/Signal.swift

+
+				action(buffer)
+
+				if Swift.zip(values, isCompleted).contains(where: { $0.isEmpty && $1 }) {


This conditional is repeated in complete below. I think it'd be good to make this into a computed property.

mdiep · 2017-05-02T13:02:50Z

Sources/Signal.swift

+
+		@discardableResult
+		func add<U>(_ signal: Signal<U, Error>) -> Self {
+			precondition(startHandlers != nil, "Signals cannot be registered after the state has started.")


Can we do this as part of init instead?

I think each make below could become an init that calls a top-level make with the signals that it wants to start.

mdiep · 2017-05-02T13:03:48Z

Sources/Signal.swift

+	}
+
+	private final class AggregateBuilder<Strategy: SignalAggregateStrategy> {
+		private var startHandlers: [(Int, CompositeDisposable, Atomic<Strategy>, @escaping (Event<Never, Error>) -> Void) -> Void]?


This seems like it would benefit from a struct. It's a bit large as a type.

NachoSoto · 2017-05-02T14:20:57Z

Sources/Signal.swift

+			return self
+		}
+
+		private func make<Composite>(count: Int, _ transform: @escaping (ContiguousArray<Any>) -> Composite) -> Signal<Composite, Error> {


Yikes :( it makes me sad that we lose type information here.

Yeah. 😞 But until the type system is powerful enough, I don't mind some internal type erasure.

andersio · 2017-05-02T14:51:58Z

@mdiep 6ee1fe7 probably still doesn't do what you asked, but it further encapsulates the unsafe implementation details in one place.

I have no idea how to make nested calls possible, but I don't think it could be more compact than packing by hand as it is now.

andersio · 2017-05-02T16:10:55Z

Managed to find a way. But we would have to pull back repack in TupleExtensions.swift. It also takes 2n - 3 function calls for a n-arity operator to pack its results. This would cause unnecessary copying across the function call boundaries, since the compiler does not have magic to flatten ((...T), U) to (...T, U), and we have to manually un- and repack the tuples.

6ee1fe7 (this PR) packs the tuple in one call regardless of the arity, since it is hardcoded. It is also space efficient, since the transform is a thin closure and does not need to capture closure references to repack stuff.

IMO nesting isn't worth it.

struct Builder<Composite, E> {
    private let count: Int
    private let transform: (ContiguousArray<Any>) -> Composite

    func append<U, NewComposite>(_ signal: S<U, E>, _ pack: @escaping (Composite, Any) -> NewComposite) -> Builder<NewComposite, E> {
        let transform = self.transform
        let count = self.count 
        return .init(count: count + 1) { pack(transform($0), $0[count]) }
    }
}

let _: Builder<(A, B, C, D, E), E> = Builder<(A, B), E>(a, b)
    .append(c) { ($0.0, $0.1, $1 as! C) } // or use `repack`.
    .append(d) { ($0.0, $0.1, $0.2, $1 as! D) } // or use `repack`.
    .append(e) { ($0.0, $0.1, $0.2, $0.3, $1 as! E) } // or use `repack`.

mdiep · 2017-05-03T12:55:59Z

This is what I was thinking:

extension Signal {
  func erased() -> Signal<Any, Error> { … }
}

class Combiner<Strategy> {
  var startHandlers: […]

  init<Composite>(
    signals: [Signal<Any, Error>],
    _ transform: @escaping (ContiguousArray<Any>) -> Composite
  ) -> Signal<Composite, Error> {
    // the code that was in make(count:_:), but also the code from add(_:)
  }

  convenience init<A, B>(
    _ a: Signal<A, Error>,
    _ b: Signal<B, Error>
  ) -> Signal<(A, B>), Error> {
    self.init([a.erased(), b.erased()]) { ($0[0] as! A, $0[1] as! B) }
  }

  …
}

andersio · 2017-05-03T15:20:40Z

I feel like 6ee1fe7 is pretty close, except for not being in the same place.

extension Signal {
	private convenience init<Strategy, A, B, C, D>(_ strategy: Strategy.Type, _ a: Signal<A, Error>, _ b: Signal<B, Error>, _ c: Signal<C, Error>, _ d: Signal<D, Error>) where Value == (A, B, C, D), Strategy: SignalAggregateStrategy {
		self.init(AggregateBuilder<Strategy>().add(a).add(b).add(c).add(d)) {
			return ($0[0] as! A, $0[1] as! B, $0[2] as! C, $0[3] as! D)
	 	}
	}
}

mdiep

I'm onboard with the builder design you've used, but I've got a few other requests.

mdiep · 2017-05-05T13:16:02Z

Sources/SignalProducer.swift

+	}
+}
+
+private func flattenStart<A, B, Error>(_ disposable: CompositeDisposable, _ a: SignalProducer<A, Error>, _ b: SignalProducer<B, Error>, _ start: (Signal<A, Error>, Signal<B, Error>) -> Void) {


Can you add some documentation to these?

Could these be structured after lift instead?

mdiep · 2017-05-05T13:17:51Z

Sources/SignalProducer.swift

@@ -2200,3 +2212,66 @@ extension SignalProducer where Value == Date, Error == NoError {
 		}
 	}
 }
+
+extension SignalProducer {
+	fileprivate func startWithSignal(interruptingBy disposable: CompositeDisposable, setup: (Signal<Value, Error>) -> Void) {


Can you add some documentation for this?

mdiep · 2017-05-05T13:20:27Z

Sources/Signal.swift

+		private let action: (ContiguousArray<Any>) -> Void
+
+		private var _hasAllSentInitial: Bool
+		private var hasAllSentInitial: Bool {


This reads a little funny. Maybe rename to haveAllSentInitial or hasAllInitialValues?

mdiep · 2017-05-05T13:21:23Z

Sources/Signal.swift

+
+		private var hasCompletedAndEmptiedSignal: Bool { return Swift.zip(values, isCompleted).contains(where: { $0.isEmpty && $1 }) }
+		private var canEmit: Bool { return values.reduce(true) { $0 && !$1.isEmpty } }
+		private var isAllCompleted: Bool { return isCompleted.reduce(true) { $0 && $1 } }


Maybe rename this to areAllCompleted so it reads nicely in English?

mdiep · 2017-05-05T13:21:54Z

Sources/Signal.swift

+		private var isCompleted: ContiguousArray<Bool>
+		private let action: (ContiguousArray<Any>) -> Void
+
+		private var hasCompletedAndEmptiedSignal: Bool { return Swift.zip(values, isCompleted).contains(where: { $0.isEmpty && $1 }) }


Would you mind putting these over multiple lines? I think that makes them more readable.

Binary to 10-ary `combineLatest` and `zip` operators now share a unified implementation, which has a constant calling and locking overhead, regardless of the arity.

These operators now call their `Signal` or `SignalProducer` counterpart of the same arity, instead of composing with tuple repacking on their own.

andersio added the work in progress label Apr 23, 2017

andersio changed the title ~~Streamline high-arity combineLatest and zip.~~ Streamline high-arity combineLatest. Apr 23, 2017

andersio added work in progress and removed work in progress labels Apr 23, 2017

andersio force-pushed the combine-latest-refactor branch from 331f0ca to 2e9a468 Compare April 23, 2017 13:46

andersio removed the work in progress label Apr 23, 2017

andersio force-pushed the combine-latest-refactor branch 8 times, most recently from 84638f1 to 4b7e7c2 Compare April 24, 2017 10:44

andersio changed the title ~~Streamline high-arity combineLatest.~~ Streamline high-arity combineLatest and zip. Apr 24, 2017

andersio force-pushed the combine-latest-refactor branch 9 times, most recently from b1a932e to d99848b Compare April 24, 2017 14:19

mdiep requested changes May 2, 2017

View reviewed changes

NachoSoto reviewed May 2, 2017

View reviewed changes

mdiep requested changes May 5, 2017

View reviewed changes

andersio added 4 commits May 11, 2017 18:00

Streamline high-arity combineLatest and zip.

3f9e931

Binary to 10-ary `combineLatest` and `zip` operators now share a unified implementation, which has a constant calling and locking overhead, regardless of the arity.

Refactor high-arity lifted producer and property operators.

e0105df

These operators now call their `Signal` or `SignalProducer` counterpart of the same arity, instead of composing with tuple repacking on their own.

Revamped the signal aggregate building API.

b924fe2

Refactor the signal aggregate strategies.

efd0d87

andersio force-pushed the combine-latest-refactor branch from cfab84d to c9fc25f Compare May 11, 2017 10:01

Addressed comments.

2b1fe54

andersio force-pushed the combine-latest-refactor branch from c9fc25f to 2b1fe54 Compare May 11, 2017 10:05

andersio requested a review from mdiep May 11, 2017 10:05

mdiep approved these changes May 13, 2017

View reviewed changes

mdiep merged commit f509bfd into master May 13, 2017

mdiep deleted the combine-latest-refactor branch May 13, 2017 22:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Streamline high-arity `combineLatest` and `zip`. #345

Streamline high-arity `combineLatest` and `zip`. #345

andersio commented Apr 23, 2017 •

edited

Loading

mdiep left a comment

mdiep May 2, 2017

mdiep May 2, 2017

mdiep May 2, 2017

mdiep May 2, 2017

andersio May 2, 2017

mdiep May 2, 2017

mdiep May 2, 2017

andersio May 2, 2017 •

edited

Loading

mdiep May 2, 2017

mdiep May 2, 2017

mdiep May 2, 2017

NachoSoto May 2, 2017

mdiep May 2, 2017

andersio commented May 2, 2017 •

edited

Loading

andersio commented May 2, 2017 •

edited

Loading

mdiep commented May 3, 2017

andersio commented May 3, 2017 •

edited

Loading

mdiep left a comment

mdiep May 5, 2017

mdiep May 5, 2017

mdiep May 5, 2017

mdiep May 5, 2017

mdiep May 5, 2017

mdiep May 5, 2017


		action(buffer)

		if Swift.zip(values, isCompleted).contains(where: { $0.isEmpty && $1 }) {

Streamline high-arity combineLatest and zip. #345

Streamline high-arity combineLatest and zip. #345

Conversation

andersio commented Apr 23, 2017 • edited Loading

Note

mdiep left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andersio May 2, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andersio commented May 2, 2017 • edited Loading

andersio commented May 2, 2017 • edited Loading

mdiep commented May 3, 2017

andersio commented May 3, 2017 • edited Loading

mdiep left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Streamline high-arity `combineLatest` and `zip`. #345

Streamline high-arity `combineLatest` and `zip`. #345

andersio commented Apr 23, 2017 •

edited

Loading

andersio May 2, 2017 •

edited

Loading

andersio commented May 2, 2017 •

edited

Loading

andersio commented May 2, 2017 •

edited

Loading

andersio commented May 3, 2017 •

edited

Loading