Fan-In/Out Relationships

In streaming architectures that are compatible with the Reactive Streams protocol information travels along a sequence of streaming stages in both directions. Data elements flow downstream and demand signals flow upstream. This has interesting consequences for fan-outs and fan-ins as well as injecting and flattening.

Symmetry between Fan-Outs and Fan-Ins

Fan-Ins mirror Fan-Outs and vice versa

When incoming data elements are to be split (i.e. distributed) to several downstream stages (as in a fan-out) the demand signaled by those downstreams has to be merged. When data elements coming in from several upstreams are to be merged (as in a fan-in) the demand signaled by the downstream stage has to be split (i.e. distributed) to the upstreams. Both structures are quite symmetric.

This symmetry is not superficial. It manifests itself also in the semantics of the individual transformations available for both sides. For example, let’s compare the fanOutRoundRobin and the fanInRoundRobin transformations:

The fanOutRoundRobin stage distributes data elements coming in from its upstream across its downstreams in a round-robin fashion. The first element goes to the first downstream, the second to the second, and so on. It doesn’t matter whether the respective target downstream has already signaled demand. If it hasn’t the fan-out stage waits until demand is signalled or the downstreams cancels.

The fanInRoundRobin stage distributes demand coming in from its downstream across its upstreams in a round-robin fashion. The request for the first element goes to the first upstream, the request for the second to the second, and so on. It doesn’t matter whether the respective target upstream has already delivered an element. If it hasn’t the fan-in stage waits until an element arrives or the upstream completes.

As you can see, the descriptions of the two transformations directly mirror each other.

Correspondance between Fan-Out/In and Injecting/Flattening

In addition to the symmetry between fan-outs and fan-ins there is a strong correlation between fan-out and injecting as well as between fan-in and flattening.

Since Injecting Transformations produce a “stream of downstreams” they can be regarded as a kind of “dynamic fan-out”, where the number of downstreams can vary across the life-time of the stream.
And similarly Flattening Transformations must usually be able to deal with a dynamic number of active upstreams, as they consume a “stream of upstreams”.

This diagram shows the high-level relationships between the discussed transformation categories:

Relationship Diagram

Transformation Table

The following table shows the relationships between all primary fan-out, fan-in, injecting and flattening transformations along with a short description of their respective core semantics:

to/from Fan-Out Fan-In Inject Flatten
the first
available
fanOutToAny

fanInMerge
fanInToCoproduct
fanInToSum
injectToAny flattenMerge
all at once

fanOutBroadcast
fanOutUnZip
fanInToTuple
fanInToHList
fanInToProduct
injectBroadcast flattenToSeq
the next in
cycling order
fanOutRoundRobin fanInRoundRobin injectRoundRobin flattenRoundRobin
the current
(alive) one
fanOutSequential fanInConcat injectSequential flattenConcat
the one selected
depending on the
element
fanOutSwitch
fanInSorted
groupBy
split
flattenSorted