In streaming architectures that are compatible with the Reactive Streams protocol information travels along a sequence of streaming stages in both directions. Data elements flow downstream and demand signals flow upstream. This has interesting consequences for fan-outs and fan-ins as well as injecting and flattening.
When incoming data elements are to be split (i.e. distributed) to several downstream stages (as in a fan-out) the demand signaled by those downstreams has to be merged. When data elements coming in from several upstreams are to be merged (as in a fan-in) the demand signaled by the downstream stage has to be split (i.e. distributed) to the upstreams. Both structures are quite symmetric.
This symmetry is not superficial. It manifests itself also in the semantics of the individual transformations available for both sides. For example, let’s compare the fanOutRoundRobin and the fanInRoundRobin transformations:
The fanOutRoundRobin stage distributes data elements coming in from its upstream across its downstreams in a round-robin fashion. The first element goes to the first downstream, the second to the second, and so on. It doesn’t matter whether the respective target downstream has already signaled demand. If it hasn’t the fan-out stage waits until demand is signalled or the downstreams cancels.
The fanInRoundRobin stage distributes demand coming in from its downstream across its upstreams in a round-robin fashion. The request for the first element goes to the first upstream, the request for the second to the second, and so on. It doesn’t matter whether the respective target upstream has already delivered an element. If it hasn’t the fan-in stage waits until an element arrives or the upstream completes.
As you can see, the descriptions of the two transformations directly mirror each other.
Since Injecting Transformations produce a “stream of downstreams” they can be regarded as a kind of “dynamic fan-out”, where the number of downstreams can vary across the life-time of the stream.
And similarly Flattening Transformations must usually be able to deal with a dynamic number of active upstreams, as they consume a “stream of upstreams”.
This diagram shows the high-level relationships between the discussed transformation categories:
|all at once
|the next in
|the one selected
depending on the