On NoFlo database applications

A few thoughts on design of NoFlo backends frequently talking to a RDBMS

A 2 minute read. Published on July 27, 2016

On NoFlo Database Applications

Thinking over common parts of database applications…

Scrappy notes

… I came to some conclusions below

Common application architecture

Steps common to all route graphs

  • The only processing step that is often present and easy to decompose is Validate (fig. I and II).

String of pearls or Pipeline or Request backbone

  • Request flow should be serial, it simplifies synchronization and understanding of the flow.
  • Each bone in the spine of the graph or step on the pipeline can process some data in parallel. But as the flow should be unidirectional, scatter-gather topology (fig. IV) is preferred over side jobs (fig. III).

Database operations

  • Database queries are all different across the application. Input data needs to be extracted/filtered first, then inserted into specific parts of the query, then the result rowset needs to be filtered too (fig. V) to be used down the stream.
  • FBP-SQL would need either a lot of components to cover every specific query configuration, or a rather bushy query builder to compose complex queries in the graph.
  • Database operations are very granular in the app and appear in any domain of the app, so Flux Store makes no sense.
  • Using a single Database component in the graph makes too many wires.
  • Using one instance per query uses more memory.