Meta Programming with Streams and Pipes
-
Upload
darach -
Category
Technology
-
view
1.837 -
download
2
description
Transcript of Meta Programming with Streams and Pipes
Essentially, programs that can be created by themselves.
We’re twisting the term a little for streams
We will be defining streaming constructs using streaming constructs
By streams I mean beams which are just streams and pipes for compute as opposed to IO
Any info? Please contact #nodeland police at @cianomaidin on twitter…
Readable Streams IO / throughput oriented
Events – data, end, error, close
Methods – pause, resume, end, destroy
Writeable Streams IO / throughput oriented
Events - drain, pause, resume, error, close
Methods – write, end, destroy
IO Oriented
Finite, not infinite
Asynchronous
Lazy
Assumes in transit data, not in memory forms
Great for IO. Not efficient for compute
Sources Where data pours in
Sinks Where results pop out
Operators Filters – Drop or pass on data based on a UDF Transforms – Mutate data based on a UDF Branch with Filters Combine with Sinks
Compute Oriented
Infinite, not finite
Synchronous
Eager
Assumes in memory form, not encoded or wire data
Convenient for compute. Not good for IO
Eager: a.pipe(operation).pipe(z).pipe(a);
Implies:
RangeError: Maximum call stack size exceeded
npm install beam Compute oriented streams Branch, Combine, Filter, Transform data Extend with goto and modules.
npm install eep Beam will be integrated into eep soon Eep provides aggregate windows Slide, Tumble, Monotonic, Periodic Stats Library Ordering now supported
npm install nodesapiens Robosapiens V1 for node.js
Code on github https://github.com/darach