Streamz helps you build pipelines to manage continuous streams of data. It is simple to use in simple cases, but also supports complex pipelines that involve branching, joining, flow control, feedback, back pressure, and so on.
Optionally, Streamz can also work with both
Pandas and cuDF dataframes, to provide sensible streaming operations on continuous tabular data.
To learn more about how to use streams, visit (Core documentation) . Motivation
Continuous data streams arise in many applications like the following:
- Log processing from web servers
- Financial time series
- Scientific instrument data like telemetry or image processing pipelines
Machine learning pipelines for real-time and on-line learning
Sometimes these pipelines are very simple. , with a linear sequence of processing steps:
Why not Python generator expressions?
¶
Python users often manage continuous sequences of data with iterators or generator expressions.
def : a ,
b =
, 1 while True : yield a a ,
b =b) , (a
b sequence =
( (f)
n ) for (n
in fib ()
However iterators become challenging when you want to fork them or control the flow of data. Typically people rely on tools like itertools.tee , and
zip .
Python users often manage continuous sequences of data with iterators or generator expressions.
However iterators become challenging when you want to fork them or control the flow of data. Typically people rely on tools like itertools.tee , and
GIPHY App Key not set. Please check settings