Composing Effects into Tasks and Workflows
Data science applications tend to be built by composing \emph{tasks}:
discrete manipulations of data. These tasks are arranged in directed
acyclic graphs, and many frameworks exist within the data science
community supporting such a structure, which is called a \emph{workflow}.
In realistic applications, we want to be able to both \emph{analyze}
a workflow in the absence of data, and to \emph{execute} the workflow
with data.
This paper combines effect handlers with arrow-like structures to abstract out
data science tasks. This combination of techniques enables a modular design of
workflows. Additionally, these workflows can both be analyzed prior to running (e.g., to
provide early failure) and run conveniently. Our work is directly motivated by
real-world scenarios, and we believe that our approach is applicable to new
data science and machine learning applications and frameworks.
Fri 28 AugDisplayed time zone: Eastern Time (US & Canada) change
11:30 - 13:00 | |||
11:30 30mTalk | A Graded Monad for Deadlock-Free Concurrency (Functional Pearl) Haskell DOI | ||
12:00 30mTalk | Composing Effects into Tasks and Workflows Haskell Yves Parès Tweag I/O, France, Jean-Philippe Bernardy University of Gothenburg, Sweden, Richard A. Eisenberg Tweag I/O DOI | ||
12:30 30mTalk | Effect Handlers in Haskell, Evidently Haskell DOI |