Is your feature request related to a problem? Please describe.
Currently Linea can extract ("slice") source code required to produce some variable.
i.e. linea.slice("c") in
a = 2
b = 2
c = a + b
d = c / a
would produce
as a does not depend ond
However, one may want to output the entire code, including d
Describe the solution you'd like
"Slicing" the entire code would produce the code identical to the input, so what is the point?
In the trivial example above - none, however, leveraging additional capabilities of Linea we can produce useful outcomes:
- Airflow (Perfect. Argo, Ray ...) DAG encompassing the entire code
- Modularization for effectiveness and efficiency - we may output 2 separate functions producing
a and d, which can be run parallel, restarted/cache independently etc.
- Replacing in-mem pandas object by distributed datasets (Dask, Spar, Ray ...) for scalability.
Is your feature request related to a problem? Please describe.
Currently Linea can extract ("slice") source code required to produce some variable.
i.e.
linea.slice("c")inwould produce
as
adoes not depend ondHowever, one may want to output the entire code, including
dDescribe the solution you'd like
"Slicing" the entire code would produce the code identical to the input, so what is the point?
In the trivial example above - none, however, leveraging additional capabilities of Linea we can produce useful outcomes:
aandd, which can be run parallel, restarted/cache independently etc.