-
First of all: congrats for the amazing tool you have developed! A question I have is whether it is possible to define global variables. Use case is as follows: globals:
varA: hello
varB: 2
default_kwargs:
task1:
x: 1
y: 2
steps:
- name: mystep
in:
- taskname: task3
- A: $varA
- B: $varB
- name: mystep
in:
- taskname: task1
- kwargs: $default_kwargs.task1
- x: 55
In my case, I'm writing my own Do you think it is possible to do that? Many thanks in advance! |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 5 replies
-
thanks for you kind words @mtazzari, much appreciated 🙌 ! it's absolutely possible to do what you want. in pypyr the easiest is to use formatting substitution expressions - these support complex types like your own dict/kwarg style arguments. I normally use the pypyr.steps.default step to set "global" values in context that I want to use later in the pipeline. You can also use pypyr.steps.contextsetf - the only difference is that So, just taking your quick example: # ./mypipeline.yaml
context_parser: pypyr.parser.keyvaluepairs
steps:
- name: pypyr.steps.default
in:
defaults:
varA: value A
varB: value B
varC: value C
default_kwargs:
task1:
x: 1
y: 2
z: '{varC}'
- name: mystep
in:
mystep:
taskname: task3
A: '{varA}'
B: '{varB}'
- name: mystep
in:
mystep:
taskname: task1
kwargs: '{default_kwargs[task1]}'
A: '{varA}'
B: '{varB}' So this you can run like this: $ pypyr mypipeline In this case it will just use the default values. If you want to override, you can do: $ pypyr mypipeline varA="value from cmd line" varB=arb in which case Notice that you can also nest formatting expressions - so I arbitrarily added an extra Hope this helps? [edit] Of course, depending on exactly what you're doing, but since you're writing your own step you don't even have to pass def run_step(context):
task_name = context['mystep']['taskname']
default_kwargs_for_this_step = context['default_kwargs'][task_name] |
Beta Was this translation helpful? Give feedback.
thanks for you kind words @mtazzari, much appreciated 🙌 !
it's absolutely possible to do what you want. in pypyr the easiest is to use formatting substitution expressions - these support complex types like your own dict/kwarg style arguments.
I normally use the pypyr.steps.default step to set "global" values in context that I want to use later in the pipeline. You can also use pypyr.steps.contextsetf - the only difference is that
default
only sets the values if they don't exist already, which is handy if you want to combine this with something like the pypyr.parser.keyvaluepairs parser to allow the user to override some of the defaults from the cmd line.contextsetf
by comparison, always …