You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In _utils.py the create_environ() helper produces a dict that can be passed as the "env" arg to subprocess.run(), etc. It is used in Master.spawn_worker() (in _master.py). Currently it takes an "inherit_environ" arg (Bool) that corresponds to the "--inherit-environ" flag in the Runner CLI (see Runner.init() in _runner.py). This results in a potentially problematic situation.
The Problem
Let's say you have a benchmark script that internally relies on some environment variable that is defined relative to the commandline args given to the script. This environment variable may be set already or it might not. Regardless, you will be setting it to some new value. To make this work you need to do something like the following:
# This is a concrete example.os.environ["MY_VAR"] ="spam"ifrunner.args.my_flagelse"eggs"ifrunner.args.inherit_environisNone:
runner.args.inherit_environ= ["MY_VAR"]
else:
runner.args.inherit_environ.append("MY_VAR")
However, in some cases you can't leave the env var set (or maybe the env var could cause pyperf to break). Plus things are more complicated if you have more than one such env var.
The Solution
Consequently, in a benchmark script it would be nice to be able to give the actual env var pairs to Runner rather than doing the dance above. Here are some possible approaches to solve the problem:
allow Runner.args.inherit_environ to be a dict
create_environ() would be updated to do the right thing
this is probably too messy to be part of pyperf's public API
add something like Runner.env_vars to allow benchmarks to explicitly set env var values to be used in workers
create_environ() would grow a new "env" arg or "inherit_environ" would updated as above
add Runner.add_env_var(name, value=INHERIT)
this is like either of the two above, but probably a better public API for the functionality
The text was updated successfully, but these errors were encountered:
FWIW, I ended up using a small helper as a stopgap. (I had to set a different PYTHONPATH value, along with others.) It looks something like this:
defbench_command_env(runner, name, command, env):
ifrunner.args.inherit_environ:
runner.args.inherit_environ.extend(env)
else:
runner.args.inherit_environ=list(env)
before=dict(os.environ)
os.environ.update(env)
# At this point any use of env vars defined in "env" should be considered tainted.try:
returnrunner.bench_command(name, command)
finally:
os.environ.clear()
os.environ.extend(before)
ericsnowcurrently
changed the title
Allow
Allow setting updated/new env var values on Runner (rather than just inherit_environ).
Mar 19, 2021
ericsnowcurrently
changed the title
Allow setting updated/new env var values on Runner (rather than just inherit_environ).
Allow setting updated/new env var values on Runner.
Mar 19, 2021
ericsnowcurrently
changed the title
Allow setting updated/new env var values on Runner.
Allow setting updated/new env var *values* on Runner.
Mar 19, 2021
I know this feature may not cover all your cases, but please take a look at the new feature.
And also I think that your suggestion is also reasonable, I want to listen to other opinions from @vstinner and @pablogsal
In _utils.py the
create_environ()
helper produces a dict that can be passed as the "env" arg tosubprocess.run()
, etc. It is used inMaster.spawn_worker()
(in _master.py). Currently it takes an "inherit_environ" arg (Bool) that corresponds to the "--inherit-environ" flag in theRunner
CLI (see Runner.init() in _runner.py). This results in a potentially problematic situation.The Problem
Let's say you have a benchmark script that internally relies on some environment variable that is defined relative to the commandline args given to the script. This environment variable may be set already or it might not. Regardless, you will be setting it to some new value. To make this work you need to do something like the following:
However, in some cases you can't leave the env var set (or maybe the env var could cause pyperf to break). Plus things are more complicated if you have more than one such env var.
The Solution
Consequently, in a benchmark script it would be nice to be able to give the actual env var pairs to
Runner
rather than doing the dance above. Here are some possible approaches to solve the problem:Runner.args.inherit_environ
to be a dictcreate_environ()
would be updated to do the right thingRunner.env_vars
to allow benchmarks to explicitly set env var values to be used in workerscreate_environ()
would grow a new "env" arg or "inherit_environ" would updated as aboveRunner.add_env_var(name, value=INHERIT)
The text was updated successfully, but these errors were encountered: