-
-
Notifications
You must be signed in to change notification settings - Fork 373
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixed "UnsupportedOperation" exception in highs_api.py #675
base: master
Are you sure you want to change the base?
Conversation
This patch solves two issues in HiGHS API (actualSolve method): 1) Code for parsing a list of additional options had a typo in method name (should be startswith) 2) The new method of starting a subprocess caused UnsupportedOperation exception in Windows (tested with Anaconda Python and WinPython). I replaced this code with the code from PuLP 2.7.0 release. Here's the stack trace for the error that previously appeared --------------------------------------------------------------------------- UnsupportedOperation Traceback (most recent call last) Cell In[30], line 1 ----> 1 prob.solve(solver=highs) 2 print(f"Status: {pl.LpStatus[prob.status]}") 3 print(f"EV = {-pl.value(prob.objective): .0f}") File ~\.conda\envs\opt\Lib\site-packages\pulp\pulp.py:1920, in LpProblem.solve(self, solver, **kwargs) 1918 # time it 1919 self.startClock() -> 1920 status = solver.actualSolve(self, **kwargs) 1921 self.stopClock() 1922 self.restoreObjective(wasNone, dummyVar) File ~\.conda\envs\opt\Lib\site-packages\pulp\apis\highs_api.py:145, in HiGHS_CMD.actualSolve(self, lp) 143 with open(tmpOptions, "w") as options_file: 144 options_file.write("\n".join(file_options)) --> 145 process = subprocess.run(command, stdout=sys.stdout, stderr=sys.stderr) 147 # HiGHS return code semantics (see: ERGO-Code/HiGHS#527 (comment)) 148 # - -1: error 149 # - 0: success 150 # - 1: warning 151 if process.returncode == -1: File ~\.conda\envs\opt\Lib\subprocess.py:548, in run(input, capture_output, timeout, check, *popenargs, **kwargs) 545 kwargs['stdout'] = PIPE 546 kwargs['stderr'] = PIPE --> 548 with Popen(*popenargs, **kwargs) as process: 549 try: 550 stdout, stderr = process.communicate(input, timeout=timeout) File ~\.conda\envs\opt\Lib\subprocess.py:992, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group) 973 raise ValueError(f"User ID cannot be negative, got {uid}") 975 # Input and output objects. The general principle is like 976 # this: 977 # (...) 987 # are -1 when not using PIPEs. The child objects are -1 988 # when not redirecting. 990 (p2cread, p2cwrite, 991 c2pread, c2pwrite, --> 992 errread, errwrite) = self._get_handles(stdin, stdout, stderr) 994 # From here on, raising exceptions may cause file descriptor leakage 995 996 # We wrap OS handles *before* launching the child, otherwise a 997 # quickly terminating child could make our fds unwrappable 998 # (see #8458). 1000 if _mswindows: File ~\.conda\envs\opt\Lib\subprocess.py:1384, in Popen._get_handles(self, stdin, stdout, stderr) 1381 c2pwrite = msvcrt.get_osfhandle(stdout) 1382 else: 1383 # Assuming file-like object -> 1384 c2pwrite = msvcrt.get_osfhandle(stdout.fileno()) 1385 c2pwrite = self._make_inheritable(c2pwrite) 1387 if stderr is None: File ~\.conda\envs\opt\Lib\site-packages\ipykernel\iostream.py:311, in OutStream.fileno(self) 309 return self._original_stdstream_copy 310 else: --> 311 raise io.UnsupportedOperation("fileno") UnsupportedOperation: fileno
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. The string method I think was already fixed in a previous PR. I've made some suggestions so the style matches the style of other solve APIs (CBC, CPLEX, GUROBI). You can check the other files to get inspiration.
if self.msg: | ||
pipe = None | ||
else: | ||
pipe = open(os.devnull, "w") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you define pipe
but then you don't use it.
# The following method of starting the subprocess causes exception UnsupportedOperation("fileno") in iostream.py in Windows | ||
# process = subprocess.run(command, stdout=sys.stdout, stderr=sys.stderr, universal_newlines=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please delete these lines
else: | ||
pipe = open(os.devnull, "w") | ||
|
||
lp_status = None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is not used
with subprocess.Popen( | ||
command, | ||
stdout=subprocess.PIPE, | ||
stderr=subprocess.STDOUT, | ||
universal_newlines=True, | ||
) as proc, open(tmpLog, "w") as log_file: | ||
for line in proc.stdout: | ||
if self.msg: | ||
sys.__stdout__.write(line) | ||
log_file.write(line) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Check how it's done in the solve method of coin_api.COIN_CMD class
. Ideally we would like to have the same style:
if self.msg:
pipe = None
else:
pipe = open(os.devnull, "w")
process = subprocess.Popen(args, stdout=pipe, stderr=pipe, stdin=devnull)
if process.wait() == -1:
if pipe:
pipe.close()
raise PulpSolverError(
"Pulp: Error while trying to execute, use msg=True for more details"
+ self.path
)
if pipe:
pipe.close()
This patch solves two issues in HiGHS API (actualSolve method):
Here's the stack trace for the error that previously appeared ---------------------------------------------------------------------------
UnsupportedOperation Traceback (most recent call last)
Cell In[30], line 1
----> 1 prob.solve(solver=highs)
2 print(f"Status: {pl.LpStatus[prob.status]}")
3 print(f"EV = {-pl.value(prob.objective): .0f}")
File ~.conda\envs\opt\Lib\site-packages\pulp\pulp.py:1920, in LpProblem.solve(self, solver, **kwargs)
1918 # time it
1919 self.startClock()
-> 1920 status = solver.actualSolve(self, **kwargs)
1921 self.stopClock()
1922 self.restoreObjective(wasNone, dummyVar)
File ~.conda\envs\opt\Lib\site-packages\pulp\apis\highs_api.py:145, in HiGHS_CMD.actualSolve(self, lp)
143 with open(tmpOptions, "w") as options_file:
144 options_file.write("\n".join(file_options))
--> 145 process = subprocess.run(command, stdout=sys.stdout, stderr=sys.stderr)
147 # HiGHS return code semantics (see: ERGO-Code/HiGHS#527 (comment))
148 # - -1: error
149 # - 0: success
150 # - 1: warning
151 if process.returncode == -1:
File ~.conda\envs\opt\Lib\subprocess.py:548, in run(input, capture_output, timeout, check, *popenargs, **kwargs)
545 kwargs['stdout'] = PIPE
546 kwargs['stderr'] = PIPE
--> 548 with Popen(*popenargs, **kwargs) as process:
549 try:
550 stdout, stderr = process.communicate(input, timeout=timeout)
File ~.conda\envs\opt\Lib\subprocess.py:992, in Popen.init(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group)
973 raise ValueError(f"User ID cannot be negative, got {uid}")
975 # Input and output objects. The general principle is like
976 # this:
977 #
(...)
987 # are -1 when not using PIPEs. The child objects are -1
988 # when not redirecting.
990 (p2cread, p2cwrite,
991 c2pread, c2pwrite,
--> 992 errread, errwrite) = self._get_handles(stdin, stdout, stderr)
994 # From here on, raising exceptions may cause file descriptor leakage
995
996 # We wrap OS handles before launching the child, otherwise a
997 # quickly terminating child could make our fds unwrappable
998 # (see #8458).
1000 if _mswindows:
File ~.conda\envs\opt\Lib\subprocess.py:1384, in Popen._get_handles(self, stdin, stdout, stderr)
1381 c2pwrite = msvcrt.get_osfhandle(stdout)
1382 else:
1383 # Assuming file-like object
-> 1384 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
1385 c2pwrite = self._make_inheritable(c2pwrite)
1387 if stderr is None:
File ~.conda\envs\opt\Lib\site-packages\ipykernel\iostream.py:311, in OutStream.fileno(self)
309 return self._original_stdstream_copy
310 else:
--> 311 raise io.UnsupportedOperation("fileno")
UnsupportedOperation: fileno