Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed "UnsupportedOperation" exception in highs_api.py #675

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

postlogist
Copy link

This patch solves two issues in HiGHS API (actualSolve method):

  1. Code for parsing a list of additional options had a typo in method name (should be startswith)
  2. The new method of starting a subprocess caused UnsupportedOperation exception in Windows (tested with Anaconda Python and WinPython). I replaced this code with the code from PuLP 2.7.0 release.

Here's the stack trace for the error that previously appeared ---------------------------------------------------------------------------
UnsupportedOperation Traceback (most recent call last)
Cell In[30], line 1
----> 1 prob.solve(solver=highs)
2 print(f"Status: {pl.LpStatus[prob.status]}")
3 print(f"EV = {-pl.value(prob.objective): .0f}")

File ~.conda\envs\opt\Lib\site-packages\pulp\pulp.py:1920, in LpProblem.solve(self, solver, **kwargs)
1918 # time it
1919 self.startClock()
-> 1920 status = solver.actualSolve(self, **kwargs)
1921 self.stopClock()
1922 self.restoreObjective(wasNone, dummyVar)

File ~.conda\envs\opt\Lib\site-packages\pulp\apis\highs_api.py:145, in HiGHS_CMD.actualSolve(self, lp)
143 with open(tmpOptions, "w") as options_file:
144 options_file.write("\n".join(file_options))
--> 145 process = subprocess.run(command, stdout=sys.stdout, stderr=sys.stderr)
147 # HiGHS return code semantics (see: ERGO-Code/HiGHS#527 (comment))
148 # - -1: error
149 # - 0: success
150 # - 1: warning
151 if process.returncode == -1:

File ~.conda\envs\opt\Lib\subprocess.py:548, in run(input, capture_output, timeout, check, *popenargs, **kwargs)
545 kwargs['stdout'] = PIPE
546 kwargs['stderr'] = PIPE
--> 548 with Popen(*popenargs, **kwargs) as process:
549 try:
550 stdout, stderr = process.communicate(input, timeout=timeout)

File ~.conda\envs\opt\Lib\subprocess.py:992, in Popen.init(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group)
973 raise ValueError(f"User ID cannot be negative, got {uid}")
975 # Input and output objects. The general principle is like
976 # this:
977 #
(...)
987 # are -1 when not using PIPEs. The child objects are -1
988 # when not redirecting.
990 (p2cread, p2cwrite,
991 c2pread, c2pwrite,
--> 992 errread, errwrite) = self._get_handles(stdin, stdout, stderr)
994 # From here on, raising exceptions may cause file descriptor leakage
995
996 # We wrap OS handles before launching the child, otherwise a
997 # quickly terminating child could make our fds unwrappable
998 # (see #8458).
1000 if _mswindows:

File ~.conda\envs\opt\Lib\subprocess.py:1384, in Popen._get_handles(self, stdin, stdout, stderr)
1381 c2pwrite = msvcrt.get_osfhandle(stdout)
1382 else:
1383 # Assuming file-like object
-> 1384 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
1385 c2pwrite = self._make_inheritable(c2pwrite)
1387 if stderr is None:

File ~.conda\envs\opt\Lib\site-packages\ipykernel\iostream.py:311, in OutStream.fileno(self)
309 return self._original_stdstream_copy
310 else:
--> 311 raise io.UnsupportedOperation("fileno")

UnsupportedOperation: fileno

This patch solves two issues in HiGHS API (actualSolve method):
1) Code for parsing a list of additional options had a typo in method name (should be startswith)
2) The new method of starting a subprocess caused UnsupportedOperation exception in Windows (tested with Anaconda Python and WinPython). I replaced this code with the code from PuLP 2.7.0 release.

Here's the stack trace for the error that previously appeared
---------------------------------------------------------------------------
UnsupportedOperation                      Traceback (most recent call last)
Cell In[30], line 1
----> 1 prob.solve(solver=highs)
      2 print(f"Status: {pl.LpStatus[prob.status]}")
      3 print(f"EV = {-pl.value(prob.objective): .0f}")

File ~\.conda\envs\opt\Lib\site-packages\pulp\pulp.py:1920, in LpProblem.solve(self, solver, **kwargs)
   1918 # time it
   1919 self.startClock()
-> 1920 status = solver.actualSolve(self, **kwargs)
   1921 self.stopClock()
   1922 self.restoreObjective(wasNone, dummyVar)

File ~\.conda\envs\opt\Lib\site-packages\pulp\apis\highs_api.py:145, in HiGHS_CMD.actualSolve(self, lp)
    143 with open(tmpOptions, "w") as options_file:
    144     options_file.write("\n".join(file_options))
--> 145 process = subprocess.run(command, stdout=sys.stdout, stderr=sys.stderr)
    147 # HiGHS return code semantics (see: ERGO-Code/HiGHS#527 (comment))
    148 # - -1: error
    149 # -  0: success
    150 # -  1: warning
    151 if process.returncode == -1:

File ~\.conda\envs\opt\Lib\subprocess.py:548, in run(input, capture_output, timeout, check, *popenargs, **kwargs)
    545     kwargs['stdout'] = PIPE
    546     kwargs['stderr'] = PIPE
--> 548 with Popen(*popenargs, **kwargs) as process:
    549     try:
    550         stdout, stderr = process.communicate(input, timeout=timeout)

File ~\.conda\envs\opt\Lib\subprocess.py:992, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group)
    973         raise ValueError(f"User ID cannot be negative, got {uid}")
    975 # Input and output objects. The general principle is like
    976 # this:
    977 #
   (...)
    987 # are -1 when not using PIPEs. The child objects are -1
    988 # when not redirecting.
    990 (p2cread, p2cwrite,
    991  c2pread, c2pwrite,
--> 992  errread, errwrite) = self._get_handles(stdin, stdout, stderr)
    994 # From here on, raising exceptions may cause file descriptor leakage
    995
    996 # We wrap OS handles *before* launching the child, otherwise a
    997 # quickly terminating child could make our fds unwrappable
    998 # (see #8458).
   1000 if _mswindows:

File ~\.conda\envs\opt\Lib\subprocess.py:1384, in Popen._get_handles(self, stdin, stdout, stderr)
   1381     c2pwrite = msvcrt.get_osfhandle(stdout)
   1382 else:
   1383     # Assuming file-like object
-> 1384     c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
   1385 c2pwrite = self._make_inheritable(c2pwrite)
   1387 if stderr is None:

File ~\.conda\envs\opt\Lib\site-packages\ipykernel\iostream.py:311, in OutStream.fileno(self)
    309     return self._original_stdstream_copy
    310 else:
--> 311     raise io.UnsupportedOperation("fileno")

UnsupportedOperation: fileno
@CLAassistant
Copy link

CLAassistant commented Aug 4, 2023

CLA assistant check
All committers have signed the CLA.

Copy link
Collaborator

@pchtsp pchtsp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. The string method I think was already fixed in a previous PR. I've made some suggestions so the style matches the style of other solve APIs (CBC, CPLEX, GUROBI). You can check the other files to get inspiration.

Comment on lines +146 to +149
if self.msg:
pipe = None
else:
pipe = open(os.devnull, "w")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you define pipe but then you don't use it.

Comment on lines +153 to +154
# The following method of starting the subprocess causes exception UnsupportedOperation("fileno") in iostream.py in Windows
# process = subprocess.run(command, stdout=sys.stdout, stderr=sys.stderr, universal_newlines=True)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please delete these lines

else:
pipe = open(os.devnull, "w")

lp_status = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not used

Comment on lines +156 to 166
with subprocess.Popen(
command,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
universal_newlines=True,
) as proc, open(tmpLog, "w") as log_file:
for line in proc.stdout:
if self.msg:
sys.__stdout__.write(line)
log_file.write(line)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check how it's done in the solve method of coin_api.COIN_CMD class. Ideally we would like to have the same style:

if self.msg:
    pipe = None
else:
    pipe = open(os.devnull, "w")
process = subprocess.Popen(args, stdout=pipe, stderr=pipe, stdin=devnull)
if process.wait() == -1:
    if pipe:
        pipe.close()
    raise PulpSolverError(
        "Pulp: Error while trying to execute, use msg=True for more details"
        + self.path
    )
if pipe:
    pipe.close()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants