Skip to content

Commit

Permalink
PERF: Implicit check for field count in loadtxt.
Browse files Browse the repository at this point in the history
In the fast-path of loadtxt, the conversion to np.void implicitly checks
the number of fields.  Removing the explicit length check saves ~5% for
the largest loads (100_000 rows) of numeric scalar types.
  • Loading branch information
anntzer committed Aug 26, 2021
1 parent 65fee8e commit a126896
Showing 1 changed file with 7 additions and 5 deletions.
12 changes: 7 additions & 5 deletions numpy/lib/npyio.py
Expand Up @@ -1137,18 +1137,20 @@ def convert_row(vals):
for i, (lineno, words) in enumerate(lineno_words_iter):
if usecols:
words = usecols_getter(words)
elif len(words) != ncols:
raise ValueError(
f"Wrong number of columns at line {lineno}")
try:
X[i] = tuple(words) # Try implicit conversion of strs.
continue # OK, done.
except IndexError:
# Resize, and, for simplicity, use explicit converters too.
X.resize(2 * len(X), refcheck=False)
except ValueError:
# Fallback to explicit converters.
pass
# ValueError can be raised either by a length mismatch...
if len(words) != ncols:
raise ValueError(
f"Wrong number of columns at line {lineno}"
) from None
# Or because the explicit (more lenient) converter (below)
# is needed.
X[i] = convert_row(words)
if i is None:
X = None
Expand Down

0 comments on commit a126896

Please sign in to comment.