Masks, where, and the slice that is not a copy

Selecting with a condition

scores = np.array([0.91, 0.12, 0.77, 0.45, 0.98])
mask = scores > 0.5
mask                 # [ True, False,  True, False,  True]
scores[mask]         # [0.91, 0.77, 0.98]
mask.sum()           # 3 — True counts as 1

A comparison on an array produces a boolean array of the same shape, and indexing with it keeps the elements where it is True. Two lines replace a loop with an if, and they run at NumPy speed. Combine conditions with &, | and ~, each side in brackets — and/or do not work on arrays, and forgetting the brackets makes & bind tighter than > and raise a confusing error:

scores[(scores > 0.3) & (scores < 0.9)]      # [0.77, 0.45]
scores[~mask]                                # [0.12, 0.45]

Assignment through a mask changes only the selected elements:

scores[scores < 0.2] = 0.0

That is the array form of "zero out the weak scores", and it is the pattern under clipping, thresholding, and replacing bad values with a sentinel.

np.where

labels = np.where(scores > 0.5, "positive", "negative")

where(condition, a, b) picks from a where the condition holds and b elsewhere, elementwise, broadcasting all three. With one argument it returns the indices where the condition is true — np.where(scores > 0.5)[0] gives [0, 2, 4] — which is what you need to map a match back to the row of a table it came from. np.nonzero is the same thing under a clearer name. np.argwhere returns them one per row for multi-dimensional arrays.

np.clip(a, lo, hi) is the special case of "no smaller than, no larger than", and np.nan_to_num replaces nan and inf with numbers, for the aftermath of the previous lesson.

Fancy indexing

idx = np.array([4, 0, 2])
scores[idx]              # [0.98, 0.91, 0.77] — in the order you asked

Indexing with an integer array picks those positions, in that order, with repeats allowed. Combined with argsort it sorts one array by another:

order = np.argsort(scores)[::-1]     # indices, largest first
names[order][:3]                     # names of the top three

argsort returns the permutation that would sort; applying it to a different array of the same length carries the ordering across. This is how the top-k results of a similarity search become the top-k documents: sort the scores, apply the permutation to the IDs. np.argpartition(scores, -k)[-k:] finds the top k without sorting everything, which matters once the array is millions long.

Two-dimensional fancy indexing takes one array per axis: m[rows, cols] with two equal-length arrays picks the elements at those (row, col) pairs, not the sub-grid. To get the sub-grid, use np.ix_(rows, cols). This surprises everyone once.

Views and copies

Here is the fact that separates people who have been bitten from people who have not. A basic slice of a NumPy array is a view. It does not copy; it is a window onto the same memory.

a = np.arange(10)
b = a[2:5]
b[0] = 99
a               # [0, 1, 99, 3, 4, ...]  — a changed

For a list this would be a copy, as module 2 established. For an array, slicing with : or a step gives a view, because copying a slice of a hundred-million-element array every time would be ruinous. The view shares the data; b.base is a confirms it.

Masks and fancy indexing, by contrast, return copies. a[a > 5] cannot be a view because the selected elements are not evenly spaced in memory. Writing to that result does nothing to a — but writing through the mask in a single statement, a[a > 5] = 0, does, because that is an assignment into a, not into a copy.

The trap in one line:

top = scores[:100]
top *= 0          # clears the first 100 of scores too

The fix is .copy() when you mean a copy: top = scores[:100].copy(). reshape and ravel also return views when they can; flatten always copies. arr.T is a view. When in doubt, np.shares_memory(a, b) answers the question.

The same fact has a benefit: a[:, 0] = 0 clears the first column of a matrix in place without allocating, and a function that receives a view of a large array can write results into it without a copy. Knowing which you have is the whole skill.

Contiguity and the cost of a strided view

A view produced by a[::2] or a.T is not contiguous in memory. Most operations still work but run slower, and a few libraries refuse it. np.ascontiguousarray(v) makes a packed copy. If a routine that is usually fast is slow on a particular array, v.flags["C_CONTIGUOUS"] is worth a look.

Try this now

Take a random (1000,) array, zero everything below its median with a mask, and confirm the count with mask.sum(). Then take a slice, modify it, and watch the original change; repeat with .copy(). Finally, argsort it descending and pull the top five indices, then do the same with argpartition.