Looks like there was some change to macOS runner setup where installing pip packages via python3 instead of python leads to some issues.
There's so many possible things it could have to do with
- the macOS installation of python
- the python venv installation of python
- pip config
- etc etc
and I didn't spend any time narrowing it down.
I experimentally tried python instead of python3 and that seemed to work. That's also what it shows in the docs in example command to install pip dependencies, so I'm just going with that for now.