Repository navigation
platform.processor() and platform.machine() report inconsistent values under Rosetta on M1 macOS #96993
Description
Bug report
When I run the Python interpreter in x86_64 mode on an M1 MacBook, I get an unexpected result from platform.machine().
In arm64 mode:
ross@Rosss-MBP ~ % arch -arm64 python
Python 3.10.6 (main, Sep 21 2022, 15:43:11) [Clang 14.0.0 (clang-1400.0.29.102)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.processor(), platform.machine()
('arm', 'arm64')
In x86_64 mode:
ross@Rosss-MBP ~ % arch -x86_64 python
Python 3.10.6 (main, Sep 21 2022, 15:43:11) [Clang 14.0.0 (clang-1400.0.29.102)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.processor(), platform.machine()
('i386', 'arm64')
Not that the processor is correct, but the machine is not.
Using an x86_64 cling binary gives the expected result:
ross@Rosss-MBP ~ % file /usr/local/Cellar/cling/0.9/bin/cling
/usr/local/Cellar/cling/0.9/bin/cling: Mach-O 64-bit executable x86_64
ross@Rosss-MBP ~ % /usr/local/Cellar/cling/0.9/bin/cling
[cling]$ #include <sys/utsname.h>
[cling]$ struct utsname u;
[cling]$ uname(&u);
[cling]$ u.machine
(char [256]) "x86_64\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
As does calling out to the uname utility as a subprocess:
ross@Rosss-MBP ~ % arch -x86_64 python
Python 3.10.6 (main, Sep 21 2022, 15:43:11) [Clang 14.0.0 (clang-1400.0.29.102)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> subprocess.check_output(['uname', '-p'], stderr=subprocess.DEVNULL, text=True,)
'i386\n'
>>> subprocess.check_output(['uname', '-m'], stderr=subprocess.DEVNULL, text=True,)
'x86_64\n'
It looks like the implementation of platform.process() works the same way as this - by calling out to a subprocess - whereas platform.machine() uses the uname syscall directly.
Not sure what causes this - perhaps it's a fundamental problem for Universal Binaries? In any case, it might be a good idea to patch for this case (since the output makes no sense at the moment).
Edit: not a universal binary issue - see below.
Your environment
- CPython versions tested on: 3.10.6, installed through pyenv
- Operating system and architecture: macOS 12.6, M1 MacBook Pro
Activity
rossng commentedon Sep 21, 2022
A quick experiment shows that this does not appear to be inherent to universal binaries:
main.c:
rossng commentedon Sep 21, 2022
Aha - it seems that pyenv did not in fact install a universal Python binary, which would explain this behaviour.
I do still think the output is quite confusing, so maybe this is a case where it would be better to clarify the documentation.
ronaldoussoren commentedon Sep 22, 2022
The documentation is pretty vague and doesn't consider the case where the program runs under emulation (such as rosetta on macOS, but likewise for binfmt* on Linux). Worse, it is far from clear what the specification should be: I have use cases for both behaviours (sometimes I need to know the real CPU, sometimes I need to know whatever CPU-type is used by the current proces) and sometimes I need to know which CPU-types are supported by the interpreter which is functionality that's not exposed in platform at all.
kwsp commentedon Oct 17, 2022
Interesting... on my M1 mac with Python 3.8.9 (from
/usr/bin/python3) that I assume is shipped as part of macOS Monterey, I get the expected (?) output:So this is possibly a regression. I can do a bisect when I get some time.
I agree with @ronaldoussoren that the documentation needs an update to clarify cases when the program runs under emulation. Possibly with the fix of this bug.
kwsp commentedon Oct 17, 2022
@rossng if pyenv didn't install a universal binary, how did the binary even execute under x86_64? When I try this on a arm64 binary, I get
With python 3.10.6 installed from pyenv,
I also built a universal binary from the current
mainbranch withand cannot reproduce the behaviour:
rossng commentedon Nov 17, 2022
I think the problem may stem from my default
python3executable being the one from miniconda, which appears to be anx86-64binary. Seems like I thought I was using the pyenv-installed Python binary, but Conda actually hijacked my PATH with its own (higher-priority) entry.I just ran into this issue again when trying to compile Blender - also reported before at https://developer.blender.org/T78710#1160643
ronaldoussoren commentedon Dec 8, 2023
On my laptop running macOS 14.1 I get:
This happens with all version I've installed, including 3.9 upto 3.13 installed using our installer. This might be a change of behaviour in macOS though, can someone test this on an M1 system running macOS 11, 12 or 13?
There doesn't seem to be anything to do here, the behaviour is consistent with the expected system behaviour when running under Rosetta 2: the launched application sees the system as if it is running on an x86_64 system.
ronaldoussoren commentedon Dec 30, 2023
Closing this issue because as mentioned earlier the behaviour I'm seeing on macOS 14 is consistent with the expected behaviour for applications running under Rosetta 2.