Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-102255: Improve build support on xbox #102256

Merged
merged 104 commits into from
Mar 9, 2023
Merged
Show file tree
Hide file tree
Changes from 100 commits
Commits
Show all changes
104 commits
Select commit Hold shift + click to select a range
58ab04a
better support building on non desktop windows systems
maxbachmann Feb 24, 2023
66a68bd
stub posix apis
maxbachmann Feb 25, 2023
26f5a44
add basic isabs implementation
maxbachmann Feb 25, 2023
68a1f67
implement isxfile
maxbachmann Feb 25, 2023
492bac0
fix static linking
maxbachmann Feb 25, 2023
26b4b24
fix guards
maxbachmann Feb 25, 2023
467bf40
add MS_XBOX
maxbachmann Feb 25, 2023
e734492
cleanup
maxbachmann Feb 25, 2023
fad23bc
use winsock2 on xbox
maxbachmann Feb 26, 2023
b6b28ec
stub winreg
maxbachmann Feb 26, 2023
cdbb1f2
rename to MS_GAMES
maxbachmann Feb 26, 2023
e7622cc
fix guard
maxbachmann Feb 26, 2023
7812891
add missing semicolon
maxbachmann Feb 26, 2023
3b005f4
remove windows console
maxbachmann Feb 26, 2023
fa526b6
do not read version from kernel32.dll
maxbachmann Feb 26, 2023
36887bc
apply part of code review
maxbachmann Feb 26, 2023
6fe3018
reduce diff
maxbachmann Feb 26, 2023
8499940
regenerate argument clinic
maxbachmann Feb 26, 2023
69f0a2c
include winioctl
maxbachmann Feb 26, 2023
f43ce54
exclude dll_directory
maxbachmann Feb 26, 2023
43cf0a5
patch mscvrtmodule
maxbachmann Feb 26, 2023
8f717ea
patch _winapi
maxbachmann Feb 26, 2023
29d5b65
📜🤖 Added by blurb_it.
blurb-it[bot] Feb 26, 2023
8a73500
patch out tzset
maxbachmann Feb 26, 2023
92ae5ef
patch support_wsa_no_inherit
maxbachmann Feb 26, 2023
f27fe95
patch scoketmodule
maxbachmann Feb 26, 2023
f15d9e2
PathCchCombineEx is unavailable on xbox
maxbachmann Feb 26, 2023
d816baa
patch out remaining funcs
maxbachmann Feb 26, 2023
2ba0a87
Merge branch 'main' into xbox
maxbachmann Feb 26, 2023
ed17e03
apply code review
maxbachmann Feb 26, 2023
c360b4a
cleanup
maxbachmann Feb 26, 2023
3b5b119
remove unneded defined
maxbachmann Feb 26, 2023
975191a
regenerate argument clinic
maxbachmann Feb 26, 2023
91a95f2
remove unused function
maxbachmann Feb 26, 2023
3be5fa7
use processsnapshot api
maxbachmann Feb 26, 2023
6bd1207
simplify implementation
maxbachmann Feb 26, 2023
ca98366
simplify implementation
maxbachmann Feb 26, 2023
fac3431
follow pep7
maxbachmann Feb 26, 2023
c1bf0e2
follow pep7
maxbachmann Feb 26, 2023
ace99f6
Update Modules/posixmodule.c
maxbachmann Feb 26, 2023
e83d398
remove duplicated comment
maxbachmann Feb 26, 2023
76ea310
test error message as well
maxbachmann Feb 26, 2023
2f1d19c
fix condition
maxbachmann Feb 26, 2023
559199f
use sys._base_executable
maxbachmann Feb 26, 2023
b119bf5
do not build mmap on unsupported platforms
maxbachmann Feb 26, 2023
1dd69e8
Apply suggestions from code review
maxbachmann Feb 27, 2023
a1e6dba
add inheritable back
maxbachmann Feb 27, 2023
7a45988
Update Python/fileutils.c
maxbachmann Feb 27, 2023
846f8eb
Update Python/fileutils.c
maxbachmann Feb 27, 2023
a56f215
add back some winreg APIs
maxbachmann Feb 27, 2023
191cab2
regenerate argument clinic
maxbachmann Feb 27, 2023
144280e
do not name platform xbox
maxbachmann Feb 27, 2023
ca9f7d9
exclude functions
maxbachmann Feb 27, 2023
30c0818
implement without nt._path_splitroot
maxbachmann Feb 27, 2023
6ebb05b
Update Modules/socketmodule.c
maxbachmann Feb 27, 2023
fc5c642
apply some code review change requests
maxbachmann Feb 27, 2023
5f286a7
get rid of support_wsa_no_inherit
maxbachmann Feb 27, 2023
3be5e3c
implement _path_splitroot
maxbachmann Feb 27, 2023
d9e1663
Merge branch 'main' into xbox
maxbachmann Feb 28, 2023
2a2a8fd
fix revert of changes to _bootstrap_external
maxbachmann Feb 28, 2023
165286d
reimplement winreg_QueryValue_impl and winreg_SetValue_impl
maxbachmann Feb 28, 2023
1413849
Update Python/fileutils.c
maxbachmann Feb 28, 2023
6465d73
move skip root to own function
maxbachmann Feb 28, 2023
3f0c6f0
cleanup
maxbachmann Feb 28, 2023
8f3a262
better handle joinfile on xbox
maxbachmann Feb 28, 2023
6b8074e
cleanup
maxbachmann Feb 28, 2023
99b6a1d
Update Modules/socketmodule.c
maxbachmann Feb 28, 2023
d0e548d
define missing flags on non desktop builds
maxbachmann Feb 28, 2023
c569a18
set inherit in WASSocketW
maxbachmann Feb 28, 2023
3e86c9e
add back dynamic loading
maxbachmann Feb 28, 2023
d5cb524
more cleanup
maxbachmann Feb 28, 2023
63267d8
rename platform defines
maxbachmann Feb 28, 2023
79f2bfe
add missing inherit flag
maxbachmann Feb 28, 2023
fc6b942
fix guard
maxbachmann Feb 28, 2023
05e9859
use api partititions
maxbachmann Feb 28, 2023
9769662
fix guards
maxbachmann Feb 28, 2023
8f12bf8
add missing include
maxbachmann Mar 1, 2023
509d0f5
Merge branch 'main' into xbox
maxbachmann Mar 1, 2023
f20fea8
apply code review
maxbachmann Mar 1, 2023
e33a42f
move replacement functions into fileutils
maxbachmann Mar 1, 2023
556a8e1
disable dynamic load of pythoncore for static lib
maxbachmann Mar 1, 2023
aa8964b
Merge branch 'main' into xbox
maxbachmann Mar 1, 2023
74258ca
use core dll for windows specific
maxbachmann Mar 1, 2023
686da59
Update Python/fileutils.c
maxbachmann Mar 1, 2023
b151f7f
apply code review
maxbachmann Mar 1, 2023
39d2d13
rename to PathCchCombineEx
maxbachmann Mar 1, 2023
2895563
define for all BUILD_CORE
maxbachmann Mar 1, 2023
8e3d3c5
cleanup
maxbachmann Mar 1, 2023
659d1a3
Update Modules/_randommodule.c
maxbachmann Mar 1, 2023
e9ff60e
add MS_WINDOWS_CRT_DESKTOP
maxbachmann Mar 2, 2023
32c4886
Update Modules/posixmodule.c
maxbachmann Mar 2, 2023
2f1fe40
Update Modules/posixmodule.c
maxbachmann Mar 2, 2023
5854e91
revert MS_WINDOWS_CRT_DESKTOP
maxbachmann Mar 2, 2023
d3d2e1c
use MS_WINDOWS_SYSTEM
maxbachmann Mar 3, 2023
b1ffce9
Apply suggestions from code review
maxbachmann Mar 3, 2023
0b4d407
Update Modules/socketmodule.c
maxbachmann Mar 3, 2023
d9976dc
handle failing malloc
maxbachmann Mar 3, 2023
427de79
Update Modules/posixmodule.c
maxbachmann Mar 4, 2023
603cea7
Update Python/fileutils.c
maxbachmann Mar 6, 2023
6db80a3
Update Include/internal/pycore_fileutils.h
maxbachmann Mar 6, 2023
4d78066
Update Include/internal/pycore_fileutils.h
maxbachmann Mar 6, 2023
65cdd56
Update Python/fileutils.c
maxbachmann Mar 6, 2023
35217ab
Update Python/fileutils.c
maxbachmann Mar 7, 2023
331b1f4
dynamically import from api-ms-win-core-path-l1-1-0.dll
maxbachmann Mar 9, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions Include/internal/pycore_fileutils.h
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,14 @@ extern int _Py_add_relfile(wchar_t *dirname,
extern size_t _Py_find_basename(const wchar_t *filename);
PyAPI_FUNC(wchar_t *) _Py_normpath(wchar_t *path, Py_ssize_t size);

// The Windows Games API family does not provide these functions
// so provide our own implementations. Remove them in case they get added
// to the Games API family
#if defined(MS_WINDOWS_GAMES) && !defined(MS_WINDOWS_DESKTOP)
#include <winerror.h>

extern HRESULT PathCchSkipRoot(const wchar_t *pszPath, const wchar_t **ppszRootEnd);
#endif /* defined(MS_WINDOWS) && !defined(MS_WINDOWS_APP) && !defined(MS_WINDOWS_SYSTEM) */
maxbachmann marked this conversation as resolved.
Show resolved Hide resolved

// Macros to protect CRT calls against instant termination when passed an
// invalid parameter (bpo-23524). IPH stands for Invalid Parameter Handler.
Expand Down
8 changes: 5 additions & 3 deletions Lib/test/test_os.py
Original file line number Diff line number Diff line change
Expand Up @@ -3064,11 +3064,13 @@ def test_device_encoding(self):
class PidTests(unittest.TestCase):
@unittest.skipUnless(hasattr(os, 'getppid'), "test needs os.getppid")
def test_getppid(self):
p = subprocess.Popen([sys.executable, '-c',
p = subprocess.Popen([sys._base_executable, '-c',
'import os; print(os.getppid())'],
stdout=subprocess.PIPE)
stdout, _ = p.communicate()
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
stdout, error = p.communicate()
# We are the parent of our subprocess
self.assertEqual(error, b'')
self.assertEqual(int(stdout), os.getpid())
eryksun marked this conversation as resolved.
Show resolved Hide resolved

def check_waitpid(self, code, exitcode, callback=None):
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Improve build support for the Xbox. Patch by Max Bachmann.
6 changes: 3 additions & 3 deletions Modules/_io/_iomodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,7 @@ _io_open_impl(PyObject *module, PyObject *file, const char *mode,
_PyIO_State *state = get_io_state(module);
{
PyObject *RawIO_class = (PyObject *)state->PyFileIO_Type;
#ifdef MS_WINDOWS
#ifdef HAVE_WINDOWS_CONSOLE_IO
const PyConfig *config = _Py_GetConfig();
if (!config->legacy_windows_stdio && _PyIO_get_console_type(path_or_fd) != '\0') {
RawIO_class = (PyObject *)&PyWindowsConsoleIO_Type;
Expand Down Expand Up @@ -660,7 +660,7 @@ static PyTypeObject* static_types[] = {

// PyRawIOBase_Type(PyIOBase_Type) subclasses
&_PyBytesIOBuffer_Type,
#ifdef MS_WINDOWS
#ifdef HAVE_WINDOWS_CONSOLE_IO
&PyWindowsConsoleIO_Type,
#endif
};
Expand Down Expand Up @@ -718,7 +718,7 @@ PyInit__io(void)
}

// Set type base classes
#ifdef MS_WINDOWS
#ifdef HAVE_WINDOWS_CONSOLE_IO
PyWindowsConsoleIO_Type.tp_base = &PyRawIOBase_Type;
#endif

Expand Down
6 changes: 3 additions & 3 deletions Modules/_io/_iomodule.h
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ extern PyType_Spec fileio_spec;
extern PyType_Spec stringio_spec;
extern PyType_Spec textiowrapper_spec;

#ifdef MS_WINDOWS
#ifdef HAVE_WINDOWS_CONSOLE_IO
extern PyTypeObject PyWindowsConsoleIO_Type;
#endif /* MS_WINDOWS */
#endif /* HAVE_WINDOWS_CONSOLE_IO */

/* These functions are used as METH_NOARGS methods, are normally called
* with args=NULL, and return a new reference.
Expand Down Expand Up @@ -178,7 +178,7 @@ find_io_state_by_def(PyTypeObject *type)

extern _PyIO_State *_PyIO_get_module_state(void);

#ifdef MS_WINDOWS
#ifdef HAVE_WINDOWS_CONSOLE_IO
extern char _PyIO_get_console_type(PyObject *);
#endif

Expand Down
42 changes: 21 additions & 21 deletions Modules/_io/clinic/winconsoleio.c.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Modules/_io/fileio.c
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,9 @@
#ifdef MS_WINDOWS
/* can simulate truncate with Win32 API functions; see file_truncate */
#define HAVE_FTRUNCATE
#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN
#endif
#include <windows.h>
#endif

Expand Down
6 changes: 4 additions & 2 deletions Modules/_io/winconsoleio.c
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
#include "pycore_fileutils.h" // _Py_BEGIN_SUPPRESS_IPH
#include "pycore_object.h" // _PyObject_GC_UNTRACK()

#ifdef MS_WINDOWS
#ifdef HAVE_WINDOWS_CONSOLE_IO

#include "structmember.h" // PyMemberDef
#ifdef HAVE_SYS_TYPES_H
Expand All @@ -22,7 +22,9 @@
#endif
#include <stddef.h> /* For offsetof */

#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN
#endif
#include <windows.h>
#include <fcntl.h>

Expand Down Expand Up @@ -1177,4 +1179,4 @@ PyTypeObject PyWindowsConsoleIO_Type = {
0, /* tp_finalize */
};

#endif /* MS_WINDOWS */
#endif /* HAVE_WINDOWS_CONSOLE_IO */
8 changes: 5 additions & 3 deletions Modules/_localemodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,9 @@ This software comes with no warranty. Use at your own risk.
#endif

#if defined(MS_WINDOWS)
#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN
#endif
#include <windows.h>
#endif

Expand Down Expand Up @@ -457,12 +459,12 @@ _locale__getdefaultlocale_impl(PyObject *module)

PyOS_snprintf(encoding, sizeof(encoding), "cp%u", GetACP());

if (GetLocaleInfo(LOCALE_USER_DEFAULT,
if (GetLocaleInfoA(LOCALE_USER_DEFAULT,
LOCALE_SISO639LANGNAME,
locale, sizeof(locale))) {
Py_ssize_t i = strlen(locale);
locale[i++] = '_';
if (GetLocaleInfo(LOCALE_USER_DEFAULT,
if (GetLocaleInfoA(LOCALE_USER_DEFAULT,
LOCALE_SISO3166CTRYNAME,
locale+i, (int)(sizeof(locale)-i)))
return Py_BuildValue("ss", locale, encoding);
Expand All @@ -474,7 +476,7 @@ _locale__getdefaultlocale_impl(PyObject *module)

locale[0] = '0';
locale[1] = 'x';
if (GetLocaleInfo(LOCALE_USER_DEFAULT, LOCALE_IDEFAULTLANGUAGE,
if (GetLocaleInfoA(LOCALE_USER_DEFAULT, LOCALE_IDEFAULTLANGUAGE,
locale+2, sizeof(locale)-2)) {
return Py_BuildValue("ss", locale, encoding);
}
Expand Down
4 changes: 3 additions & 1 deletion Modules/_multiprocessing/multiprocessing.h
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@
*/

#ifdef MS_WINDOWS
# define WIN32_LEAN_AND_MEAN
# ifndef WIN32_LEAN_AND_MEAN
# define WIN32_LEAN_AND_MEAN
# endif
# include <windows.h>
# include <winsock2.h>
# include <process.h> /* getpid() */
Expand Down
6 changes: 6 additions & 0 deletions Modules/_pickle.c
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,12 @@ enum {
#define FLOAT FLOAT_
#define INT INT_
#define LONG LONG_

/* This can already be defined on Windows to set the character set
the Windows header files treat as default */
#ifdef UNICODE
#undef UNICODE
#endif
#endif

/* Pickle opcodes. These must be kept updated with pickle.py.
Expand Down
6 changes: 5 additions & 1 deletion Modules/_randommodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,10 @@
# include <process.h> // getpid()
#endif

#ifdef MS_WINDOWS
# include <windows.h>
#endif

/* Period parameters -- These are all magic. Don't change. */
#define N 624
#define M 397
Expand Down Expand Up @@ -259,7 +263,7 @@ random_seed_time_pid(RandomObject *self)
key[0] = (uint32_t)(now & 0xffffffffU);
key[1] = (uint32_t)(now >> 32);

#ifdef MS_WINDOWS_NON_DESKTOP
#if defined(MS_WINDOWS) && !defined(MS_WINDOWS_DESKTOP) && !defined(MS_WINDOWS_SYSTEM)
key[2] = (uint32_t)GetCurrentProcessId();
#elif defined(HAVE_GETPID)
key[2] = (uint32_t)getpid();
Expand Down
4 changes: 4 additions & 0 deletions Modules/_ssl.c
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@
/* Include symbols from _socket module */
#include "socketmodule.h"

#ifdef MS_WINDOWS
# include <wincrypt.h>
#endif

#include "_ssl.h"

/* Redefined below for Windows debug builds after important #includes */
Expand Down
13 changes: 13 additions & 0 deletions Modules/_winapi.c
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,11 @@
#include "structmember.h" // PyMemberDef


#ifndef WINDOWS_LEAN_AND_MEAN
#define WINDOWS_LEAN_AND_MEAN
#endif
#include "windows.h"
#include <winioctl.h>
#include <crtdbg.h>
#include "winreparse.h"

Expand All @@ -63,6 +66,14 @@

#define T_HANDLE T_POINTER

// winbase.h limits the STARTF_* flags to the desktop API as of 10.0.19041.
#ifndef STARTF_USESHOWWINDOW
#define STARTF_USESHOWWINDOW 0x00000001
#endif
#ifndef STARTF_USESTDHANDLES
#define STARTF_USESTDHANDLES 0x00000100
#endif

typedef struct {
PyTypeObject *overlapped_type;
} WinApiState;
Expand Down Expand Up @@ -1201,8 +1212,10 @@ _winapi_ExitProcess_impl(PyObject *module, UINT ExitCode)
/*[clinic end generated code: output=a387deb651175301 input=4f05466a9406c558]*/
{
#if defined(Py_DEBUG)
#ifdef MS_WINDOWS_DESKTOP
SetErrorMode(SEM_FAILCRITICALERRORS|SEM_NOALIGNMENTFAULTEXCEPT|
SEM_NOGPFAULTERRORBOX|SEM_NOOPENFILEERRORBOX);
#endif
_CrtSetReportMode(_CRT_ASSERT, _CRTDBG_MODE_DEBUG);
#endif

Expand Down
Loading