gh-156780: Emscripten: support building CPython with static linking - #156781
gh-156780: Emscripten: support building CPython with static linking#156781clementperon wants to merge 1 commit into
Conversation
…king CPython's Emscripten glue was written for the -sMAIN_MODULE configuration it is built and tested in upstream, and reaches for helpers that only exist there. Because EM_JS bodies are emitted verbatim, Emscripten's dependency tracker cannot see those references: a static link silently emits JS referring to helpers it never included, and dies with a ReferenceError during initRuntime rather than failing at link time. Four such references: * emscripten_syscalls.c hooks resolveGlobalSymbol() to wrap main() in WebAssembly.promising(). That helper lives in libdylink.js, which src/modules.mjs only links in for MAIN_MODULE builds. It sits below a WebAssembly.promising check, so the break is invisible until a runtime exposes JSPI. * emscripten_trampoline.c uses wasmTable and addFunction(), neither of which is included by default; MAIN_MODULE pulls them in via libdylink.js. * The promising-main wrapper calls emscripten_exit_with_live_runtime(), which is likewise dropped from a static link. * The trampoline's preRun hook reads _PyRuntime, which is bound into the JS glue only because configure lists it in -sEXPORTED_FUNCTIONS. An embedder linking libpython gets no such flag, so this one fails on every runtime. Declare the library symbols with EM_JS_DEPS, export _PyRuntime's address from the trampoline itself so it needs no link flags, define Py_EMSCRIPTEN_DYNAMIC_LINKING from configure so the entry-point hook can be written for whichever configuration is in use, and pass -sALLOW_TABLE_GROWTH when not linking with MAIN_MODULE, as addFunction() requires a growable table. Statically, Emscripten calls the `_main` binding directly instead of resolving main through libdylink, so that is what the hook rebinds. It must wrap the raw wasm export: WebAssembly.promising() rejects the createExportWrapper() shim that `_main` normally holds. A static libpython is usually embedded in a host application whose main() is not Python's, where wrapping the entry point would be wrong, so this now requires the embedder to set Module.Py_EmscriptenPromisingMain. The async stdin device is needed in every configuration, so it moves to its own constructor instead of riding along with the entry-point hook. Verified by embedding a static libpython in a small browser app: an interactive console runs under Chrome with JSPI available, where the same app built against unpatched sources fails at startup. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
@hoodmane 👀 maybe ? |
|
Maybe let's start with a PR that adds the I think rather than using #ifdef __EMSCRIPTEN__
__attribute__((constructor)) void _PyEmscripten_Constructor(void) {
_PyEmscripten_BeforeMain();
}
And of course as traditional we need much less prose since Claude is as rambly as usual. Please rewrite all the comments and the commit message by hand. I find that helps cut them down to a more reasonable length. Of course, it's likely that any of this will bit rot without test coverage. cc @freakboy3742 WDYT? |
Fixes #156780.
CPython's Emscripten glue is written for the
-sMAIN_MODULEdynamic-linking configuration it is built and tested in, and reaches for helpers that only exist there. BecauseEM_JSbodies are emitted verbatim, Emscripten's dependency tracker cannot see those references: a static link silently emits JS calling helpers it never included, and dies with aReferenceErrorduringinitRuntimerather than failing at link time.Four such references, all detailed in the issue:
emscripten_syscalls.chooksresolveGlobalSymbol(), which lives inlibdylink.jsand is only linked in forMAIN_MODULEbuilds.emscripten_trampoline.cuseswasmTableandaddFunction(), neither included by default.emscripten_exit_with_live_runtime(), also dropped from a static link._PyRuntime, bound into the glue only becauseconfigurelists it in-sEXPORTED_FUNCTIONSfor CPython's own link. An embedder gets no such flag, so this one fails on every runtime.The library symbols are now declared with
EM_JS_DEPS, so referencing them in a configuration that cannot provide them is a link error rather than a runtime surprise._PyRuntime's address is exported from the trampoline itself, so that code needs no link flags of its own.configuredefinesPy_EMSCRIPTEN_DYNAMIC_LINKINGwhen-sMAIN_MODULEis used, letting the entry-point hook be written for whichever configuration is in effect, and passes-sALLOW_TABLE_GROWTHotherwise, sinceaddFunction()requires a growable table.Statically, Emscripten calls the
_mainbinding directly instead of resolvingmainthrough libdylink, so that is what the hook rebinds. It has to wrap the raw wasm export:WebAssembly.promising()rejects thecreateExportWrapper()shim that_mainnormally holds.Behaviour change for embedders
A static
libpythonis normally linked into a host application whosemain()is not Python's, where wrapping the entry point inWebAssembly.promising()and callingemscripten_exit_with_live_runtime()would be actively wrong. Static builds therefore only install the wrapper when the embedder opts in by settingModule.Py_EmscriptenPromisingMainbefore the runtime starts.MAIN_MODULEbuilds — includingpython.mjsas built byPlatforms/emscripten— are unaffected: they keep theresolveGlobalSymbolhook and wrapmain()unconditionally, exactly as before.The async stdin device is needed in every configuration, so it moves to its own constructor rather than riding along with the entry-point hook.
Testing
configureandpyconfig.h.inwere regenerated withmake regen-configure.Verified end to end by building
libpython3.14.awith--disable-wasm-dynamic-linking MODULE_BUILDTYPE=static CFLAGS=-DPY_CALL_TRAMPOLINEand linking it into a small C application that runs an interactive Python console in a browser — an embedder whosemain()is not Python's. With this change the console runs under Chrome with JSPI available and under node with and without--experimental-wasm-jspi; the same application built from unpatched sources aborts during startup.The
MAIN_MODULEpath was checked in the same harness to confirm theresolveGlobalSymbolhook is still installed and still wrapsmain().Worth noting for reviewers: nothing in the test suite covers this today. Every wasm test builds through
Platforms/emscripten/__main__.py, which hard-codes--enable-wasm-dynamic-linking, soweb_example,web_example_pyrepl_jspiandbrowser_testall exerciseMAIN_MODULEonly. I am happy to follow up with a static build target if that would be welcome.