WASM: functions count exceeds internal limit / too many functions

[derofim]

Got "functions count exceeds internal limit" / "too many functions" while building large apps like more info for Qt or more info for skia (reproduced both in firefox & chrome).

Is it possible to overcome that issue somehow?

You can see all v8 limits here (not only functions count is limited): https://github.com/v8/v8/blob/master/src/wasm/wasm-limits.h

[sbc100]

How many functions to do you have?

It looks like the limit is current set at 1,000,000.

There is no way to overcome that other than update the limit and waiting for that change to filter down into a new chrome and firefix release.

[kripken]

If this is a serious problem then we can try to work around it in the toolchain, by merging functions and such (but it will have downsides).

Btw, does this happen in an optimized build? Inlining and duplicate function elimination might help a lot here.

[derofim]

If this is a serious problem then we can try to work around it in the toolchain, by merging functions and such (but it will have downsides).

Btw, does this happen in an optimized build? Inlining and duplicate function elimination might help a lot here.

Problem may be serious for very large projects like Qt and AutoCAD, more info (they may unexpectedly reach limits someday even for optimized builds, in worst case - losing some clients)

Fully optimized build fixes issue for my use case (but app may add more functionality and release build can`t produce good debug info).

May dynamic linking be used to solve WASM function count issues?

Also i don`t know how to see current WASM function count in release build. Is there any ways to check how close my app to browser limits?

[derofim]

How many functions to do you have?

It looks like the limit is current set at 1,000,000.

There is no way to overcome that other than update the limit and waiting for that change to filter down into a new chrome and firefix release.

Project based on skia.googlesource.com and dom&css parser (without dom renderer) from http://cobalt.foo. Also ported BLINK platform from chromium (may be enabled with ENABLE_BLINK in CMakeLists).

Errors produced:

'CompileError: WebAssembly.compile(): functions count of 1087002 exceeds internal limit of 1000000 @+20668' - in chrome

CompileError: wasm validation error: at offset 20671: too many functions - in firefox ()

Built https://github.com/blockspacer/skia-opengl-emscripten/pull/new/emscripten_issue_8755 with -O1

In repo you can find parts of chromium/blink ported to WASM & WEBGL2. (also - enabling pthreads - ENABLE_WEB_PTHREADS in cmake - crashes chrome silently, while firefox runs fine. chrome broken somehow after "added cobalt files" commits). If you are interested - you can build project replacing files from "TODO: patches" dir and building via:

cmake -DRUN_APP=ON -DBUILD_APP=ON -DEXTRA_EMMAKE_OPTS="-j;9" -DBUILD_TYPE="Release" -DCLEAN_BUILD=OFF -P tools/buildWeb.cmake

Full build flags (skia with almost all features disabled):

CLICK ME

``` -- SKIA_C_FLAGS="","","","-DHAS_ICU=1","-DENABLE_HARFBUZZ=1","-DENABLE_GNET=1","-DENABLE_NETWORK=1","-DENABLE_SKIA=1","-DENABLE_BLINK_PLATFORM=1","-DENABLE_BLINK=1","-DENABLE_GIPC=1","-DENABLE_MOJO_IPC=1","-frtti","-DNO_CHROMIUM_WINDOW=1","-DHAS_LIBJPEG_TURBO=1","-DUSE_LIBJPEG_TURBO=1","-DENABLE_COBALT=1","-DPOEM_NO_EMULATION=1","-D_FILE_OFFSET_BITS=64","-D_LARGEFILE_SOURCE=1","-DNVALGRIND=1","-DENABLE_BASE=1","-DENABLE_WTF=1","","-DSK_SUPPORT_GPU=1","-DSK_SUPPORT_GPU","-DSK_DISABLE_READBUFFER=1","-DLIBEGL_IMPLEMENTATION=1","-DEGL_EGLEXT_PROTOTYPES=1","-DSK_RELEASE","-DSK_ASSUME_WEBGL=1","-DSKNX_NO_SIMD","-DSK_DISABLE_AAA","-DSK_DISABLE_EFFECT_DESERIALIZATION","-DWEB_ASSEMBLY=1","-DSK_SUPPORT_OPENCL=0","-DSK_SAMPLES_FOR_X=1","-DSK_HAS_PNG_LIBRARY=1","","-DPORT_OWN_DLOG=1","-s","DISABLE_DEPRECATED_FIND_EVENT_TARGET_BEHAVIOR=0","-DTARGET_EMSCRIPTEN","-s","WASM=1","-s","USE_FREETYPE=1","-DDISABLE_PTHREADS=1","-DUSE_FAKE_THREAD_LS=1","-DDISABLE_LOCKS=1","-s","USE_ZLIB=1","-s","USE_LIBPNG=1","-s","USE_SDL=2","-s","USE_SDL_IMAGE=2","-s","USE_SDL_TTF=2","-s","USE_WEBGL2=1","-DWEBGL2_SUPPORT=1","-s","WEBGL2_BACKWARDS_COMPATIBILITY_EMULATION=1","-s","FULL_ES3=1","-s","GL_ASSERTIONS=1","-s","DEMANGLE_SUPPORT=1","-frtti","-DIS_WEBGL=1","-s","SAFE_HEAP=1","-s","ALIASING_FUNCTION_POINTERS=0","--source-map-base","http://localhost:9090/","","-g4","--emit-symbol-map","--memoryprofiler","-DPATHKIT_TESTING","","-O1","-s","ERROR_ON_MISSING_LIBRARIES=1","-s","ERROR_ON_UNDEFINED_SYMBOLS=1","-s","ASSERTIONS=2","-s","TOTAL_MEMORY=1024MB","-s","NO_EXIT_RUNTIME=1","-Wno-error" -- SKIA_CXX_FLAGS="","-DNEED_SHORT_EXTERNAL_NAMES=1","-std=c++17","-DCODEC_IMPLEMENTATION=1","","-DHAS_ICU=1","-DENABLE_HARFBUZZ=1","-DENABLE_GNET=1","-DENABLE_NETWORK=1","-DENABLE_SKIA=1","-DENABLE_BLINK_PLATFORM=1","-DENABLE_BLINK=1","-DENABLE_GIPC=1","-DENABLE_MOJO_IPC=1","-frtti","-DNO_CHROMIUM_WINDOW=1","-DHAS_LIBJPEG_TURBO=1","-DUSE_LIBJPEG_TURBO=1","-DENABLE_COBALT=1","-DPOEM_NO_EMULATION=1","-D_FILE_OFFSET_BITS=64","-D_LARGEFILE_SOURCE=1","-DNVALGRIND=1","-DENABLE_BASE=1","-DENABLE_WTF=1","","-DSK_SUPPORT_GPU=1","-DSK_SUPPORT_GPU","-DSK_DISABLE_READBUFFER=1","-DLIBEGL_IMPLEMENTATION=1","-DEGL_EGLEXT_PROTOTYPES=1","-DSK_RELEASE","-DSK_ASSUME_WEBGL=1","-DSKNX_NO_SIMD","-DSK_DISABLE_AAA","-DSK_DISABLE_EFFECT_DESERIALIZATION","-DWEB_ASSEMBLY=1","-DSK_SUPPORT_OPENCL=0","-DSK_SAMPLES_FOR_X=1","-DSK_HAS_PNG_LIBRARY=1","","-DPORT_OWN_DLOG=1","-s","DISABLE_DEPRECATED_FIND_EVENT_TARGET_BEHAVIOR=0","-DTARGET_EMSCRIPTEN","-s","WASM=1","-s","USE_FREETYPE=1","-DDISABLE_PTHREADS=1","-DUSE_FAKE_THREAD_LS=1","-DDISABLE_LOCKS=1","-s","USE_ZLIB=1","-s","USE_LIBPNG=1","-s","USE_SDL=2","-s","USE_SDL_IMAGE=2","-s","USE_SDL_TTF=2","-s","USE_WEBGL2=1","-DWEBGL2_SUPPORT=1","-s","WEBGL2_BACKWARDS_COMPATIBILITY_EMULATION=1","-s","FULL_ES3=1","-s","GL_ASSERTIONS=1","-s","DEMANGLE_SUPPORT=1","-frtti","-DIS_WEBGL=1","-s","SAFE_HEAP=1","-s","ALIASING_FUNCTION_POINTERS=0","--source-map-base","http://localhost:9090/","","-g4","--emit-symbol-map","--memoryprofiler","-DPATHKIT_TESTING","","-O1","-s","ERROR_ON_MISSING_LIBRARIES=1","-s","ERROR_ON_UNDEFINED_SYMBOLS=1","-s","ASSERTIONS=2","-s","TOTAL_MEMORY=1024MB","-s","NO_EXIT_RUNTIME=1","-Wno-error" -- GN_ARGS=target_cpu="wasm" skia_gl_standard="webgl" ar="/home/denis/job/emsc/emsdk/emscripten/1.38.31/emar" cc="/home/denis/job/emsc/emsdk/emscripten/1.38.31/emcc" cxx="/home/denis/job/emsc/emsdk/emscripten/1.38.31/em++" extra_cflags=["-I/home/denis/workspace/skia-opengl-emscripten/src/chromium/third_party/libjpeg_turbo/"] extra_cflags_c=["","","","-DHAS_ICU=1","-DENABLE_HARFBUZZ=1","-DENABLE_GNET=1","-DENABLE_NETWORK=1","-DENABLE_SKIA=1","-DENABLE_BLINK_PLATFORM=1","-DENABLE_BLINK=1","-DENABLE_GIPC=1","-DENABLE_MOJO_IPC=1","-frtti","-DNO_CHROMIUM_WINDOW=1","-DHAS_LIBJPEG_TURBO=1","-DUSE_LIBJPEG_TURBO=1","-DENABLE_COBALT=1","-DPOEM_NO_EMULATION=1","-D_FILE_OFFSET_BITS=64","-D_LARGEFILE_SOURCE=1","-DNVALGRIND=1","-DENABLE_BASE=1","-DENABLE_WTF=1","","-DSK_SUPPORT_GPU=1","-DSK_SUPPORT_GPU","-DSK_DISABLE_READBUFFER=1","-DLIBEGL_IMPLEMENTATION=1","-DEGL_EGLEXT_PROTOTYPES=1","-DSK_RELEASE","-DSK_ASSUME_WEBGL=1","-DSKNX_NO_SIMD","-DSK_DISABLE_AAA","-DSK_DISABLE_EFFECT_DESERIALIZATION","-DWEB_ASSEMBLY=1","-DSK_SUPPORT_OPENCL=0","-DSK_SAMPLES_FOR_X=1","-DSK_HAS_PNG_LIBRARY=1","","-DPORT_OWN_DLOG=1","-s","DISABLE_DEPRECATED_FIND_EVENT_TARGET_BEHAVIOR=0","-DTARGET_EMSCRIPTEN","-s","WASM=1","-s","USE_FREETYPE=1","-DDISABLE_PTHREADS=1","-DUSE_FAKE_THREAD_LS=1","-DDISABLE_LOCKS=1","-s","USE_ZLIB=1","-s","USE_LIBPNG=1","-s","USE_SDL=2","-s","USE_SDL_IMAGE=2","-s","USE_SDL_TTF=2","-s","USE_WEBGL2=1","-DWEBGL2_SUPPORT=1","-s","WEBGL2_BACKWARDS_COMPATIBILITY_EMULATION=1","-s","FULL_ES3=1","-s","GL_ASSERTIONS=1","-s","DEMANGLE_SUPPORT=1","-frtti","-DIS_WEBGL=1","-s","SAFE_HEAP=1","-s","ALIASING_FUNCTION_POINTERS=0","--source-map-base","http://localhost:9090/","","-g4","--emit-symbol-map","--memoryprofiler","-DPATHKIT_TESTING","","-O1","-s","ERROR_ON_MISSING_LIBRARIES=1","-s","ERROR_ON_UNDEFINED_SYMBOLS=1","-s","ASSERTIONS=2","-s","TOTAL_MEMORY=1024MB","-s","NO_EXIT_RUNTIME=1","-Wno-error"] extra_cflags_cc=["","-DNEED_SHORT_EXTERNAL_NAMES=1","-std=c++17","-DCODEC_IMPLEMENTATION=1","","-DHAS_ICU=1","-DENABLE_HARFBUZZ=1","-DENABLE_GNET=1","-DENABLE_NETWORK=1","-DENABLE_SKIA=1","-DENABLE_BLINK_PLATFORM=1","-DENABLE_BLINK=1","-DENABLE_GIPC=1","-DENABLE_MOJO_IPC=1","-frtti","-DNO_CHROMIUM_WINDOW=1","-DHAS_LIBJPEG_TURBO=1","-DUSE_LIBJPEG_TURBO=1","-DENABLE_COBALT=1","-DPOEM_NO_EMULATION=1","-D_FILE_OFFSET_BITS=64","-D_LARGEFILE_SOURCE=1","-DNVALGRIND=1","-DENABLE_BASE=1","-DENABLE_WTF=1","","-DSK_SUPPORT_GPU=1","-DSK_SUPPORT_GPU","-DSK_DISABLE_READBUFFER=1","-DLIBEGL_IMPLEMENTATION=1","-DEGL_EGLEXT_PROTOTYPES=1","-DSK_RELEASE","-DSK_ASSUME_WEBGL=1","-DSKNX_NO_SIMD","-DSK_DISABLE_AAA","-DSK_DISABLE_EFFECT_DESERIALIZATION","-DWEB_ASSEMBLY=1","-DSK_SUPPORT_OPENCL=0","-DSK_SAMPLES_FOR_X=1","-DSK_HAS_PNG_LIBRARY=1","","-DPORT_OWN_DLOG=1","-s","DISABLE_DEPRECATED_FIND_EVENT_TARGET_BEHAVIOR=0","-DTARGET_EMSCRIPTEN","-s","WASM=1","-s","USE_FREETYPE=1","-DDISABLE_PTHREADS=1","-DUSE_FAKE_THREAD_LS=1","-DDISABLE_LOCKS=1","-s","USE_ZLIB=1","-s","USE_LIBPNG=1","-s","USE_SDL=2","-s","USE_SDL_IMAGE=2","-s","USE_SDL_TTF=2","-s","USE_WEBGL2=1","-DWEBGL2_SUPPORT=1","-s","WEBGL2_BACKWARDS_COMPATIBILITY_EMULATION=1","-s","FULL_ES3=1","-s","GL_ASSERTIONS=1","-s","DEMANGLE_SUPPORT=1","-frtti","-DIS_WEBGL=1","-s","SAFE_HEAP=1","-s","ALIASING_FUNCTION_POINTERS=0","--source-map-base","http://localhost:9090/","","-g4","--emit-symbol-map","--memoryprofiler","-DPATHKIT_TESTING","","-O1","-s","ERROR_ON_MISSING_LIBRARIES=1","-s","ERROR_ON_UNDEFINED_SYMBOLS=1","-s","ASSERTIONS=2","-s","TOTAL_MEMORY=1024MB","-s","NO_EXIT_RUNTIME=1","-Wno-error"] extra_ldflags=["-Wno-error"] is_official_build=true is_component_build=false is_debug=false skia_enable_gpu=true skia_use_egl=true skia_use_vulkan=false skia_enable_vulkan_debug_layers=false skia_enable_spirv_validation=false skia_use_harfbuzz=false skia_use_icu=false skia_enable_ccpr=false skia_enable_nvpr=false skia_use_expat=false skia_enable_skottie=false skia_use_libjpeg_turbo=true skia_use_system_libjpeg_turbo=true skia_use_libpng=true skia_use_system_libpng=true skia_use_zlib=true skia_use_wuffs=false skia_use_libwebp=false skia_enable_pdf=false skia_use_sfntly=false skia_enable_atlas_text=false skia_use_fontconfig=false skia_use_freetype=true skia_use_system_freetype2=true skia_enable_tools=false skia_use_lua=false skia_use_piex=false skia_use_angle=false skia_use_dng_sdk=false skia_use_metal=false skia_enable_fontmgr_empty=false skia_enable_fontmgr_custom=true skia_use_libheif=false skia_enable_skpicture=true skia_enable_skshaper=false skia_use_x11=true skia_lex=false skia_compile_processors=false skia_generate_workarounds=false ```

[sbc100]

So you are attempting run a WASM-compiled version of blink inside of chrome? Thats amazing.

If programs with more than 1,000,000 function are actually feasible to run in chrome then I would suggest making a PR to increase the maximum number of functions in https://github.com/v8/v8/blob/master/src/wasm/wasm-limits.h. It might be that such programs will also run up against other resource limits in the browser, but its hard to know up front.

[kripken]

Previous discussion on the wasm design repo is here: WebAssembly/design#1138 it may be a good idea to raise these concerns there.

May dynamic linking be used to solve WASM function count issues?

It can be a workaround, yes, since the limit is just in each module. (Does add some overhead, though.)

Also i don`t know how to see current WASM function count in release build. Is there any ways to check how close my app to browser limits?

A simple way is to run binaryen's wasm-opt file.wasm --metrics, which will print a line with the number of functions.

[jgravelle-google]

It's possible that we're doing something pathological in the toolchain. I recall there being a lot of dynCall thunks generated with the asm2wasm path, something like O(signatures^2)?
How big is the compiled wasm module? Is it feasible to upload it somewhere?

[blockspacer]

Using -s ASSERTIONS=1 instead of -s ASSERTIONS=2 fixes issue for my use case (but not AvaloniaUI issue or Qt issue, as i suspect).

From >1000000 to <100000 functions count (and 40MiB to 14MiB). Is it expected behavior for '-s ASSERTIONS=2' comparing with '-s ASSERTIONS=1'?

Metrics for debug build (i always used -O1 cause -O0 is too huge/slow for browser/llvm linker/my computer):

1. Without SOURCE_MAP/PROFILING/SYMBOL_MAP/MEMORY_PROFILER/THREAD_PROFILER & ASSERTION_LEVEL=1

[funcs]        : 37351

2. With --source-map-base, --emit-symbol-map & -s ASSERTIONS=1:

[funcs]        : 37351

3. With --source-map-base, --emit-symbol-map & -s ASSERTIONS=2:

[funcs]        : 1256304

wasm file size: 40MiB

full wasm-opt report for -s ASSERTIONS=2

total
 [funcs]        : 1256304
 [imports]      : 1021
 [memory-data]  : 1188090
 [table-data]   : 1228803
 [total]        : 6107268
 [vars]         : 96128
 binary         : 341278
 block          : 522642
 break          : 39426
 call           : 1645335
 call_indirect  : 8195
 const          : 2272674
 drop           : 12976
 get_global     : 65735
 get_local      : 732758
 if             : 93263
 load           : 128
 loop           : 10076
 nop            : 12
 return         : 25649
 set_global     : 44029
 set_local      : 259292
 store          : 32
 switch         : 2921
 unary          : 30847

4. With --source-map-base, --emit-symbol-map, --memoryprofiler & -s ASSERTIONS=1:

[funcs]        : 37351

wasm file size: 14MiB

full wasm-opt report for -s ASSERTIONS=1

total
 [funcs]        : 37351
 [imports]      : 1024
 [memory-data]  : 1188090
 [table-data]   : 1228803
 [total]        : 2832661
 [vars]         : 96129
 binary         : 341278
 block          : 104267
 break          : 39426
 call           : 426395
 call_indirect  : 8195
 const          : 635345
 drop           : 12976
 get_global     : 65735
 get_local      : 732784
 if             : 93263
 load           : 128
 loop           : 10076
 nop            : 12
 return         : 25649
 set_global     : 44029
 set_local      : 259303
 store          : 32
 switch         : 2921
 unary          : 30847

5. -s ASSERTIONS=1 with cobalt & BLINK platform & skia modules (skottie/skshaper/sksg/utils) & pthreads & --threadprofiler enabled:

[funcs]        : 45911

wasm size: 15MiB

[kripken]

Makes sense about ASSERTIONS=2, that does create a great many extra functions (so that any failing indirect call can be reported accurately). But I think that is only really excessive with asm2wasm because of the separate asm.js tables for each signature. Without that (EMULATED_FUNCTION_POINTERS will use a single table), or with the llvm wasm backend, there shouldn't be that many (I actually am not sure if we even support these extra assertion notifications in the wasm backend). If this problem happens in those last 2 modes then we should investigate that more.

[stale]

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 7 days. Feel free to re-open at any time if this issue is still relevant.