[WIP] Experimental WebAssembly Support #2561

1div0 · 2023-03-25T15:19:43Z

emcmake cmake -B WebAssembly
cmake --build WebAssembly

Exiv2 WebAssembly.log

Failure by linking stage. Need to dig deeper.

ghost · 2023-03-25T15:21:28Z

👇 Click on the image for a new way to code review

Legend

neheb · 2023-03-25T15:39:41Z

Uhhh try with meson instead

edit: need to pass -Ddefault_library=static

ERROR: ld.wasm does not support shared libraries.

edit2: nvm:

em++: error: lib_convert.a: Unknown format, not a static library!

1div0 · 2023-03-25T19:30:02Z

meson setup --cross-file wasm32.text WebAssembly
ninja -C WebAssembly

ninja: Entering directory `WebAssembly'
[1/1] Linking target exiv2
FAILED: exiv2
em++ -o exiv2 exiv2.p/app_actions.cpp.o exiv2.p/app_app_utils.cpp.o exiv2.p/app_exiv2.cpp.o exiv2.p/app_getopt.cpp.o -sERROR_ON_UNDEFINED_SYMBOLS=1 -Wl,--start-group libexiv2.a -Wl,--end-group
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::indent(unsigned long)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::indent(unsigned long)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::indent(unsigned long)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::indent(unsigned long)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::TiffMapping::findDecoder(std::__2::basic_string<char, std::__2::char_traits, std::__2::allocator> const&, unsigned int, Exiv2::IfdId)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::TiffParserWorker::decode(Exiv2::ExifData&, Exiv2::IptcData&, Exiv2::XmpData&, unsigned char const*, unsigned long, unsigned int, void (Exiv2::Internal::TiffDecoder::* ()(std::__2::basic_string<char, std::__2::char_traits, std::__2::allocator> const&, unsigned int, Exiv2::IfdId))(Exiv2::Internal::TiffEntryBase const), Exiv2::Internal::TiffHeaderBase*)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::TiffMapping::findDecoder(std::__2::basic_string<char, std::__2::char_traits, std::__2::allocator> const&, unsigned int, Exiv2::IfdId)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::TiffParserWorker::decode(Exiv2::ExifData&, Exiv2::IptcData&, Exiv2::XmpData&, unsigned char const*, unsigned long, unsigned int, void (Exiv2::Internal::TiffDecoder::* ()(std::__2::basic_string<char, std::__2::char_traits, std::__2::allocator> const&, unsigned int, Exiv2::IfdId))(Exiv2::Internal::TiffEntryBase const), Exiv2::Internal::TiffHeaderBase*)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_exif.cpp.o): undefined symbol: Exiv2::Internal::printValue(std::__2::basic_ostream<char, std::__2::char_traits>&, Exiv2::Value const&, Exiv2::ExifData const*)
wasm-ld: error: libexiv2.a(src_exif.cpp.o): undefined symbol: Exiv2::Internal::tagInfo(unsigned short, Exiv2::IfdId)
wasm-ld: error: libexiv2.a(src_exif.cpp.o): undefined symbol: Exiv2::Internal::TiffHeader::TiffHeader(Exiv2::ByteOrder, unsigned int, bool)
wasm-ld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)
em++: error: '/2TB/usr/src/github.com/emscripten-core/emsdk/upstream/bin/wasm-ld -o exiv2.wasm exiv2.p/app_actions.cpp.o exiv2.p/app_app_utils.cpp.o exiv2.p/app_exiv2.cpp.o exiv2.p/app_getopt.cpp.o libexiv2.a -L/2TB/usr/src/github.com/emscripten-core/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten -lGL -lal -lhtml5 -lstubs-debug -lnoexit -lc-debug -ldlmalloc -lcompiler_rt -lc++-noexcept -lc++abi-debug-noexcept -lsockets -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --allow-undefined-file=/tmp/tmptp3dz3hq.undefined --strip-debug --export-if-defined=main --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__start_em_lib_deps --export-if-defined=__stop_em_lib_deps --export-if-defined=__start_em_js --export-if-defined=__stop_em_js --export-if-defined=__main_argc_argv --export-if-defined=fflush --export=emscripten_stack_get_end --export=emscripten_stack_get_free --export=emscripten_stack_get_base --export=emscripten_stack_get_current --export=emscripten_stack_init --export=stackSave --export=stackRestore --export=stackAlloc --export=__errno_location --export=__get_temp_ret --export=__set_temp_ret --export=__wasm_call_ctors --export=malloc --export=__cxa_is_pointer_type --export=ntohs --export=htons --export=emscripten_builtin_memalign --export-table -z stack-size=65536 --initial-memory=16777216 --no-entry --max-memory=16777216 --stack-first' failed (returned 1)
ninja: build stopped: subcommand failed.

neheb · 2023-03-25T20:33:47Z

meson.build

  gnu_symbol_visibility: 'hidden',
-  dependencies: [deps, exiv2int_dep],


That's your problem

1div0 · 2023-03-26T05:36:22Z

em++ -o exiv2.js exiv2.p/app_actions.cpp.o exiv2.p/app_app_utils.cpp.o exiv2.p/app_exiv2.cpp.o exiv2.p/app_getopt.cpp.o libexiv2.a.p/*.o libexiv2int.a.p/*.o -sERROR_ON_UNDEFINED_SYMBOLS=1

peter.kovar@Pascal /2TB/usr/src/github.com/1div0/exiv2/WebAssembly
€ file exiv2.js exiv2.wasm
exiv2.js: ASCII text, with very long lines (357)
exiv2.wasm: WebAssembly (wasm) binary module version 0x1 (MVP)

1div0 · 2023-03-26T06:33:52Z

neheb · 2023-03-26T18:36:42Z

What's the issue?

1div0 · 2023-03-26T18:42:08Z

This was output to the developer console in Brave. Just a confirmation that Exiv2 library is executing via WebAssembly.

neheb · 2023-03-26T18:45:01Z

OK. The purpose of the meson build is simply to build the library. The goal is not to fully replace the CMake build. I have no idea whether or not it works properly.

1div0 · 2023-03-27T12:15:39Z

Well, my intent was to make sure that WebAssembly can be built and viable option. Maybe even for 1.0. Thoughts?

neheb · 2023-06-07T14:55:25Z

Sure, why not.

I can’t help much with CMake though

neheb · 2023-08-14T01:49:59Z

I tried this again. Needs:

--- a/src/meson.build
+++ b/src/meson.build
@@ -115,12 +115,6 @@ if host_machine.system() == 'windows' and get_option('default_library') != 'stat
   dllapi = '-DEXIV2API=__declspec(dllimport)'
 endif
 
-cmake = import('cmake')
-cmake.write_basic_package_version_file(
-  name: meson.project_name(),
-  version: meson.project_version(),
-)
-
 pkg = import('pkgconfig')
 pkg.generate(
   exiv2,

Maybe I should completely get rid of it. It doesn't even work for cmake.

meson ran with

meson --cross-file=em.txt -Ddefault_library=static em

em.txt:

[binaries]
c = 'emcc'
cpp = 'em++'
ar = 'emar'
nm = 'emnm'

[host_machine]
system = 'emscripten'
cpu_family = 'x86_64'
cpu = 'x86_64'
endian = 'little'

neheb · 2023-08-14T01:58:50Z

if you want to build the subprojects as well, -Dauto_features=enable and -DXXX:default_library=static for each subproject.

neheb · 2023-08-14T02:00:29Z

>>> defined as (i32, i32, i32) -> i32 in subprojects/zlib-1.2.13/libz.a(gzlib.c.o)
>>> defined as (i32, i64, i32) -> i64 in /usr/share/emscripten/cache/sysroot/lib/wasm32-emscripten/libc-debug.a(lseek.o)

great, a bug in the wrap.

neheb · 2023-08-14T02:05:17Z

hmm tests don't compile

[212/212] Linking target unitTests/unit_tests.js
FAILED: unitTests/unit_tests.js 
em++  -o unitTests/unit_tests.js unitTests/unit_tests.js.p/test_DateValue.cpp.o unitTests/unit_tests.js.p/test_Error.cpp.o unitTests/unit_tests.js.p/test_FileIo.cpp.o unitTests/unit_tests.js.p/test_ImageFactory.cpp.o unitTests/unit_tests.js.p/test_IptcKey.cpp.o unitTests/unit_tests.js.p/test_LangAltValueRead.cpp.o unitTests/unit_tests.js.p/test_Photoshop.cpp.o unitTests/unit_tests.js.p/test_TimeValue.cpp.o unitTests/unit_tests.js.p/test_XmpKey.cpp.o unitTests/unit_tests.js.p/test_basicio.cpp.o unitTests/unit_tests.js.p/test_bmpimage.cpp.o unitTests/unit_tests.js.p/test_cr2header_int.cpp.o unitTests/unit_tests.js.p/test_datasets.cpp.o unitTests/unit_tests.js.p/test_enforce.cpp.o unitTests/unit_tests.js.p/test_futils.cpp.o unitTests/unit_tests.js.p/test_helper_functions.cpp.o unitTests/unit_tests.js.p/test_image_int.cpp.o unitTests/unit_tests.js.p/test_jp2image.cpp.o unitTests/unit_tests.js.p/test_jp2image_int.cpp.o unitTests/unit_tests.js.p/test_safe_op.cpp.o unitTests/unit_tests.js.p/test_slice.cpp.o unitTests/unit_tests.js.p/test_tiffheader.cpp.o unitTests/unit_tests.js.p/test_types.cpp.o unitTests/unit_tests.js.p/test_utils.cpp.o unitTests/unit_tests.js.p/test_asfvideo.cpp.o unitTests/unit_tests.js.p/test_matroskavideo.cpp.o unitTests/unit_tests.js.p/test_riffVideo.cpp.o unitTests/unit_tests.js.p/test_pngimage.cpp.o unitTests/unit_tests.js.p/.._subprojects_googletest-1.13.0_googletest_src_gtest-all.cc.o unitTests/unit_tests.js.p/.._subprojects_googletest-1.13.0_googlemock_src_gmock-all.cc.o unitTests/unit_tests.js.p/.._subprojects_googletest-1.13.0_googlemock_src_gmock_main.cc.o -s ERROR_ON_UNDEFINED_SYMBOLS=1 -Wl,--start-group subprojects/expat-2.5.0/libexpat.a subprojects/zlib-1.2.13/libz.a src/libexiv2.a src/lib_convert.a src/libexiv2int.a -Wl,--end-group -s USE_PTHREADS=1 -s PTHREAD_POOL_SIZE=4
wasm-ld-13: error: --shared-memory is disallowed by datasets.cpp.o because it was not compiled with 'atomics' or 'bulk-memory' features.
em++: error: '/usr/bin/wasm-ld-13 -o unitTests/unit_tests.wasm unitTests/unit_tests.js.p/test_DateValue.cpp.o unitTests/unit_tests.js.p/test_Error.cpp.o unitTests/unit_tests.js.p/test_FileIo.cpp.o unitTests/unit_tests.js.p/test_ImageFactory.cpp.o unitTests/unit_tests.js.p/test_IptcKey.cpp.o unitTests/unit_tests.js.p/test_LangAltValueRead.cpp.o unitTests/unit_tests.js.p/test_Photoshop.cpp.o unitTests/unit_tests.js.p/test_TimeValue.cpp.o unitTests/unit_tests.js.p/test_XmpKey.cpp.o unitTests/unit_tests.js.p/test_basicio.cpp.o unitTests/unit_tests.js.p/test_bmpimage.cpp.o unitTests/unit_tests.js.p/test_cr2header_int.cpp.o unitTests/unit_tests.js.p/test_datasets.cpp.o unitTests/unit_tests.js.p/test_enforce.cpp.o unitTests/unit_tests.js.p/test_futils.cpp.o unitTests/unit_tests.js.p/test_helper_functions.cpp.o unitTests/unit_tests.js.p/test_image_int.cpp.o unitTests/unit_tests.js.p/test_jp2image.cpp.o unitTests/unit_tests.js.p/test_jp2image_int.cpp.o unitTests/unit_tests.js.p/test_safe_op.cpp.o unitTests/unit_tests.js.p/test_slice.cpp.o unitTests/unit_tests.js.p/test_tiffheader.cpp.o unitTests/unit_tests.js.p/test_types.cpp.o unitTests/unit_tests.js.p/test_utils.cpp.o unitTests/unit_tests.js.p/test_asfvideo.cpp.o unitTests/unit_tests.js.p/test_matroskavideo.cpp.o unitTests/unit_tests.js.p/test_riffVideo.cpp.o unitTests/unit_tests.js.p/test_pngimage.cpp.o unitTests/unit_tests.js.p/.._subprojects_googletest-1.13.0_googletest_src_gtest-all.cc.o unitTests/unit_tests.js.p/.._subprojects_googletest-1.13.0_googlemock_src_gmock-all.cc.o unitTests/unit_tests.js.p/.._subprojects_googletest-1.13.0_googlemock_src_gmock_main.cc.o subprojects/expat-2.5.0/libexpat.a subprojects/zlib-1.2.13/libz.a src/libexiv2.a src/lib_convert.a src/libexiv2int.a -L/usr/share/emscripten/cache/sysroot/lib/wasm32-emscripten /usr/share/emscripten/cache/sysroot/lib/wasm32-emscripten/crtbegin.o -lGL-mt -lal -lhtml5 -lstubs-debug -lnoexit -lc-mt-debug -ldlmalloc-mt -lcompiler_rt-mt -lc++-mt-noexcept -lc++abi-mt-noexcept -lsockets-mt -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --import-undefined --import-memory --shared-memory --strip-debug --export-if-defined=main --export-if-defined=_emscripten_thread_init --export-if-defined=_emscripten_thread_exit --export-if-defined=_emscripten_thread_crashed --export-if-defined=emscripten_tls_init --export-if-defined=emscripten_current_thread_process_queued_calls --export-if-defined=pthread_self --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__stdio_exit --export=emscripten_stack_get_end --export=emscripten_stack_get_free --export=emscripten_stack_get_base --export=emscripten_stack_init --export=stackSave --export=stackRestore --export=stackAlloc --export=__wasm_call_ctors --export=__errno_location --export=emscripten_dispatch_to_thread_ --export=_emscripten_thread_free_data --export=_emscripten_allow_main_runtime_queued_calls --export=emscripten_main_browser_thread_id --export=emscripten_main_thread_process_queued_calls --export=emscripten_run_in_main_runtime_thread_js --export=emscripten_stack_set_limits --export=emscripten_sync_run_in_main_thread_2 --export=emscripten_sync_run_in_main_thread_4 --export=emscripten_builtin_memalign --export=malloc --export=free --export=ntohs --export=htons --export=htonl --export-table -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=16777216 --global-base=1024' failed (returned 1)
ninja: build stopped: subcommand failed

neheb · 2023-09-05T20:02:48Z

The reason for the tests is because gtest uses std::atomic. Might be fixed in a newer version of emscripten. Who knows.

1div0 · 2023-09-06T02:12:35Z

Will check ASAP.

1div0 · 2023-09-24T17:37:56Z

meson setup --cross-file wasm32.text -Ddefault_library=static WebAssembly
ninja -C WebAssembly

codecov · 2023-09-24T17:44:33Z

Codecov Report

Merging #2561 (8bb5636) into main (fe44c8c) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #2561   +/-   ##
=======================================
  Coverage   63.96%   63.96%           
=======================================
  Files         103      103           
  Lines       22344    22344           
  Branches    10835    10835           
=======================================
  Hits        14293    14293           
  Misses       5828     5828           
  Partials     2223     2223

1div0 added compilers Related with compiler options, definitions, support, etc. build wasm WebAssembly labels Mar 25, 2023

1div0 requested review from kevinbackhouse and kmilos March 25, 2023 15:19

1div0 self-assigned this Mar 25, 2023

1div0 marked this pull request as draft March 25, 2023 15:21

1div0 mentioned this pull request Mar 25, 2023

WASM support #2550

Open

neheb reviewed Mar 25, 2023

View reviewed changes

meson.build Outdated

gnu_symbol_visibility: 'hidden',

dependencies: [deps, exiv2int_dep],

Copy link

Collaborator

neheb Mar 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's your problem

1div0 added 4 commits September 24, 2023 19:13

Updated the cmake configuration

fc93650

Add simple HTML page in order to check loading of the WebAssembly

5d7bc9f

Added WebAssembly compilation options

325c234

Added simple test page

c1e49fd

1div0 force-pushed the WebAssembly branch from 68e26cd to c1e49fd Compare September 24, 2023 17:34

1div0 and others added 2 commits September 24, 2023 19:49

Removed the test page from subdirectory

c1286b7

Merge branch 'Exiv2:main' into WebAssembly

8bb5636

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Experimental WebAssembly Support #2561

[WIP] Experimental WebAssembly Support #2561

1div0 commented Mar 25, 2023

ghost commented Mar 25, 2023 •

edited by ghost

Legend

neheb commented Mar 25, 2023 •

edited

1div0 commented Mar 25, 2023 •

edited

neheb Mar 25, 2023

1div0 commented Mar 26, 2023

1div0 commented Mar 26, 2023

neheb commented Mar 26, 2023

1div0 commented Mar 26, 2023

neheb commented Mar 26, 2023

1div0 commented Mar 27, 2023

neheb commented Jun 7, 2023

neheb commented Aug 14, 2023

neheb commented Aug 14, 2023

neheb commented Aug 14, 2023

neheb commented Aug 14, 2023

neheb commented Sep 5, 2023

1div0 commented Sep 6, 2023

1div0 commented Sep 24, 2023

codecov bot commented Sep 24, 2023 •

edited

		gnu_symbol_visibility: 'hidden',
		dependencies: [deps, exiv2int_dep],

[WIP] Experimental WebAssembly Support #2561

Are you sure you want to change the base?

[WIP] Experimental WebAssembly Support #2561

Conversation

1div0 commented Mar 25, 2023

ghost commented Mar 25, 2023 • edited by ghost

Legend

neheb commented Mar 25, 2023 • edited

1div0 commented Mar 25, 2023 • edited

neheb Mar 25, 2023

Choose a reason for hiding this comment

1div0 commented Mar 26, 2023

1div0 commented Mar 26, 2023

neheb commented Mar 26, 2023

1div0 commented Mar 26, 2023

neheb commented Mar 26, 2023

1div0 commented Mar 27, 2023

neheb commented Jun 7, 2023

neheb commented Aug 14, 2023

neheb commented Aug 14, 2023

neheb commented Aug 14, 2023

neheb commented Aug 14, 2023

neheb commented Sep 5, 2023

1div0 commented Sep 6, 2023

1div0 commented Sep 24, 2023

codecov bot commented Sep 24, 2023 • edited

Codecov Report

ghost commented Mar 25, 2023 •

edited by ghost

neheb commented Mar 25, 2023 •

edited

1div0 commented Mar 25, 2023 •

edited

codecov bot commented Sep 24, 2023 •

edited