Drop gyp, move to CMake #359

ibc · 2020-02-13T07:28:25Z

Rationale:

Nobody is maintaining gyp.
libuv is dropping gyp.
We (mediasoup authors) have to provide and maintain our own xxx.gyp for libsrtp, usrsctp and more deps we use, while those libraries do support CMake out of the box.
There is gyp issues in Windows that nobody fixes (even if reported in gyp project) and affect mediasoup (see [Windows] Parent project GUID not found in "mediasoup-worker" project dependency section #345 and [Windows] Support for Visual Studio 2019 #347).

So let's move to CMake:

Should choose a CMake version available in most common platforms (Debian, Ubuntu, OSX, etc).
Avoid the very latest and coolest CMake version since some of its features are not implemented by most common versions such as 3.7.

Ping @saghul because I can.

The text was updated successfully, but these errors were encountered:

saghul · 2020-02-13T08:36:58Z

Current worker deps:

Name	CMake support upstream?
catch	✅
getopt	✅
json	✅
lcov	🚫
libsrtp	✅
libuv	✅
libwebrtc	🚫
netstring	🚫
openssl	🚫
usrsctp	✅

Out of these, I'd say OpenSSL is the trickiest one. An option would be to move to BoringSSL, which does support CMake out of the box. The rest are pretty small and writing CMakefiles for them should be trivial.

ibc · 2020-02-13T08:56:27Z

Wow, thanks! BTW:

lcov is not really used in mediasoup (I mean, it's not built).
Regarding libwebrtc we only use a subset of files so it shouldn't be hard to make a CMakelists.txt for it (wait, it depends on abseil-cpp lib but it does provide CMakelists.txt).
netstring is two files code :)
openssl... wow, didn't know that... Moving to BoringSSL is an option, of course, but somehow I don't want to rely on so many Google-managed projects... sad.

saghul · 2020-02-13T11:41:51Z

* `openssl`... wow, didn't know that... Moving to BoringSSL is an option, of course, but somehow I don't want to rely on so many Google-managed projects... sad.

It already has the necessary APIs for QUIC unlike OpenSSL (still in PR) so maybe it's a sign from, the future! 😎

torlarse · 2020-04-05T09:31:08Z

Please correct me if I am interrupting or misunderstanding, this is just a friendly attempt to contribute. I copied this from the BoringSSL Git 1:

Although BoringSSL is an open source project, it is not intended for general use, as OpenSSL is. We don't recommend that third parties depend upon it. Doing so is likely to be frustrating because there are no guarantees of API or ABI stability.

barry-ran · 2020-04-10T11:42:36Z

openssl-cmake

barry-ran · 2020-05-05T11:56:28Z

I am trying to move mediasoup to cmake. I have made some attempts and it should be successful (mediasoup-cmake)

ibc · 2020-05-05T12:24:14Z

Interesting, This will be very useful for us.

Just wondering: Did you use openssl-cmake project into it? And what else did you have to modify (other than creating the CMakeLists.txt file)?

BTW I tried it in OSX and it seems to work, but I get these warnings:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../../../../libcrypto.a(ebcdic.c.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../../../../libcrypto.a(async_null.c.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../../../../libcrypto.a(async_win.c.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../../../../libcrypto.a(rsaz_exp.c.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../../../../libcrypto.a(cms_cd.c.o) has no symbols

[...]

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../../../../libssl.a(ssl_utst.c.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../../../../libssl.a(t1_trce.c.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../../../../libssl.a(ssl_utst.c.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../../../../libssl.a(t1_trce.c.o) has no symbols

barry-ran · 2020-05-05T12:50:51Z

libssl.a(ssl_utst.c.o) has no symbols

Yes, I used openssl-cmake. Now the mediasoup-cpp worker can only be compiled and run. It has not been rigorously tested. I will continue to improve it later (for example, there are some compilation options not added, I am not sure What question), now I just hope it can help you.

ibc · 2020-05-05T14:29:49Z

Thanks, don't hesitase in updating this issue with your progress, please.

jmillan · 2020-06-01T15:08:03Z

Hi @barry-ran,

I see that mediasoup-cpp is a separate project, not a mediasoup branch.

Could you please make it a mediasoup branch (out of current mediasoup v3 version) so it's easier to compare, collaborate and actually merge? We would like to put some efforts on this.

Thanks

barry-ran · 2020-06-02T06:35:44Z

Hi @jmillan ,
Ok i will do it, give me some time

jmillan · 2020-06-02T07:10:20Z

Ok i will do it, give me some time

👍

barry-ran · 2020-06-14T11:16:10Z

Hi @jmillan ,
Ok i will do it, give me some time

Hi @jmillan ,
Sorry for not finishing till now，I cut off the feat/gyp-move-to-camke branch based on 3.6.7, I try to small changes:

replace openssl with openssl-cmake
add CMakeLists.txt
build on github action

It has not been rigorously tested. There are some compilation options not added, I am not sure What question (Because I don’t know much about these options).

jmillan · 2020-06-15T09:26:22Z

Thanks @barry-ran

jmillan · 2020-06-17T15:35:42Z

Hi @barry-ran,

I'm getting the following error when trying to compile your branch:

Such file is not present in deps/openssl/openssl/crytpo

barry-ran · 2020-06-18T13:34:48Z

Hi @barry-ran,

I'm getting the following error when trying to compile your branch:

Such file is not present in deps/openssl/openssl/crytpo

Hi @jmillan ,
You should build it with cmake:

mkdir build
cd build
cmake ..
cmake --build .

mabuchner · 2021-03-05T04:52:37Z

@barry-ran I tried to compile your branch on Windows with Visual Studio 2017, but CMake gave me the following error during configuration

-- Building for: Visual Studio 15 2017
-- Check if the system is big endian
-- Searching 16 bit integer
CMake Error at C:/Program Files/CMake/share/cmake-3.19/Modules/TestBigEndian.cmake:30 (message):
  TEST_BIG_ENDIAN needs either C or CXX language enabled
Call Stack (most recent call first):
  CMakeLists.txt:4 (TEST_BIG_ENDIAN)

To fix this, I had to move the project definition to the top of the CMakeLists.txt file

cmake_minimum_required(VERSION 3.16)

set(PROJECT_NAME "mediasoup-worker")
project(${PROJECT_NAME})

...

Afterwards I was able to configure and build with CMake

cmake -S . -B build -G "Visual Studio 15 2017" -A x64
cmake --build build --config Release

Finally, I had to copy mediasoup-worker.exe, usrsctp.dll and uv.dll to mediasoup/worker/out/Release. I also tried to specify the CMAKE_INSTALL_PREFIX and build the INSTALL target, but nothing was installed.

jmillan · 2021-03-11T11:27:36Z

Hi @barry-ran,

Sorry for the long time without feedback.

Would you be willing to make a PR agains current v3?

jmillan · 2021-03-11T11:35:23Z

I've just tried and had the same issue @mabuchner had.

barry-ran · 2021-04-07T07:24:59Z

@barry-ran I tried to compile your branch on Windows with Visual Studio 2017, but CMake gave me the following error during configuration
-- Building for: Visual Studio 15 2017
-- Check if the system is big endian
-- Searching 16 bit integer
CMake Error at C:/Program Files/CMake/share/cmake-3.19/Modules/TestBigEndian.cmake:30 (message):
  TEST_BIG_ENDIAN needs either C or CXX language enabled
Call Stack (most recent call first):
  CMakeLists.txt:4 (TEST_BIG_ENDIAN)
To fix this, I had to move the project definition to the top of the CMakeLists.txt file
cmake_minimum_required(VERSION 3.16)

set(PROJECT_NAME "mediasoup-worker")
project(${PROJECT_NAME})

...
Afterwards I was able to configure and build with CMake
cmake -S . -B build -G "Visual Studio 15 2017" -A x64
cmake --build build --config Release
Finally, I had to copy mediasoup-worker.exe, usrsctp.dll and uv.dll to mediasoup/worker/out/Release. I also tried to specify the CMAKE_INSTALL_PREFIX and build the INSTALL target, but nothing was installed.

try this?

cmake --build . --config Release --target install

barry-ran · 2021-04-07T07:28:01Z

Hi @barry-ran,

Sorry for the long time without feedback.

Would you be willing to make a PR agains current v3?

Ok i will do it

jmillan · 2021-04-07T11:49:56Z

Thank you!

mabuchner · 2021-04-08T09:50:08Z

try this?

cmake --build . --config Release --target install

I tried

cmake -S . -B build -G "Visual Studio 15 2017" -A x64 -D CMAKE_INSTALL_PREFIX="build/install"
cmake --build build --config Release --target install

The CMake summary says

-- Selecting Windows SDK version 10.0.17763.0 to target Windows 10.0.18363.
-- summary of build options:
    Install prefix:  C:/Users/user.name/project-name/node_modules/mediasoup/worker/build/install
    Target system:   Windows
    Compiler:
      C compiler:    C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe
      CFLAGS:         /DWIN32 /D_WINDOWS

But as before nothing was installed.

barry-ran · 2021-04-18T01:30:36Z

Hi @barry-ran,
Sorry for the long time without feedback.
Would you be willing to make a PR agains current v3?

Ok i will do it

Hi @jmillan @mabuchner

I rebased current v3 and fixed some error, this pr
I've tested the following command:

cmake .. -D CMAKE_INSTALL_PREFIX=./dist
cmake --build . --target install

mediasoup-worker.exe and uv.dll is installed in the ./dist/lib/Debug directory (usrsctp is static now)

nazar-pc · 2021-05-02T19:33:01Z

BTW, was Meson under consideration?
I see many projects transition to it eventually.

ibc · 2021-05-02T20:04:31Z

BTW, was Meson under consideration?

I see many projects transition to it eventually.

We want something that it's already integrated into most of the C/C++ libraries used by mediasoup. Meson is not.

nazar-pc · 2021-05-02T20:12:52Z

I'm far from either of those, but it seems like using CMake projects in Meson is relatively straightforward: https://mesonbuild.com/CMake-module.html

So you'd use dependencies that already have CMake support as is.

ibc · 2021-05-02T20:18:27Z

I don't even know CMake so I'm not gonna learn Meson to integrate CMake into it XD

nazar-pc · 2021-06-06T14:32:02Z

I found this option for OpenSSL: https://github.com/viaduck/openssl-cmake
It will download desired OpenSSL version during build step, thus should be easier to maintain and keep up to date with OpenSSL updates. Will also reduce size of NPM package and Rust crate (I had to request to bump the limit for crate size to even publish mediasoup-sys).

ibc · 2021-07-08T09:48:38Z

Late thanks, @nazar-pc. Unclear whether such a OpenSSL will come with the optimized build options that have been merged into mediasoup today. Will see when we come back to this topic.

nazar-pc · 2021-07-08T22:07:03Z

I'm learning CMake to make this happen, but approaching from a different angle.

GYP can actually produce CMakeLists.txt files, with current GYP config it actually produces two (Release and Debug).

After minor tweaks it seems to build most of the stuff just fine as is. I'll try to combine both files into one and then swapping explicit instructions for building dependencies with includes/imports/whatever it is called in CMake. That will make ensure there is a working build all along and wouldn't necessarily block CMake introduction on upstream CMake support on every dependency too much.

I'll see where it goes, but no promises, no ETA.

ibc · 2021-07-08T22:29:15Z

Are you learning CMake? 😀😀😀😀

ibc · 2021-07-09T09:25:22Z

Just in case, here another approach: #602

nazar-pc · 2021-07-29T11:24:06Z

If some of subscribed to this issue have time to look at and maybe even test #622, it would be very helpful!

ibc added feature gyp meson Meson build system related labels Feb 13, 2020

ibc added this to the v3 updates milestone Feb 13, 2020

This was referenced Feb 13, 2020

[Windows] Support for Visual Studio 2019 #347

Closed

[Windows] Parent project GUID not found in "mediasoup-worker" project dependency section #345

Closed

versatica deleted a comment from barry-ran Apr 10, 2020

ibc mentioned this issue Mar 10, 2021

Remove unnecessary files from gyp configs to fix linking issues #522

Merged

nazar-pc mentioned this issue Jul 11, 2021

RFC: Meson build system #606

Closed

This was referenced Jul 18, 2021

Add OpenSSL mesonbuild/wrapdb#113

Merged

Meson build system 💪 #622

Merged

nazar-pc closed this as completed in #622 Oct 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Drop gyp, move to CMake #359

Drop gyp, move to CMake #359

ibc commented Feb 13, 2020

saghul commented Feb 13, 2020

ibc commented Feb 13, 2020

saghul commented Feb 13, 2020

torlarse commented Apr 5, 2020 •

edited

barry-ran commented Apr 10, 2020 •

edited

barry-ran commented May 5, 2020 •

edited

ibc commented May 5, 2020 •

edited

barry-ran commented May 5, 2020 •

edited

ibc commented May 5, 2020

jmillan commented Jun 1, 2020

barry-ran commented Jun 2, 2020

jmillan commented Jun 2, 2020

barry-ran commented Jun 14, 2020 •

edited

jmillan commented Jun 15, 2020

jmillan commented Jun 17, 2020

barry-ran commented Jun 18, 2020

mabuchner commented Mar 5, 2021 •

edited

jmillan commented Mar 11, 2021 •

edited

jmillan commented Mar 11, 2021

barry-ran commented Apr 7, 2021

barry-ran commented Apr 7, 2021

jmillan commented Apr 7, 2021

mabuchner commented Apr 8, 2021

barry-ran commented Apr 18, 2021

nazar-pc commented May 2, 2021

ibc commented May 2, 2021

nazar-pc commented May 2, 2021

ibc commented May 2, 2021

nazar-pc commented Jun 6, 2021

ibc commented Jul 8, 2021

nazar-pc commented Jul 8, 2021

ibc commented Jul 8, 2021

ibc commented Jul 9, 2021

nazar-pc commented Jul 29, 2021

Drop gyp, move to CMake #359

Drop gyp, move to CMake #359

Comments

ibc commented Feb 13, 2020

saghul commented Feb 13, 2020

ibc commented Feb 13, 2020

saghul commented Feb 13, 2020

torlarse commented Apr 5, 2020 • edited

barry-ran commented Apr 10, 2020 • edited

barry-ran commented May 5, 2020 • edited

ibc commented May 5, 2020 • edited

barry-ran commented May 5, 2020 • edited

ibc commented May 5, 2020

jmillan commented Jun 1, 2020

barry-ran commented Jun 2, 2020

jmillan commented Jun 2, 2020

barry-ran commented Jun 14, 2020 • edited

jmillan commented Jun 15, 2020

jmillan commented Jun 17, 2020

barry-ran commented Jun 18, 2020

mabuchner commented Mar 5, 2021 • edited

jmillan commented Mar 11, 2021 • edited

jmillan commented Mar 11, 2021

barry-ran commented Apr 7, 2021

barry-ran commented Apr 7, 2021

jmillan commented Apr 7, 2021

mabuchner commented Apr 8, 2021

barry-ran commented Apr 18, 2021

nazar-pc commented May 2, 2021

ibc commented May 2, 2021

nazar-pc commented May 2, 2021

ibc commented May 2, 2021

nazar-pc commented Jun 6, 2021

ibc commented Jul 8, 2021

nazar-pc commented Jul 8, 2021

ibc commented Jul 8, 2021

ibc commented Jul 9, 2021

nazar-pc commented Jul 29, 2021

torlarse commented Apr 5, 2020 •

edited

barry-ran commented Apr 10, 2020 •

edited

barry-ran commented May 5, 2020 •

edited

ibc commented May 5, 2020 •

edited

barry-ran commented May 5, 2020 •

edited

barry-ran commented Jun 14, 2020 •

edited

mabuchner commented Mar 5, 2021 •

edited

jmillan commented Mar 11, 2021 •

edited