Test environment (using `conan`) #235

aarlt · 2022-02-27T04:19:48Z

Same as #234 but using Conan as package manager.

basic integration
adapt GitHub workflow

aarlt · 2022-02-27T04:25:58Z

At least so far I can tell that Conan is also very nice. However, the interesting part will be the setup of the GitHub workflows, but theoretically there should not be any major problem - but lets see.

aarlt · 2022-02-27T04:29:16Z

Build instructions for this PR:

# checkout
git clone git@github.com:aarlt/autobahn-cpp.git
cd autobahn-cpp
git checkout test-environment-conan

# build directory
mkdir build
cd build

# initialise conan
conan install --build missing ../conanfile.py

# build
cmake ..
make

# test (if docker daemon is running)
make test

aarlt · 2022-02-27T05:12:55Z

Theoretically we could even support conan and vcpkg at the same time: the user could just select it's favourite package manager, where let's say conan is default.

aarlt · 2022-02-27T05:40:30Z

Maybe https://github.com/conan-io/cmake-conan is very interesting, because it seem to allow using conan without manually running conan before cmake invocation.

oberstet · 2022-02-28T07:33:38Z

oh, this is fantastic!! really cool that you give conan a try, and if it works with vcpkg at the same time - well, that's what I'd call "best of both worlds";)

gimme a bit time to digest it, try it etc, I post here!

first need to get over this:

CMake 3.21 or higher is required.  You are running version 3.18.5

cmake/focal,now 3.16.3-1ubuntu1 amd64  [installiert]
  Plattformübergreifendes, quelloffenes Make-System

(base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.4 LTS
Release:	20.04
Codename:	focal

comfort docs for our readme then:

wget https://github.com/Kitware/CMake/releases/download/v3.22.2/cmake-3.22.2-linux-x86_64.sh
chmod +x cmake-3.22.2-linux-x86_64.sh
./cmake-3.22.2-linux-x86_64.sh
echo "export PATH=${HOME}/cmake-3.22.2-linux-x86_64/bin/:${PATH}" >> ~/.profile

oberstet · 2022-02-28T07:44:18Z

awesome! it generates the cmake build files

websocketpp/0.8.2: Package installed 5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
websocketpp/0.8.2: Downloaded package revision 0
bzip2/1.0.8: Appending PATH environment variable: /home/oberstet/.conan/data/bzip2/1.0.8/_/_/package/da606cf731e334010b0bf6e85a2a6f891b9f36b0/bin
conanfile.py (autobahn-cpp/v20.8.1): Generator txt created conanbuildinfo.txt
conanfile.py (autobahn-cpp/v20.8.1): Generator cmake created conanbuildinfo.cmake
conanfile.py (autobahn-cpp/v20.8.1): Aggregating env generators
conanfile.py (autobahn-cpp/v20.8.1): Generated conaninfo.txt
conanfile.py (autobahn-cpp/v20.8.1): Generated graphinfo
(cpy39_1) (base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$

only other thing I had to do: set compiler.libcxx=libstdc++11 in $HOME/.conan/profiles/default

because

(cpy39_1) (base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$ conan install --build missing ../conanfile.py
Auto detecting your dev setup to initialize the default profile (/home/oberstet/.conan/profiles/default)
Found gcc 9
Found clang 10.0
gcc>=5, using the major as version

************************* WARNING: GCC OLD ABI COMPATIBILITY ***********************
 
Conan detected a GCC version > 5 but has adjusted the 'compiler.libcxx' setting to
'libstdc++' for backwards compatibility.
Your compiler is likely using the new CXX11 ABI by default (libstdc++11).

If you want Conan to use the new ABI for the default profile, run:

    $ conan profile update settings.compiler.libcxx=libstdc++11 default

Or edit '/home/oberstet/.conan/profiles/default' and set compiler.libcxx=libstdc++11

************************************************************************************

....

Installing (downloading, building) binaries...
ERROR: There are invalid packages (packages that cannot exist for this configuration):
botan/2.19.1: Invalid ID: Using Botan with GCC >= 5 on Linux requires "compiler.libcxx=libstdc++11"
(cpy39_1) (base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$

now, to build, I had to add a env var like

cmake -DCMAKE_BUILD_TYPE=Release ..

and it builds ... without any hickups ... I can't believe it. with all the deps, this is a pretty big and complex code base. really great work!

...
Requirements
    boost/1.78.0 from 'conancenter' - Cache
    botan/2.19.1 from 'conancenter' - Cache
    bzip2/1.0.8 from 'conancenter' - Cache
    catch2/2.13.8 from 'conancenter' - Cache
    libbacktrace/cci.20210118 from 'conancenter' - Cache
    msgpack-cxx/4.0.3 from 'conancenter' - Cache
    openssl/3.0.1 from 'conancenter' - Cache
    websocketpp/0.8.2 from 'conancenter' - Cache
    zlib/1.2.11 from 'conancenter' - Cache
...
-- Configuring done
-- Generating done
-- Build files have been written to: /home/oberstet/scm/3rdparty/autobahn-cpp/build
(cpy39_1) (base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$ 
(cpy39_1) (base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$ 
(cpy39_1) (base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$ make
[  3%] Building CXX object examples/CMakeFiles/examples_parameters.dir/parameters.cpp.o
...
[ 83%] Building CXX object test/CMakeFiles/test_pubsub.dir/pubsub/pubsub.cpp.o
[ 87%] Building CXX object test/CMakeFiles/test_pubsub.dir/test_main.cpp.o
[ 90%] Linking CXX executable ../bin/test_pubsub
[ 90%] Built target test_pubsub
[ 93%] Building CXX object test/CMakeFiles/test_calls.dir/calls/calls.cpp.o
[ 96%] Building CXX object test/CMakeFiles/test_calls.dir/test_main.cpp.o
[100%] Linking CXX executable ../bin/test_calls
[100%] Built target test_calls
(cpy39_1) (base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$

aarlt · 2022-02-28T14:49:26Z

Nice! I still need to fix the windows build (see https://github.com/aarlt/autobahn-cpp/runs/5355028210?check_suite_focus=true) ;) I will try to fix it when I find some time.

Beside the build system stuff, the test environment can be found in test/.

BTW I was trying to build crossbar for windows (because I was not able to find a way to run docker images in windows with GitHub workflow) - but it was not just working, sadly I forgot what was the exact issue there. Using Python & crossbar would be in general the other alternative - to not use docker. However, I think at least for Linux & macOS the usage of the docker images is the best solution. But I also think that it would be nice if we could find a way to also run the test cases on windows.

oberstet · 2022-02-28T16:07:05Z

BTW I was trying to build crossbar for windows

very brave;) honestly, it's long ago I did that. it is possible, but only for masochists, and also pointless. just using docker is a lot better. use windows as a glorified docker driver is good enough.

here is an example how that can be used in CI / GH actions:

https://github.com/crossbario/autobahn-js/blob/master/.github/workflows/main.yml
https://github.com/crossbario/autobahn-js/blob/master/.crossbar/config.json

oberstet · 2022-02-28T16:34:53Z

rgd test strategy vs a running router, as part of CI, "end to end integration tests", just as some background/details:

over the years, we've developed and tried different approaches in autobahn-python, autobahn-js, autobahn-java and crossbar itself.

the approach in autobahn-js is probably the simplest. the tests

will start crossbar via docker
then tests are driven from https://github.com/crossbario/autobahn-js/blob/master/packages/autobahn/test/test.js
each of these test will open multiple WAMP connections (from one process), do WAMP stuff, and then log messages to stdout.
those messages are designed to indicated that sth that should happen actually did happen, and the messages are "fixed" (don't have changing timestamps or similar)
the stdout of each test is the diff'ed against a "known good" file, eg https://github.com/crossbario/autobahn-js/blob/master/packages/autobahn/test/test_pubsub_basic.txt

the test passes, if the log matches known good exactly.

now, this works and does indeed catch many issues, and in a way positively proves that the basics are working.

however, it obviously also has shortcomings.

how do you initially come up with a "known good" file?
how do you prevent testing crossbar specific implementation behavior vs WAMP spec only?
how do you test what should not happen (since then there is no message logged ..)
...

but the upside is: it is a straight-forward test strategy that is easy to implement;)

if you are interested, I can explain the approaches in the other repos ... the stuff in crossbar is the most advanced. it contains tests to configure and boot a cluster of router nodes with management node and all the shit. however, for the client libraries testing, this is overkill.

the client lib test strategy IMO should have one important feature: simple and "just works". not introducing even more problems, unrelated to the client library itself.

aarlt marked this pull request as draft February 27, 2022 04:22

aarlt changed the title ~~Test environment (Conan)~~ Test environment (using conan) Feb 27, 2022

aarlt force-pushed the test-environment-conan branch 2 times, most recently from b542d0f to 177df28 Compare February 27, 2022 04:40

aarlt force-pushed the test-environment-conan branch 3 times, most recently from 57617bc to 31d94e8 Compare February 28, 2022 02:47

aarlt added 7 commits February 27, 2022 21:53

Add simple test environment.

e8996fd

Initial vcpkg support.

f78be89

Add vcpkg as submodule.

8835c54

Add vcpkg manifest vcpkg.json.

2e9353c

Add new workflow.

d1d0b6d

Basic conan integration.

3f5d760

Use of cmake-conan.

3e37056

aarlt force-pushed the test-environment-conan branch from 31d94e8 to 641098d Compare February 28, 2022 02:59

Adapt workflow for conan.

6c02721

aarlt force-pushed the test-environment-conan branch from 641098d to 6c02721 Compare February 28, 2022 03:49

aarlt added 3 commits February 27, 2022 23:40

mingw test.

3e91548

iiiiiiiiiiii

86abe8f

iiiiiiiiii

67b2c63

oberstet marked this pull request as ready for review February 28, 2022 07:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test environment (using `conan`) #235

Test environment (using `conan`) #235

aarlt commented Feb 27, 2022 •

edited

aarlt commented Feb 27, 2022

aarlt commented Feb 27, 2022 •

edited

aarlt commented Feb 27, 2022 •

edited

aarlt commented Feb 27, 2022 •

edited

oberstet commented Feb 28, 2022 •

edited

oberstet commented Feb 28, 2022 •

edited

aarlt commented Feb 28, 2022 •

edited

oberstet commented Feb 28, 2022

oberstet commented Feb 28, 2022

Test environment (using conan) #235

Are you sure you want to change the base?

Test environment (using conan) #235

Conversation

aarlt commented Feb 27, 2022 • edited

aarlt commented Feb 27, 2022

aarlt commented Feb 27, 2022 • edited

aarlt commented Feb 27, 2022 • edited

aarlt commented Feb 27, 2022 • edited

oberstet commented Feb 28, 2022 • edited

oberstet commented Feb 28, 2022 • edited

aarlt commented Feb 28, 2022 • edited

oberstet commented Feb 28, 2022

oberstet commented Feb 28, 2022

Test environment (using `conan`) #235

Test environment (using `conan`) #235

aarlt commented Feb 27, 2022 •

edited

aarlt commented Feb 27, 2022 •

edited

aarlt commented Feb 27, 2022 •

edited

aarlt commented Feb 27, 2022 •

edited

oberstet commented Feb 28, 2022 •

edited

oberstet commented Feb 28, 2022 •

edited

aarlt commented Feb 28, 2022 •

edited