Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

crash with ffmpeg 3.x #7

Closed
chris1705 opened this issue May 2, 2017 · 5 comments
Closed

crash with ffmpeg 3.x #7

chris1705 opened this issue May 2, 2017 · 5 comments

Comments

@chris1705
Copy link

Hello Scott,

first I'd like to thank you for sharing your awesome work!

I have compiled your project on my Raspberry Pi 3 and it starts well but after configuring my first camera, the moonfire-nvr stops working while opening the input stream. Here is the output:

I0502 172118.091 main moonfire_nvr::db] Loading video sample entries
I0502 172118.091 main moonfire_nvr::db] Loaded 0 video sample entries
I0502 172118.092 main moonfire_nvr::db] Loading cameras
I0502 172118.092 main moonfire_nvr::db] Loaded 1 cameras
I0502 172118.092 main moonfire_nvr::db] Loading recordings for camera Terrasse
I0502 172118.092 main moonfire_nvr::db] Loaded 0 recordings for camera Terrasse
I0502 172118.092 main moonfire_nvr::cmds::run] Database is loaded.
I0502 172118.104 stream-Terrasse moonfire_nvr::streamer] Terrasse: Opening input: rtsp://acrux:redacted@192.168.1.7:8554/acrux-cam-1
I0502 172118.104 main moonfire_nvr::cmds::run] Ready to serve HTTP requests
[rtsp @ 0x73e18000] Estimating duration from bitrate, this may be inaccurate
I0502 172118.616 stream-Terrasse moonfire_nvr::stream] Discarding the first packet to work around https://trac.ffmpeg.org/ticket/5018

I have tried to configure it for serval RTSP streams but it always stops with the same error (I also tried this one: rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov)

I'm using version 1.16.0 (30cf806ef 2017-03-10) of rustc.

Do you have any idea? If you need any more information please let me know! Thanks a lot!

Regards
Chris
@scottlamb
Copy link
Owner

Thanks for the bug report!

I don't recognize the symptoms, so yes, I need some more information.

First, what version of ffmpeg are you using? The ffmpeg FFI stuff is the part of the system I know is sketchy right now. I'm using a ffmpeg I installed from source; I'm not at all confident moonfire-nvr currently works with the one built into Raspbian. Looks like the README suggests using sudo apt-get install ... libavcodec-dev and such, but I think I just neglected to update that when I rewrote everything in Rust.

ldd path/to/moonfire-nvr would be a good start to get this information.

What exactly do you mean by stops working? No more log output, I assume? Does the web interface respond?

Could you get stack traces of all threads? I do something like this to save them to /tmp/my-backtrace:

$ gdb /path/to/moonfire-nvr $(pidof moonfire-nvr)
(gdb) set pagination off
(gdb) set logging file /tmp/my-backtrace
(gdb) set logging on
(gdb) set logging redirect on
(gdb) thread apply all bt
(gdb) quit

@chris1705
Copy link
Author

chris1705 commented May 3, 2017 •
edited

With stopping I mean it terminates, which means there is no moonfire-nvr process anymore. The web interface is not available also.

I have compiled ffmpeg myself today:

ffmpeg version N-85750-ga75ef15 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 4.9.2 (Raspbian 4.9.2-10)
  configuration: --enable-libfreetype --enable-gpl --enable-nonfree --enable-libx264 --enable-libass --enable-libmp3lame --bindir=/usr/local/bin
  libavutil      55. 61.100 / 55. 61.100
  libavcodec     57. 93.100 / 57. 93.100
  libavformat    57. 72.101 / 57. 72.101
  libavdevice    57.  7.100 / 57.  7.100
  libavfilter     6. 88.100 /  6. 88.100
  libswscale      4.  7.101 /  4.  7.101
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100

The output of ldd:

pi@einsteinpi:~/moonfire-nvr$ ldd /usr/local/bin/moonfire-nvr 
	linux-vdso.so.1 (0x7ecb7000)
	/usr/lib/arm-linux-gnueabihf/libarmmem.so (0x76d98000)
	libssl.so.1.0.0 => /usr/lib/arm-linux-gnueabihf/libssl.so.1.0.0 (0x76d35000)
	libcrypto.so.1.0.0 => /usr/lib/arm-linux-gnueabihf/libcrypto.so.1.0.0 (0x76bc7000)
	libncursesw.so.5 => /lib/arm-linux-gnueabihf/libncursesw.so.5 (0x76b8b000)
	libtinfo.so.5 => /lib/arm-linux-gnueabihf/libtinfo.so.5 (0x76b5d000)
	libsqlite3.so.0 => /usr/lib/arm-linux-gnueabihf/libsqlite3.so.0 (0x76aa3000)
	libavutil.so.54 => /usr/lib/arm-linux-gnueabihf/neon/vfp/libavutil.so.54 (0x76a6e000)
	libavcodec.so.56 => /usr/lib/arm-linux-gnueabihf/neon/vfp/libavcodec.so.56 (0x75e10000)
	libavformat.so.56 => /usr/lib/arm-linux-gnueabihf/neon/vfp/libavformat.so.56 (0x75ced000)
	libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0x75cda000)
	librt.so.1 => /lib/arm-linux-gnueabihf/librt.so.1 (0x75cc3000)
	libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x75c9a000)
	libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0x75c6d000)
	libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x75b2c000)
	/lib/ld-linux-armhf.so.3 (0x54ab3000)
	libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0x75ab1000)
	libavresample.so.2 => /usr/lib/arm-linux-gnueabihf/neon/vfp/libavresample.so.2 (0x75a92000)
	libxvidcore.so.4 => /usr/lib/arm-linux-gnueabihf/libxvidcore.so.4 (0x75992000)
	libx264.so.142 => /usr/lib/arm-linux-gnueabihf/neon/vfp/libx264.so.142 (0x75826000)
	libvpx.so.1 => /usr/lib/arm-linux-gnueabihf/vfp/neon/libvpx.so.1 (0x7569d000)
	libvorbisenc.so.2 => /usr/lib/arm-linux-gnueabihf/libvorbisenc.so.2 (0x7560b000)
	libvorbis.so.0 => /usr/lib/arm-linux-gnueabihf/libvorbis.so.0 (0x755d3000)
	libtheoraenc.so.1 => /usr/lib/arm-linux-gnueabihf/libtheoraenc.so.1 (0x7558b000)
	libtheoradec.so.1 => /usr/lib/arm-linux-gnueabihf/libtheoradec.so.1 (0x75568000)
	libspeex.so.1 => /usr/lib/arm-linux-gnueabihf/libspeex.so.1 (0x75542000)
	libschroedinger-1.0.so.0 => /usr/lib/arm-linux-gnueabihf/libschroedinger-1.0.so.0 (0x75496000)
	libz.so.1 => /lib/arm-linux-gnueabihf/libz.so.1 (0x7546f000)
	libopus.so.0 => /usr/lib/arm-linux-gnueabihf/libopus.so.0 (0x75420000)
	libopenjpeg.so.5 => /usr/lib/arm-linux-gnueabihf/libopenjpeg.so.5 (0x753f1000)
	libmp3lame.so.0 => /usr/lib/arm-linux-gnueabihf/libmp3lame.so.0 (0x7538b000)
	libgsm.so.1 => /usr/lib/arm-linux-gnueabihf/libgsm.so.1 (0x75379000)
	libva.so.1 => /usr/lib/arm-linux-gnueabihf/libva.so.1 (0x7534c000)
	librtmp.so.1 => /usr/lib/arm-linux-gnueabihf/librtmp.so.1 (0x75322000)
	libgnutls-deb0.so.28 => /usr/lib/arm-linux-gnueabihf/libgnutls-deb0.so.28 (0x75215000)
	libbz2.so.1.0 => /lib/arm-linux-gnueabihf/libbz2.so.1.0 (0x751fd000)
	libogg.so.0 => /usr/lib/arm-linux-gnueabihf/libogg.so.0 (0x751ef000)
	liborc-0.4.so.0 => /usr/lib/arm-linux-gnueabihf/liborc-0.4.so.0 (0x7516e000)
	libstdc++.so.6 => /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 (0x75092000)
	libhogweed.so.2 => /usr/lib/arm-linux-gnueabihf/libhogweed.so.2 (0x75059000)
	libnettle.so.4 => /usr/lib/arm-linux-gnueabihf/libnettle.so.4 (0x75019000)
	libgmp.so.10 => /usr/lib/arm-linux-gnueabihf/libgmp.so.10 (0x74fa7000)
	libp11-kit.so.0 => /usr/lib/arm-linux-gnueabihf/libp11-kit.so.0 (0x74f65000)
	libtasn1.so.6 => /usr/lib/arm-linux-gnueabihf/libtasn1.so.6 (0x74f45000)
	libffi.so.6 => /usr/lib/arm-linux-gnueabihf/libffi.so.6 (0x74f35000)

I'm using the current master of moonfire.

I could not pass a PID to gdb, so I executed it like this (I added your commands to my gbdinit file):

pi@einsteinpi:~/moonfire-nvr$ gdb -ex=r  --args  /usr/local/bin/moonfire-nvr run --sample-file-dir=/var/lib/moonfire-nvr/samples --db-dir=/var/lib/moonfire-nvr/db --http-addr=0.0.0.0:8081

The output ist the following:

Starting program: /usr/local/bin/moonfire-nvr run --sample-file-dir=/var/lib/moonfire-nvr/samples --db-dir=/var/lib/moonfire-nvr/db --http-addr=0.0.0.0:8081
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
Cannot access memory at address 0x0

Program received signal SIGILL, Illegal instruction.
[New Thread 0x749ff290 (LWP 26386)]
[New Thread 0x745ff290 (LWP 26387)]
[New Thread 0x741ff290 (LWP 26388)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x741ff290 (LWP 26388)]
0x76fbcc1c in memcmp () from /usr/lib/arm-linux-gnueabihf/libarmmem.so
A debugging session is active.

	Inferior 1 [process 26383] will be killed.

Regards
Chris

@scottlamb scottlamb changed the title moonfire stops when opening input crash with ffmpeg 3.x May 4, 2017
@scottlamb
Copy link
Owner

scottlamb commented May 4, 2017 •
edited

I don't see the backtrace (you redirected it to a file?) but I see the problem from the ffmpeg command output you quoted (thanks for that). ffmpeg's version numbers seem inscrutable to me, but you wrote:

libavutil 55. 61.100 / 55. 61.100

and ffmpeg.org says that ffmpeg 3.3 has this version:

libavutil 55. 58.100

so you're presumably newer than ffmpeg 3.3. moonfire-nvr is still stuck on ffmpeg 2.x right now.

It's frustrating that Moonfire requires a narrow range of ffmpeg versions (the version bundled with Raspbian is too old to work, anything 3.x is too new to work) and that if you violate its (poorly stated) requirements, it just crashes. Rust intends to be a safe language, and for the most part it is, but the FFI is decidedly unsafe, both in general and for moonfire's use of ffmpeg in particular. (I'm using an old version of the rust-ffmpeg bug; I filed a similar bug against that package here.) There's Rust code with structure definitions that have to match the C ones or bad things will happen. And at present, those definitions expect ffmpeg 2.x.

I've been tempted to start using rust-bindgen to improve the situation by automatically producing the Rust bindings from the C ones at compile time, so that it should be correct at least if you're running against the same version you compiled against. And then either statically linking against ffmpeg (to guarantee that) or trusting that the underlying C library handled ABI compatibility properly.

The downside is that rust-bindgen's installation is another step to do when compiling Moonfire. For Raspbian Jessie, I mean this sentence on their Requirements page:

If you are using older version of Ubuntu or other Debian-based distros, you may need to add the LLVM repos to get version 3.9. See http://apt.llvm.org/.

@scottlamb scottlamb mentioned this issue May 10, 2017
Closed
@scottlamb
Copy link
Owner

I've been thinking about this problem. I think I can solve it without introducing rust-bindgen into the picture. I can write a small C wrapper around ffmpeg's Rust-unfriendly interface (macros and struct member accesses) that provides a C interface that's both inherently easier to wrap and versioned with the rest of moonfire-nvr, eliminating any unsafety.

I'm not quite sure how to do some of the details, though, and asked for help here.

@scottlamb
Copy link
Owner

Should work properly with ffmpeg 3.x now (and still work with ffmpeg 2.x). Let me know if you see otherwise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@chris1705 @scottlamb