SIGFPE crash in libbcg729 / bcg729Encoder with rtpengine 12.5.1 on Debian 12

[descartin]

rtpengine version the issue has been seen with

Version: 12.5.1.51+0~mr12.5.1.51

Used distribution and its version

Debian 12

Linux kernel version used

6.1.0-42-amd64

CPU architecture issue was seen on (see uname -m)

x86_64

Expected behaviour you didn't see

Hello,

This may be related to #1827, but I am opening a new issue because this occurrence was seen on a newer rtpengine and OS version.

we have seen what looks like the same kind of crash previously reported in that issue, but now on a newer setup:

• rtpengine version: 12.5.1.51+0~mr12.5.1.51

• OS: Debian GNU/Linux 12 (bookworm)

• Kernel: Linux 6.1.0-42-amd64

• Architecture: x86_64

• Hardware: Dell PowerEdge R450

• libbcg729 packages:

  • libbcg729-0:amd64 1.1.1-2
  • libbcg729-dev:amd64 1.1.1-2

The process generated a core dump and terminated with SIGFPE:

Program terminated with signal SIGFPE, Arithmetic exception.
#0  0x00007fad67640fb2 in ?? () from /lib/x86_64-linux-gnu/libbcg729.so.0
#1  0x00007fad6763e104 in bcg729Encoder () from /lib/x86_64-linux-gnu/libbcg729.so.0
#2  0x00005583ddb21b4f in bcg729_encoder_input (enc=0x7fad3c56c7d0, frame=0x7fad5edf4f78) at ../lib/codeclib.c:3562
#3  0x00005583ddb16d05 in encoder_input_data (...) at ../lib/codeclib.c:2108
#4  0x00005583ddb1e37e in encoder_fifo_flush (...) at ../lib/codeclib.c:2149
#5  encoder_input_fifo (...) at ../lib/codeclib.c:2169
#6  0x00005583ddadd27a in __buffer_delay_frame (...) at ./daemon/codec.c:2811
#7  packet_decoded_common (...) at ./daemon/codec.c:4258
#8  0x00005583ddb1df76 in __decoder_input_data (...) at ../lib/codeclib.c:1245
#9  0x00005583ddad768a in decoder_input_data_ptime (...) at ../lib/codeclib.c:1268
#10 __rtp_decode_direct (...) at ./daemon/codec.c:4314
#11 0x00005583ddad4d4c in __handler_func_sequencer (...) at ./daemon/codec.c:2008
#12 0x00005583ddab49aa in stream_packet (...) at ./daemon/media_socket.c:2822
#13 0x00005583ddab5ff0 in __stream_fd_readable (...) at ./daemon/media_socket.c:3001
#14 0x00005583ddab6636 in stream_fd_readable (...) at ./daemon/media_socket.c:3079
#15 0x00005583ddb0e727 in poller_poll (...) at ./daemon/poller.c:240
#16 poller_loop (...) at ./daemon/poller.c:345
#17 0x00005583dda303cb in thread_detach_func (...) at ./daemon/helpers.c:264
#18 0x00007fad673f71f5 in start_thread (...) at ./nptl/pthread_create.c:442
#19 0x00007fad674778dc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

From bt full, the crash happens while processing/transcoding RTP media. Some relevant details from the crashing thread:

#2 bcg729_encoder_input (...) at ../lib/codeclib.c:3562
        len = 0 '\000'

#8 __decoder_input_data (...)
        frames = {head = 0x0, tail = 0x0, length = 0}
        ret = 0
        samples = 160

#14 stream_fd_readable (...)
        phc.s.len = 172
        payload.len = 160
        payload_type = 0
        sink.attrs.transcoding = 1
        ptime = 0

The pattern looks very similar to the previous crash reported here: SIGFPE inside libbcg729.so.0, then bcg729Encoder(), then bcg729_encoder_input() in rtpengine.

Given that this is now happening on rtpengine 12.5.1 with Debian 12 and kernel 6.1, it does not look like this can simply be attributed to an old rtpengine or old OS version.

Could you please advise whether this should be handled on the rtpengine side, for example by validating or guarding the input passed to bcg729Encoder(), or whether this should be reported upstream to the bcg729 project?

I can provide the full bt full, core details, rtpengine config, or packet/log context if needed.

Thanks a lot and regards,

David

Unexpected behaviour you saw

No response

Steps to reproduce the problem

No response

Additional program output to the terminal or logs illustrating the issue

Anything else?

No response

[rfuchs]

Unless there is some kind of usage error of the library, this is most likely a problem of the bcg729 library. Doesn't look like Debian provides debugging symbols for the library, so this won't be easy to track down.

[rfuchs]

Maybe the encoder state gives a clue, post the output of:

frame 2
x/80xh (*frame)->extended_data[0]
x/209xg 0x7fad3c56c7d0

[descartin]

Hello Richard

let me share the output of those commands

2115_bt_output.txt
thanks a lot and regards
david

[rfuchs]

Ah sorry, one more please (I misread the code)

frame 2
x/209xg enc->bcg729

Thanks!

[descartin]

hello Richard

here you have

thanks a lot

2115_bt_output_2.txt

[rfuchs]

Sadly when calling the encoder with this exact state, I don't get a SIGFPE ☹️

Let's try:

frame 0
x/200i 0x00007fad67640f00

[descartin]

hello

i add the output 👍

2115_bt_output_frame0.txt

best regards
david

[rfuchs]

Ok, getting closer. Also post info registers please

[descartin]

here you have

thanks a lot
david

2115_bt_output_registers.txt

[rfuchs]

AFAICT the culprit is in this line: https://github.com/BelledonneCommunications/bcg729/blob/master/src/LP2LSPConversion.c#L103

where Cx and previousCx are the same value (both -1 AFAICT) and so SUB32(Cx, previousCx) comes out as zero, resulting in division by zero.

I'll open a bug report there, will see if there's any activity.