Segmentation fault in GetBaseBatches()

Hi,

We are running Urho3d 1.5 on Debian Jessie on Raspberry PI. We have a set up of ten running units.
They all run fine, but after leaving them running a day or two, they tend to stop or die.

One common reason seem to be a segmentation fault deep in the Renderer/View.
We have managed to get this stack trace from them:

[2016-05-05 06:54:08][ERROR] Segmentation fault (SIGSEGV) stacktrace:

Unmangled stack trace:

Urho3D::View::GetBaseBatches()+0x6c
Urho3D::View::Update(Urho3D::FrameInfo const&)+0x3a4
Urho3D::Renderer::Update(float)+0x48c
Urho3D::EventHandlerImpl<Urho3D::Renderer>::Invoke(Urho3D::HashMap<Urho3D::StringHash, Urho3D::Variant>&)+0x44
Urho3D::Object::SendEvent(Urho3D::StringHash, Urho3D::HashMap<Urho3D::StringHash, Urho3D::Variant>&)+0xc50
Urho3D::Engine::Update()+0x108
Urho3D::Engine::RunFrame()+0xe0

Does anyone have any idea what might be going wrong? Any suggestions on how to pin down the cause of the problem further?

Sincerely
alexander

Welcome to our forum.

You didn’t say which application was running in your post but I presume it is one of yours. In any case, it would be easier to troubleshoot if we know the exact line number in the source code instead of offset. Also have you tried to use valgrind?

Hi Weitjong,
Thanks for taking the time to try to help us.

You are right in presuming thet it is our program that runs the Urho3D engine. We use Urho3D::ApplicationExt::Run() to get things going.

We would also like to have the line number, but so far we havnt managed to get it. This could wery well be due to our lack of skills when it comes to c++ and the development enviroment. We have tried Valgrind, but again havnt got it to work with our set up.

Any further suggestions are welcome.

Sincerely
Alexander

Application failing over a long time sounds like a potential memory leak. Once a dynamic memory allocation through new() fails (and if it doesn’t throw bad_alloc but just returns null) all bets are basically off. The memory leak can either be in Urho or in your application, or even in the graphics driver or other system software; you could compare by letting some of the Urho examples run over a long time, though this is not definitive as each example exercises different parts of the engine, and thus might not uncover the engine leak (if any) in question.

I get the line number shown in the stacktrace when I use Debug build configuration (i.e. having “-g” compiler flag set and without having any optimization flags set). I am using Fedora (and Pidora on RPI) but I think it should be the same for Debian too. I haven’t tried valgrind on RPI yet, but on a desktop version it only detected so far leak coming from SDL and not from Urho3D library. I was using one of the Urho3D samples as the test target. This is the command that I use: valgrind --tool=memcheck --leak-check=yes -v --leak-check=full --show-reachable=yes ./test.

HTH

Hi again,

Thank you for your suggestions on parameters for valgrind.
We did get valgrind running eventually, and it showed some memory leaks but nothing alarming,if that is a valid statement

We also got the system to make core dumps, and from those we have the line number on where it happens.
It is at: GetBaseBatches Source/Urho3D/Graphics/View.cpp:1182
I understand the concept of “all bets are off” when memory leaks start to be a factor, but even so, i find it suspicious that the crashes keep apearing in same place in the code. Can the affected code be extra sencitive?
We have looked at the code in question, and it seems to be copying some objects from one arry to onther. And most of the time it succeds… But then it suddenly crashes instead.

Is there something we can do to safeguard the problematic code section?

Here is information from the core dump. Perhaps it can give you some clue.
Or is there some other info we could gather that might be helpful in solving this?

Core was generated by `urhoplayer /root/playlist/'.
Program terminated with signal SIGSEGV, Segmentation fault.

(gdb) bt full
#0  Urho3D::View::GetBaseBatches (this=this@entry=0x20d6700)
    at C:/Documents/branches/5304_Raspberry_Flow/FlowCinema/Urho3D/Source/Urho3D/Graphics/View.cpp:1182
        drawable = 0x71b004b8
        type = <optimized out>
        vertexLightsProcessed = <optimized out>
        i = {ptr_ = 0x71b004b4}
        profile_GetBaseBatches = {profiler_ = 0x1}
#1  0x0017a560 in Urho3D::View::GetBatches (this=this@entry=0x20d6700)
    at C:/Documents/branches/5304_Raspberry_Flow/FlowCinema/Urho3D/Source/Urho3D/Graphics/View.cpp:967
No locals.
#2  0x0017a9a8 in Urho3D::View::Update (this=this@entry=0x20d6700, frame=...)
    at C:/Documents/branches/5304_Raspberry_Flow/FlowCinema/Urho3D/Source/Urho3D/Graphics/View.cpp:579
        eventData = @0x205b180: {<Urho3D::HashBase> = {static MIN_BUCKETS = 8, static MAX_LOAD_FACTOR = 4, head_ = 0x205dd44,
            tail_ = 0x205abac, ptrs_ = 0x205a9e0, allocator_ = 0x205ab98}, <No data fields>}
#3  0x0019ac64 in Urho3D::Renderer::Update (this=0x2058760, timeStep=0.042088002)
    at C:/Documents/branches/5304_Raspberry_Flow/FlowCinema/Urho3D/Source/Urho3D/Graphics/Renderer.cpp:656
        viewport = @0x20d3790: {ptr_ = 0x2067578, refCount_ = 0x20d4ec8}
        octree = 0x207f4e0
        renderTarget = @0x20d3788: {ptr_ = 0x2077f00, refCount_ = 0x2077f40}
        view = 0x20d6700
        scene = 0x2082a00
        i = 1
        profile_UpdateViews = {profiler_ = 0x0}
#4  0x0019bb74 in Urho3D::EventHandlerImpl<Urho3D::Renderer>::Invoke (this=<optimized out>, eventData=...)
    at C:/Documents/branches/5304_Raspberry_Flow/FlowCinema/Urho3D/Source/Urho3D/Graphics/../Core/../Core/Object.h:309
        receiver = <optimized out>
#5  0x0022be00 in Urho3D::Object::OnEvent (this=<optimized out>, sender=0x2032850, eventType=..., eventData=...)
    at C:/Documents/branches/5304_Raspberry_Flow/FlowCinema/Urho3D/Source/Urho3D/Core/Object.cpp:121
        context = 0x20361b8
        specific = 0x0
        nonSpecific = <optimized out>
        handler = <optimized out>
#6  0x0022d50c in Urho3D::Object::SendEvent (this=this@entry=0x2032850, eventType=..., eventData=...)
    at C:/Documents/branches/5304_Raspberry_Flow/FlowCinema/Urho3D/Source/Urho3D/Core/Object.cpp:347
        current = <optimized out>
        receiver = <optimized out>
        next = <optimized out>
        i = {<Urho3D::HashIteratorBase> = {ptr_ = 0x2031d34}, <No data fields>}
        self = {ptr_ = <optimized out>, refCount_ = 0x20358d0}
        context = 0x0
        processed = {<Urho3D::HashBase> = {static MIN_BUCKETS = 8, static MAX_LOAD_FACTOR = 4, head_ = 0x20d32cc, tail_ = 0x20d32cc,
            ptrs_ = 0x0, allocator_ = 0x20d32b8}, <No data fields>}
        group = 0x2031cc4
#7  0x00211244 in Urho3D::Engine::Update (this=this@entry=0x2032850)
    at C:/Documents/branches/5304_Raspberry_Flow/FlowCinema/Urho3D/Source/Urho3D/Engine/Engine.cpp:642
        profile_Update = {profiler_ = 0x2036108}
        eventData = @0x2054ce8: {<Urho3D::HashBase> = {static MIN_BUCKETS = 8, static MAX_LOAD_FACTOR = 4, head_ = 0x205ab6c,
---Type <return> to continue, or q <return> to quit---
            tail_ = 0x205aae4, ptrs_ = 0x205aa40, allocator_ = 0x205aad0}, <No data fields>}
#8  0x00216898 in Urho3D::Engine::RunFrame (this=0x2032850)
    at C:/Documents/branches/5304_Raspberry_Flow/FlowCinema/Urho3D/Source/Urho3D/Engine/Engine.cpp:464
        time = 0x2035808
        input = 0x2034b98
        audio = 0x20319d0
#9  0x000eb2ec in Urho3D::ApplicationExt::Run (this=this@entry=0x20324f0)
    at C:/Documents/branches/5304_Raspberry_Flow/FlowCinema/Projector2/Source/Players/UrhoPlayer/ApplicationExt.cpp:90
No locals.
#10 0x000d28a8 in main (argc=<optimized out>, argv=<optimized out>)
    at C:/Documents/branches/5304_Raspberry_Flow/FlowCinema/Projector2/Source/Players/UrhoPlayer/UrhoPlayer.cpp:28
        context = {ptr_ = 0x20361b8}
        flow = {ptr_ = 0x20324f0}
        cmdLine = {static NPOS = 4294967295, static MIN_CAPACITY = 8, static EMPTY = {static NPOS = 4294967295, static MIN_CAPACITY = 8,
            static EMPTY = <same as static member of an already seen type>, length_ = 0, capacity_ = 0,
            buffer_ = 0x4d4e54 <Urho3D::String::endZero> "", static endZero = 0 '\000'}, length_ = 35, capacity_ = 44,
          buffer_ = 0x20367a8 "-p \"Data;CoreData;/root/playlist/\" ", static endZero = 0 '\000'}
        arguments = {<Urho3D::VectorBase> = {size_ = 2, capacity_ = 2, buffer_ = 0x2035f70 "\002"}, <No data fields>}
        exitCode = <optimized out>
(gdb)

(gdb) info threads
  Id   Target Id         Frame
  9    Thread 0x73025450 (LWP 6461) 0x76b33360 in nanosleep () at ../sysdeps/unix/syscall-template.S:81
  8    Thread 0x76a3c450 (LWP 6454) 0x76b5e02c in ioctl () at ../sysdeps/unix/syscall-template.S:81
  7    Thread 0x74825450 (LWP 6458) 0x76b33364 in nanosleep () at ../sysdeps/unix/syscall-template.S:81
  6    Thread 0x74025450 (LWP 6459) 0x76b33364 in nanosleep () at ../sysdeps/unix/syscall-template.S:81
  5    Thread 0x73825450 (LWP 6460) 0x76b33364 in nanosleep () at ../sysdeps/unix/syscall-template.S:81
  4    Thread 0x750ff450 (LWP 6457) 0x76f29a40 in do_futex_wait (isem=isem@entry=0x76ddbaf4 <cecservice_notify_available_event+24>)
    at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:48
  3    Thread 0x758ff450 (LWP 6456) 0x76f29a40 in do_futex_wait (isem=isem@entry=0x76ddad6c <tvservice_notify_available_event+24>)
    at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:48
  2    Thread 0x760ff450 (LWP 6455) 0x76f29a40 in do_futex_wait (isem=isem@entry=0x76ddbbf0 <dispmanx_notify_available_event+24>)
    at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:48
* 1    Thread 0x76f67000 (LWP 6453) Urho3D::View::GetBaseBatches (this=this@entry=0x20d6700)
    at C:/Documents/branches/5304_Raspberry_Flow/FlowCinema/Urho3D/Source/Urho3D/Graphics/View.cpp:1182

We will mow try to place a watchdog around the urho player, so we can restart it if it suddenly crashes. Hopfully this will help in our case. But if something can be done to prevent this crash in the first place, that would be great!

Thanks again for your comittment and support!

This is an iteration of the drawable objects in view. I don’t think it can be safeguarded any more than any other code in the engine; the engine just has to make the assumption that the array & objects in question are intact.

Instead of just a memory leak, there could also be heap corruption in play, which may be harder to diagnose.