Crash on heap dump

I need to get a heap dump of my app but it keeps crashing either by using Chrome dev tools or node-heapdump module. After several tests I figured out that it’s some thing related to mediasoup. The core dump shows that the crash is happening in Channel::WriteBuffer() and when I don’t create any Transport/Producer it does not crash.

Hard to help as you may imagine. Do you have exact steps to reproduce it?

Tested on mediasoup demo and it doesn’t crash. So no, it’s not about mediasoup. Sorry for disturbing.

BTW having heapdump in the available commands of the mediasoup demo would be useful for memory leak diagnostics.

I think this really means a crash in mediasoup. Do you have the coredump?

Also, no idea which “heapdump” tool you mean.

Not sure, may be my bad usage. The coredump gives not much information but I’m struggling with valgrind to figure it out :grimacing:. Here’s the report:

==23318== ERROR SUMMARY: 460 errors from 20 contexts (suppressed: 0 from 0)
==23318==
==23318== 1 errors in context 1 of 20:
==23318== Invalid read of size 8
==23318==    at 0xFCFA73: v8::internal::V8HeapExplorer::AddEntry(v8::internal::HeapObject*) (in /root/.nvm/versions/node/v8.16.0/bin/node)
==23318==    by 0xFC775F: v8::internal::V8HeapExplorer::SetContextReference(v8::internal::HeapObject*, int, v8::internal::String*, v8::internal::Object*, int) (in /root/.nvm/versions/node/v8.16.0/bin/node)
==23318==    by 0xFC9B8F: v8::internal::V8HeapExplorer::ExtractContextReferences(int, v8::internal::Context*) (in /root/.nvm/versions/node/v8.16.0/bin/node)
==23318==    by 0xFC9C94: v8::internal::V8HeapExplorer::ExtractReferencesPass2(int, v8::internal::HeapObject*) (in /root/.nvm/versions/node/v8.16.0/bin/node)
==23318==    by 0xFD0E64: bool v8::internal::V8HeapExplorer::IterateAndExtractSinglePass<&v8::internal::V8HeapExplorer::ExtractReferencesPass2>() (in /root/.nvm/versions/node/v8.16.0/bin/node)
==23318==    by 0xFD1427: v8::internal::V8HeapExplorer::IterateAndExtractReferences(v8::internal::SnapshotFiller*) (in /root/.nvm/versions/node/v8.16.0/bin/node)
==23318==    by 0xFD1811: v8::internal::HeapSnapshotGenerator::GenerateSnapshot() (in /root/.nvm/versions/node/v8.16.0/bin/node)
==23318==    by 0xFBC24B: v8::internal::HeapProfiler::TakeSnapshot(v8::ActivityControl*, v8::HeapProfiler::ObjectNameResolver*) (in /root/.nvm/versions/node/v8.16.0/bin/node)
==23318==    by 0x993733C: (anonymous namespace)::OnSIGUSR2(uv_signal_s*, int) (in /var/conference/server/node_modules/heapdump/build/Release/addon.node)
==23318==    by 0x9C3475: uv__signal_event (signal.c:468)
==23318==    by 0x9CA781: uv__io_poll (linux-core.c:382)
==23318==    by 0x9BA264: uv_run (core.c:370)
==23318==  Address 0x2 is not stack'd, malloc'd or (recently) free'd
==23318==
==23318==
==23318== 4 errors in context 2 of 20:
==23318== Syscall param epoll_ctl(event) points to uninitialised byte(s)
==23318==    at 0x5E6E9BA: epoll_ctl (syscall-template.S:84)
==23318==    by 0x9CA3AD: uv__io_poll (linux-core.c:249)
==23318==    by 0x9BA264: uv_run (core.c:370)
==23318==    by 0x8D6814: node::Start(uv_loop_s*, int, char const* const*, int, char const* const*) (in /root/.nvm/versions/node/v8.16.0/bin/node)
==23318==    by 0x8D5B6F: node::Start(int, char**) (in /root/.nvm/versions/node/v8.16.0/bin/node)
==23318==    by 0x5D8782F: (below main) (libc-start.c:291)
==23318==  Address 0xffeffc298 is on thread 1's stack
==23318==  in frame #1, created by uv__io_poll (linux-core.c:189)
==23318==  Uninitialised value was created by a stack allocation
==23318==    at 0x9CA2E0: uv__io_poll (linux-core.c:189)
==23318==
==23318==
==23318== 45 errors in context 3 of 20:
==23318== Syscall param epoll_ctl(event) points to uninitialised byte(s)
==23318==    at 0x5E6E9BA: epoll_ctl (syscall-template.S:84)
==23318==    by 0x9CA379: uv__io_poll (linux-core.c:242)
==23318==    by 0x9BA264: uv_run (core.c:370)
==23318==    by 0x8D6814: node::Start(uv_loop_s*, int, char const* const*, int, char const* const*) (in /root/.nvm/versions/node/v8.16.0/bin/node)
==23318==    by 0x8D5B6F: node::Start(int, char**) (in /root/.nvm/versions/node/v8.16.0/bin/node)
==23318==    by 0x5D8782F: (below main) (libc-start.c:291)
==23318==  Address 0xffeffc298 is on thread 1's stack
==23318==  in frame #1, created by uv__io_poll (linux-core.c:189)
==23318==  Uninitialised value was created by a stack allocation
==23318==    at 0x9CA2E0: uv__io_poll (linux-core.c:189)
==23318==
==23318==
==23318== 394 errors in context 4 of 20:
==23318== Syscall param epoll_pwait(sigmask) points to unaddressable byte(s)
==23318==    at 0x5E6E627: epoll_pwait (epoll_pwait.c:42)
==23318==    by 0x9CA456: uv__io_poll (linux-core.c:275)
==23318==    by 0x9BA264: uv_run (core.c:370)
==23318==    by 0x8D6814: node::Start(uv_loop_s*, int, char const* const*, int, char const* const*) (in /root/.nvm/versions/node/v8.16.0/bin/node)
==23318==    by 0x8D5B6F: node::Start(int, char**) (in /root/.nvm/versions/node/v8.16.0/bin/node)
==23318==    by 0x5D8782F: (below main) (libc-start.c:291)
==23318==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==23318==
==23318== ERROR SUMMARY: 460 errors from 20 contexts (suppressed: 0 from 0)

This cool tool: GitHub - bnoordhuis/node-heapdump: Make a dump of the V8 heap for later inspection.

I’ve added it to the mediasoup-demo server:

Now, no idea how to inspect such a .heapsnapshot in Chrome. Yes, I know, open “Memory” tab and etc. But no idea about how to understand the data it provides.

Can you explain it without requiring me to become an expert in this topic?

I’m not an expert too but it needs a little practice and you can find lots of guides on the Internet like this one. The most useful part is the comparison view for finding leakages.

The crash does not happen on node.js v10 neither on v12 so I think it must be a bug with v8.

BTW should I rebuild mediasoup for different node.js versions? Some native addons, like heapdump, are strict and prevent from using a compiled addon with different version while mediasoup is working fine on various versions without need to rebuild.

But I don’t think this is useful at all for the mediasoup worker subprocess, right?

The mediasoup C++ component is not a Node addon but a separate process. So this tool is not useful for it

Correct, just useful for the node.js part of the app and this is why mediasoup works with different node.js versions without need to rebuild. :ok_hand: