Skip to content

sio::client destructor stuck indefinitely #254

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bugale opened this issue Mar 25, 2020 · 2 comments · Fixed by #290
Closed

sio::client destructor stuck indefinitely #254

bugale opened this issue Mar 25, 2020 · 2 comments · Fixed by #290

Comments

@bugale
Copy link

bugale commented Mar 25, 2020

Hi, I encountered an issue that happens sometimes, when the connection gets closed (i.e. a cable gets unplugged) during a session. Even in this scenatio it only happens sporadically.
In these cases, the destructor of sio::client is stuck forever.

The stack traces I found when trying to debug it are:

00 00000000`05c7eda8 000007fe`fd6710ac ntdll!ZwWaitForSingleObject+0xa
01 00000000`05c7edb0 000007fe`ed0595a7 KERNELBASE!WaitForSingleObjectEx+0x79
02 00000000`05c7ee50 000007fe`ecad38ed SentinelAgentCore!_Thrd_join+0x1f [d:\agent\_work\5\s\src\vctools\crt\github\stl\src\cthread.cpp @ 56] 
03 00000000`05c7ee80 000007fe`ed0fb567 SentinelAgentCore!std::thread::join+0x2d [C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.24.28314\include\thread @ 113] 
04 (Inline Function) --------`-------- SentinelAgentCore!sio::client_impl::sync_close+0x7c [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\buildtrees\socket-io-client\src\92649efe88-6a0711c271\src\internal\sio_client_impl.cpp @ 154] 
05 00000000`05c7eec0 000007fe`ed0d4506 SentinelAgentCore!sio::client_impl::~client_impl+0xa7 [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\buildtrees\socket-io-client\src\92649efe88-6a0711c271\src\internal\sio_client_impl.cpp @ 63] 
06 00000000`05c7ef50 000007fe`ecf21922 SentinelAgentCore!sio::client::~client+0x16 [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\buildtrees\socket-io-client\src\92649efe88-6a0711c271\src\sio_client.cpp @ 23] 

The thread it waits for is:

00 00000000`0a11fb38 000007fe`fd6716ad ntdll!NtRemoveIoCompletion+0xa
01 00000000`0a11fb40 00000000`77859991 KERNELBASE!GetQueuedCompletionStatus+0x39
02 00000000`0a11fba0 000007fe`ed110f14 kernel32!GetQueuedCompletionStatusStub+0x11
03 00000000`0a11fbe0 000007fe`ed12efa1 SentinelAgentCore!boost::asio::detail::win_iocp_io_context::do_one+0x2b4 [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\installed\x64-windows-static\include\boost\asio\detail\impl\win_iocp_io_context.ipp @ 381] 
04 (Inline Function) --------`-------- SentinelAgentCore!boost::asio::detail::win_iocp_io_context::run+0xb2 [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\installed\x64-windows-static\include\boost\asio\detail\impl\win_iocp_io_context.ipp @ 163] 
05 00000000`0a11fcd0 000007fe`ed12f645 SentinelAgentCore!boost::asio::io_context::run+0xe1 [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\installed\x64-windows-static\include\boost\asio\impl\io_context.ipp @ 62] 
06 (Inline Function) --------`-------- SentinelAgentCore!websocketpp::transport::asio::endpoint<websocketpp::config::asio_tls_client::transport_config>::run+0xc [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\installed\x64-windows-static\include\websocketpp\transport\asio\endpoint.hpp @ 642] 
07 00000000`0a11fd50 000007fe`ed102929 SentinelAgentCore!sio::client_impl::run_loop+0x15 [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\buildtrees\socket-io-client\src\92649efe88-6a0711c271\src\internal\sio_client_impl.cpp @ 195] 
08 (Inline Function) --------`-------- SentinelAgentCore!boost::_mfi::mf0<void,sio::client_impl>::operator()+0x6 [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\installed\x64-windows-static\include\boost\bind\mem_fn_template.hpp @ 49] 
09 (Inline Function) --------`-------- SentinelAgentCore!boost::_bi::list1<boost::_bi::value<sio::client_impl *> >::operator()+0x6 [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\installed\x64-windows-static\include\boost\bind\bind.hpp @ 259] 
0a (Inline Function) --------`-------- SentinelAgentCore!boost::_bi::bind_t<void,boost::_mfi::mf0<void,sio::client_impl>,boost::_bi::list1<boost::_bi::value<sio::client_impl *> > >::operator()+0x6 [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\installed\x64-windows-static\include\boost\bind\bind.hpp @ 1294] 
0b (Inline Function) --------`-------- SentinelAgentCore!std::_Invoker_functor::_Call+0x6 [c:\program files (x86)\microsoft visual studio\2017\professional\vc\tools\msvc\14.16.27023\include\thr\xthread @ 230] 
0c (Inline Function) --------`-------- SentinelAgentCore!std::invoke+0x6 [c:\program files (x86)\microsoft visual studio\2017\professional\vc\tools\msvc\14.16.27023\include\thr\xthread @ 230] 
0d (Inline Function) --------`-------- SentinelAgentCore!std::_LaunchPad<std::unique_ptr<std::tuple<boost::_bi::bind_t<void,boost::_mfi::mf0<void,sio::client_impl>,boost::_bi::list1<boost::_bi::value<sio::client_impl *> > > >,std::default_delete<std::tuple<boost::_bi::bind_t<void,boost::_mfi::mf0<void,sio::client_impl>,boost::_bi::list1<boost::_bi::value<sio::client_impl *> > > > > > >::_Execute+0x6 [c:\program files (x86)\microsoft visual studio\2017\professional\vc\tools\msvc\14.16.27023\include\thr\xthread @ 238] 
0e (Inline Function) --------`-------- SentinelAgentCore!std::_LaunchPad<std::unique_ptr<std::tuple<boost::_bi::bind_t<void,boost::_mfi::mf0<void,sio::client_impl>,boost::_bi::list1<boost::_bi::value<sio::client_impl *> > > >,std::default_delete<std::tuple<boost::_bi::bind_t<void,boost::_mfi::mf0<void,sio::client_impl>,boost::_bi::list1<boost::_bi::value<sio::client_impl *> > > > > > >::_Run+0x52 [c:\program files (x86)\microsoft visual studio\2017\professional\vc\tools\msvc\14.16.27023\include\thr\xthread @ 245] 
0f 00000000`0a11fd80 000007fe`ed1021d9 SentinelAgentCore!std::_LaunchPad<std::unique_ptr<std::tuple<boost::_bi::bind_t<void,boost::_mfi::mf0<void,sio::client_impl>,boost::_bi::list1<boost::_bi::value<sio::client_impl *> > > >,std::default_delete<std::tuple<boost::_bi::bind_t<void,boost::_mfi::mf0<void,sio::client_impl>,boost::_bi::list1<boost::_bi::value<sio::client_impl *> > > > > > >::_Go+0x69 [c:\program files (x86)\microsoft visual studio\2017\professional\vc\tools\msvc\14.16.27023\include\thr\xthread @ 230] 
10 00000000`0a11fdc0 000007fe`ed0b0da0 SentinelAgentCore!std::_Pad::_Call_func+0x9 [c:\program files (x86)\microsoft visual studio\2017\professional\vc\tools\msvc\14.16.27023\include\thr\xthread @ 209] 
11 00000000`0a11fdf0 00000000`778659cd SentinelAgentCore!thread_start<unsigned int (__cdecl*)(void *),1>+0x50 [minkernel\crts\ucrt\src\appcrt\startup\thread.cpp @ 97] 
12 00000000`0a11fe20 00000000`779c385d kernel32!BaseThreadInitThunk+0xd
13 00000000`0a11fe50 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

And I also found these two:

00 00000000`0a5efd18 000007fe`fd6710ac ntdll!ZwWaitForSingleObject+0xa
01 00000000`0a5efd20 000007fe`ed0de9d3 KERNELBASE!WaitForSingleObjectEx+0x79
02 (Inline Function) --------`-------- SentinelAgentCore!boost::asio::detail::win_iocp_io_context::timer_thread_function::operator()+0x2d [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\installed\x64-windows-static\include\boost\asio\detail\impl\win_iocp_io_context.ipp @ 53] 
03 00000000`0a5efdc0 000007fe`ed0df74f SentinelAgentCore!boost::asio::detail::win_thread::func<boost::asio::detail::win_iocp_io_context::timer_thread_function>::run+0x33 [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\installed\x64-windows-static\include\boost\asio\detail\win_thread.hpp @ 122] 
04 00000000`0a5efdf0 000007fe`ed0b0da0 SentinelAgentCore!boost::asio::detail::win_thread_function+0x2f [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\installed\x64-windows-static\include\boost\asio\detail\impl\win_thread.ipp @ 127] 
05 00000000`0a5efe30 00000000`778659cd SentinelAgentCore!thread_start<unsigned int (__cdecl*)(void *),1>+0x50 [minkernel\crts\ucrt\src\appcrt\startup\thread.cpp @ 97] 
06 00000000`0a5efe60 00000000`779c385d kernel32!BaseThreadInitThunk+0xd
07 00000000`0a5efe90 00000000`00000000 ntdll!RtlUserThreadStart+0x1d
00 00000000`0a76fc38 000007fe`fd6716ad ntdll!NtRemoveIoCompletion+0xa
01 00000000`0a76fc40 00000000`77859991 KERNELBASE!GetQueuedCompletionStatus+0x39
02 00000000`0a76fca0 000007fe`ed110f14 kernel32!GetQueuedCompletionStatusStub+0x11
03 00000000`0a76fce0 000007fe`ed12efa1 SentinelAgentCore!boost::asio::detail::win_iocp_io_context::do_one+0x2b4 [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\installed\x64-windows-static\include\boost\asio\detail\impl\win_iocp_io_context.ipp @ 381] 
04 (Inline Function) --------`-------- SentinelAgentCore!boost::asio::detail::win_iocp_io_context::run+0xb2 [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\installed\x64-windows-static\include\boost\asio\detail\impl\win_iocp_io_context.ipp @ 163] 
05 00000000`0a76fdd0 000007fe`ed0df74f SentinelAgentCore!boost::asio::io_context::run+0xe1 [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\installed\x64-windows-static\include\boost\asio\impl\io_context.ipp @ 62] 
06 00000000`0a76fe50 000007fe`ed0b0da0 SentinelAgentCore!boost::asio::detail::win_thread_function+0x2f [c:\temp\2019-02-18\21\socket-io-client\temp\vcpkg\installed\x64-windows-static\include\boost\asio\detail\impl\win_thread.ipp @ 127] 
07 00000000`0a76fe90 00000000`778659cd SentinelAgentCore!thread_start<unsigned int (__cdecl*)(void *),1>+0x50 [minkernel\crts\ucrt\src\appcrt\startup\thread.cpp @ 97] 
08 00000000`0a76fec0 00000000`779c385d kernel32!BaseThreadInitThunk+0xd
09 00000000`0a76fef0 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

These specific stacktraces are from Win7x64, but I could reproduce it also on Win7x86 and Win10x64 with socket.io-client-cpp v1.6.1 (built using vcpkg).

To reproduce it what I did is:

  1. Create a connection to a server
  2. Send some data in both directions
  3. Cut off the connection (I used iptables on the serverside to block any IP communication with the client)
  4. Send some data from the client to the server
  5. Call the destructor of sio::client in the client side.

As I said, only sporadically, the destructor gets stuck.
If you need any more information about this that I can get for you, please tell me.

@DraconPern
Copy link

I think I ran into a similar issue. This problem is that that there's a timing problem with deleting the client when the code is trying to do a connection. How I fixed it was the following. Have a conditional variable called connecting and set it to false; In the code that you use to call connect(), do connecting = true;. In your set_socket_open_listener listener, have connecting = false; In your .socket()->on_error() listener, have connecting = false; Now check connecting before you delete sio::client. If it's true, you have to wait until it's false to delete.

@AdrianDC
Copy link
Contributor

@darrachequesne

We faced the issue here too, on CentOS 7 / CentOS 6 hosts using the C++ client,
while all hosts were constantly connected through LAN to a server.

The CPU was stuck at 100% in the sync_close function for hours, breaking the enclosing software system.

Here's an unit test proving this issue in a permanent way: test.cpp
and implementing the workaround logic mentioned by @DraconPern

docker run --rm -i -v "${PWD}:${PWD}" -w "${PWD}" centos:7.6.1810 <<EOF
yum install -y cmake gcc gcc-c++ make openssl-devel
cmake  ./
make install
g++ -std=c++11 -Wall -lpthread -o test254 test.cpp /usr/local/lib/libsioclient.a
timeout -s9 10 ./test254
timeout -s9 10 ./test254 true
EOF

Nelson-numerical-software added a commit to nelson-lang/nelson that referenced this issue Feb 9, 2021
druckdev added a commit to DeerSoftDevelopment/socket.io-client-cpp that referenced this issue Apr 6, 2021
* upstream/master:
  docs: update compatibility table for Socket.IO v4
  docs: fix title format (socketio#297)
  feat: allow resource path to be set in connection URI (socketio#134)
  fix: resolve client_impl::ping LOG call syntax in debug builds
  fix: resolve socketio#254: handle closing sockets upon on_fail events
  fix: lower the minimum CMake supported version
  feat: implement socketio#45: add support for logging configuration
  feat: add support for Socket.IO v3
  refactor: use correct Engine.IO protocol revision
  refactor: use standard install paths (socketio#58)
  ci: migrate to GitHub Actions
  chore: update .gitignore with cmake output
  refactor: remove Boost dependency (socketio#176)
cleancoderob added a commit to SolinkCorp/socket.io-client-cpp that referenced this issue Apr 3, 2023
Very similar problem as described, here, but no work-arounds would fix the issue.
socketio#254

Similar fix described here:
zaphoyd/websocketpp#805
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants