[omniORB] OMNIORB_ASSERT(num_active_oas == 0); with omniORB 4.1.2 and 4.1.4

Martin B. 0xCDCDCDCD at gmx.at
Mon Jan 10 12:40:00 GMT 2011


Hi all, Hi Duncan,

Today I got OMNIORB_ASSERT(num_active_oas == 0) for the 3rd time within 
one week on my development machine. Sadly logging isn't upped, but at 
least I have configured VS to immediately stop at any 
omniORB::fatalException ... so I'll try to give as much details as possible:

* Last week it happened with 4.1.2 but I'm now using 4.1.4
* Debug Build with Visual C++ 2005
* Pre-Built Binaries omniORB414_vc8_rtd.dll
* Windows XP sp3
* local connection
* Multithreaded MFC GUI app.
* ORB_init and shutdown(true) are called from the main (=GUI) thread
* The ORB::run() function is called from a separate thread (not the main 
thread)

The error occurs during ORB shutdown which is initiated by the main 
(GUI) thread which is the same thread that also called ORB_init.
All CORBA connections are already terminated during shutdown (that is, 
the only client I had connected was already disconnected several seconds 
before ORB shutdown is called.

objectadapter.cc : 509 - OMNIORB_ASSERT(num_active_oas == 0);

Find all omniORB related threads+callstacks at the end!

My observations:
* I call shutdown(true) from the main thread (#380), while the run() 
function is running from a separate thread (#2588)
* There seem to be several worker threads: omniAsyncWorker::run : #3068, 
#1372, #456, #4608
* Thread #5840 shows the destruction of one of the POAs I used. (Which 
is correct, as I shut down this POA by calling destroy(true, false)
   + Note that there should've been 2 POAs involved in this test run: 
the omniINS POA, and a newly created POA that used the RootPOA a parent.
   + Inspecting the POA object under destruction confirms it's the POA I 
created and not the INS POA.
* From the state my process is in at the moment, it is clear the the 
ASSERT is triggered when:
   + omniObjAdapter::shutdown() (called from 
omniOrbORB::actual_shutdown()) is holding the lock sync(oa_lock) and the 
POA shutdown thread #5840 is waiting for this lock to shut down the POA.
* What is weird is that omniOrbPOA::shutdown() (in 
omniOrbORB::actual_shutdown()) should have waited for all object 
adapters to shut down, but apparently this did not work.
* Maybe it's a similar timing problem to the one I asked about in my 
mail from 05.01.2011 -> 
http://www.omniorb-support.com/pipermail/omniorb-list/2011-January/030868.html
	
Note that my shutdown sequence (in thread 380) goes like this:
1) deactivate_object(..) of my main object in the omniINS POA
2) PortableServer::POAManager_var pman = other_poa->the_POAManager();
    pman->deactivate(true, /*wait=*/false);
    other_poa->destroy(true, /*wait=*/false);
    other_poa = PortableServer::POA::_nil();
    the_orb->shutdown(true); <<< assert is triggered here

The other_poa POA was created with these options:
   root_poa->create_lifespan_policy(TRANSIENT);
   root_poa->create_id_assignment_policy(SYSTEM_ID);
   root_poa->create_id_uniqueness_policy(UNIQUE_ID);
   root_poa->create_implicit_activation_policy(IMPLICIT_ACTIVATION); // 
!= default
   root_poa-> create_request_processing_policy(USE_ACTIVE_OBJECT_MAP_ONLY);
   root_poa->create_servant_retention_policy(RETAIN);
   root_poa->create_thread_policy(ORB_CTRL_MODEL);

***
Since this is pretty non-reproducible (read: I started and stopped the 
app under the debugger dozens and dozens of times during the last week) 
it would be great if I could get some tips how to influence omniORBs 
timing behaviour to repro this behaviour.
***

cheers,
Martin

Calls stack follow for reference:

#	380	wWinMainCRTStartup	_CxxThrowException	Normal	0
- - - - -
omniORB414_vc8_rtd.dll!omni::assertFail(const char * file=0x67c809dc, 
int line=509, const char * expr=0x67c809c8)  Line 1300 + 0x46 bytes	C++
omniORB414_vc8_rtd.dll!omni::omniObjAdapter::shutdown()  Line 509 + 
0x1d bytes	C++
omniORB414_vc8_rtd.dll!omniOrbORB::actual_shutdown()  Line 1020	C++
omniORB414_vc8_rtd.dll!omniOrbORB::do_shutdown(bool 
wait_for_completion=true)  Line 1078	C++
omniORB414_vc8_rtd.dll!omniOrbORB::shutdown(bool 
wait_for_completion=true)  Line 871	C++
my.dll!ShutdownDmORB()  Line 214	C++
my.dll!StopDatamanager()  Line 2477 + 0xd bytes	C++
my.exe!CMainFrame::OnClose()  Line 856	C++
mfc80ud.dll!CWnd::OnWndMsg(unsigned int message=16, unsigned int 
wParam=0, long lParam=0, long * pResult=0x0013ee14)  Line 2028	C++
...
mfc80ud.dll!CWinThread::PumpMessage()  Line 896	C++
mfc80ud.dll!CWinThread::Run()  Line 625 + 0xd bytes	C++
mfc80ud.dll!CWinApp::Run()  Line 894	C++
mfc80ud.dll!AfxWinMain(HINSTANCE__ * hInstance=0x00400000, HINSTANCE__ * 
hPrevInstance=0x00000000, wchar_t * lpCmdLine=0x00020d8e, int 
nCmdShow=1)  Line 47 + 0xd bytes	C++
my.exe!wWinMain(HINSTANCE__ * hInstance=0x00400000, HINSTANCE__ * 
hPrevInstance=0x00000000, wchar_t * lpCmdLine=0x00020d8e, int 
nCmdShow=1)  Line 33	C++
my.exe!__tmainCRTStartup()  Line 589 + 0x35 bytes	C
my.exe!wWinMainCRTStartup()  Line 414	C
kernel32.dll!_BaseProcessStart at 4()  + 0x23 bytes	
- - - - -

#	2588	_threadstartex	omni_condition::wait	Normal	0
- - - - -
omnithread34_vc8_rtd.dll!omni_condition::wait()  Line 187 + 0x16 bytes	C++
omniORB414_vc8_rtd.dll!omniOrbORB::run()  Line 823 + 0x16 bytes	C++
my.dll!run_thread_fn(void * param=0x02e991b8)  Line 315 + 0x23 bytes	C++
msvcr80d.dll!_callthreadstartex()  Line 348 + 0xf bytes	C
- - - - -

#	3068	_threadstartex	omni::SocketCollection::Select	Normal	0
- - - - -
omniORB414_vc8_rtd.dll!omni::SocketCollection::Select()  Line 929 + 
0x19 bytes	C++
omniORB414_vc8_rtd.dll!omni::tcpEndpoint::AcceptAndMonitor(void (void *, 
omni::giopConnection *)* func=0x67b64f02, void * cookie=0x02ea22e0) 
Line 721 + 0xb bytes	C++
omniORB414_vc8_rtd.dll!omni::giopRendezvouser::execute()  Line 96 + 
0x1e bytes	C++
omniORB414_vc8_rtd.dll!omniAsyncWorker::real_run()  Line 236 + 0x14 
bytes	C++
omniORB414_vc8_rtd.dll!omniAsyncWorkerInfo::run()  Line 285	C++
omniORB414_vc8_rtd.dll!omniAsyncWorker::run(void * __formal=0x00000000) 
  Line 164	C++
omnithread34_vc8_rtd.dll!omni_thread_wrapper(void * ptr=0x02ea2338) 
Line 502 + 0x16 bytes	C++
msvcr80d.dll!_callthreadstartex()  Line 348 + 0xf bytes	C
- - - - -

#	1372	_threadstartex	omni_condition::timedwait	Normal	0
- - - - -
omnithread34_vc8_rtd.dll!omni_condition::timedwait(unsigned long 
abs_sec=1294654111, unsigned long abs_nsec=322000000)  Line 230 + 0x18 
bytes	C++
omniORB414_vc8_rtd.dll!omni::Scavenger::execute()  Line 736 + 0x16 bytes	C++
omniORB414_vc8_rtd.dll!omniAsyncWorker::real_run()  Line 236 + 0x14 
bytes	C++
omniORB414_vc8_rtd.dll!omniAsyncWorkerInfo::run()  Line 285	C++
omniORB414_vc8_rtd.dll!omniAsyncWorker::run(void * __formal=0x00000000) 
  Line 164	C++
omnithread34_vc8_rtd.dll!omni_thread_wrapper(void * ptr=0x0a0f6df0) 
Line 502 + 0x16 bytes	C++
msvcr80d.dll!_callthreadstartex()  Line 348 + 0xf bytes	C
- - - - -

#	456	_threadstartex	omni_condition::timedwait	Normal	0
- - - - -
omnithread34_vc8_rtd.dll!omni_condition::timedwait(unsigned long 
abs_sec=1294654115, unsigned long abs_nsec=760000000)  Line 230 + 0x18 
bytes	C++
omniORB414_vc8_rtd.dll!omniAsyncWorker::real_run()  Line 197 + 0x16 
bytes	C++
omniORB414_vc8_rtd.dll!omniAsyncWorkerInfo::run()  Line 285	C++
omniORB414_vc8_rtd.dll!omniAsyncWorker::run(void * __formal=0x00000000) 
  Line 164	C++
omnithread34_vc8_rtd.dll!omni_thread_wrapper(void * ptr=0x0b62c468) 
Line 502 + 0x16 bytes	C++
msvcr80d.dll!_callthreadstartex()  Line 348 + 0xf bytes	C
- - - - -

#	4608	_threadstartex	omni_condition::timedwait	Normal	0
- - - - -
omnithread34_vc8_rtd.dll!omni_condition::timedwait(unsigned long 
abs_sec=1294654115, unsigned long abs_nsec=760000000)  Line 230 + 0x18 
bytes	C++
omniORB414_vc8_rtd.dll!omniAsyncWorker::real_run()  Line 197 + 0x16 
bytes	C++
omniORB414_vc8_rtd.dll!omniAsyncWorkerInfo::run()  Line 285	C++
omniORB414_vc8_rtd.dll!omniAsyncWorker::run(void * __formal=0x00000000) 
  Line 164	C++
omnithread34_vc8_rtd.dll!omni_thread_wrapper(void * ptr=0x0b635d70) 
Line 502 + 0x16 bytes	C++
msvcr80d.dll!_callthreadstartex()  Line 348 + 0xf bytes	C
- - - - -

#	5840	_threadstartex	omni_mutex::lock	Normal	0
- - - - -
omnithread34_vc8_rtd.dll!omni_mutex::lock()  Line 237 + 0x1b bytes	C++
omnithread34_vc8_rtd.dll!omni_mutex_lock::omni_mutex_lock(omni_mutex & 
m={...})  Line 281 + 0x20 bytes	C++
omniORB414_vc8_rtd.dll!omni::omniObjAdapter::adapterInactive()  Line 562 
+ 0x10 bytes	C++
omniORB414_vc8_rtd.dll!omni::omniOrbPOA::do_destroy(bool 
etherealize_objects=true)  Line 2513 + 0xf bytes	C++
omniORB414_vc8_rtd.dll!destroyer_thread_fn(void * args=0x09edfed0) 
Line 4169	C++
omnithread34_vc8_rtd.dll!omni_thread_wrapper(void * ptr=0x0a0f5fc8) 
Line 492 + 0x11 bytes	C++
msvcr80d.dll!_callthreadstartex()  Line 348 + 0xf bytes	C
- - - - -



More information about the omniORB-list mailing list