[omniORB] Python and omniORB thread issues

[Thomas Nugent]

Hello, we are very puzzled by omniORB behaviour as of late with respect
to handling threads in a Python servant.  We are hoping you can shed
some light on our current state of confusion.

Issue:
  Python threads spawned from an omniORB created thread are being
  trashed by the omnipyThreadScavenger (pyThreadCache.cc).

Software Config:
  Python 1.6
  OmniORB 3.0.1

Details/Scenario:
  - omni3.0.1/lib/python/omniORB/__init__.py WorkerThread class has been
    modified as follows for debugging:

    def __init__(self):
        _thr_init(self, name="omniORB")
        self._Thread__started = 1
        self.id = _thr_id()
        _thr_acq()
        if _thr_act.has_key(self.id):
            self.add = 0
        else:
            print "\nOMNI - add worker thread not in threading._active(%d)" % \
                  self.id
            self.add = 1
            _thr_act[self.id] = self
        _thr_rel()
        if self.add:
            for hook in self.hooks:
                hook(WTHREAD_CREATED, self)

  - A client invokes a method on a Python implemented server object.
  - The Python implemented server object (thread ID 176) executes the
    following code:
        import thread
        thread.start_new_thread(self._tracePolling, ())
  - The spawned thread uses:
        import thread
        print "DataSource - CtHWIC polling in thread %d" % thread.get_ident()
    to print out:
        DataSource - CtHWIC polling in thread 557
  - omniORB creates a Python state for the new thread e.g.
        OMNI - add worker thread not in threading._active(557)
        omniORB: Creating new Python state for thread id 557
  - After 'n' seconds (greater than omnipyThreadCache::scanPeriod
    apparently), omnipyThreadScavenger deletes the Python thread state
    for thread 557 as evidenced by these debug printouts:
        omniORB: Scanning Python thread states.
        omniORB: Deleting Python state for thread id 557
  - The Python thread 557 is not done and continues executing.  At 
    a given point, thread 557 attempts to release a thread RLock.
    This causes an assertion exception to be raised from _RLock.release
    in Python's threading.py module e.g. 

      File "..\win\python-1.6\lib\threading.py", line 100, in release 
        assert self.__owner is me, "release() of un-acquire()d lock"
      AssertionError: release() of un-acquire()d lock

  - Investigation and debug printing showed that the WorkerThread class
    delete method in omni3.0.1/lib/python/omniORB/__init__.py removed
    the thread entry for thread 557 from Python's threading._active 
    dictionary before thread 557 had a chance to complete.

Questions:
  1.) Why is omniORB creating a Python state for a thread that was not
      created via an ORB call?  
  2.) Why is omniORB adding an entry to Python's threading._active
      dictionary when the thread was not created by omniORB?
  3.) Why is there a fixed 30 second scavenger scan interval? e.g.

      unsigned int   omnipyThreadCache::scanPeriod = 30;
    
      Since our polling thread may or may not be finished in 60 seconds,
      we are having unexpected failures.

Cheers

Tom Nugent
Applied Microsystems Corp.