[omniORB] Python and omniORB thread issues

[Ian Searle]

I just wanted to say thanks for the support.  Not only did you discover
our own folly, but you fixed another problem we were experiencing (we
thought we would tackle one at a time).

-Ian

Duncan Grisby wrote:
> 
> On Friday 6 October, Thomas Nugent wrote:
> 
> >   Python threads spawned from an omniORB created thread are being
> >   trashed by the omnipyThreadScavenger (pyThreadCache.cc).
> 
> The root of your problem is that you are mixing threads created with
> the thread module with primitives from the threading module. You
> should not do this. If you are going to use things from threading, you
> should start all your threads with threading.Thread. I'm not sure it
> can be considered a bug in omniORB that it masks this incorrect use,
> and causes a later error. There is in fact a bug which is related to
> this, but it is not what is affecting you.
> 
> To explain what's going wrong, I need to explain a few things about
> how Python and omniORB manage threads.
> 
> For each thread, Python maintains two separate bits of "thread
> state". One is a C structure called PyThreadState which contains
> various "global" things like the Python exception status, and so on.
> Whenever a particular thread is running Python code, the global
> interpreter lock must be held, and a suitable PyThreadState must be
> installed for that thread.
> 
> The second piece of state is a Python object, derived from
> threading.Thread, created by the threading module. It is used by the
> primitives in threading, such as RLock, to track what each thread is
> doing. The threading module maintains a dictionary, called _active,
> which maps thread ids to these Thread objects. This state only exists
> if a thread is created with the threading module. If you try to use
> one of threading's primitives on a non-threading thread, it creates a
> DummyThread object and prints the "currentThread(): no current thread
> for n" message. The DummyThread object is never deleted, so you get a
> memory leak.
> 
> Whenever omniORB makes a call from C++ into Python code, it has to
> install a suitable PyThreadState struct. Often, it is the case that
> the thread in question was created by omniORB, and therefore has never
> had a PyThreadState. A suitable struct is therefore created. In other
> cases, the call into Python has resulted from an earlier call _from_
> Python, in meaning there is already a suitable PyThreadState.
> However, in some circumstances omniORB has no way of getting hold of
> the original PyThreadState, so it creates another one. As far as
> Python is concerned, a different thread makes the call into Python,
> but that's OK since CORBA doesn't make any guarantees about how
> threads are used. Once omniORB has created a PyThreadState, it caches
> it to save the overhead of creating it again. Cached thread states are
> removed after a minute with no use.
> 
> Now the problematic bit. People expect to be able to use the
> primitives from the threading module within threads created by
> omniORB. Therefore, when omniORB creates a new PyThreadState for a
> thread, it also looks to see if it needs to create a threading.Thread
> object to put in threading._active. In the case of threads created by
> omniORB or by the thread module, it does; in the case of threads
> created by threading, it does not. If a threading.Thread object is
> added to _active, then it is removed again when the cached thread
> state is deleted.
> 
> So, the problem that you have is that your thread is created with the
> thread module, so it has no entry in threading._active. Now you make a
> call into some omniORB function which performs an up-call into Python
> (a call to _this(), for example). omniORB is unable to get hold of the
> existing PyThreadState for the thread, so it creates a new one. It
> discovers that there is no entry in threading._active, so it creates
> one. This would not be a problem if you did not use anything from the
> threading module. However, since you _do_ use threading.RLock, things
> go wrong when the thread state scavenger does its thing. I don't think
> I can do anything to detect this situation, but I'll think about it.
> 
> If you create your thread with threading.Thread, rather than
> thread.start_new_thread(), then there will already be an entry in
> threading._active when you call into omniORB, and everything will be
> fine.
> 
> The related bug is that the thread state would sometimes be deleted
> prematurely for a thread created by omniORB. This would occur if you
> did some call which re-entered Python, then spent more than a minute
> doing other things. It's fixed in CVS.
> 
> Cheers,
> 
> Duncan.
> 
> --
>  -- Duncan Grisby  \  Research Engineer  --
>   -- AT&T Laboratories Cambridge          --
>    -- http://www.uk.research.att.com/~dpg1 --