[omniORB] Lost one-way calls

[Helge Penne]

This is all quite interesting.  If I have a server that sends oneway messages
to a client (subscriber) at a fixed frequnecy (set by the subscriber), what
would I have to do to make this reliable (for all practical purposes)?  (This
question applies to omniORB 2.7.1 only, not to ORBs in general as specified in
the standard...)

Would I have to set disable the "incoming connection shutdown" (scavenger) on
the client (receiver) side by setting idleConnectioScanPeriod to zero, or would
it be enough to set this period an order of magnitude higher than what it is
for outgoing connections on the sender side, as suggested by Nick Reeves in his
message yesterday?

And now for some strong opinions:

Even though the CORBA standard states that oneway calls are not 100% reliable,
my opinion is that failure to deliver a oneway message should occur if the
connection is down for some reason.  Due to the nature of oneway calls, it is
then natural to accept that the sender may not be informed that the
transmission failed.

However, if messages are lost even if the socket connection is perfectly OK
(ORB internal/related issues not included) and both client and server are alive
and well, and the message is actuallu lost due to some more or less rare timing
glitch in the ORB (as seems to be the case here), this can only be regarded as
an ORB bug that should be carefully diagnosed and corrected.

- Helge (no offense intended)

Sai-Lai Lo wrote:

> >>>>> Gary D Duzan writes:
>
> >    Right. I was just scanning the source code looking for the
> > behavior. It is a bit tough to sort through everything, but it appears
> > that you are correct and the scavenger algorithm doesn't take into
> > account dispatch time, so if a call takes longer than the scavenger
> > interval it will shutdown your connection, discarding any oneway
> > messages queued up behind it in the TCP buffer. It seems to me that it
> > would be worth it to add a "dispatching" flag to Strand and have
> > Strand::Sync::WrTimedLock() always set heartbeat to zero if it is set.
> > That way we shouldn't have any prematurely shutdown connections.
>
> Let me fill in a bit more details.
>
> On the server side this is what happens:
>
> The server thread blocks on socket recv(). When it has got a complete
> request and is about to dispatch an upcall, it acquires the WrLock().
> Once the WrLock() is acquired, the scavenger thread won't touch the
> connection. Therefore the connection would not go away while the server
> thread is working inside an object implementation.
>
> It is however possible that the connection may be shutdown by the scavenger
> while the server thread is still receiving parts of a request
> message. This is because at this time the WrLock() is not held.
>
> The reason behind this behaviour is to protect the server from denial of
> service attack. Think of a malicious/buggy client that just send part of a
> request message and stop sending the rest. The server should not wait
> indefinitely for the bits that will never come. The current behaviour is
> that the scavenger will clean up these wedged connections.
>
> I do take your point that the scavenger is actually better to sync the
> timeout with the moment the server thread receives the first byte of a
> request message. However, this would not solve the race condition in
> general because the scavenger may close the connection at the same time as
> the client starts sending bytes down its tcp socket. The client may have
> finish with the send() call before the TCP close status is propagated back
> to the client's OS.
>
> Sai-Lai
>
> --
> Sai-Lai Lo                                   S.Lo@uk.research.att.com
> AT&T Laboratories Cambridge           WWW:   http://www.uk.research.att.com
> 24a Trumpington Street                Tel:   +44 223 343000
> Cambridge CB2 1QA                     Fax:   +44 223 313542
> ENGLAND

--
Helge Penne (helge.penne@datarespons.no)
Senior Software Development Engineer
Data Respons AS, Postboks 489, N-1323 Høvik, Norway
Phone: +47 67112081 / 22719957 (work/home)