[omniORB] bulk data transfer of files

[Steven W. Brenneis]

Smith, Norman wrote:
> 
> Have a question about sending bulk data transfer via omniORB. I want to send
> large chunks of data between processes, using omniORB as the IPC mechanism.
> I've successfully implemented a test case using the array datatype, but read
> the postings in this forum regarding arrays being marshalled item-by-item,
> and the efficiency of using sequences instead.  However, the sequence data
> type doesn't seem to have any interface for pushing data into it except on
> an item-by-item basis, and it seems rather inefficient to transfer the
> contents of a 64K buffer (i.e. an array) item-by-item into a sequence, then
> pass the sequence to the ORB.
> 
> I've thought about creating a "sequence of array" datatype (i.e.
> sequence<char[65535]>), but I would think the ORB would be smart enough to
> marshall each member of the sequence according to its declared type (in this
> case an array of char's) and I'm back where I started with the array being
> marshalled character-by-character.
> 
> Question is this: what is the best way to perform this bulk transfer? Will
> the above sequence declaration marshall the entire array at once? Should I
> just stick to arrays, or is there some alternative that I'm not seeing?
> 

Norman,

There is a non-portable method of transferring bulk quantities of scalar
data using sequences that is very fast and very efficient.  I have been
using it for a quite a while and it works well.  I'm not sure if this
will be the answer you are looking for, but here goes:

In your IDL:

typedef sequence<octet> buffer;

interface My_Class {
  attribute buffer Buf;
};

The implementation goes something like this:

class My_Class_I : public _sk_My_Class {
public:
  My_Class_I();
  ~My_Class_I();

  void Buf(const buffer&);
  buffer* Buf();

private:
  buffer mybuffer;
};

The mutator is implemented thus:

void My_Class_I::Buf(const buffer& inbuffer) {
  mybuffer.length(inbuffer.length());
  ::memcpy(mybuffer.NP_Data(),inbuffer.NP_data(),inbuffer.length());
}

The accessor can be implemented as either:

buffer* My_Class_I::Buf() {
  return new buffer(mybuffer);
}

or:

buffer* My_Class_I::Buf() {
  buffer* out = new buffer;
  out->length(mybuffer.length());
  ::memcpy(out->NP_data(),mybuffer.NP_data(),mybuffer.length());
  return out;
}

I think on most platforms the performance difference of either method is
nominal.  Since the marshalling operation on this sequence is performed
into a memory buffered stream, the code is very efficient and quick.

Hope this helps.

Steve Brenneis