[Vortex] Using vortex for IPC

[Francis Brosnan Blazquez]

> Hello Francis!

Hi Mikhail!

> Thanks a lot for your response! It was very helpful!
>
> > > I'm trying to use vortex for exchanging large amount of data
> between
> > > two processes on the same computer. I have found 2 problems.
> > >
> > >      1. Very low transfer speed (about 20-30 Mb/sec)
> > >
> > > After some changes in vortex I have speed about  100 Mb/sec, and
> it
> > > looks much better than 30 Mb/sec.
> > 
> > Great job Mikhail. At this point, which are the changes introduced?
> You
> > can post and describe them?
> 
> Looks like I was using vortex in wrong way. I was trying to extend
> window size, but vortex fails because of small buffer in VortexCtx
> (sequencer_send_buffer). That's why I made changes in vortex to make
> default buffer size much bigger - I replace values like 4096 and 4095
> in source with bigger values. I know it was wrong, but it was a way to
> try to speed up vortex. Now I clearly understand that I need to do to
> make vortex working faster. May be my experience helps someone else:
> 
> 1. I created frame size handler function:
> 
> int my_frame_size_maker (VortexChannel *channel, unsigned int
> next_seq_no, int message_size, unsigned int max_seq_no, axlPointer
> user_data) 
> {
>         int res = VORTEX_MIN (PTR_TO_INT(user_data), 
>                     VORTEX_MIN (message_size, max_seq_no - next_seq_no
> + 1));
>         return res;
> };
> 
> Main idea of this function is to tell frame size as big as possible.

Ok.

> 2. In listener and initiator I added these calls after I have
> connection created:
> 
> vortex_connection_set_next_frame_size_handler (connection,
> my_frame_size_maker,                                                                                                         INT_TO_PTR(BIG_BUFFER_SIZE));
> vortex_channel_set_window_size(channel, BIG_BUFFER_SIZE); // set
> bigger window size
> 
> After some tests I've got that the best buffer size is the value near
> #define BIG_BUFFER_SIZE 61440
> I made this value less than 64k to eliminate possibility of creating
> frame bigger than MAX_BUFFER_SIZE (64k).

Ok.

So, with this changes, you reach an interprocess transfer rate of
100MB/sec? If so, that's really great!

I'll place this comments into the documentation. Thanks for your
contribution.
 
> > > - sequencer_send_buffer and sequencer_send_buffer_size from
> VortexCtx
> > > inits only once in vortex_sequencer_run function and after
> extending
> > > channel window size sequencer_send_buffer seems to be too small.
> > 
> > This is not a bug. Take a look on
> vortex_sequencer_build_packet_to_send
> > [2]. It dynamically increase the buffer size according to the value
> > returned by vortex_channel_get_next_frame_size, which is,
> configurable
> > via setting a handler that can return your desired value by
> > vortex_connection_set_next_frame_size_handler.
> > 
> 
> Looks like there is a bug in vortex_sequencer_build_packet_to_send
> buffer increasing algorithm, because it only doubles buffer. In my
> case I want buffer greater or equal 60k, but if we have initial value
> 4k, doubled value will be only 8k. I changed 
> 
> /* check if we have to realloc buffer */
> if (size_to_copy + 100 > ctx->sequencer_send_buffer_size) {
>         /* update buffer size */
>         ctx->sequencer_send_buffer_size =
> (ctx->sequencer_send_buffer_size - 100) * 2 + 100;
>         /* free current buffer */
>         axl_free (ctx->sequencer_send_buffer);
>         /* allocate new buffer */
>         ctx->sequencer_send_buffer      = axl_new (char,
> ctx->sequencer_send_buffer_size);
> } /* end if */
> 
> to 
> 
> /* check if we have to realloc buffer */
> while (size_to_copy + 100 > ctx->sequencer_send_buffer_size) {
>         /* update buffer size */
>         ctx->sequencer_send_buffer_size =
> (ctx->sequencer_send_buffer_size - 100) * 2 + 100;
>         /* free current buffer */
>         axl_free (ctx->sequencer_send_buffer);
>         /* allocate new buffer */
>         ctx->sequencer_send_buffer      = axl_new (char,
> ctx->sequencer_send_buffer_size);
> } /* end if */
> 
> and now it work perfectly!

Nice, you hit the bug. I'll apply the change (only reply if by while).

> > > Have anyone some ideas how to fix vortex for extending window
> size?
> > >
> > >      2. Transfer limit is 2Gb.
> > >
> > > As I understand, the problem is that vortex uses variables of
> "int"
> > > type instead of "unsigned int" for saving sequence numbers.  Then
> the
> > > transfer size becomes more that 2Gb, vortex tries to increase
> > sequence
> > > number and it becomes greater than INT_MAX, so  we get sequence
> > number
> > > value near -INT_MAX. After that vortex closes connection.
> > 
> > You are right, this is a bug. Reading the RFC it is clear that seq
> no
> > values must use unsigned int (2^32 - 1). I'll take a look on this.
> > 
> > > To fix this problem I made a lot of changes in vortex sources
> (about
> > > 30 places in 10 files), but I'm not sure that I did all necessary
> > > changes.  If someone want to fix this bug in vortex, I can send my
> > > changes to him.
> > 
> > Sure. You can always post your changes to the mailing list. I'll
> take a
> > look on them ASAP.
> > 
> 
> I think it will be better if I send them to you directly, because
> message will be ~700k, don't think this size applicable to mail
> list :)

Ok, as you wish. Great job Mikhail. Cheers!

> WBR, Mikhail.
-- 
Francis Brosnan Blazquez <francis at aspl.es>
Advanced Software Production Line, S.L.