[Vortex] BUG: Split greetings message aborts connection

[Jens Alfke]

On 24 Apr '08, at 8:32 AM, Francis Brosnan Blazquez wrote:

> This requires more digging at the problem since reducing the frame
> splitting size down to 1k (I've checked several sizes) shows really  
> poor
> performance.

Hm. Well, let's think about why that might happen. I can think of four  
reasons:

* Stream bandwidth? Seems unlikely, as the overhead per frame is only  
~20 bytes for the "MSG..." or "RPY..." line. Using 1k frames instead  
of 4k only adds three of these headers per 4k of payload, about 1%  
overhead.

* Inefficient use of packets? Does writing 1k at a time to the  
socket's file descriptor instead of 4k affect the way the TCP stack  
sends packets out? I don't have enough expertise with the guts of TCP  
to know. I know that normally the socket will buffer together multiple  
writes instead of sending packets after each one, but maybe Vortex is  
forcing a flush after each frame? (Packet tracing, a la tcpdump, would  
show this, but that's a black art I've never learned.)

* CPU overhead? It definitely takes more CPU time to split up and  
reassemble messages from smaller frames. With today's processor speeds  
this ought to be pretty negligible, but maybe there's an inefficient  
algorithm somewhere that's turning into a bottleneck. (Profiling /  
sampling would show if there's any problem here. I haven't noticed any  
excessive CPU usage, but then I've mostly been testing with low  
bandwidths.)

* Thread scheduling latency? The way I implemented my patch, every  
time the sequencer chews another 1k off a larger message it puts the  
remainder back in the channel's queue, signals a wake-up, and goes  
back to wait. That was the simplest way to do it, but perhaps too much  
time's being lost while the sequencer thread goes into and out of  
waiting on its queue.

That said, my personal bias is to maxing out the throughput over a  
slow line, not a fast one, as that's the domain my app is going to be  
in. So I can't justify taking my own time to research Vortex  
performance over high-speed connections, beyond giving armchair advice  
like this ;-) Fortunately my patches are pretty small, and it should  
be easy to configure the 1k-frame one to be optional and off by default.

—Jens
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1875 bytes
Desc: not available
Url : http://lists.aspl.es/pipermail/vortex/attachments/20080424/d41ec8c3/attachment.bin