<html><head></head><body><div>Hello Rahul,</div><div><br></div><div>No, I didn't have enough time to look into this issue, </div><div>I hope to review it soon. </div><div><br></div><div>Best Regards,</div><div><br></div><div><br></div><blockquote type="cite"><pre>Hello,
Were you able to reproduce any of the issues I had raised recently?
I would like to know if I am using the noPoll library in the wrong fashion.
Regards,
Rahul
-----Original Message-----
From: Kale, Rahul
Sent: Tuesday, June 14, 2016 9:51 AM
To: <a href="mailto:nopoll@lists.aspl.es">nopoll@lists.aspl.es</a>
Cc: <a href="mailto:francis.brosnan@aspl.es">francis.brosnan@aspl.es</a>
Subject: RE: Stress Test -- Final Flag after calling nopoll_conn_get_msg
Hello,
Any update on this issue yet?
On second thoughts, I believe that noPoll library should also attempt to consolidate multiple fragments into a complete message when
nopoll_conn_get_msg() is used.
A typical use case for websockets is to send JSON or XML text messages back and forth. Forcing the user to worry about coding up stitching together potentially fragmented messages is an unnecessary burden. Websockets specifically provides a way to have a message based API and indicate end of message using final bit.
The noPoll library already provides a stream based API. The user should be able to pick the appropriate API (message based vs. stream based) depending on the use case. Presumably the user of the library is fully aware about the nature of messages transferred and the expected size of the messages. To prevent blow up of the memory, the library should expose configurable parameters for maximum size of each message and total size of fragmented messages. Once these are exceeded, it should be OK for the noPoll library to abort the connection. The user can configure these parameters based on domain knowledge and tolerance for memory consumption.
I noticed that other websocket libraries already follow this approach.
For example:
<a href="https://github.com/theturtle32/WebSocket-Node/blob/master/docs/WebSocketClient.md">https://github.com/theturtle32/WebSocket-Node/blob/master/docs/WebSocketClient.md</a>
Best Regards,
Rahul
<blockquote type="cite">
-----Original Message-----
From: Kale, Rahul
Sent: Friday, June 03, 2016 1:22 PM
To: <a href="mailto:nopoll@lists.aspl.es">nopoll@lists.aspl.es</a>
Cc: <a href="mailto:francis.brosnan@aspl.es">francis.brosnan@aspl.es</a>
Subject: RE: Stress Test -- Final Flag after calling
nopoll_conn_get_msg
Hello,
I believe I have a resolution for the incorrect final flag reported on
the message returned by nopoll_conn_get_msg();
First off, I determined that this is a receiver side issue since the
problem was reproducible even with the Node.js version of the server side.
>From the documentation for noPoll:
"In general noPoll tries to consolidate internal frame fragments but
only for some particular functions like (nopoll_conn_get_msg). For example, ..."
Looking at the code for nopoll_conn_get_msg() I believe that this is
not what is being done. When a partial frame (header and part of the
body) is received, it is handed back directly to the caller. It is
made to look like a fragmented message with adjustments made for
is_fragment() and is_final().
Indeed your approach is reasonable. We should NOT attempt to
automatically integrate fragments into a single message. Otherwise
there is a danger of running out of memory. On the wire, a sender is
free to send an arbitrarily long message. Potentially, an infinitely
long stream of fragmented messages can be send with a 'final' flag set
after even gigabytes. I believe currently there is no upper limit set
on the size of a complete message in the Websocket specification.
The adjustments made for is_final() flag for partial frame received
did not seem to be correct. I have attempted to fix this by doing the
following changes. First, I introduced a new 'is_fin' flag in the noPollMsg structure.
I ensure that the original 'has_fin' flag is only initialized from the
first header received. That flag is transfered to the next message
structure created in case the previous one was partial frame. The new
'is_fin' flag is set to false when delivering a partial frame. It is
set to the original 'has_fin' value when the final portion of the
frame is delivered. Finally, the nopoll_msg_is_final() method now returns the new 'is_fin' flag instead of the 'has_fin' flag.
The reasoning is that the original incoming message frame might also
be itself fragmented and the value for the final flag in the last
segment delivered needs to reflect that.
After the above changes, the reported is_final() flag is always
correct for all messages received even under stress.
I do not have a full understanding of the code so you may have a
better approach to fix this. Anyway, below is a unified diff of the changes I made:
diff -u -r -w ../tmp/nopoll-0.4.1.b265/src/nopoll_conn.c ./src/nopoll_conn.c
--- ../tmp/nopoll-0.4.1.b265/src/nopoll_conn.c 2016-05-23
12:01:54.000000000 -0700
+++ ./src/nopoll_conn.c 2016-06-03 12:06:28.502322481 -0700
@@ -2752,7 +2752,8 @@
msg->is_fragment = nopoll_true;
/* get fin bytes */
- msg->has_fin = 1; /* for now final message */
+ msg->has_fin = conn->previous_msg->has_fin;
+ msg->is_fin = nopoll_false;
msg->op_code = 0; /* continuation frame */
/* copy initial mask indication */ @@ -3069,11 +3070,16 @@
/* flag this message as a fragment */
msg->is_fragment = nopoll_true;
+ msg->is_fin = nopoll_false;
/* flag that this message doesn't have FIN = 0 because
* we wasn't able to read it entirely */
/* msg->has_fin = 0; */
+
} /* end if */
+ else {
+ msg->is_fin = msg->has_fin;
+ }
/* flag the message was being a fragment according to previous flag
*/
msg->is_fragment = msg->is_fragment || conn-
<blockquote type="cite">
previous_was_fragment || msg->has_fin == 0;
</blockquote>
diff -u -r -w ../tmp/nopoll-0.4.1.b265/src/nopoll_msg.c ./src/nopoll_msg.c
--- ../tmp/nopoll-0.4.1.b265/src/nopoll_msg.c 2015-11-15
01:57:52.000000000 -0800
+++ ./src/nopoll_msg.c 2016-06-03 11:52:28.102344864 -0700
@@ -180,7 +180,7 @@
if (msg == NULL)
return nopoll_false;
- return msg->has_fin;
+ return msg->is_fin;
}
/**
diff -u -r -w ../tmp/nopoll-0.4.1.b265/src/nopoll_private.h
./src/nopoll_private.h
--- ../tmp/nopoll-0.4.1.b265/src/nopoll_private.h 2016-04-21
01:18:52.000000000 -0700
+++ ./src/nopoll_private.h 2016-06-03 11:25:52.000716385 -0700
@@ -358,6 +358,7 @@
int remain_bytes;
nopoll_bool is_fragment;
+ nopoll_bool is_fin;
int unmask_desp;
};
Regards,
Rahul
Rahul Kale
IP Video Systems
Barco, Inc
1287 Anvilwood Ave
Sunnyvale, CA 94089
Tel +1 408 400 4238
</blockquote>
This message is subject to the following terms and conditions: MAIL DISCLAIMER<<a href="http://www.barco.com/en/maildisclaimer">http://www.barco.com/en/maildisclaimer</a>>
_______________________________________________
noPoll mailing list
<a href="mailto:noPoll@lists.aspl.es">noPoll@lists.aspl.es</a>
<a href="http://lists.aspl.es/cgi-bin/mailman/listinfo/nopoll">http://lists.aspl.es/cgi-bin/mailman/listinfo/nopoll</a>
</pre></blockquote><div><span>-- <br><div>Francis Brosnan Blázquez - ASPL</div><div>91 134 14 22 - 91 134 14 45 - 91 116 07 57</div><div><br></div><div><a href="http://aspl.es">http://aspl.es</a></div><div><a href="http://asplhosting.com">http://asplhosting.com</a></div><div><a href="http://twitter.com/aspl_es">http://twitter.com/aspl_es</a></div><div><a href="http://twitter.com/asplhosting">http://twitter.com/asplhosting</a></div><div><br></div><div>AVISO LEGAL</div><div><br></div><div>Este mensaje se dirige exclusivamente a su destinatario. Los datos</div><div>incluidos en el presente correo son confidenciales y sometidos a secreto</div><div>profesional, se prohíbe divulgarlos, en virtud de las leyes vigentes. Si</div><div>usted no lo es y lo ha recibido por error o tiene conocimiento del mismo</div><div>por cualquier motivo, le rogamos que nos lo comunique por este medio y</div><div>proceda a destruirlo o borrarlo.</div><div><br></div><div>En virtud de lo dispuesto en la Ley Orgánica 15/1999, de 13 de</div><div>diciembre, de Protección de Datos de Carácter Personal, le informamos de</div><div>que sus datos de carácter personal, recogidos de fuentes accesibles al</div><div>público o datos que usted nos ha facilitado previamente, proceden de</div><div>bases de datos propiedad de Advanced Software Production Line, S.L.</div><div>(ASPL). No obstante, usted puede ejercitar sus derechos de acceso,</div><div>rectificación, cancelación y oposición dispuestos en la mencionada Ley</div><div>Orgánica, notificándolo por escrito a:</div><div>ASPL - Protección Datos, C/Antonio Suárez 10 A-102, 28802, Alcalá de</div><div>Henares (Madrid).</div><div><br></div></span></div></body></html>