<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/3.30.3">
</HEAD>
<BODY>
El jue, 09-01-2014 a las 12:06 +0530, ayush jain escribió:
<BLOCKQUOTE TYPE=CITE>
<PRE>
Hello Francis,
</PRE>
</BLOCKQUOTE>
<PRE>
</PRE>
Hi,<BR>
<BR>
<BLOCKQUOTE TYPE=CITE>
<PRE>
Would be really glad, if you can throw some lights on my 2 more
queries/doubts related to nopoll implementation.
1) In case, we happen to use our existing connection handling
mechanism (socket, bind, listen) with our own I/O mechanism (epoll)
for tcp/tls, let's say we call our own accept(), as mentioned in the
nopoll library manual, if we use nopoll_listener_from_socket(),
nopoll_conn_accept_complete(), followed by nopoll_conn_get_msg() to
take care of websocket handshake and data, would it take care of the
locking across multiple threads provided we initialize the
'nopoll_thread_handlers' ? As we won't use in this case
'nopoll_loop_wait' right?
</PRE>
</BLOCKQUOTE>
<BR>
Yes, it should. For example, we are using this in the context of Vortex Library with has<BR>
an intensive thread usage.<BR>
<BR>
<BLOCKQUOTE TYPE=CITE>
<PRE>
Also, I believe,
nopoll_conn_accept_complete() would only be required in case of TLS
(for ssl related stuff), for tcp I guess there is nothing getting done
as of now.
</PRE>
</BLOCKQUOTE>
<PRE>
</PRE>
Ok, it is required in any situation were you might have a previous protocol that has to <BR>
happen first before WebSocket enters....TLS is a good example of that.<BR>
<BR>
Additionally, if you are not going to use PortSharing, that is, mixing WebSocket and <BR>
YourSpecificProtocolHere, then this function isn't required.<BR>
<BR>
<BLOCKQUOTE TYPE=CITE>
<PRE>
2) As per the manual, we are very much interested in nopoll API's to
receive our protocol message over websocket with websocket preserving
the boundary.
</PRE>
</BLOCKQUOTE>
<BR>
Ok, as a general concept, this is not possible/reliable. As defined by WS standard, any<BR>
intermediary may split or coalesce a WebSocket message...and by intermediary, it <BR>
mean any WS stack, which includes the browser itself. <BR>
<BR>
The direct consequence if this is that you may send message A and then B, but they<BR>
arrive as a single consolidated unit at the server. The same happens in the opposite direction <BR>
(specially in this direction). Maybe it is because it more usual to have the server sending more<BR>
data to the client instead of the opposite.<BR>
<BR>
<BLOCKQUOTE TYPE=CITE>
<PRE>
As in our case, each application message must be carried
within a single WebSocket message, and a WebSocket message would not
contain more than one application message.
</PRE>
</BLOCKQUOTE>
<BR>
No. See section 1.2, at the end of page 4, beginning of 5:<BR>
<A HREF="http://tools.ietf.org/html/rfc6455">http://tools.ietf.org/html/rfc6455</A><BR>
<BR>
On the wire, a message is composed of one or more frames. <BR>
The WebSocket message does not necessarily correspond to a<BR>
particular network layer framing, as a fragmented message may be<BR>
coalesced or split by an intermediary.<BR>
<BR>
Apart from what it says, we have found that there is not a correspondence between what<BR>
the client/server sends and what arrives at the other peer. In general, WS has been designed<BR>
as a stream oriented protocol (despite it has a framing design :-?). <BR>
<BR>
In any case, you must include some short of framing at your application level to <BR>
handle boundaries in a reliable manner. We do it using BEEP on top of noPoll. At the <BR>
web client we use jsVortex [1]. At the server side we use Vortex Library [2] plus Turbulence [3].<BR>
<BR>
[1] <A HREF="http://www.aspl.es/jsVortex">http://www.aspl.es/jsVortex</A><BR>
[2] <A HREF="http://www.aspl.es/vortex">http://www.aspl.es/vortex</A><BR>
[3] <A HREF="http://www.aspl.es/jsVortex">http://www.aspl.es/turbulence</A><BR>
<BR>
<BLOCKQUOTE TYPE=CITE>
<PRE>
Also, in our case,
application protocol will not define the message boundary using some
delimiter or for that matter any header in case its over
websockets/TCP or websockets/TLS (similar to UDP and SCTP). I was of
the opinion that nopoll_conn_get_msg() at server would return 1
complete websocket message (NULL in case message is not completed)
even if the same is coming in various frames.
</PRE>
</BLOCKQUOTE>
<BR>
Ok, this is true if you are using noPoll at the other end. In the case these frames are<BR>
coming from a browser, this is not going to happen. You may receive different frames<BR>
as those that were sent by the peer.<BR>
<BR>
<BLOCKQUOTE TYPE=CITE>
<PRE>
But looking at
nopoll-regression-listener.c, I found out that we are again checking
at application if its a final fragment or not else return, and we keep
joining frames till we get one single complete message. Am little
confused here now.
</PRE>
</BLOCKQUOTE>
<BR>
I think, at this point, you know why that is done...specially because that regression<BR>
listener is used with a web regression client which runs on top of a browser which <BR>
doesn't respect message boundaries as sent by the javascript client.<BR>
<BR>
<BLOCKQUOTE TYPE=CITE>
<PRE>
Would really appreciate your response on the same.
In case we need to join the message, we would have to define per
connection 'previous' buffer if am right else would cause overwrite of
data in single threaded as well as multi-threaded environment in case
of thousands of clients sending data to a single server right?
</PRE>
</BLOCKQUOTE>
<PRE>
</PRE>
That's right. Best Regards,<BR>
<BR>
<BLOCKQUOTE TYPE=CITE>
<PRE>
Thanks & Best Regards,
_Ayush
On 12/31/13, Francis Brosnan Blazquez <<A HREF="mailto:francis@aspl.es">francis@aspl.es</A>> wrote:
> Hi,
>
>> Sure Francis. Would definitely do that. By POC, I meant Proof of
>> Concepts.
>
> Ok, ;-)
>
>> We need websocket support in one of our product wherein we
>> may go ahead and try to integrate noPoll as it tends to suit our
>> requirements as well. I have been going through the same a lot from
>> last 2 weeks. Would definitely let you know in case we happen to
>> finalize the same into our product.
>
> Perfect!
>
>> I also believe, that in case
>> required, we can opt for commercial support for the same (I guess I
>> saw sometime back on ASPL website).
>
> Sure! That would be a good option to provide direct and fast support for
> noPoll integration. You've got more information here:
>
> <A HREF="http://www.aspl.es/nopoll/commercial.html">http://www.aspl.es/nopoll/commercial.html</A>
>
> Contact us when you are ready!
>
>> Please correct me if am wrong
>> here. Last but not the least, would also take this time to wish you a
>> very Happy new year 2014 :) Thanks again!
>
> Same to you! Cheers!
>
>> >
>> >> Regards,
>> >> _Ayush
>> >>
>> >> On 12/26/13, Francis Brosnan Blazquez <<A HREF="mailto:francis@aspl.es">francis@aspl.es</A>> wrote:
>> >> >> Hello Francis,
>> >> >
>> >> > Hi there,
>> >> >
>> >> > Sorry for the delay, been busy ;-)
>> >> >
>> >> >> Thanks a lot for this info. I was also of the impression that
>> >> >> noPoll
>> >> >> provides it internal select,poll and epoll I/O based on some
>> >> >> switch.
>> >> >
>> >> > That's right..
>> >> >
>> >> >> But traversing the code I found out that only 'select' is
>> >> >> implemented
>> >> >> as part of default. Does that mean we need to write our own waiting
>> >> >> mechanism in case we wish to use 'NOPOLL_IO_ENGINE_EPOLL' etc, and
>> >> >> also need to change e.g. 'nopoll_io_get_engine' as it didn't use
>> >> >> 'engine_type' currently.
>> >> >
>> >> > Ok, several things here.
>> >> >
>> >> > You could go and update noPoll to add support for epoll but, if you
>> >> > already have a I/O waiting mechanism or you are planning you have
>> >> > one,
>> >> > do not update noPoll and put your all efforts into your I/O waiting
>> >> > mechnism (using epoll).
>> >> >
>> >> > This way you'll have an independent I/O waiting mechanism that will
>> >> > play
>> >> > well with all next noPoll releases (because you are using its API).
>> >> >
>> >> > Once you have your own I/O waiting mechanism, you take a look at the
>> >> > following link to have each noPoll socket watched:
>> >> >
>> >> > <A HREF="http://www.aspl.es/nopoll/html/nopoll_core_library_manual.html#nopoll_manual_reading_content_from_connection">http://www.aspl.es/nopoll/html/nopoll_core_library_manual.html#nopoll_manual_reading_content_from_connection</A>
>> >> >
>> >> >> So my understanding is that in case we wish to use internal noPoll
>> >> >> I/O, we need to write wrappers and waiting mechanism (would need to
>> >> >> change library functions as well), and in case we need to integrate
>> >> >> to
>> >> >> our existing I/O, we need to go as you have described.
>> >> >
>> >> > Ok, to this particular matter, you only have one waiting I/O in your
>> >> > app. It could be noPoll's or yours but not both at the same time
>> >> > (excuse
>> >> > me if I got it wrong from your words).
>> >> >
>> >> > Because that, I recommend you to have your own I/O waiting poll
>> >> > mechanism (plus all the features you may want to add it)...so you
>> >> > can
>> >> > use it to wait and watch noPoll sockets...plus the fact that it will
>> >> > allow you easy upgrades when noPoll releases come out.
>> >> >
>> >> >> When I tried to test concurrent connections using my client with
>> >> >> noPoll, I could not proceed more than 1050 connections (I guess the
>> >> >> limit due to default 'select' and my kernel limitation for the
>> >> >> same).
>> >> >
>> >> > Ok, that's because you have to update FD_SETSIZE to a bigger value
>> >> > at
>> >> > compilation time....but I wouldn't go that way; epoll is your best
>> >> > bet
>> >> > (at least for linux ;-)
>> >> >
>> >> >> >>> 3) Let's say if we implement websocket server, Can we do the
>> >> >> >>> receive
>> >> >> >>> handling in 1 thread for all the (say max 200000 sockets) and
>> >> >> >>> continue
>> >> >> >>> with send operation using multiple threads. In that case do we
>> >> >> >>> need
>> >> >> >>> any callback to check if the connection is ready?
>> >> >> >>
>> >> >> >> Under previous links you'll find everything about this. In
>> >> >> >> short:
>> >> >> >> you
>> >> >> >> can watch noPoll sockets as regular sockets (in fact they are).
>> >> >> >> Once
>> >> >> >> movement is detected, call appropriate functions to get data
>> >> >> >> received...
>> >> >> >>
>> >> >> >>> And also, would I
>> >> >> >>> need some locking before sending my data from any thread or its
>> >> >> >>> already taken care of in noPoll library?
>> >> >> >>
>> >> >> >> Ok, we've designed the library so it can work in threaded and
>> >> >> >> non-threaded scenarios. In this case you must provide the right
>> >> >> >> functions to let noPoll to create mutexes, destroy, lock and
>> >> >> >> unlock
>> >> >> >> them... with this:
>> >> >> >>
>> >> >> >> <A HREF="http://www.aspl.es/nopoll/html/group__nopoll__support_ga7f2a26a896c1416b23933097a993a435.html#ga7f2a26a896c1416b23933097a993a435">http://www.aspl.es/nopoll/html/group__nopoll__support_ga7f2a26a896c1416b23933097a993a435.html#ga7f2a26a896c1416b23933097a993a435</A>
>> >> >> >>
>> >> >> >> Then noPoll will use these at the right places...
>> >> >> >>
>> >> >> Nice! That's great. So we need to just initialize the global mutex,
>> >> >> and have to insert my lock/unlock etc API's inside those mutex
>> >> >> wrappers during 'nopoll_thread_handlers' and noPoll would take care
>> >> >> of
>> >> >> appropriate locking and synchronization no matter which thread is
>> >> >> accessing the noPoll context at the same time. Please correct me if
>> >> >> my
>> >> >> understanding is wrong here.
>> >> >
>> >> > That's right.
>> >> >
>> >> >> >> Let us to know your progress and any product using noPoll. We
>> >> >> >> will
>> >> >> >> be
>> >> >> >> glad to publish some references about it. BR
>> >> >> >>
>> >> >> :) Sure Francis. Would definitely update once we get a go ahead
>> >> >> with
>> >> >> the websocket library.
>> >> >
>> >> > Great,
>> >> > Best Regards!
>> >> >
>> >> >> >>> Would really appreciate your responses on the same.
>> >> >> >>>
>> >> >> >>> Regards,
>> >> >> >>> _Ayush
>> >> >> >>> _______________________________________________
>> >> >> >>> noPoll mailing list
>> >> >> >>> <A HREF="mailto:noPoll@lists.aspl.es">noPoll@lists.aspl.es</A>
>> >> >> >>> <A HREF="http://lists.aspl.es/cgi-bin/mailman/listinfo/nopoll">http://lists.aspl.es/cgi-bin/mailman/listinfo/nopoll</A>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >
>> >> > --
>> >> > Francis Brosnan Blázquez <<A HREF="mailto:francis.brosnan@aspl.es">francis.brosnan@aspl.es</A>>
>> >> > ASPL
>> >> > 91 134 14 22 - 91 134 14 45 - 91 116 07 57
>> >> >
>> >> > AVISO LEGAL
>> >> >
>> >> > Este mensaje se dirige exclusivamente a su destinatario. Los datos
>> >> > incluidos en el presente correo son confidenciales y sometidos a
>> >> > secreto
>> >> > profesional, se prohíbe divulgarlos, en virtud de las leyes
>> >> > vigentes.
>> >> > Si
>> >> > usted no lo es y lo ha recibido por error o tiene conocimiento del
>> >> > mismo
>> >> > por cualquier motivo, le rogamos que nos lo comunique por este medio
>> >> > y
>> >> > proceda a destruirlo o borrarlo.
>> >> >
>> >> > En virtud de lo dispuesto en la Ley Orgánica 15/1999, de 13 de
>> >> > diciembre, de Protección de Datos de Carácter Personal, le
>> >> > informamos
>> >> > de
>> >> > que sus datos de carácter personal, recogidos de fuentes accesibles
>> >> > al
>> >> > público o datos que usted nos ha facilitado previamente, proceden de
>> >> > bases de datos propiedad de Advanced Software Production Line, S.L.
>> >> > (ASPL). No obstante, usted puede ejercitar sus derechos de acceso,
>> >> > rectificación, cancelación y oposición dispuestos en la mencionada
>> >> > Ley
>> >> > Orgánica, notificándolo por escrito a:
>> >> > ASPL - Protección Datos, C/Antonio Suárez 10 A-102, 28802, Alcalá de
>> >> > Henares (Madrid).
>> >> >
>> >> >
>> >
>> > --
>> > Francis Brosnan Blázquez <<A HREF="mailto:francis.brosnan@aspl.es">francis.brosnan@aspl.es</A>>
>> > ASPL
>> > 91 134 14 22 - 91 134 14 45 - 91 116 07 57
>> >
>> > AVISO LEGAL
>> >
>> > Este mensaje se dirige exclusivamente a su destinatario. Los datos
>> > incluidos en el presente correo son confidenciales y sometidos a
>> > secreto
>> > profesional, se prohíbe divulgarlos, en virtud de las leyes vigentes.
>> > Si
>> > usted no lo es y lo ha recibido por error o tiene conocimiento del
>> > mismo
>> > por cualquier motivo, le rogamos que nos lo comunique por este medio y
>> > proceda a destruirlo o borrarlo.
>> >
>> > En virtud de lo dispuesto en la Ley Orgánica 15/1999, de 13 de
>> > diciembre, de Protección de Datos de Carácter Personal, le informamos
>> > de
>> > que sus datos de carácter personal, recogidos de fuentes accesibles al
>> > público o datos que usted nos ha facilitado previamente, proceden de
>> > bases de datos propiedad de Advanced Software Production Line, S.L.
>> > (ASPL). No obstante, usted puede ejercitar sus derechos de acceso,
>> > rectificación, cancelación y oposición dispuestos en la mencionada Ley
>> > Orgánica, notificándolo por escrito a:
>> > ASPL - Protección Datos, C/Antonio Suárez 10 A-102, 28802, Alcalá de
>> > Henares (Madrid).
>> >
>> >
>
> --
> Francis Brosnan Blázquez <<A HREF="mailto:francis.brosnan@aspl.es">francis.brosnan@aspl.es</A>>
> ASPL
> 91 134 14 22 - 91 134 14 45 - 91 116 07 57
>
> AVISO LEGAL
>
> Este mensaje se dirige exclusivamente a su destinatario. Los datos
> incluidos en el presente correo son confidenciales y sometidos a secreto
> profesional, se prohíbe divulgarlos, en virtud de las leyes vigentes. Si
> usted no lo es y lo ha recibido por error o tiene conocimiento del mismo
> por cualquier motivo, le rogamos que nos lo comunique por este medio y
> proceda a destruirlo o borrarlo.
>
> En virtud de lo dispuesto en la Ley Orgánica 15/1999, de 13 de
> diciembre, de Protección de Datos de Carácter Personal, le informamos de
> que sus datos de carácter personal, recogidos de fuentes accesibles al
> público o datos que usted nos ha facilitado previamente, proceden de
> bases de datos propiedad de Advanced Software Production Line, S.L.
> (ASPL). No obstante, usted puede ejercitar sus derechos de acceso,
> rectificación, cancelación y oposición dispuestos en la mencionada Ley
> Orgánica, notificándolo por escrito a:
> ASPL - Protección Datos, C/Antonio Suárez 10 A-102, 28802, Alcalá de
> Henares (Madrid).
>
>
</PRE>
</BLOCKQUOTE>
<BR>
<TABLE CELLSPACING="0" CELLPADDING="0" WIDTH="100%">
<TR>
<TD>
<PRE>
--
Francis Brosnan Blázquez <<A HREF="mailto:francis.brosnan@aspl.es">francis.brosnan@aspl.es</A>>
ASPL
91 134 14 22 - 91 134 14 45 - 91 116 07 57
AVISO LEGAL
Este mensaje se dirige exclusivamente a su destinatario. Los datos
incluidos en el presente correo son confidenciales y sometidos a secreto
profesional, se prohíbe divulgarlos, en virtud de las leyes vigentes. Si
usted no lo es y lo ha recibido por error o tiene conocimiento del mismo
por cualquier motivo, le rogamos que nos lo comunique por este medio y
proceda a destruirlo o borrarlo.
En virtud de lo dispuesto en la Ley Orgánica 15/1999, de 13 de
diciembre, de Protección de Datos de Carácter Personal, le informamos de
que sus datos de carácter personal, recogidos de fuentes accesibles al
público o datos que usted nos ha facilitado previamente, proceden de
bases de datos propiedad de Advanced Software Production Line, S.L.
(ASPL). No obstante, usted puede ejercitar sus derechos de acceso,
rectificación, cancelación y oposición dispuestos en la mencionada Ley
Orgánica, notificándolo por escrito a:
ASPL - Protección Datos, C/Antonio Suárez 10 A-102, 28802, Alcalá de
Henares (Madrid).
</PRE>
</TD>
</TR>
</TABLE>
</BODY>
</HTML>