[noPoll] NOPOLL: API update to support SSLv3 and TLSv1 at the same time -- Re: 5th step: "wss://"
Francis Brosnan Blázquez
francis en aspl.es
Jue Oct 16 17:19:18 CEST 2014
El mar, 14-10-2014 a las 16:53 +0200, Alexander Eisenhuth escribió:
> Hi Francis,
Hi Alexander,
> thanks a lot for your detailed mail. Finally I got it working. It was a
> failure of mine with the certificates. (used the private key of CA
> instead the private key of the Request)
Perfect,
> One question about the root CA. You also have the possibility to support
> the root CA in:
>
> nopoll_listener_set_certificate(_listener, _tlsCertificate, _privateKey,
> _caFile)
Ok, there's still no support for that. I'm checking this to add support
to configure chain certificate...I'll let you know when it is done.
> I suppose the need is to check weather _tlsCertificate and _privateKey
> are derived from _caFile. Right?
It is more than that. In some cases, signed certificate may require
chain certificate to validate the trust path. If this chain certificate
is present at the web browser (or ssl client) there is no
problem...however, in the case it is not, the chain is needed for
successful validation.
> In my implementation now I only use TLSv1.0
Ok,
> _listener = nopoll_listener_tls_new(_context, "0.0.0.0",
> std::to_string(websocketPort).c_str());
>
> and not nopoll_listener_tls_new_opts (ctx, opts, "0.0.0.0", "1236")
Ok,
> because TLSv1.0 seems the most common protocol. (and is also mentioned
> in rfc6455. Right?)
That's right.
> If I want to support SSLv23 or SSLv3 I must use a own listener (and
> port). Right?
That's right. You need to use a different port to ensure the right TLS
protocol is activated according to the connecting client.
> What is the advantage of SSLv23 or SSLv3? Do you have knowledge about that?
Ok, what I find about SSLv23 option (which means supporting all SSL
protocols available during the handshake) is just that, a quick path to
support various scenarios at the same time, including SSLv3, TLSv1.0 and
TLSv1.1 (when available) and SSLv2 (when not disable because it is known
to be broken).
However, as defined in RFC, it is recommended to use TLSv1.0 version.
This setups a scenario where there will be no potential client trying to
use previous versions.
In essence, SSLv3 was used as base protocol to produce the standardized
RFC for TLSv1.0, however the later includes security corrections,
extension options and clarifications that aren't available on SSLv3...
...and at the same time, no one should be using other SSL protocol than
TLSv1.0 (and possible TLSv1.1) in the WebSocket context.
> One thing I also mentioned is in my implementation of wss with nopoll,
> the client connection sometimes fail (ssl.SSLError: [Errno 1]
> _ssl.c:392: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version
> number or SSL23_GET_SERVER_HELLO:unknown protocol), but when I'm using
> nopoll-regression-listener I don't observe that failure.
Ok,
> So it seems to
> me that I miss some code. Do you have any hint? (It might happen the
> first time I execute my testServer.py script or after a couple of times
> canceling and restarting it)
I'm not sure about this. Technically that error refers to wrong SSL
usage (protocol version mismatch) but it also may be a implementation
bug...
The interesting thing here is that we have noPoll running in thousand of
servers ...all of them continuously connected and sending data with a
permanent connection that lasts for months connected.
In the same direction, it seems that the client implementation should be
updated to use TLSv1.0 (and to avoid any other SSL
protocol/configuration). Try this to see if this fixes the issue.
> So far so good. Now I can do some testing about resources.
Perfect, let me know your progress,
I'm going to work on the chain SSL extension for the API so you can
configure this too,
Best Regards,
> Regards
> Alexander
>
> Am 09.10.2014 um 20:48 schrieb Francis Brosnan Blázquez:
> > Hi Alexander,
> >
> > Sorry for the delay. It took me some time to prepare the setup.
> >
> > I was able to use your test case and to have it working with the latest
> > release I sent to you. Here are my notes.
> >
> > 1) I've updated nopoll source code to avoid building TLSv1.1 support if
> > it is not present. However, I think this is not the problem. In any
> > case, please, use the following url to have a fresh installation:
> >
> > http://www.aspl.es/nopoll/preview/stacom/nopoll-0.2.7.b154.tar.gz
> >
> > 2) After installing it, I had to update your ez_setup.py file to include
> > the following flag: "--no-check-certificate" (inside download_file_wget
> > function, around 223 line). Attached file updated.
> >
> > 3) After having everything installed, I've copied your certificate files
> > overwriting the ones that are going to be used by nopoll regression
> > listener. Then I've started the latest version of the
> > nopoll-regression-listener like follows:
> >
> >>> cd nopoll-0.2.7.b154/test
> >>> cp cert.pem test-certificate.crt
> >>> cp key.pem test-private.key
> >>> ./nopoll-regression-listener
> >
> > ...this will start the following ports:
> >
> > - 1235 : TLSv1.0
> > - 1236 : support SSL3 and TLSv1.0
> > - 1237 : support SSL3 only
> > - 1238 : if TLSv1.1 is available
> >
> > 4) After this, I've updated your testServer.py to connect to the right
> > port like follows:
> >
> > ws = KsaClient ('wss://localhost:1236', protocols=['http-only', 'chat'], ssl_options={"certfile":cert, "keyfile":key})
> >
> > 5) After changing it to use, I run your testServer.py and I get what I
> > think it is the expected output:
> >
> > >> python ./testServer.py
> > websocket received: {"start":"ElevatorCarStatus"}
> >
> > ...and if I close the listener (without closing the python client first,
> > I get):
> >
> > websocket closed code: 1006 Going away
> >
> > I've tried to make it working with TLSv1.0 or SSL3 only or even TLSv1.1
> > but it does not work in any case more than 1236 port, giving the
> > following error:
> >
> > ssl.SSLError: [Errno 8] _ssl.c:502: EOF occurred in violation of protocol
> >
> > After, I've updated the library to provide some debug and I've found the
> > following:
> >
> > (proc 2388): (critical) nopoll_conn.c:2110 (nopoll_conn_get_msg) there was an error while accepting TLS connection
> > (proc 2388): (critical) nopoll_conn.c:320 (nopoll_conn_log_ssl) tls stack: error:1408F10B:lib(20):func(143):reason(267) (find reason(code) at openssl/ssl.h)
> > (proc 2388): (critical) nopoll_conn.c:335 (nopoll_conn_log_ssl) details, error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
> > (proc 2388): (critical) nopoll_conn.c:340 (nopoll_conn_log_ssl) noPoll id=6, socket: 7 (after testing errno: 0)
> >
> > I've being digging about this error but there is nothing clear apart
> > from the evident error: there is a wrong version number expected in the
> > SSL record. I understand it is a protocol mismatch.
> >
> > Knowing this, please, check again with the latest link (although it
> > should work with rev 154). Maybe you are not using the latest revision
> > by mistake? ... and also, make sure you use the following listener
> > initialization to support this python code:
> >
> > /* start listener with sslv23 */
> > printf ("Test: starting listener with TLS (SSLv23) at :1236 (all methods)\n");
> > opts = nopoll_conn_opts_new ();
> > nopoll_conn_opts_set_ssl_protocol (opts, NOPOLL_METHOD_SSLV23);
> > listener2 = nopoll_listener_tls_new_opts (ctx, opts, "0.0.0.0", "1236");
> > if (! nopoll_conn_is_ok (listener2)) {
> > printf ("ERROR: Expected to find proper listener TLS connection status (:1236, SSLv23), but found..\n");
> > return -1;
> > } /* end if */
> >
> > Let me know your progress and thoughts about this,
> > Best Regards,
> >
> >
> >> Hi Francis,
> >>
> >> I've no good news. I've build and installed
> >> www.aspl.es/nopoll/preview/stacom/nopoll-0.2.7.b152.tar.gz. and adapted
> >> the initialization. Now it looks:
> >> [...]
> >> _context = nopoll_ctx_new();
> >> _opts = nopoll_conn_opts_new ();
> >>
> >> nopoll_conn_opts_set_ssl_protocol (_opts, NOPOLL_METHOD_SSLV23);
> >> _listener = nopoll_listener_tls_new_opts(_context, _opts, "0.0.0.0",
> >> std::to_string(websocketPort).c_str());
> >>
> >> if (not nopoll_conn_is_ok (_listener)) {
> >> throw ...;
> >> }
> >> [...]
> >>
> >> And the error on the websocket client:
> >> ssl.SSLError: [Errno 8] _ssl.c:392: EOF occurred in violation of protocol
> >>
> >> A quick look to wireshark showed me no obvious difference to the last
> >> nopoll version. To reproduce my behavior I'll send you my python script,
> >> that I use for the test and again the certificates.
> >>
> >> To run the python script you need a python3 (At least python 3.2).
> >>
> >> python setup (maybe as root)
> >> ============
> >> python3 ez_setup.py (file attachen)
> >> (download and extract
> >> https://github.com/Lawouach/WebSocket-for-Python/archive/master.zip)
> >> cd <extractDir>/ws4py
> >> python3 setup.py install
> >>
> >> To Test
> >> =======
> >> python3 testServer.py
> >>
> >> I hope we getting a step further. If you need any assistance, please let
> >> me know
> >>
> >> As I'm on my own, I'm my manager and the receiver of the NDA.
> >>
> >> Regards
> >> Alexander
> >>
> >> Am 03.10.2014 um 17:54 schrieb Francis Brosnan Blázquez:
> >>> Hi Alexander,
> >>>
> >>> Sorry for the delay. It took me some time to update the noPoll API and
> >>> other internal structures to provide a set of functions that allows to
> >>> configure the SSL method that is accepted by your listener.
> >>>
> >>> At the heart of the problem the issue is located at the TLS method
> >>> provided by default which is currently TLSv1. In this case, only
> >>> connections using that version of the protocol will work (the one stated
> >>> by the RFC).
> >>>
> >>> However, if you have a client talking SSLv3 (which seems to be case), it
> >>> won't work. It is important to note that this will never happen with a
> >>> browser (the major client actor in the WebSocket thing).
> >>>
> >>> Please, use the following update to update your setup:
> >>>
> >>> www.aspl.es/nopoll/preview/stacom/nopoll-0.2.7.b152.tar.gz
> >>>
> >>> Then, at your listener use the following code to make your listener to
> >>> accept both SSLv3 and TLSv1 (and TLSv1.1 which is rather new).
> >>>
> >>> /* declared somewhere at the start of your main function */
> >>> noPollConnOpts * opts;
> >>>
> >>> /* then init a connection option, set SSL method and call
> >>> extended listener creation function */
> >>> opts = nopoll_conn_opts_new ();
> >>> nopoll_conn_opts_set_ssl_protocol (opts, NOPOLL_METHOD_SSLV23);
> >>> listener2 = nopoll_listener_tls_new_opts (ctx, opts, "0.0.0.0", "<port>");
> >>>
> >>> The rest of code remains the same including the way how a listener
> >>> connection is checked.
> >>>
> >>> Please, let me know if this update fixes your issue.
> >>>
> >>> Looking forward your reply,
> >>> Best Regards,
> >>>
> >>> PS: ca and rest of key, crt files deleted including mails received.
> >>> Maybe we should sign an NDA. If you have it, please, let me know it to
> >>> pass it to the manager for review.
> >>>
> >>>> Hi Francis,
> >>>>
> >>>> her are:
> >>>> cakey_opass.pem: private key of CA
> >>>> key-cert.pem: public key and client certificate
> >>>>
> >>>> and if you need:
> >>>> key.pem: public key
> >>>> cert.pem: client certificate
> >>>>
> >>>> The are newly generted, so you have no "real" secrets. But please throw
> >>>> them to trash after we solved the issue.
> >>>>
> >>>> Regards
> >>>> Alexander
> >>>>
> >>>> Am 01.10.2014 um 11:11 schrieb Francis Brosnan Blázquez:
> >>>>>
> >>>>>> Hi Francis,
> >>>>>
> >>>>>
> >>>>> Hi Alexander,
> >>>>>
> >>>>>
> >>>>>> thanks for the feedback. Please let me know if I can assist you. (Maybe I should share my SSL certificates?)
> >>>>>
> >>>>>
> >>>>> That certainly will help. If possible, please, send me certificates you
> >>>>> are use.
> >>>>>
> >>>>> Best Regards,
> >>>>>
> >>>>>
> >>>>>> Regards
> >>>>>> Alexander
> >>>>>>
> >>>>>> ----------------ursprüngliche Nachricht-----------------
> >>>>>> Von: "Francis Brosnan Blázquez" francis en aspl.es
> >>>>>> An: "Alexander Eisenhuth" ae en stacom-software.de
> >>>>>> Kopie: nopoll-tech-support en aspl.es
> >>>>>> Datum: Tue, 30 Sep 2014 13:17:43 +0200
> >>>>>> -------------------------------------------------
> >>>>>>
> >>>>>>
> >>>>>>> El vie, 26-09-2014 a las 17:18 +0200, Alexander Eisenhuth escribió:
> >>>>>>>> Hi Francis,
> >>>>>>>
> >>>>>>>
> >>>>>>> Hi Alexander,
> >>>>>>>
> >>>>>>>
> >>>>>>>> (while writing I got the mail from you ...)
> >>>>>>>>
> >>>>>>>> I got a little step further as I managed to remove the pass phrase from
> >>>>>>>> the private key, the (nopoll) socket server is using. Now the system
> >>>>>>>> behaves like that:
> >>>>>>>>
> >>>>>>>> client (python) side:
> >>>>>>>> =====================
> >>>>>>>>> ssl.SSLError: [Errno 8] _ssl.c:392: EOF occurred in violation of
> >>>>>>>> protocol
> >>>>>>>>
> >>>>>>>> server side:
> >>>>>>>> ============
> >>>>>>>> (no connection could be established)
> >>>>>>>>
> >>>>>>>> I used wireshark to analyse. I recognized, that the connection is closed
> >>>>>>>> by the server, before a TLS handshake takes place. I attached the
> >>>>>>>> wireshark dump. (Hopefully you can figure out something)
> >>>>>>>
> >>>>>>>
> >>>>>>> I'm still working on this. I'll let you know when it is fixed or shorted
> >>>>>>> out. Maybe there is an issue while working with ssl3 and tls1.
> >>>>>>>
> >>>>>>> I'll keep you updated,
> >>>>>>> Best Regards,
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> Regards
> >>>>>>>> Alexander
> >>>>>>>>
> >>>>>>>> Am 25.09.2014 um 17:10 schrieb Alexander Eisenhuth:
> >>>>>>>>> Hi Francis,
> >>>>>>>>>
> >>>>>>>>> I didn't manage it to setup a "wss:/" connection. What ssl variant
> >>>>>>>>> implements nopoll (PROTOCOL_SSLv2, PROTOCOL_SSLv3, PROTOCOL_SSLv23,
> >>>>>>>>> PROTOCOL_TLSv1)?
> >>>>>>>>>
> >>>>>>>>> Is the ssl variant specified in RFC 6455?
> >>>>>>>>>
> >>>>>>>>> I currently tried to use PROTOCOL_SSLv23 that is the default in my ws4py
> >>>>>>>>> (python) client. The state is that I get the following error on the
> >>>>>>>>> client (python) side:
> >>>>>>>>>> ssl.SSLError: [Errno 1] _ssl.c:392: error:140770FC:SSL
> >>>>>>>>> routines:SSL23_GET_SERVER_HELLO:unknown protocol
> >>>>>>>>>
> >>>>>>>>> and on the server (nopoll) side:
> >>>>>>>>>> Enter PEM pass phrase:
> >>>>>>>>>
> >>>>>>>>> I use a pass phrase when creating the certificate. In my https
> >>>>>>>>> client/server connection this works well (no pass phrase input appears).
> >>>>>>>>>
> >>>>>>>>> Have you any hints for the ssl hell?
> >>>>>>>>>
> >>>>>>>>> Regards
> >>>>>>>>> Alexander
> >>>>>>>>>
> >>>>>>>>> Am 17.09.2014 um 19:45 schrieb Francis Brosnan Blázquez:
> >>>>>>>>>> El mié, 17-09-2014 a las 15:51 +0200, Alexander Eisenhuth escribió:
> >>>>>>>>>>> Hi Francis,
> >>>>>>>>>>
> >>>>>>>>>> Hi Alexander,
> >>>>>>>>>>
> >>>>>>>>>>> we wonder what performance (latancy, cpu load) overhead we are facing,
> >>>>>>>>>>> if we use wss:/ instead of ws:/. Mabe you can share your experience?
> >>>>>>>>>>
> >>>>>>>>>> Ok, here are different performance measurements according to the cpu,
> >>>>>>>>>> the libc6 and other crypto options associated to your current setup,
> >>>>>>>>>> however the the overhead between a single connection without TLS and one
> >>>>>>>>>> running TLS is about 25 to 50 times (cpu overhead) and around 3 times
> >>>>>>>>>> for memory usage.
> >>>>>>>>>>
> >>>>>>>>>> The best thing here is to test that are the actual values for for your
> >>>>>>>>>> setup taking a look at the memory used every time a connection is
> >>>>>>>>>> accepted...but certainly, TLS isn't something "soft"...
> >>>>>>>>>>
> >>>>>>>>>>> What nopoll API I have to use?
> >>>>>>>>>>
> >>>>>>>>>> Ok, here the the listener, use have to use the following functions:
> >>>>>>>>>>
> >>>>>>>>>> - nopoll_listener_tls_new to star a listener accepting connection with
> >>>>>>>>>> TLS by default.
> >>>>>>>>>>
> >>>>>>>>>> - nopoll_listener_set_certificate to setup certificates to be used by
> >>>>>>>>>> the provided context (noPollCtx)
> >>>>>>>>>>
> >>>>>>>>>> Take a look at the following link, inside the main () function for a
> >>>>>>>>>> complete working example using those functions:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>> https://dolphin.aspl.es/svn/publico/nopoll/trunk/test/nopoll-regre
> >>>>>>>> ssion-listener.c
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> (As I have already https for the web
> >>>>>>>>>>> server to receive the request, I already have ssl certificates :-) )
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Yes, https certificates used by apache2 works "as is" for noPoll. As
> >>>>>>>>>> long as you use the server name that matches the certificate everything
> >>>>>>>>>> will work. But remember those certificates must be signed accepted by
> >>>>>>>>>> your browser.
> >>>>>>>>>>
> >>>>>>>>>> If you are working with self signed certificates you must add the
> >>>>>>>>>> exception in your browser to skip certificate verification
> >>>>>>>>>> part...otherwise any wss:// connection will fail.
> >>>>>>>>>>
> >>>>>>>>>> Best Regards,
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> Regards
> >>>>>>>>>>> Alexander
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Francis Brosnan Blázquez francis.brosnan en aspl.es
> >>>>>>> ASPL
> >>>>>>> 91 134 14 22 - 91 134 14 45 - 91 116 07 57
> >>>>>>>
> >>>>>>> AVISO LEGAL
> >>>>>>>
> >>>>>>> Este mensaje se dirige exclusivamente a su destinatario. Los datos
> >>>>>>> incluidos en el presente correo son confidenciales y sometidos a secreto
> >>>>>>> profesional, se prohíbe divulgarlos, en virtud de las leyes vigentes. Si
> >>>>>>> usted no lo es y lo ha recibido por error o tiene conocimiento del mismo
> >>>>>>> por cualquier motivo, le rogamos que nos lo comunique por este medio y
> >>>>>>> proceda a destruirlo o borrarlo.
> >>>>>>>
> >>>>>>> En virtud de lo dispuesto en la Ley Orgánica 15/1999, de 13 de
> >>>>>>> diciembre, de Protección de Datos de Carácter Personal, le informamos de
> >>>>>>> que sus datos de carácter personal, recogidos de fuentes accesibles al
> >>>>>>> público o datos que usted nos ha facilitado previamente, proceden de
> >>>>>>> bases de datos propiedad de Advanced Software Production Line, S.L.
> >>>>>>> (ASPL). No obstante, usted puede ejercitar sus derechos de acceso,
> >>>>>>> rectificación, cancelación y oposición dispuestos en la mencionada Ley
> >>>>>>> Orgánica, notificándolo por escrito a:
> >>>>>>> ASPL - Protección Datos, C/Antonio Suárez 10 A-102, 28802, Alcalá de
> >>>>>>> Henares (Madrid).
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
--
Francis Brosnan Blázquez <francis.brosnan en aspl.es>
ASPL
91 134 14 22 - 91 134 14 45 - 91 116 07 57
AVISO LEGAL
Este mensaje se dirige exclusivamente a su destinatario. Los datos
incluidos en el presente correo son confidenciales y sometidos a secreto
profesional, se prohíbe divulgarlos, en virtud de las leyes vigentes. Si
usted no lo es y lo ha recibido por error o tiene conocimiento del mismo
por cualquier motivo, le rogamos que nos lo comunique por este medio y
proceda a destruirlo o borrarlo.
En virtud de lo dispuesto en la Ley Orgánica 15/1999, de 13 de
diciembre, de Protección de Datos de Carácter Personal, le informamos de
que sus datos de carácter personal, recogidos de fuentes accesibles al
público o datos que usted nos ha facilitado previamente, proceden de
bases de datos propiedad de Advanced Software Production Line, S.L.
(ASPL). No obstante, usted puede ejercitar sus derechos de acceso,
rectificación, cancelación y oposición dispuestos en la mencionada Ley
Orgánica, notificándolo por escrito a:
ASPL - Protección Datos, C/Antonio Suárez 10 A-102, 28802, Alcalá de
Henares (Madrid).
Más información sobre la lista de distribución noPoll