[noPoll] New user of nopoll seeking clarification

[Tony Mountifield]

I am a new user of nopoll for the last couple of weeks, and am using it to
add a websocket server to an existing threaded C program. The client side
will be a web page, and for testing I am using the Simple-Websocket-Client
add-on for Google Chrome.

I have the server mostly working, but wanted to ask for advice on a few
things.

1. The websocket listener thread creates its listeners in the same context
using nopoll_listener_new() or nopoll_listener_tls_new() as appropriate,
and then does a nopoll_ctx_set_on_open() to register a handler for
incoming connections. It then calls nopoll_loop_wait(ctx, 0); to handle
everything until the program ends.

2. The on_open handler sets up on_ready, on_msg and on_close handlers for
the connection.

3. Because nopoll_loop_wait() is a single thread, but incoming messages
will cause blocking operations such as database lookups before sending any
response messages, the on_open handler also creates a separate thread for
the connection, and associates it with the connection via a data structure
attached using nopoll_conn_set_hook().

4. The on_msg handler takes a reference to the msg, then passes the msg
pointer to the connection thread, and then returns.

5. The connection thread receives the msg, processes it, does any required
data lookups, and then sends any response to the client using
nopoll_conn_send_text() and nopoll_conn_flush_writes (). Finally it unrefs
the received msg.

This all seems to work very well. If any of the above sounds wrong, please
let me know!

The requirement that is puzzling me is this: under some circumstances, the
connection thread needs to initiate a close of the connection, rather than
waiting for the remote client to do so.

Initially I tried calling nopoll_conn_close(), but this resulted in a
connection disappearing while still being used in nopoll_conn_get_msg().

So I changed it to nopoll_conn_shutdown(), which does seem to cause the
connection to closed by the main wait loop. However, I understand that
nopoll_conn_shutdown() is an immediate hard shutdown without doing a
websocket protocol close. So that doesn't feel quite right too.

More recently, I added nopoll_conn_ref() and nopoll_conn_unref() to the
start and end of the connection thread, because I figured it should make a
note of its interest to avoid the connection disappearing.

So when I want the connection thread on the server to initiate a
controlled connection close, should I call nopoll_conn_close() instead of
nopoll_conn_shutdown(), but then avoid doing a nopoll_conn_unref()? It
seems that close includes an implicit unref.

Actually, since writing the paragraph above, I have tried using
nopoll_conn_close() within the connection thread. On my faster CentOS 5
system, it seems to work, but on a slower CentOS 4 system, I get the
on_close handler invoked twice on the same connection, once from the
connection thread and once from the main thread that calls
nopoll_loop_wait(). And I get errors logged too. It seems like calling
nopoll_conn_close() independently of the main wait loop causes a conflict,
because it wants to execute protocol rather than just flag the socket for
shutdown.  So I may have to just stick with nopoll_conn_shutdown() when
the connection thread wants to terminate the connection.

Is there a better way?

Thanks in advance for any hints!

Regards
Tony
-- 
Tony Mountifield
Work: tony at softins.co.uk - http://www.softins.co.uk
Play: tony at mountifield.org - http://tony.mountifield.org