[Vortex] Anybody have binding of vortex into Lua?

[Sam Roberts]

On Wed, Oct 04, 2006 at 02:11:34PM +0200, Francis Brosnan Blazquez wrote:
> To ilustrate this, attached goes a BEEP echo profile server
> implementation entirely written without using callbacks (actually it
> uses callbacks, but they won't get inside the ruby runtime). The example
> shows how a function like "queue_reply" could be used to make vortex to
> work as a loop, even at the server side.

Thanks.

I will look at glib's queue, but I would need a way to do a "wait"
on multiple socket descriptors and the glib async queue in a single
select()/poll(). I guess this can be simulated by the callback pushing
a byte into the write end of a pipe after putting something in the
queue, and the select() blocking on the read end (using "readable" to
mean "try to read something from the asynch queue).

> > Give the above, what approach would you suggest?
> 
> Uhmmn..So, if I understand, the problem is reduced to make the sockets
> handled by vortex to not be watched by its own select(2) code but by
> the ruby one (replace ruby by the preferred event-loop based
> environment). 
>
> Then, when ruby select(2) handling wakeups, it gives the control to
> the vortex-ruby binding, making a call to the vortex API to notify
> that something happend on a particular socket that must be handled
> (for the example those tasks that must be done by the vortex reader
> process).
> 
> Supposing this, this will require to update vortex reader internals to
> support current configuration plus a way to activate vortex reader
> process handling on a socket that was handled by external select(2)
> code. 
> 
> Would this work Sam?

Yes, I think this would work very well.

One clarification, though. By "reader process" I hope you do not mean
a seperate thread will get woken up.

I think threads are useful for a number of applications, and that
whether threads should be used at all is something of a religious
debate. I don't want to get into that discussion :-),  but I'm
integrating the example "echo" client into a Qt app, and its a bit
painful. I have to do pretty much what you do in the example you gave,
except your code makes it look really easy!

It becomes more work when you are interested in all the events
(on_connect, on_close, on_framereceived), have to build custom QtEvent
objects that encapsulate the payload and can be posted to the main Qt
application loop, and need specific event handlers for each custom event
so the handlers can effect the UI (Qt is basically single-threaded - it
allows threads, but only the main thread can interact with any classes
they provide in their library).

I got much of this done last weekend, but there are already Qt events
for IO readiness, I would prefer to just add a single handler for a
connection fd, and tell vortex when the fd is ready. This would
minimize the amount of glue code required.

> > I don't need a chat application, of course, but it seems a good way
> > to get up to speed on Qt, vortex, and lua. If I can get my lua and Qt
> > apps chatting over vortex, I can throw the code away and do something
> > interesting.
> 
> I see. This approach looks good. The lua binding will be easier than
> the ruby binding, knowing that it performs well with threaded
> libraries.

I may have given the wrong impression about how well this works, it is
"possible" with lua, but I wouldn't say it works "well". :-)

The lua authors are actually biased against the use of threads in
application code. They provide the hooks to rebuild the lua run-time
with mutex's around the VM, but if I built a lua "module" that provided
vortex interfaces and the callbacks did not all occur in the single main
thread, it wouldn't work with any standard lua run-time.

I would like to integrate Vortex into lua in a way that inter-works with
other lua libraries, like LuaSocket, this would require a completely
single-threaded model, where Vortex would be driven by an explicit
"vortex handle i/o" call when the main lua loop wakes up from it's
select.

A lua-ish interface would combine elements of LuaSocket (non-blocking
i/o support) and the lua Expat wrappers (expat is sax-like, so the lua
wrappers are callback based). Pseudo-lua usage would be:

v_table = {
	on_create = function (channel, ....)
	   do stuff...
	end,
	on_receive = function (channel, frame, ....)
	   print(frame:data())
	end,
	...
}

v = vortex.new(... profiles supported ...)

v:set_up_various_stuff(...)

l = v.connection_new(host, port, ...)

s = socket.tcp(...)

f = socket.fd(0) -- wrap fd 0 to be selectable

selectable = { l, s, f }

readable, writable = socket.select(selectable, selectable)

for _ in pairs(readable) do
	if _ == l then
		l:process() -- will call functions in v_table
	end
	if _ == s then
		-- handle i/o on the socket
	end
	if _ == f then
		-- handle i/o on fd 0
	end
end


I don't know how much work this is, I can see it being a lot for you!
Still, I think that it would be useful for things other than binding
into lua and ruby. There are lots of people who like to use threads
in networking code, but there are lots that don't. You've got good docs,
well thought-out APIs, good threading support, I think (thread pools,
etc.). You could completely corner the open-source BEEP market if you
added a non-threaded API usage pattern!

Cheers,
Sam

Co-routines, and how to use select:

http://www.lua.org/pil/9.4.html

LuaSocket:

http://www.cs.princeton.edu/~diego/professional/luasocket/socket.html#select

Expat:

http://www.lua.org/pil/29.2.html