[Vortex] feeder-api question
Francis Brosnan Blazquez
francis at aspl.es
Fri Mar 18 16:47:28 CET 2011
> Hello
Hi Martin,
> Thanks for the help. It could be that I don't fully understand how to
> use the API.
>
> One thing that I misunderstood (fixing it does not solve my problem)
> is that in the feed handler, the feed_op PAYLOAD_FEEDER_GET_SIZE is
> implemented in file feeder to always return the entire file size but I
> misunderstood the documentation and returned the number of bytes left
> to feed.
Ok, documentation is now updated with some additional comments.
Currently PAYLOAD_FEEDER_GET_SIZE should return the entire message that
was meant to be sent...however you can play with feeder by "increasing"
dynamically this value (always increasing) if you want to send more
content than the one initially expected...
As conclusion, you can always return the entire message or a bigger
value than the last one reported.
> Inspecting debug output and doing some tests indicates the problem for
> me is that in my original implementation I send ANS frames having size
> of about 16k which on the client side becomes handle-frame events.
> Using the feeder API the vortex_channel_send_ans_rpy_from_feeder
> apparently send ANS frames but message size isn't 4k or 16k or similar
> but the entire file size so on the client side I eventually get a
> handle-frame event but it's so unexpectedly big the client runs into
> trouble.
Ok, two things. A VortexPayloadFeeder represents a single message, so,
triggering a single operation with a feeder results into a single
message delivered to the remote peer (that may or may not be fragmented,
see next), which would be the same results as sending a single ANS with
the entire message.
In the other hand, you might want to check [1] to enable fragmentation
(or disable complete frame delivery) so even if the message is
fragmented, you get all pieces notified at the frame received as they
come.
[1] http://www.aspl.es/fact/files/af-arch/vortex-1.1/html/group__vortex__channel_gf352be87d3fc33fb5cf3eb4607fde99c.html#gf352be87d3fc33fb5cf3eb4607fde99c
> So, perhaps I should call vortex_channel_send_ans_rpy_from_feeder
> repeatedly with the feeder_op PAYLOAD_FEEDER_GET_SIZE returning about
> 16k to obtain handle-frame event?
You can do so, but you'll get poor results and eventually it will become
more work, see [1] to get all pieces as they come.
> If this is so, might I suggest a small improvment to the feeder API,
> that there is a feeder-op PAYLOAD_FEEDER_GET_CHUNK_SIZE that makes the
> feeder API send the entire file as a sequence of messages of size
> returned by the feed-op?
Ok, I see your proposal, but consider this. The payload feeder is
designed in a way it get the optimal size from the feeder at the right
moment. This collides with the chunk size concept mainly because the
sequencer may truncate that value to the value that is already proposed
by the feeder...
Keep us updated Martin, cheers!
> Thanks for all help.
> /Martin
>
>
>
>
>
>
>
> 2011/3/18 Francis Brosnan Blázquez <francis at aspl.es>
> > Hello
>
> Hi Martin,
>
> > Lately I've been experimenting with the feeder-api which
> seems like
> > the right way to go. However, I do have a problem I am
> trying to
> > understand.
>
>
> Ok,
>
> > The scenario is roughly this:
> >
> > There is a file-server serving large media files (about 500
> - 1500 mb
> > each) and clients connect to this server requesting to
> download an
> > image file. Nothing fancy really.
> >
> > In the original implementation, the client connects, send a
> msg and
> > the server replies with a series of ANS. The client gets a
> > handle-frame callback for each ANS and stores it to some
> local
> > storage.
> >
> > I just tried to do this the feeder-api way. It looks like
> the server
> > sends a series of ANS like it should but on the client side
> I never
> > get any handle-frame callbacks, instead the client just
> grows in
> > memory size so I assume it's queueing all recieved ANS.
>
>
> ..so frames somehow aren't delivered..
>
> > I would appreciate some hints on ways forward to debug this.
> I don't
> > need desperately it but its the right way to do it so I want
> to try to
> > get it to work.
>
>
> Ok, as a general hint I would try enabling debug (either
> calling to
> vortex_log_enable, or setting environment variable
> VORTEX_DEBUG=1 ./your-app). This will give you lot of
> output...
>
> You can also check [1] and [2] to see how those examples are
> implemented
> to get hints about what is failing..
>
> I thinking about ways to make frames to be not notified is to
> not set a
> frame received on the particular channel/profile/ctx, or to
> send some
> replies with some of them missing (ie, sending replies 2 and
> 3, without
> sending 1), and having vortex_channel_set_serialize
> activated...
>
> > By the way, I don't use the supplied file-feeder, I wrote my
> own
> > because the source of the media files isn't really the file
> system.
>
>
> Nice Martin...
>
> > Regards and thanks for a great library.
>
>
> Thanks!
>
> [1]
> https://dolphin.aspl.es/svn/publico/af-arch/trunk/libvortex-1.1/test/vortex-file-transfer-client.c
> [2]
> https://dolphin.aspl.es/svn/publico/af-arch/trunk/libvortex-1.1/test/vortex-file-transfer-server.c
>
> > /Martin
> >
> > --
> > Martin Eliasson
> > +46 (0) 739 97 87 33
> > http://asplunden.org
>
> > _______________________________________________
> > Vortex mailing list
> > Vortex at lists.aspl.es
> > http://lists.aspl.es/cgi-bin/mailman/listinfo/vortex
>
>
>
>
> --
> Martin Eliasson
> +46 (0) 739 97 87 33
> http://asplunden.org
--
Francis Brosnan Blázquez <francis.brosnan at aspl.es>
ASPL
91 134 14 22 - 91 134 14 45 - 91 116 07 57
AVISO LEGAL
Este mensaje se dirige exclusivamente a su destinatario. Los datos
incluidos en el presente correo son confidenciales y sometidos a secreto
profesional, se prohíbe divulgarlos, en virtud de las leyes vigentes. Si
usted no lo es y lo ha recibido por error o tiene conocimiento del mismo
por cualquier motivo, le rogamos que nos lo comunique por este medio y
proceda a destruirlo o borrarlo.
En virtud de lo dispuesto en la Ley Orgánica 15/1999, de 13 de
diciembre, de Protección de Datos de Carácter Personal, le informamos de
que sus datos de carácter personal, recogidos de fuentes accesibles al
público o datos que usted nos ha facilitado previamente, proceden de
bases de datos propiedad de Advanced Software Production Line, S.L.
(ASPL). No obstante, usted puede ejercitar sus derechos de acceso,
rectificación, cancelación y oposición dispuestos en la mencionada Ley
Orgánica, notificándolo por escrito a:
ASPL - Protección Datos, C/Antonio Suárez 10 A-102, 28802, Alcalá de
Henares (Madrid).
More information about the Vortex
mailing list