[Vortex] Thread pool in vortex

[subrahmanya]

Hi Francis,

Which version of vortex has the API "vortex_thread_pool_setup"? As of now I
am using version 1.1.8. It does not have this API.

I have one connection created per instance of vortex context. The reason I
have 1-1 mapping between context and session is that it gives flexibility
to set different timeout for different connections. Having a single context
would mean I've to use same timeout for all connections.

I looked at the implementation of "vortex_connection_connect_timeout" API.
This API does not seem to set the timeout to the context instead it sets
the value to an environment variable. The timeout will be set to context
only if  "vortex_connection_get_connect_timeout" API is invoked after
invocation of "vortex_connection_connect_timeout" API. Is this behavior
intentional?

Another issue is that  "vortex_connection_connect_timeout" API sets an
environment variable which is later read in "
vortex_connection_get_connect_timeout" API. How does it work when two
threads try to set timeout for two different contexts at the same time?

Here is one of my observations:
Vortex creates joinable threads. It looks like RedHat Linux OS has a limit
on the number of joinable threads that can be created in a process. The
limit looks to be around 250 (probably 256).

Thanks
Subrahmanya

On Wed, Feb 29, 2012 at 7:04 PM, Francis Brosnan Blazquez
<francis at aspl.es>wrote:

> > Hi
>
> Hi Subrahmanya,
>
> > I wanted to understand thread pool mechanism in vortex.
>
> Ok,
>
> > I see that vortex creates a pool of threads for each context and the
> > default pool size is 5. This pool is shared across all connections
> > created using this context. Is that correct?
>
> Yes,
>
> > Does it expand/shrink automatically?
>
> By default no, but you can use vortex_thread_pool_setup to do so (even
> at runtime).
>
> > Ideally I would not like to create 5 threads when context is created
> > instead it has to be created only when a new thread is required.
>
> You can't anticipate this (for example, a thread from the pool is
> required to process channel start reply). You need at least 1 thread in
> the pool.
>
> See documentation on [1] for more details:
>
> [1]
> http://www.aspl.es/fact/files/af-arch/vortex-1.1/html/group__vortex__thread__pool.html
>
> > If the thread pool does not expand automatically then creating many
> > connections over the same context wouldn't be a good idea since that
> > can lead to performance issues. I see that pool size can be configured
> > through environment variable which means the value is applicable to
> > any context created in that process. If the pool is not
> > expandable/shrinkable then it may not be a good idea to allocate 5
> > threads per context. This could be a burden in some scenarios when
> > multiple contexts are created in the process. Activities over
> > different connections created with different contexts might not be
> > uniform. This means creating 5 threads (or whatever value is
> > configured through the environment variable) for some of the contexts
> > may be optimum and for some this value would be high and for some it
> > could be low as well. So not having a expandable/shrinkable pool would
> > mean high resource usage even in case of very less activity.
>
> Ok, this is not the case: vortex includes facilities for automatic
> thread pool expansion/reduction. At the same time you can also do it
> yourself on top of vortex thread pool using
> vortex_thread_pool_add/vortex_thread_pool_remove (even at runtime).
>
> > For me the solution seems to be to maintain a global thread pool
> > rather than having a thread pool per context and the global thread
> > pool has to be shared by different contexts. And, the thread pool has
> > to be expanded when there is high activity and there are no threads
> > available and reduced when there is less activity. Yes there should be
> > a limit on the max number of threads to be created in this pool. This
> > way resource usage can be controlled when there is no activity on the
> > connections.
>
> Ok, why do you want to create several separated VortexCtx objects but
> sharing a global thread pool? I mean, why don't use a single VortexCtx?
>
> > Thanks
> > Subrahmanya
> > _______________________________________________
> > Vortex mailing list
> > Vortex at lists.aspl.es
> > http://lists.aspl.es/cgi-bin/mailman/listinfo/vortex
>
> --
> Francis Brosnan Blázquez <francis.brosnan at aspl.es>
> ASPL
> 91 134 14 22 - 91 134 14 45 - 91 116 07 57
>
> AVISO LEGAL
>
> Este mensaje se dirige exclusivamente a su destinatario. Los datos
> incluidos en el presente correo son confidenciales y sometidos a secreto
> profesional, se prohíbe divulgarlos, en virtud de las leyes vigentes. Si
> usted no lo es y lo ha recibido por error o tiene conocimiento del mismo
> por cualquier motivo, le rogamos que nos lo comunique por este medio y
> proceda a destruirlo o borrarlo.
>
> En virtud de lo dispuesto en la Ley Orgánica 15/1999, de 13 de
> diciembre, de Protección de Datos de Carácter Personal, le informamos de
> que sus datos de carácter personal, recogidos de fuentes accesibles al
> público o datos que usted nos ha facilitado previamente, proceden de
> bases de datos propiedad de Advanced Software Production Line, S.L.
> (ASPL). No obstante, usted puede ejercitar sus derechos de acceso,
> rectificación, cancelación y oposición dispuestos en la mencionada Ley
> Orgánica, notificándolo por escrito a:
> ASPL - Protección Datos, C/Antonio Suárez 10 A-102, 28802, Alcalá de
> Henares (Madrid).
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.aspl.es/pipermail/vortex/attachments/20120302/ed5689ee/attachment.htm>