From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <hmf@inescporto.pt>
X-Original-To: caml-list@yquem.inria.fr
Delivered-To: caml-list@yquem.inria.fr
Received: from mail4-relais-sop.national.inria.fr (mail4-relais-sop.national.inria.fr [192.134.164.105])
	by yquem.inria.fr (Postfix) with ESMTP id 8E9D6BC57
	for <caml-list@yquem.inria.fr>; Fri, 19 Mar 2010 12:07:31 +0100 (CET)
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AqUBADX4okvVhjEXkWdsb2JhbACPJ4wfAQEBAQkLCgcTAx+7L4R6BA
X-IronPort-AV: E=Sophos;i="4.51,273,1267398000"; 
   d="scan'208";a="59350743"
Received: from ihsmtp01voda.lis.interhost.com (HELO ihsmtp01cons.lis.interhost.com) ([213.134.49.23])
  by mail4-smtp-sop.national.inria.fr with ESMTP; 19 Mar 2010 12:07:17 +0100
Received: from [192.168.1.64] ([95.136.106.136]) by ihsmtp01cons.lis.interhost.com with Microsoft SMTPSVC(6.0.3790.3959);
	 Fri, 19 Mar 2010 11:07:16 +0000
Message-ID: <4BA35ADD.3050702@inescporto.pt>
Date: Fri, 19 Mar 2010 11:07:09 +0000
From: Hugo Ferreira <hmf@inescporto.pt>
User-Agent: Thunderbird 2.0.0.24 (X11/20100317)
MIME-Version: 1.0
To: Goswin von Brederlow <goswin-v-b@web.de>
Cc: caml-list@yquem.inria.fr
Subject: Re: [Caml-list] Shared data space: How to manage Ancient keys?
References: <4BA271D2.9030602@inescporto.pt> <87hboc4o2q.fsf@frosties.localdomain>
In-Reply-To: <87hboc4o2q.fsf@frosties.localdomain>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
X-OriginalArrivalTime: 19 Mar 2010 11:07:16.0994 (UTC) FILETIME=[56253620:01CAC754]
X-Spam: no; 0.00; beginner's:01 ocaml:01 bug:01 mfg:98 beginners:01 wrote:01 caml-list:01 caml-list:01 writes:01 writes:01 bin:01 data:02 data:02 caml:02 caml:02 

Goswin von Brederlow wrote:
> Hugo Ferreira <hmf@inescporto.pt> writes:
>
snip...
> 
> How about a simple solution. Only one thread handles indexes.
> 
> You said you have worker processes. Add one controling process on top of
> it that spawns all the worker processes and keeps a communication line
> open with them, e.g. a pipe. Each worker would then tell the controller
> to remove an index or request an unused index to store a result. Its
> probably a good idea to request unused indexes in bunches, say 100 at a
> time to cut down on latenzy.
>

To be honest I have already considered (but not discarded) this option
(which may provide other advantages). This however has several issues:

1) I still need to deal with problem of managing indexes (identifying
    and reusing slots).
2) Each worker has to scan all sequences that have been generated but
    not consumed. So I will also need to generate and send a list of
    available indexes. This may be a very long list.
3) Interaction between the controller and workers need to exchange
    several types of messages:
    a) Request for a sequence to process
    b) Request for a list of sequences to scan
    c) Request an index to use
    d) Acknowledge a used index (maybe)
    e) Request an index to free
    f) Acknowledge a freed index (maybe)
    This implies a complicated protocol with state (worker aware).

Advantages:
1) The workers scan the sequences asynchronously. Scans may have
    various processes access data in read only. Insertion/deletion
    requires locking for read/writes. Use of a controller will
    facilitate this.
2) The controller may avoid having the same sequences scanned
    by two or more workers.

Needless to say these problems are more than I bargained for. 8-(

Thanks,
Hugo.

> MfG
>         Goswin
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>