9fans - fans of the OS Plan 9 from Bell Labs
 help / color / mirror / Atom feed
* [9fans] A potentially useful venti client
@ 2017-12-12  9:33 Ole-Hjalmar Kristensen
  2017-12-12 14:07 ` Steve Simon
  0 siblings, 1 reply; 27+ messages in thread
From: Ole-Hjalmar Kristensen @ 2017-12-12  9:33 UTC (permalink / raw)
  To: Fans of the OS Plan 9 from Bell Labs

[-- Attachment #1: Type: text/plain, Size: 3636 bytes --]

Based on copy.c and readlist.c, I have cobbled together a venti client to
copy a list of venti blocks from one venti server to another. I am thinking
of using it to incrementally replicate the contents on one site site to
another. It could even be used for two-way replication, since the CAS and
deduplicating properties of venti ensures that you will never have write
conflicts at a block level.

I have tried it out by feeding it with the output from printarenas, and it
seems to work reasonably well. Does anyone have any good ideas about how to
incrementally extract the set of scores that has been added to a venti
server? You could extract the whole set of scores and do a diff with an old
set of course, but that's rather inefficient.

Ole-Hj.


#include <u.h>
#include <libc.h>
#include <thread.h>
#include <venti.h>
#include <bio.h>

enum
{
    // XXX What to do here?
    VtMaxLumpSize = 65535,
};

char *srchost;
char *dsthost;
Biobuf b;
VtConn *zsrc;
VtConn *zdst;
uchar *buf;
void run(Biobuf*);
int nn;

void
usage(void)
{
    fprint(2, "usage: copylist srchost dsthost list\n");
    threadexitsall("usage");
}

int
parsescore(uchar *score, char *buf, int n)
{
    int i, c;

    memset(score, 0, VtScoreSize);

    if(n != VtScoreSize*2){
        werrstr("score wrong length %d", n);
        return -1;
    }
    for(i=0; i<VtScoreSize*2; i++) {
        if(buf[i] >= '0' && buf[i] <= '9')
            c = buf[i] - '0';
        else if(buf[i] >= 'a' && buf[i] <= 'f')
            c = buf[i] - 'a' + 10;
        else if(buf[i] >= 'A' && buf[i] <= 'F')
            c = buf[i] - 'A' + 10;
        else {
            c = buf[i];
            werrstr("bad score char %d '%c'", c, c);
            return -1;
        }

        if((i & 1) == 0)
            c <<= 4;

        score[i>>1] |= c;
    }
    return 0;
}

void
threadmain(int argc, char *argv[])
{
    int fd, i;

    ARGBEGIN{
    default:
        usage();
        break;
    }ARGEND

    if(argc < 2)
        usage();

    fmtinstall('V', vtscorefmt);
    buf = vtmallocz(VtMaxLumpSize);

    srchost = argv[0];
    zsrc = vtdial(srchost);
    if(zsrc == nil)
        sysfatal("could not dial src server: %r");
    if(vtconnect(zsrc) < 0)
        sysfatal("vtconnect src: %r");

    dsthost = argv[1];
    zdst = vtdial(dsthost);
    if(zdst == nil)
        sysfatal("could not dial dst server: %r");
    if(vtconnect(zdst) < 0)
        sysfatal("vtconnect dst: %r");

    if(argc == 2){
        Binit(&b, 0, OREAD);
        run(&b);
    }else{
        for(i=2; i<argc; i++){
            if((fd = open(argv[i], OREAD)) < 0)
                sysfatal("open %s: %r", argv[i]);
            Binit(&b, fd, OREAD);
            run(&b);
        }
    }
    threadexitsall(nil);
}

void
run(Biobuf *b)
{
    char *p, *f[10];
    int nf;
    uchar score[20];
    int type, n;

    while((p = Brdline(b, '\n')) != nil){
        p[Blinelen(b)-1] = 0;
        nf = tokenize(p, f, nelem(f));
        if(nf != 2)
            sysfatal("syntax error in work list");
        if(parsescore(score, f[0], strlen(f[0])) < 0)
            sysfatal("bad score %s in work list", f[0]);
        type = atoi(f[1]);
        n = vtread(zsrc, score, type, buf, VtMaxLumpSize);
        if(n < 0)
            sysfatal("could not read %s %s: %r", f[0], f[1]);
        n = vtwrite(zdst, score, type, buf, n);
        if(n < 0)
            sysfatal("could not write %s %s: %r", f[0], f[1]);
        if(++nn%1000 == 0)
            print("%d...", nn);
    }
}

[-- Attachment #2: Type: text/html, Size: 4834 bytes --]

^ permalink raw reply	[flat|nested] 27+ messages in thread
end of thread, other threads:[~2017-12-13 14:13 UTC | newest]

Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-12-12  9:33 [9fans] A potentially useful venti client Ole-Hjalmar Kristensen
2017-12-12 14:07 ` Steve Simon
2017-12-12 15:45   ` Steven Stallion
2017-12-12 16:11     ` Steve Simon
2017-12-12 16:23       ` Steven Stallion
2017-12-12 18:42     ` Ole-Hjalmar Kristensen
2017-12-12 19:16       ` Steven Stallion
2017-12-12 20:31         ` hiro
2017-12-12 23:36         ` Skip Tavakkolian
2017-12-13 10:17           ` Bakul Shah
2017-12-12 18:33   ` Ole-Hjalmar Kristensen
2017-12-12 19:53     ` Steve Simon
2017-12-12 20:03       ` Steve Simon
2017-12-12 20:07       ` Ole-Hjalmar Kristensen
2017-12-12 20:15     ` Steve Simon
2017-12-12 20:31       ` Ole-Hjalmar Kristensen
2017-12-12 20:38         ` Steve Simon
2017-12-12 21:40           ` Ole-Hjalmar Kristensen
2017-12-13  0:03             ` Steve Simon
2017-12-13  7:29               ` Ole-Hjalmar Kristensen
2017-12-13  9:44                 ` hiro
2017-12-13 11:00                 ` Steve Simon
2017-12-13 12:22                   ` Richard Miller
2017-12-13 14:13                     ` Ole-Hjalmar Kristensen
2017-12-13 13:37                   ` Ole-Hjalmar Kristensen
2017-12-12 21:02       ` Steven Stallion
2017-12-12 21:55         ` Ole-Hjalmar Kristensen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).