[9fans] btfs, a BitTorrent client (attempt)

[9fans] btfs, a BitTorrent client (attempt)

[Mathieu L.]

Hello 9fans,

So I'm following the ericvh school and I'm "releasing" that, because I
for one badly need the peer review.  However, the program is doing
very little at the moment and it should really be considered as some
sort of alpha version.

The code is neither efficient nor elegant and it definitely is lower
quality than what you usually see around here. That does not worry me that
much as it's constantly improving thanks to 9fans and the guys on #plan9.

What it should be able to do at the current state is:
-parse the .torrent file and put everything relevant into mem
-call the tracker and parse the reply
-call a peer revealed by the tracker
-ask that peer for the pieces in order, receive them, and write them to
the files they belong to.
So if you're lucky enough that the chosen peer is a seeder, you should
get the torrent.  Anything else, you can assume it can't do.  Also, if
some files in the torrent are smaller than the typical piece size
(262144 bytes), it will probably fail.

As you can see, I'd say than less than half of the rfc is there, but
I feel like the hardest part so far was to understand the protocol,
so I'm confident the rest should follow quite easily.

Now I don't expect people to look at the code, but I'd really like
some advice for the next steps.  What mainly needs to be done, is the
capability to contact several peers at the same time, and request
different pieces from all of them, as well as to send to those peers
the pieces we already have.  I've read the docs about the csp style
and I've tried some small examples, but I'm not sure what's the best
to do for that bt program now.  Most of the processes/threads will
have to share some info, like the list of pieces we already have, and
the list of peers we're already communicating with.
So as a first approach I was thinking of having simply one thread for
each connection with a peer.  And maybe one group of threads (in one
process) would be responsible for getting the pieces, while another
one (in another process) would be responsbile for sending the pieces
we have.  How does that sound?

Anyway, for those interested enough to try it out, here it is:
http://smgl.fr.eu.org/lejatorn/plan9/btfs.tar
http://smgl.fr.eu.org/lejatorn/plan9/btfs
I should have a contrib when Geoff is back, I'll announce it when it
gets there.

You'll have to patch webfs so it accepts nuls in urls, because that
can happen with tracker infohashs (and it's not illegal according to
the rfc).  That does not seem to have caused me any trouble but then I
only use webfs for btfs.
9vx% diff /sys/src/cmd/webfs/url.c-old /sys/src/cmd/webfs/url.c
543a544
> 	/*
547a549
> 	*/

There's no documentation yet, so here's how it works:

./btfs [-d datadir] [-m mountpoint] [-v]
echo 'add /path/to/the/torrent/file.torrent' > /mnt/btfs/ctl

Of course, lots of features and user interaction are planned to happen
through the fs mounted at /mnt/btfs/, that will come later on.

I've set up a tracker and a peer that you can test against, if you use
the .torrent attached to this mail, it should work.  I suppose
torrents with one full seeder like the ones that can be found on
jamendo.com would work as well.

Finally, I'd like to thank everyone who helped me with it so far,
especially fgb, maht, quintile and cinap.

Comments, ideas, criticism all welcome please.

Enjoy,
Mathieu

Re: [9fans] btfs, a BitTorrent client (attempt)

[erik quanstrom]

> So as a first approach I was thinking of having simply one thread for
> each connection with a peer.  And maybe one group of threads (in one
> process) would be responsible for getting the pieces, while another
> one (in another process) would be responsbile for sending the pieces
> we have.  How does that sound?

why can't they communicate through the filesystem?
i would imagine you could avoid building a lot of
fancy infastructure if you did it that way.

- erik

Re: [9fans] btfs, a BitTorrent client (attempt)

[Mathieu L.]

[-- Attachment #1: Type: text/plain, Size: 114 bytes --]

and of course I forgot to attach the .torrent, so here it is:
http://smgl.fr.eu.org/lejatorn/plan9/btfs.torrent

[-- Attachment #2: Type: message/rfc822, Size: 6992 bytes --]

From: "Mathieu L." <lejatorn@gmail.com>
To: 9fans@9fans.net
Subject: [9fans] btfs, a BitTorrent client (attempt)
Date: Thu, 16 Jul 2009 20:35:08 +0200
Message-ID: <8ace18cfdc26bf96b994af920c6e77e9@smgl.fr.eu.org>

Hello 9fans,

So I'm following the ericvh school and I'm "releasing" that, because I
for one badly need the peer review.  However, the program is doing
very little at the moment and it should really be considered as some
sort of alpha version.

The code is neither efficient nor elegant and it definitely is lower
quality than what you usually see around here. That does not worry me that
much as it's constantly improving thanks to 9fans and the guys on #plan9.

What it should be able to do at the current state is:
-parse the .torrent file and put everything relevant into mem
-call the tracker and parse the reply
-call a peer revealed by the tracker
-ask that peer for the pieces in order, receive them, and write them to
the files they belong to.
So if you're lucky enough that the chosen peer is a seeder, you should
get the torrent.  Anything else, you can assume it can't do.  Also, if
some files in the torrent are smaller than the typical piece size
(262144 bytes), it will probably fail.

As you can see, I'd say than less than half of the rfc is there, but
I feel like the hardest part so far was to understand the protocol,
so I'm confident the rest should follow quite easily.

Now I don't expect people to look at the code, but I'd really like
some advice for the next steps.  What mainly needs to be done, is the
capability to contact several peers at the same time, and request
different pieces from all of them, as well as to send to those peers
the pieces we already have.  I've read the docs about the csp style
and I've tried some small examples, but I'm not sure what's the best
to do for that bt program now.  Most of the processes/threads will
have to share some info, like the list of pieces we already have, and
the list of peers we're already communicating with.
So as a first approach I was thinking of having simply one thread for
each connection with a peer.  And maybe one group of threads (in one
process) would be responsible for getting the pieces, while another
one (in another process) would be responsbile for sending the pieces
we have.  How does that sound?

Anyway, for those interested enough to try it out, here it is:
http://smgl.fr.eu.org/lejatorn/plan9/btfs.tar
http://smgl.fr.eu.org/lejatorn/plan9/btfs
I should have a contrib when Geoff is back, I'll announce it when it
gets there.

You'll have to patch webfs so it accepts nuls in urls, because that
can happen with tracker infohashs (and it's not illegal according to
the rfc).  That does not seem to have caused me any trouble but then I
only use webfs for btfs.
9vx% diff /sys/src/cmd/webfs/url.c-old /sys/src/cmd/webfs/url.c
543a544
> 	/*
547a549
> 	*/

There's no documentation yet, so here's how it works:

./btfs [-d datadir] [-m mountpoint] [-v]
echo 'add /path/to/the/torrent/file.torrent' > /mnt/btfs/ctl

Of course, lots of features and user interaction are planned to happen
through the fs mounted at /mnt/btfs/, that will come later on.

I've set up a tracker and a peer that you can test against, if you use
the .torrent attached to this mail, it should work.  I suppose
torrents with one full seeder like the ones that can be found on
jamendo.com would work as well.

Finally, I'd like to thank everyone who helped me with it so far,
especially fgb, maht, quintile and cinap.

Comments, ideas, criticism all welcome please.

Enjoy,
Mathieu