[9fans] ftpfs shows empty /n/ftp after login

[9fans] ftpfs shows empty /n/ftp after login

[Nils M Holm]

Hello everybody!

Came back to Plan 9 recently, and I'm running into some trouble.
I'm running Plan 9 on QEMU and wanted to use ftpfs to exchange
data with my FreeBSD system.

When logging in via ftpfs, though, I get an empty /n/ftp directory.

This is how far I got:

My FreeBSD machine runs inetd and logging in to my FTP account
via localhost works fine and shows my home directory.

I'm using QEMU 2.2.1 (i386 emulation, no kqemu). This is what I do:

$ qemu -hda Plan9.qcow

[ network card is being detected ]
[ log in as glenda ]
[ rio starts up ]

term% ip/ipconfig
term% ndb/dns -r
term% ftpfs 10.0.2.2
220 ananda.local FTP server (Version 6.00LS) ready
!Adding key: proto=pass server=10.0.2.2 service=ftp
user[glenda]: nmh
password: <super-secret-password>
!
331 Password required for nmh.
230 User nmh logged in.
215 UNIX Type: L8 Version: BSD-199506
257 "/u/home/nmh" is current directory.
%term

And then it gets weird, I think:

term% ls -l /n/ftp
[ there should be lots of files here ]
term%

The logfile on my FreeBSD machine says:

| Jun 25 19:16:16 ananda ftpd[1316]: connection from localhost (127.0.0.1)
| Jun 25 19:17:35 ananda ftpd[1316]: FTP LOGIN FROM localhost as nmh

and nothing more.

"ns | grep ftp" says "mount -c '#|/data1' /n/ftp"

Does anyone have any idea about what's going on here or how to debug?
It's been a while and my Plan-9-fu is quite rusty.

Thanks,
Nils

--
Nils M Holm  < n m h @ t 3 x . o r g >  www.t3x.org

Re: [9fans] ftpfs shows empty /n/ftp after login

[Ryan Gonzalez]

FYI, for what it's worth, I usually use SSH to sync files between my Linux laptop and my Plan 9 VM.

Also, last time I tried, VirtualBox was a bit faster in emulating Plan 9 than QEMU, especially without KVM.


On June 25, 2015 12:25:36 PM CDT, Nils M Holm <nmh@t3x.org> wrote:
>
>Hello everybody!
>
>Came back to Plan 9 recently, and I'm running into some trouble.
>I'm running Plan 9 on QEMU and wanted to use ftpfs to exchange
>data with my FreeBSD system.
>
>When logging in via ftpfs, though, I get an empty /n/ftp directory.
>
>This is how far I got:
>
>My FreeBSD machine runs inetd and logging in to my FTP account
>via localhost works fine and shows my home directory.
>
>I'm using QEMU 2.2.1 (i386 emulation, no kqemu). This is what I do:
>
>$ qemu -hda Plan9.qcow 
>
>[ network card is being detected ]
>[ log in as glenda ]
>[ rio starts up ]
>
>term% ip/ipconfig
>term% ndb/dns -r
>term% ftpfs 10.0.2.2
>220 ananda.local FTP server (Version 6.00LS) ready
>!Adding key: proto=pass server=10.0.2.2 service=ftp
>user[glenda]: nmh
>password: <super-secret-password>
>!
>331 Password required for nmh.
>230 User nmh logged in.
>215 UNIX Type: L8 Version: BSD-199506
>257 "/u/home/nmh" is current directory.
>%term
>
>And then it gets weird, I think:
>
>term% ls -l /n/ftp
>[ there should be lots of files here ]
>term%
>
>The logfile on my FreeBSD machine says:
>
>| Jun 25 19:16:16 ananda ftpd[1316]: connection from localhost
>(127.0.0.1)
>| Jun 25 19:17:35 ananda ftpd[1316]: FTP LOGIN FROM localhost as nmh
>
>and nothing more.
>
>"ns | grep ftp" says "mount -c '#|/data1' /n/ftp"
>
>Does anyone have any idea about what's going on here or how to debug?
>It's been a while and my Plan-9-fu is quite rusty.
>
>Thanks,
>Nils

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [9fans] ftpfs shows empty /n/ftp after login

[Bakul Shah]

On Thu, 25 Jun 2015 19:25:36 +0200 "Nils M Holm" <nmh@t3x.org> wrote:
>
> Came back to Plan 9 recently, and I'm running into some trouble.
> I'm running Plan 9 on QEMU and wanted to use ftpfs to exchange
> data with my FreeBSD system.
>
> When logging in via ftpfs, though, I get an empty /n/ftp directory.

My guess: you are using qemu's networking and can only make
outgoing connections but not accept incoming connections and

The default ftp protocol is to have the server connect back to
you. To go past firewalls and NAT gateways these days you use
ftp passive mode (i.e. force the client to make a connection to
a separate server port). My guess is ftpfs requires active
mode connection (but should be easy to change).

Otherwise use qemu in bridged networking mode so that a guest
os can accept any connection. I can't recall the details at
the moment but you have to use FreeBSD bridge device and add
tap ports to it.

In any case, here are some debugging ideas: Try tcpdump on
freebsd or the qemu host? Are you ftping to the qemu host
machine or another?  Does 9fs sources works? There is also
ratrace (sort of like strace) - pick up my modifications from
sources/contrib/bakul/cmd as it separates trace output from
normal output.

ratrace -o tracefile ftpfs ...

Re: [9fans] ftpfs shows empty /n/ftp after login

[Steve Simon]

ftpfs works fine for me.

I tried it against
	ftp.sunet.se,
	username=anaonymous,
	password=your-email-addr@domain.dom

and /n/ftp contains stuff.

Its worth realising ftpfs gets its directory listing by parsing the output
of the ls command in the ftp protocol, so it has parsers for a variety of
remote os's directory listing formats (vms and I think dec20 parsing code
still there :-).

It may be that your ftp server generates listings in a different format.

try running it with the -d option (debug) to see the directory listings
just in case they look odd/weird/non-existant.

-Steve

Re: [9fans] ftpfs shows empty /n/ftp after login

[Nils M Holm]

On 2015-06-25T11:28:12-0700, Bakul Shah wrote:
> On Thu, 25 Jun 2015 19:25:36 +0200 "Nils M Holm" <nmh@t3x.org> wrote:
> >
> > Came back to Plan 9 recently, and I'm running into some trouble.
> > I'm running Plan 9 on QEMU and wanted to use ftpfs to exchange
> > data with my FreeBSD system.
> >
> > When logging in via ftpfs, though, I get an empty /n/ftp directory.
>
> My guess: you are using qemu's networking and can only make
> outgoing connections but not accept incoming connections and

Yes, that's probably it!

So I either go through the trouble of setting up a tap network or
I find a different means of transfering files. I seem to remember
that tap caused me a lot of trouble once, so I'd prefer some
different method.

> My guess is ftpfs requires active
> mode connection (but should be easy to change).

But then it would still attempt to open a connection to the QEMU
user network, wouldn't it?

> In any case, here are some debugging ideas: Try tcpdump on
> freebsd or the qemu host? Are you ftping to the qemu host
> machine or another?  Does 9fs sources works? There is also
> ratrace (sort of like strace) - pick up my modifications from
> sources/contrib/bakul/cmd as it separates trace output from
> normal output.

Thanks! Will back get back to that later. For now, getting some
sort of file trabsfer to work has priority, and FTP does not seem
to work with QEMU user networking anyway.

--
Nils M Holm  < n m h @ t 3 x . o r g >  www.t3x.org

Re: [9fans] ftpfs shows empty /n/ftp after login

[Nils M Holm]

On 2015-06-25T13:09:00-0500, Ryan Gonzalez wrote:
> FYI, for what it's worth, I usually use SSH to sync files between
> my Linux laptop and my Plan 9 VM.

Thanks for the suggestion! Care to give me a short introduction to
ssh on Plan 9? I normally avoid ssh like the plague, so I assume I
know nothing about it.

> Also, last time I tried, VirtualBox was a bit faster in emulating
> Plan 9 than QEMU, especially without KVM.

VirtualBox is much faster on my machine, but I don't manage to
set up any network connections at all. NIC is recognized, but
ip/ipconfig hangs and after configuring manually, not even ip/ping
works. So I'll stick to QEMU for now.

--
Nils M Holm  < n m h @ t 3 x . o r g >  www.t3x.org

Re: [9fans] ftpfs shows empty /n/ftp after login

[Bakul Shah]

On Thu, 25 Jun 2015 21:51:49 +0200 "Nils M Holm" <nmh@t3x.org> wrote:
> > My guess is ftpfs requires active
> > mode connection (but should be easy to change).

Looking at the ftpfs sources, I see that it should do passive
mode fine. You should debug!

Re: [9fans] ftpfs shows empty /n/ftp after login

[Nils M Holm]

On 2015-06-25T13:16:53-0700, Bakul Shah wrote:
> On Thu, 25 Jun 2015 21:51:49 +0200 "Nils M Holm" <nmh@t3x.org> wrote:
> > > My guess is ftpfs requires active
> > > mode connection (but should be easy to change).
>
> Looking at the ftpfs sources, I see that it should do passive
> mode fine. You should debug!

OK, thanks! Will do!

--
Nils M Holm  < n m h @ t 3 x . o r g >  www.t3x.org

Re: [9fans] ftpfs shows empty /n/ftp after login

[Ryan Gonzalez]

[-- Attachment #1: Type: text/plain, Size: 1838 bytes --]

On Thu, Jun 25, 2015 at 2:57 PM, Nils M Holm <nmh@t3x.org> wrote:

>
> On 2015-06-25T13:09:00-0500, Ryan Gonzalez wrote:
> > FYI, for what it's worth, I usually use SSH to sync files between
> > my Linux laptop and my Plan 9 VM.
>
> Thanks for the suggestion! Care to give me a short introduction to
> ssh on Plan 9? I normally avoid ssh like the plague, so I assume I
> know nothing about it.
>
>
I don't use FreeBSD, so this might be slightly wrong. :)

Sources:

https://www.freebsd.org/doc/handbook/openssh.html
http://plan9.bell-labs.com/wiki/plan9/connecting_to_other_OSes/index.html

You want to do this in FreeBSD:

- Edit /etc/rc.conf and check if this line is present:

    sshd_enable="YES"

  If it isn't, add it.

- Edit /etc/ssh/sshd_config and add the following lines:

    Protocol 2,1
    HostKey /etc/ssh/ssh_host_key
    AllowUsers username@plan9_machine_ip_address

- Run:

    ssh-keygen -t rsa1 -f /etc/ssh/ssh_host_key
    chmod 644 /etc/ssh/ssh_host_key


That's the setup. Now, whenever you want to access your machine's files
from Plan 9, run this on FreeBSD:

    service sshd start

And on Plan 9:

    ssh username@bsd_machine_ip_address

Type 'yes' at any prompts that pop up.


> > Also, last time I tried, VirtualBox was a bit faster in emulating
> > Plan 9 than QEMU, especially without KVM.
>
> VirtualBox is much faster on my machine, but I don't manage to
> set up any network connections at all. NIC is recognized, but
> ip/ipconfig hangs and after configuring manually, not even ip/ping
> works. So I'll stick to QEMU for now.
>
> --
> Nils M Holm  < n m h @ t 3 x . o r g >  www.t3x.org
>



-- 
Ryan
[ERROR]: Your autotools build scripts are 200 lines longer than your
program. Something’s wrong.
http://kirbyfan64.github.io/

[-- Attachment #2: Type: text/html, Size: 3557 bytes --]

Re: [9fans] ftpfs shows empty /n/ftp after login

[Nils M Holm]

On 2015-06-25T13:16:53-0700, Bakul Shah wrote:
> On Thu, 25 Jun 2015 21:51:49 +0200 "Nils M Holm" <nmh@t3x.org> wrote:
> > > My guess is ftpfs requires active
> > > mode connection (but should be easy to change).
>
> Looking at the ftpfs sources, I see that it should do passive
> mode fine. You should debug!

Looks like ftpfs and ftpd cannot agree on a port for passive mode:

PASV
227 Entering passive mode (127,0,0,1,212,96)
passive mode connect to /net/tcp!127.0.0.1!54368 failed: connection refused
PORT 10,0,2,15,131,19
500 Illegal PORT range rejected.

Looks like on the Plan 9 side, the kernel is picking the port, so I
guess the next step would be to have a look at the ftpd sources.

--
Nils M Holm  < n m h @ t 3 x . o r g >  www.t3x.org

Re: [9fans] ftpfs shows empty /n/ftp after login

[Mark van Atten]

Admittedly not exactly the same thing, but still: 9vx works very well
on FreeBSD,
and gives immediate access to the host's files.

Mark.

Re: [9fans] ftpfs shows empty /n/ftp after login

[Nils M Holm]

On 2015-06-26T10:22:14+0200, Nils M Holm wrote:
> Looks like ftpfs and ftpd cannot agree on a port for passive mode:

Of course, here's the problem:

> PASV
> 227 Entering passive mode (127,0,0,1,212,96)
                             ^^^^^^^^^
> passive mode connect to /net/tcp!127.0.0.1!54368 failed: connection refused
> PORT 10,0,2,15,131,19
> 500 Illegal PORT range rejected.

Ftpd announces a port on localhost which, of course, ftpfs is unable
to open. What ftpfs needs is a port on 10.0.2.2 (the guest address of
QEMU).

--
Nils M Holm  < n m h @ t 3 x . o r g >  www.t3x.org

Re: [9fans] ftpfs shows empty /n/ftp after login

[Bakul Shah]

On Fri, 26 Jun 2015 10:29:29 +0200 "Nils M Holm" <nmh@t3x.org> wrote:
> On 2015-06-26T10:22:14+0200, Nils M Holm wrote:
> > Looks like ftpfs and ftpd cannot agree on a port for passive mode:
>
> Of course, here's the problem:
>
> > PASV
> > 227 Entering passive mode (127,0,0,1,212,96)
>                              ^^^^^^^^^
> > passive mode connect to /net/tcp!127.0.0.1!54368 failed: connection refused
> > PORT 10,0,2,15,131,19
> > 500 Illegal PORT range rejected.
>
> Ftpd announces a port on localhost which, of course, ftpfs is unable
> to open. What ftpfs needs is a port on 10.0.2.2 (the guest address of
> QEMU).

I just tried this on an old qemu plan9 image.  I tried connecting
to the hosts' external address, not 10.0.2.2 and it all worked.

Re: [9fans] ftpfs shows empty /n/ftp after login

[Nils M Holm]

On 2015-06-26T10:24:45+0200, Mark van Atten wrote:
> Admittedly not exactly the same thing, but still: 9vx works very well
> on FreeBSD,
> and gives immediate access to the host's files.

This was my first choice but, unfortunately, 9vx crashes on my machine
as soon as I try to do something interesting, e.g. compile a program
with 8c.

--
Nils M Holm  < n m h @ t 3 x . o r g >  www.t3x.org

Re: [9fans] ftpfs shows empty /n/ftp after login

[Nils M Holm]

On 2015-06-25T19:25:36+0200, Nils M Holm wrote:
> When logging in via ftpfs, though, I get an empty /n/ftp directory.

Turns out that ftpd on FreeBSD announces a port on localhost for
passive mode connections:

227 Entering Passive Mode (127,0,0,1,247,55)

If I understand QEMU user networking correctly, it sees incoming
traffic as originating from localhost, in which case the above
would make sense.

So I patched ftpfs and added an additional command line option to
ignore the host address part of the above message and substitute
the destination system's address.

The patch works fine for me. I'm not a networking expert, though,
so it may have some subtle side effects that I'm not aware of.

The patch is below in case anybody is interested.

Thanks for your help!

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
/sys/src/cmd/ip/ftpfs/ftpfs.c.OLD:32,39 - /sys/src/cmd/ip/ftpfs/ftpfs.c:32,37
  int	kapid = -1;
  int	dying;		/* set when any process decides to die */
  int	dokeepalive;
+ char	*system;
+ int		qemu_hack = 0;

  char	*rflush(Fid*), *rnop(Fid*), *rversion(Fid*),
  	*rattach(Fid*), *rclone(Fid*), *rwalk(Fid*),
/sys/src/cmd/ip/ftpfs/ftpfs.c.OLD:151,159 - /sys/src/cmd/ip/ftpfs/ftpfs.c:149,154
  	case 'q':
  		quiet = 1;
  		break;
+ 	case 'Q':
+ 		qemu_hack = 1;
+ 		break;
  	} ARGEND
  	if(argc != 1)
  		usage();
/sys/src/cmd/ip/ftpfs/ftpfs.c.OLD:165,171 - /sys/src/cmd/ip/ftpfs/ftpfs.c:160,165

  	/* initial handshakes with remote side */
  	hello(*argv);
+ 	system = *argv;
  	if(cpassword == 0)
  		rlogin(*argv, keyspec);
  	else
/sys/src/cmd/ip/ftpfs/proto.c.OLD:1272,1277 - /sys/src/cmd/ip/ftpfs/proto.c:1272,1279
  	char *p;
  	int x, fd;
  	TLSconn conn;
+ 	extern char *system;
+ 	extern int	qemu_hack;

  	if(nopassive)
  		return Impossible;
/sys/src/cmd/ip/ftpfs/proto.c.OLD:1297,1305 - /sys/src/cmd/ip/ftpfs/proto.c:1299,1314
  		nopassive = 1;
  		return Impossible;
  	}
- 	snprint(ds, sizeof(ds), "%s!%s.%s.%s.%s!%d", net,
- 		f[0], f[1], f[2], f[3],
- 		((atoi(f[4])&0xff)<<8) + (atoi(f[5])&0xff));
+ 	if (qemu_hack) {
+ 		snprint(ds, sizeof(ds), "%s!%s!%d", net,
+ 			system,
+ 			((atoi(f[4])&0xff)<<8) + (atoi(f[5])&0xff));
+ 	}
+ 	else {
+ 		snprint(ds, sizeof(ds), "%s!%s.%s.%s.%s!%d", net,
+ 			f[0], f[1], f[2], f[3],
+ 			((atoi(f[4])&0xff)<<8) + (atoi(f[5])&0xff));
+ 	}

  	/* open data connection */
  	fd = dial(ds, 0, 0, 0);
/sys/man/4/ftpfs.OLD:4,10 - /sys/man/4/ftpfs:4,10
  .SH SYNOPSIS
  .B ftpfs
  [
- .B -/dqnt
+ .B -/dqntKQ
  ]
  [
  .B -m
/sys/man/4/ftpfs.OLD:96,101 - /sys/man/4/ftpfs:96,107
  option causes ftp to send a NOP command every 15 seconds to attempt
  to keep the connection open.  This command can cause some servers to
  hangup, so you'll have to feel your way.
+ .PP
+ QEMU will see incoming connections as coming from localhost and therefore
+ announce a port for passive mode on 127.0.0.1, which will confuse
+ .I ftpfs.
+ The -Q option ignores the host addresses announced by FTP servers in
+ passive mode negotation and uses the destination address instead.
  .PP
  The
  .B -t
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

--
Nils M Holm  < n m h @ t 3 x . o r g >  www.t3x.org

Re: [9fans] ftpfs shows empty /n/ftp after login

[Nils M Holm]

On 2015-06-26T01:52:25-0700, Bakul Shah wrote:
> I just tried this on an old qemu plan9 image.  I tried connecting
> to the hosts' external address, not 10.0.2.2 and it all worked.

Works. Didn't know that was possible. Thanks!

--
Nils M Holm  < n m h @ t 3 x . o r g >  www.t3x.org

Re: [9fans] ftpfs shows empty /n/ftp after login

[David du Colombier]

> This was my first choice but, unfortunately, 9vx crashes on my machine
> as soon as I try to do something interesting, e.g. compile a program
> with 8c.

What version of 9vx are you using?

--
David du Colombier

Re: [9fans] ftpfs shows empty /n/ftp after login

[fgergo]

[-- Attachment #1: Type: text/plain, Size: 437 bytes --]

thanks, to Charles , u9fs is still always an option for me.
On Jun 26, 2015 11:43 AM, "Nils M Holm" <nmh@t3x.org> wrote:

> On 2015-06-26T01:52:25-0700, Bakul Shah wrote:
> > I just tried this on an old qemu plan9 image.  I tried connecting
> > to the hosts' external address, not 10.0.2.2 and it all worked.
>
> Works. Didn't know that was possible. Thanks!
>
> --
> Nils M Holm  < n m h @ t 3 x . o r g >  www.t3x.org
>
>

[-- Attachment #2: Type: text/html, Size: 767 bytes --]

Re: [9fans] ftpfs shows empty /n/ftp after login

[Nils M Holm]

On 2015-06-26T12:00:36+0200, David du Colombier wrote:
> > This was my first choice but, unfortunately, 9vx crashes on my machine
> > as soon as I try to do something interesting, e.g. compile a program
> > with 8c.
>
> What version of 9vx are you using?

I'm using version 0.12, from here: https://swtch.com/9vx/

--
Nils M Holm  < n m h @ t 3 x . o r g >  www.t3x.org

Re: [9fans] ftpfs shows empty /n/ftp after login

[David du Colombier]

>> What version of 9vx are you using?
>
> I'm using version 0.12, from here: https://swtch.com/9vx/

This is a pretty old version (2009), which have known issues.
Later, Yiyus and Ron Minnich made a lot of improvements.

I would recommend you to use this repository:

https://github.com/0intro/vx32

--
David du Colombier

Re: [9fans] ftpfs shows empty /n/ftp after login

[Nils M Holm]

On 2015-06-27T11:54:58+0200, David du Colombier wrote:
> > I'm using version 0.12, from here: https://swtch.com/9vx/
>
> This is a pretty old version (2009), which have known issues.
> Later, Yiyus and Ron Minnich made a lot of improvements.
>
> I would recommend you to use this repository:
>
> https://github.com/0intro/vx32

Will give it a try, thanks!

--
Nils M Holm  < n m h @ t 3 x . o r g >  www.t3x.org

Re: [9fans] ftpfs shows empty /n/ftp after login

[Nils M Holm]

On 2015-06-27T22:24:50+0200, Nils M Holm wrote:
> On 2015-06-27T11:54:58+0200, David du Colombier wrote:
> > > I'm using version 0.12, from here: https://swtch.com/9vx/
> >
> > This is a pretty old version (2009), which have known issues.
> > Later, Yiyus and Ron Minnich made a lot of improvements.
> >
> > I would recommend you to use this repository:
> >
> > https://github.com/0intro/vx32
>
> Will give it a try, thanks!

Tried it, works like a charm!

--
Nils M Holm  < n m h @ t 3 x . o r g >  www.t3x.org