* [TUHS] Be there a "remote diff" utility? @ 2024-05-16 6:34 Dave Horsfall 2024-05-16 6:51 ` [TUHS] " arnold 2024-05-16 19:53 ` Theodore Ts'o 0 siblings, 2 replies; 14+ messages in thread From: Dave Horsfall @ 2024-05-16 6:34 UTC (permalink / raw) To: The Eunuchs Hysterical Society Every so often I want to compare files on remote machines, but all I can do is to fetch them first (usually into /tmp); I'd like to do something like: rdiff host1:file1 host2:file2 Breathes there such a beast? I see that Penguin/OS has already taken "rdiff" which doesn't seem to do what I want. Think of it as an extension to the Unix philosophy of "Everything looks like a file"... -- Dave ^ permalink raw reply [flat|nested] 14+ messages in thread
* [TUHS] Re: Be there a "remote diff" utility? 2024-05-16 6:34 [TUHS] Be there a "remote diff" utility? Dave Horsfall @ 2024-05-16 6:51 ` arnold 2024-05-16 7:33 ` Ralph Corderoy 2024-05-16 19:53 ` Theodore Ts'o 1 sibling, 1 reply; 14+ messages in thread From: arnold @ 2024-05-16 6:51 UTC (permalink / raw) To: tuhs, dave Maybe diff -u <(ssh host1 cat file1) <(ssh host2 cat file2) ? That could be put into a shell script that does the approriate text manipulations on the original $1 and $2. HTH, Arnold Dave Horsfall <dave@horsfall.org> wrote: > Every so often I want to compare files on remote machines, but all I can > do is to fetch them first (usually into /tmp); I'd like to do something > like: > > rdiff host1:file1 host2:file2 > > Breathes there such a beast? I see that Penguin/OS has already taken > "rdiff" which doesn't seem to do what I want. > > Think of it as an extension to the Unix philosophy of "Everything looks > like a file"... > > -- Dave ^ permalink raw reply [flat|nested] 14+ messages in thread
* [TUHS] Re: Be there a "remote diff" utility? 2024-05-16 6:51 ` [TUHS] " arnold @ 2024-05-16 7:33 ` Ralph Corderoy 2024-05-16 8:59 ` George Michaelson 2024-05-16 9:01 ` arnold 0 siblings, 2 replies; 14+ messages in thread From: Ralph Corderoy @ 2024-05-16 7:33 UTC (permalink / raw) To: tuhs Hi, I've set ‘mail-followup-to: coff@tuhs.org’. > > Every so often I want to compare files on remote machines, but all > > I can do is to fetch them first (usually into /tmp); I'd like to do > > something like: > > > > rdiff host1:file1 host2:file2 > > > > Breathes there such a beast? No, nor should there. It would be slain less it beget rcmp, rcomm, rpaste, ... > > Think of it as an extension to the Unix philosophy of "Everything > > looks like a file"... Then make remote files look local as far as their access is concerned. Ideally at the system-call level. Less ideal, at libc.a. > Maybe > > diff -u <(ssh host1 cat file1) <(ssh host2 cat file2) This is annoyingly noisy if the remote SSH server has sshd_config(5)'s ‘Banner’ set which spews the contents of a file before authentication, e.g. the pointless This computer system is the property of ... Disconnect NOW if you have not been expressly authorised to use this system. Unauthorised use is a criminal offence under the Computer Misuse Act 1990. Communications on or through ...uk's computer systems may be monitored or recorded to secure effective system operation and for other lawful purposes. It appears on stderr so doesn't upset the diff but does clutter. And discarding stderr is too sloppy. -- Cheers, Ralph. ^ permalink raw reply [flat|nested] 14+ messages in thread
* [TUHS] Re: Be there a "remote diff" utility? 2024-05-16 7:33 ` Ralph Corderoy @ 2024-05-16 8:59 ` George Michaelson 2024-05-16 9:01 ` arnold 1 sibling, 0 replies; 14+ messages in thread From: George Michaelson @ 2024-05-16 8:59 UTC (permalink / raw) To: Computer Old Farts Followers; +Cc: The Eunuchs Hysterical Society [-- Attachment #1: Type: text/plain, Size: 1560 bytes --] Sshfs G On Thu, 16 May 2024, 5:34 pm Ralph Corderoy, <ralph@inputplus.co.uk> wrote: > Hi, > > I've set ‘mail-followup-to: coff@tuhs.org’. > > > > Every so often I want to compare files on remote machines, but all > > > I can do is to fetch them first (usually into /tmp); I'd like to do > > > something like: > > > > > > rdiff host1:file1 host2:file2 > > > > > > Breathes there such a beast? > > No, nor should there. It would be slain less it beget rcmp, rcomm, > rpaste, ... > > > > Think of it as an extension to the Unix philosophy of "Everything > > > looks like a file"... > > Then make remote files look local as far as their access is concerned. > Ideally at the system-call level. Less ideal, at libc.a. > > > Maybe > > > > diff -u <(ssh host1 cat file1) <(ssh host2 cat file2) > > This is annoyingly noisy if the remote SSH server has sshd_config(5)'s > ‘Banner’ set which spews the contents of a file before authentication, > e.g. the pointless > > This computer system is the property of ... > > Disconnect NOW if you have not been expressly authorised to use this > system. Unauthorised use is a criminal offence under the Computer > Misuse Act 1990. > > Communications on or through ...uk's computer systems may be > monitored or recorded to secure effective system operation and for > other lawful purposes. > > It appears on stderr so doesn't upset the diff but does clutter. > And discarding stderr is too sloppy. > > -- > Cheers, Ralph. > [-- Attachment #2: Type: text/html, Size: 2151 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* [TUHS] Re: Be there a "remote diff" utility? 2024-05-16 7:33 ` Ralph Corderoy 2024-05-16 8:59 ` George Michaelson @ 2024-05-16 9:01 ` arnold 2024-05-16 17:08 ` ron minnich 1 sibling, 1 reply; 14+ messages in thread From: arnold @ 2024-05-16 9:01 UTC (permalink / raw) To: tuhs, ralph Ralph Corderoy <ralph@inputplus.co.uk> wrote: > > Maybe > > > > diff -u <(ssh host1 cat file1) <(ssh host2 cat file2) > > This is annoyingly noisy if the remote SSH server has sshd_config(5)'s > ‘Banner’ set which spews the contents of a file before authentication, > e.g. the pointless > > [....] > > It appears on stderr so doesn't upset the diff but does clutter. All true, I didn't think about that. > And discarding stderr is too sloppy. But the author of a personal script knows his/her remote machines and can decide if diff -u <(ssh host1 cat file1 2>/dev/null) <(ssh host2 cat file2 2>/dev/null) is appropriate or not. My main point was that the problem is easily solved with a few lines of shell, so no need for a utility, especially one written in C or some other compiled language. Thanks, Arnold ^ permalink raw reply [flat|nested] 14+ messages in thread
* [TUHS] Re: Be there a "remote diff" utility? 2024-05-16 9:01 ` arnold @ 2024-05-16 17:08 ` ron minnich 2024-05-16 18:51 ` Skip Tavakkolian 0 siblings, 1 reply; 14+ messages in thread From: ron minnich @ 2024-05-16 17:08 UTC (permalink / raw) To: arnold; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 1689 bytes --] " The 9import tool allows an arbitrary file on a remote system, with the capability of running the Plan 9 exportfs(4) service, to be imported into the local name space. Usually file is a directory, so the complete file tree under the directory is made available." https://9fans.github.io/plan9port/man/man4/9import.html 9import host1 / /tmp/host1 9import host2 /tmp/host2 diff /tmp/host1/a/b/c /tmp/host2/a/b/c (or whatever command you want that works with files. No need for stuff like 'rdiff' etc.) stuff you take for granted on some systems ... I have the plan 9 cpu command working (written in Go) and I think it's time I get import working more widely, it's just too useful. On Thu, May 16, 2024 at 2:01 AM <arnold@skeeve.com> wrote: > Ralph Corderoy <ralph@inputplus.co.uk> wrote: > > > > Maybe > > > > > > diff -u <(ssh host1 cat file1) <(ssh host2 cat file2) > > > > This is annoyingly noisy if the remote SSH server has sshd_config(5)'s > > ‘Banner’ set which spews the contents of a file before authentication, > > e.g. the pointless > > > > [....] > > > > It appears on stderr so doesn't upset the diff but does clutter. > > All true, I didn't think about that. > > > And discarding stderr is too sloppy. > > But the author of a personal script knows his/her remote machines > and can decide if > > diff -u <(ssh host1 cat file1 2>/dev/null) <(ssh host2 cat file2 > 2>/dev/null) > > is appropriate or not. > > My main point was that the problem is easily solved with a > few lines of shell, so no need for a utility, especially one > written in C or some other compiled language. > > Thanks, > > Arnold > [-- Attachment #2: Type: text/html, Size: 2376 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* [TUHS] Re: Be there a "remote diff" utility? 2024-05-16 17:08 ` ron minnich @ 2024-05-16 18:51 ` Skip Tavakkolian 2024-05-16 19:51 ` Marc Donner 0 siblings, 1 reply; 14+ messages in thread From: Skip Tavakkolian @ 2024-05-16 18:51 UTC (permalink / raw) To: ron minnich; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 2056 bytes --] To add to Ron's post, Plan 9's cpu exports the origination's namespace to the destination; by convention it is mounted on /mnt/term at destination. host1% cpu -h host2 host2% diff file2 /mnt/term/usr/me/file1 On Thu, May 16, 2024 at 10:09 AM ron minnich <rminnich@gmail.com> wrote: > " The 9import tool allows an arbitrary file on a remote system, with the > capability of running the Plan 9 exportfs(4) service, to be imported into > the local name space. Usually file is a directory, so the complete file > tree under the directory is made available." > https://9fans.github.io/plan9port/man/man4/9import.html > > 9import host1 / /tmp/host1 > 9import host2 /tmp/host2 > diff /tmp/host1/a/b/c /tmp/host2/a/b/c > (or whatever command you want that works with files. No need for stuff > like 'rdiff' etc.) > > stuff you take for granted on some systems ... > > I have the plan 9 cpu command working (written in Go) and I think it's > time I get import working more widely, it's just too useful. > > On Thu, May 16, 2024 at 2:01 AM <arnold@skeeve.com> wrote: > >> Ralph Corderoy <ralph@inputplus.co.uk> wrote: >> >> > > Maybe >> > > >> > > diff -u <(ssh host1 cat file1) <(ssh host2 cat file2) >> > >> > This is annoyingly noisy if the remote SSH server has sshd_config(5)'s >> > ‘Banner’ set which spews the contents of a file before authentication, >> > e.g. the pointless >> > >> > [....] >> > >> > It appears on stderr so doesn't upset the diff but does clutter. >> >> All true, I didn't think about that. >> >> > And discarding stderr is too sloppy. >> >> But the author of a personal script knows his/her remote machines >> and can decide if >> >> diff -u <(ssh host1 cat file1 2>/dev/null) <(ssh host2 cat file2 >> 2>/dev/null) >> >> is appropriate or not. >> >> My main point was that the problem is easily solved with a >> few lines of shell, so no need for a utility, especially one >> written in C or some other compiled language. >> >> Thanks, >> >> Arnold >> > [-- Attachment #2: Type: text/html, Size: 3050 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* [TUHS] Re: Be there a "remote diff" utility? 2024-05-16 18:51 ` Skip Tavakkolian @ 2024-05-16 19:51 ` Marc Donner 0 siblings, 0 replies; 14+ messages in thread From: Marc Donner @ 2024-05-16 19:51 UTC (permalink / raw) To: Skip Tavakkolian; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 2684 bytes --] If I recall correctly, there is a combination of flags to rsync that will generate a report on a file, a set of files, or a set of directories to tell if they are different. I seem to recall DPK or RCT doing something clever with rsync and cksum to get this sort of result without having to stream a lot of data across the long-haul network back in the day. Best, Marc ===== nygeek.net mindthegapdialogs.com/home <https://www.mindthegapdialogs.com/home> On Thu, May 16, 2024 at 2:51 PM Skip Tavakkolian <fariborz.t@gmail.com> wrote: > To add to Ron's post, Plan 9's cpu exports the origination's namespace to > the destination; by convention it is mounted on /mnt/term at destination. > > host1% cpu -h host2 > host2% diff file2 /mnt/term/usr/me/file1 > > > On Thu, May 16, 2024 at 10:09 AM ron minnich <rminnich@gmail.com> wrote: > >> " The 9import tool allows an arbitrary file on a remote system, with the >> capability of running the Plan 9 exportfs(4) service, to be imported into >> the local name space. Usually file is a directory, so the complete file >> tree under the directory is made available." >> https://9fans.github.io/plan9port/man/man4/9import.html >> >> 9import host1 / /tmp/host1 >> 9import host2 /tmp/host2 >> diff /tmp/host1/a/b/c /tmp/host2/a/b/c >> (or whatever command you want that works with files. No need for stuff >> like 'rdiff' etc.) >> >> stuff you take for granted on some systems ... >> >> I have the plan 9 cpu command working (written in Go) and I think it's >> time I get import working more widely, it's just too useful. >> >> On Thu, May 16, 2024 at 2:01 AM <arnold@skeeve.com> wrote: >> >>> Ralph Corderoy <ralph@inputplus.co.uk> wrote: >>> >>> > > Maybe >>> > > >>> > > diff -u <(ssh host1 cat file1) <(ssh host2 cat file2) >>> > >>> > This is annoyingly noisy if the remote SSH server has sshd_config(5)'s >>> > ‘Banner’ set which spews the contents of a file before authentication, >>> > e.g. the pointless >>> > >>> > [....] >>> > >>> > It appears on stderr so doesn't upset the diff but does clutter. >>> >>> All true, I didn't think about that. >>> >>> > And discarding stderr is too sloppy. >>> >>> But the author of a personal script knows his/her remote machines >>> and can decide if >>> >>> diff -u <(ssh host1 cat file1 2>/dev/null) <(ssh host2 cat file2 >>> 2>/dev/null) >>> >>> is appropriate or not. >>> >>> My main point was that the problem is easily solved with a >>> few lines of shell, so no need for a utility, especially one >>> written in C or some other compiled language. >>> >>> Thanks, >>> >>> Arnold >>> >> [-- Attachment #2: Type: text/html, Size: 4390 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* [TUHS] Re: Be there a "remote diff" utility? 2024-05-16 6:34 [TUHS] Be there a "remote diff" utility? Dave Horsfall 2024-05-16 6:51 ` [TUHS] " arnold @ 2024-05-16 19:53 ` Theodore Ts'o 2024-06-19 23:20 ` Alan D. Salewski 1 sibling, 1 reply; 14+ messages in thread From: Theodore Ts'o @ 2024-05-16 19:53 UTC (permalink / raw) To: Dave Horsfall; +Cc: The Eunuchs Hysterical Society On Thu, May 16, 2024 at 04:34:54PM +1000, Dave Horsfall wrote: > Every so often I want to compare files on remote machines, but all I can > do is to fetch them first (usually into /tmp); I'd like to do something > like: > > rdiff host1:file1 host2:file2 > > Breathes there such a beast? I see that Penguin/OS has already taken > "rdiff" which doesn't seem to do what I want. rdiff is something which someone on the internet had created, as part of the librsync package[1]. Thia isn't considered part of the core package (for example, Debian consideres it as an "optional" package) but rather something which various distributions have packaged for the convenience for their users. [1] https://librsync.github.io/ So if this is considered part of Penguin/OS, would we also consider "nethack" or X11 part of BSD 4.3, since it was available and often would be commonly installed on BSD 4.3 systems? Or are all packages which are in FreeBSD's ports "part of FreeBSD"? Or all packages in MacPorts part of MacOS? In any case, the way I'd suggest that you do this that works as an extention to the Unix philosohy of "Everything looks like a file" is to use FUSE: sshfs host1:/ ~/mnt/host1 sshfs host2:/ ~/mnt/host2 diff ~/mnt/host1/file1 ~/mnt/host2/file2 Cheers, - Ted ^ permalink raw reply [flat|nested] 14+ messages in thread
* [TUHS] Re: Be there a "remote diff" utility? 2024-05-16 19:53 ` Theodore Ts'o @ 2024-06-19 23:20 ` Alan D. Salewski 0 siblings, 0 replies; 14+ messages in thread From: Alan D. Salewski @ 2024-06-19 23:20 UTC (permalink / raw) To: TUHS (The Unix Heritage Society) On Thu, May 16, 2024, at 15:53, Theodore Ts'o wrote: > On Thu, May 16, 2024 at 04:34:54PM +1000, Dave Horsfall wrote: >> Every so often I want to compare files on remote machines, but all I can >> do is to fetch them first (usually into /tmp); I'd like to do something >> like: >> >> rdiff host1:file1 host2:file2 >> >> Breathes there such a beast? I see that Penguin/OS has already taken >> "rdiff" which doesn't seem to do what I want. [...] > In any case, the way I'd suggest that you do this that works as an > extention to the Unix philosohy of "Everything looks like a file" is > to use FUSE: > > sshfs host1:/ ~/mnt/host1 > sshfs host2:/ ~/mnt/host2 > diff ~/mnt/host1/file1 ~/mnt/host2/file2 > > Cheers, > > - Ted Mentioning this since I just came across it today: Eric S. Raymond (ESR) has a 'netdiff' tool available here: http://www.catb.org/~esr/netdiff/ https://gitlab.com/esr/netdiff <quote> Perform diff across network links. Takes two arguments which may be host:path pairs in the style of ssh/scp, make temporary local copies as needed, applies diff(1). All options are passed to diff(1). </quote> Usage: netdiff [diff-options] [host1:]path1 [host2:]path2 It's 56 lines of POSIX shell script that invokes 'scp' behind the scenes and creates temp files that are fed to 'diff', with some file name post-processing. Compared to the one line of shell script mentioned by Arnold: diff -u <(ssh host1 cat file1) <(ssh host2 cat file2) the main benefits I see to 'netdiff' in its current form are that it encapsulates the mechanism behind a descriptive name that can be found in the file system, and it has a man page. So it can be found, used and/or studied by those with larval shell fu. However, the comments in the 'netdiff' source suggest openness to supporting different transport mechanisms, which (if added) might allow using the same command line interface regardless of the underlying transport being used (scp, ssh, rsync, curl, wget, whatever), possibly with different transports for each of the requested files. Of course, that would not offer a superior user experience to the '9import', 'sshfs', and NFS approaches also mentioned. But if there were an 'rf' ("remote file") tool that did just the transport abstraction portion, then tools such as 'netdiff' could use it internally, or folks could use it directly: diff -u <(rf user@host1:/path/to/file1) <(rf user@host2:/path/to/file2) The presumption here is that the user has control over some local configuration that maps user@host pairs to transports (maybe defaulting to 'ssh' invoking 'cat' on the remote host). Maybe somebody who needs more than 'ssh' would find value in such a thing... -Al -- a l a n d. s a l e w s k i ads@salewski.email salewski@att.net https://github.com/salewski ^ permalink raw reply [flat|nested] 14+ messages in thread
* [TUHS] Re: Be there a "remote diff" utility? @ 2024-05-16 12:31 Douglas McIlroy 2024-05-16 17:12 ` Bakul Shah via TUHS 0 siblings, 1 reply; 14+ messages in thread From: Douglas McIlroy @ 2024-05-16 12:31 UTC (permalink / raw) To: TUHS main list [-- Attachment #1: Type: text/plain, Size: 118 bytes --] With the disclaimer that I have never used it, I note that FUSE/sshfs allows one to mount remote file systems. Doug [-- Attachment #2: Type: text/html, Size: 165 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* [TUHS] Re: Be there a "remote diff" utility? 2024-05-16 12:31 Douglas McIlroy @ 2024-05-16 17:12 ` Bakul Shah via TUHS 2024-05-16 18:12 ` Rich Salz 0 siblings, 1 reply; 14+ messages in thread From: Bakul Shah via TUHS @ 2024-05-16 17:12 UTC (permalink / raw) To: The Unix Heritage Society mailing list [-- Attachment #1: Type: text/plain, Size: 459 bytes --] An interesting question is whether there exists a diff algorithm which *minimizes* data movement across the network. Assuming similar lengths, you can halve it by running the diff at one of the hosts but can one do better if the two files are fairly similar? Is this even a theoretical possibility? I don't see links to any such algorithm on wikipedia's diff page but I figured there might be someone on TUHS who may have speculated or know about this! Bakul [-- Attachment #2: Type: text/html, Size: 1178 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* [TUHS] Re: Be there a "remote diff" utility? 2024-05-16 17:12 ` Bakul Shah via TUHS @ 2024-05-16 18:12 ` Rich Salz 2024-05-16 18:38 ` Ben Greenfield via TUHS 0 siblings, 1 reply; 14+ messages in thread From: Rich Salz @ 2024-05-16 18:12 UTC (permalink / raw) To: Bakul Shah; +Cc: The Unix Heritage Society mailing list [-- Attachment #1: Type: text/plain, Size: 142 bytes --] The rsync protocol might be appropriate. See https://www.samba.org/~tridge/phd_thesis.pdf and https://rsync.samba.org/tech_report/node2.html [-- Attachment #2: Type: text/html, Size: 311 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* [TUHS] Re: Be there a "remote diff" utility? 2024-05-16 18:12 ` Rich Salz @ 2024-05-16 18:38 ` Ben Greenfield via TUHS 0 siblings, 0 replies; 14+ messages in thread From: Ben Greenfield via TUHS @ 2024-05-16 18:38 UTC (permalink / raw) To: The Unix Heritage Society mailing list; +Cc: Bakul Shah [-- Attachment #1: Type: text/plain, Size: 277 bytes --] I use rsync for that with the -n dry run flag > On May 16, 2024, at 2:12 PM, Rich Salz <rich.salz@gmail.com> wrote: > > The rsync protocol might be appropriate. See https://www.samba.org/~tridge/phd_thesis.pdf and https://rsync.samba.org/tech_report/node2.html > [-- Attachment #2: Type: text/html, Size: 788 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2024-06-19 23:22 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-05-16 6:34 [TUHS] Be there a "remote diff" utility? Dave Horsfall 2024-05-16 6:51 ` [TUHS] " arnold 2024-05-16 7:33 ` Ralph Corderoy 2024-05-16 8:59 ` George Michaelson 2024-05-16 9:01 ` arnold 2024-05-16 17:08 ` ron minnich 2024-05-16 18:51 ` Skip Tavakkolian 2024-05-16 19:51 ` Marc Donner 2024-05-16 19:53 ` Theodore Ts'o 2024-06-19 23:20 ` Alan D. Salewski 2024-05-16 12:31 Douglas McIlroy 2024-05-16 17:12 ` Bakul Shah via TUHS 2024-05-16 18:12 ` Rich Salz 2024-05-16 18:38 ` Ben Greenfield via TUHS
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).