From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from tb-mx1.topicbox.com (localhost.local [127.0.0.1]) by tb-mx1.topicbox.com (Postfix) with ESMTP id 69DD623D064D for <9fans@9fans.net>; Sun, 10 May 2020 15:04:26 -0400 (EDT) (envelope-from ori@eigenstate.org) Received: from tb-mx1.topicbox.com (localhost [127.0.0.1]) by tb-mx1.topicbox.com (Authentication Milter) with ESMTP id BD5ECD67CB2; Sun, 10 May 2020 15:04:26 -0400 ARC-Seal: i=1; a=rsa-sha256; cv=none; d=topicbox.com; s=arcseal; t= 1589137466; b=J7XLjp+tON2410yX8EddYRxgaF6T3WTquPPoFUexRQVpNPXrL/ /D3o4LlJEI9BOodj2Pe+ngt837K5mrCY9DwMQPXpOeksnlPN4Zb1jHpew2R5UtzP NWUjp9m0ULfNxGORwG8SN5ORWRuvmP7T2Ba/y0ugEHmW9UZkDJYQNxGi9bLYKsxi 4lPaAtXZ7orFzAzyYXI3YKFQejFVvUJdiWlBkqL1jPzJ93wgj4WcKHP/Uf/2FeeL K4Lj3QjbzXTNXW2720YtR0zGowpb0bOo5Yp+82fCSqhtY1kM2Qqx0UbUiowPhtsh RHt4a990ojvsmWQ3zXcTk6eVnCr0S4grGN9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d= topicbox.com; h=message-id:to:subject:date:from:in-reply-to :mime-version:content-type:content-transfer-encoding; s=arcseal; t=1589137466; bh=MzxJ1u/V/sxJjhMdm9LXBRH+XTtmJq54enuuVGsvJ28=; b= LwX97d9k/Sg6CjurQahg+AH2EYXi8p8qyL9XMJrV04owUdyBG/pUfkiV2mCoxdsW ejp+PRP3ts9EewBbDHhknLyCRrBiz6Nq/vBGNGN94xCWtWPvKrVTEpQcc7wymLUw 19iIyzGyAXPT1oOW8FkWUKFTnRtDmynLVYopoJfS8136AW9CeSc4rEandLNDtsL9 d03BRhN11H4r/agiRC09HBGoJcGZJA7lXyvifOP9ouwTCR2lAULG03NF0E+Qg6MG 1yeLdQhWNBGFHG9PpeOhaWjZbkBoTnC+AC1tJ9B898w+bjBjzshmw+LrtKxKe2zV r+nexwYPDWZ07fR+UyIdzw== ARC-Authentication-Results: i=1; tb-mx1.topicbox.com; arc=none (no signatures found); bimi=none (Domain is not BIMI enabled); dkim=none (no signatures found); dmarc=pass policy.published-domain-policy=none policy.applied-disposition=none policy.evaluated-disposition=none (p=none,d=none,d.eval=none) policy.policy-from=p header.from=eigenstate.org; iprev=pass smtp.remote-ip=206.124.132.107 (mimir.eigenstate.org); spf=pass smtp.mailfrom=ori@eigenstate.org smtp.helo=mimir.eigenstate.org; x-aligned-from=pass (Address match); x-ptr=pass smtp.helo=mimir.eigenstate.org policy.ptr=mimir.eigenstate.org; x-return-mx=pass header.domain=eigenstate.org policy.is_org=yes (MX Records found: kusuri.pikopiko.org,mail.pikopiko.org,nokogiri.pikopiko.org,eigenstate.org); x-return-mx=pass smtp.domain=eigenstate.org policy.is_org=yes (MX Records found: kusuri.pikopiko.org,mail.pikopiko.org,nokogiri.pikopiko.org,eigenstate.org); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES256-GCM-SHA384 smtp.bits=256/256; x-vs=clean score=0 state=0 Authentication-Results: tb-mx1.topicbox.com; arc=none (no signatures found); bimi=none (Domain is not BIMI enabled); dkim=none (no signatures found); dmarc=pass policy.published-domain-policy=none policy.applied-disposition=none policy.evaluated-disposition=none (p=none,d=none,d.eval=none) policy.policy-from=p header.from=eigenstate.org; iprev=pass smtp.remote-ip=206.124.132.107 (mimir.eigenstate.org); spf=pass smtp.mailfrom=ori@eigenstate.org smtp.helo=mimir.eigenstate.org; x-aligned-from=pass (Address match); x-ptr=pass smtp.helo=mimir.eigenstate.org policy.ptr=mimir.eigenstate.org; x-return-mx=pass header.domain=eigenstate.org policy.is_org=yes (MX Records found: kusuri.pikopiko.org,mail.pikopiko.org,nokogiri.pikopiko.org,eigenstate.org); x-return-mx=pass smtp.domain=eigenstate.org policy.is_org=yes (MX Records found: kusuri.pikopiko.org,mail.pikopiko.org,nokogiri.pikopiko.org,eigenstate.org); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES256-GCM-SHA384 smtp.bits=256/256; x-vs=clean score=0 state=0 X-ME-VSCause: gggruggvucftvghtrhhoucdtuddrgeduhedrkeehucetufdoteggodetrfdotffvucfrrh hofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfurfetoffk rfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpefkvffufffhjgggtg fgsehtjeejtddttddvnecuhfhrohhmpehorhhisegvihhgvghnshhtrghtvgdrohhrghen ucggtffrrghtthgvrhhnpeefgfefgeekleeijefggeejueetveekgedvudelffdvvdeite eihfejgffggfeuleenucffohhmrghinhepghhithhhuhgsrdgtohhmnecukfhppedvtdei rdduvdegrddufedvrddutdejpdduiedvrdekfedrudefvddrvdegheenucevlhhushhtvg hrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpedvtdeirdduvdegrddufedvrddutdej pdhhvghlohepmhhimhhirhdrvghighgvnhhsthgrthgvrdhorhhgpdhmrghilhhfrhhomh epoehorhhisegvihhgvghnshhtrghtvgdrohhrgheq X-ME-VSScore: 0 X-ME-VSCategory: clean Received-SPF: pass (eigenstate.org: 206.124.132.107 is authorized to use 'ori@eigenstate.org' in 'mfrom' identity (mechanism 'mx' matched)) receiver=tb-mx1.topicbox.com; identity=mailfrom; envelope-from="ori@eigenstate.org"; helo=mimir.eigenstate.org; client-ip=206.124.132.107 Received: from mimir.eigenstate.org (mimir.eigenstate.org [206.124.132.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tb-mx1.topicbox.com (Postfix) with ESMTPS for <9fans@9fans.net>; Sun, 10 May 2020 15:04:25 -0400 (EDT) (envelope-from ori@eigenstate.org) Received: from abbatoir.fios-router.home (pool-162-83-132-245.nycmny.fios.verizon.net [162.83.132.245]) by mimir.eigenstate.org (OpenSMTPD) with ESMTPSA id eeea999d (TLSv1.2:ECDHE-RSA-AES256-SHA:256:NO); Sun, 10 May 2020 12:04:24 -0700 (PDT) Message-ID: To: staal1978@gmail.com, 9fans@9fans.net Subject: Re: [9fans] Software preservation in the post-hg era Date: Sun, 10 May 2020 12:04:23 -0700 From: ori@eigenstate.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Topicbox-Policy-Reasoning: allow: sender is a member Topicbox-Message-UUID: 13c0cf50-92f1-11ea-9db4-dcd2998714c7 > Den tors 7 maj 2020 16:17Dave MacFarlane skrev: > >> >> On Mon, Mar 30, 2020 at 9:12 PM Sean Hinchee wrote: >> >>> As a footnote, there's a decent git client written in Go that works >>> alright on plan9 [4], but it's slow and memory intensive at the >>> moment. >>> >>> >> [...] >> >> [4] https://github.com/driusan/dgit >> >> This (and the fact that the speed of Go on Plan9/amd64 seems to be finally >> be useable enough to do development again as of 1.14..) finally gave me the >> kick I needed to fix some of the hacks that were causing performance >> problems on clone. The self-clone time went from ~160s to ~13s on my >> machine (compared to ~8s with "real" git) If there's other parts that you >> were referring to as being slow and memory intensive let me know (or if you >> still find it's memory intensive, I didn't benchmark that part..) >> >> - Dave >> > > > How does it compare performance wise with git9 ? > > https://github.com/oridb/git9 I'll be honest, I'm using git9 because of the improved interactions, rather than performance -- it's fast enough for most of my usage. Still, this got curious enough to test a bit. Here are the results: It's close for cloning dgit -- I'm seeing about 3 seconds for dgit with git/clone, 4.5 using dgit to clone itself. % time git/clone https://github.com/driusan/dgit 0.81u 1.08s 2.70r (Looking closer, about 1.5 seconds of that comes from the dircp to pull data out of /mnt/git/ and into the working directory.) When testing dgit, I redirected output to /dev/null, since it printed enough that it affected the time. It's *really* chatty -- for the larger test, it produced more than 50 megabytes of status text. cpu% time rc -c 'dgit clone https://github.com/driusan/dgit >[2]/dev/null' 0.47u 0.55s 4.32r It seems like there's something accidentally quadratic, though. Cloning a larger repository -- in this case, perl5 -- takes 160s on git9, and 1200 seconds on dgit. For comparison, git on OpneBSD with different (but comparable) hardware takes about 90 seconds. cpu% time git/clone https://github.com/Perl/perl5.git 94.40u 14.16s 159.30r cpu% time ./dgit clone https://github.com/Perl/perl5.git >[2]/tmp/dgit.log 121.93u 22.16s 1211.30r I only skimmed the dgit code quickly, and didn't see an obvious answer: do you cache objects that you've decompressed, or do you iterate over full delta chains every time? One other bug report -- it seems that dgit hard-codes the default branch as origin/master, but perl uses 'origin/blead', so the checkout fails with 'Could not find origin/master' There are still places where git9 is very slow. Sending lots of commits at once in big repositories stands out. Two reasons for this: we don't deltify, and we walk too much data deciding what should go into the pack. There's also a bug that causes certain kinds of merge to push the whole history spuriously, which is.. only wasteful rather than incorrect -- but wasteful isn't good. Pushing all perl commits to an empty repository, for example: # this is the size of the packfile git gives us cpu% du -sh .git 297.043M .git # pushing to git is slow cpu% git/push -u git+ssh://192.168.1.10/tmp/p5.git 1783.08u 444.15s 2835.86r # and our undeltified packfiles are 10x the size # that they should be $ du -sh p5.git; 4.2Gp5.git I can't compare with dgit, since dgit doesn't support ssh pushes, and I'm not going to set up http pushes right now.