* Repository/mirrors question @ 2022-01-22 19:15 Jim 2022-01-22 19:24 ` Bart Schaefer 0 siblings, 1 reply; 6+ messages in thread From: Jim @ 2022-01-22 19:15 UTC (permalink / raw) To: devs [-- Attachment #1: Type: text/plain, Size: 749 bytes --] Hi All, Not a git expert, so I'm not sure what I'm seeing. Maybe nothing, if so sorry for the bother. Building a new system and cloned the git repository and(for whatever reason) one of the mirrors(github). I noticed the size of the two was over 15M difference. So I cloned the other mirror. It is only ~1.5M larger. So after some googling found and executed "git count-objects -v -H' which reported a difference in the number of objects in one of the mirrors(gitlab). As I said, not a git expert so don't know if this means anything. Hopefully someone with git expertise can tell me if there is or isn't something amiss or if mirror(s) need an update/maintenance/whatever. Sorry if this is just noise, or my lack of knowledge of git, Jim [-- Attachment #2: Type: text/html, Size: 1010 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Repository/mirrors question 2022-01-22 19:15 Repository/mirrors question Jim @ 2022-01-22 19:24 ` Bart Schaefer 2022-01-23 21:42 ` Jim 0 siblings, 1 reply; 6+ messages in thread From: Bart Schaefer @ 2022-01-22 19:24 UTC (permalink / raw) To: linuxtechguy; +Cc: devs On Sat, Jan 22, 2022 at 11:16 AM Jim <linux.tech.guy@gmail.com> wrote: > > Building a new system and cloned the git repository and(for whatever reason) one of > the mirrors(github). I noticed the size of the two was over 15M difference. I think the difference is in the number of forks. There are 341 forks of the mirror on github and only 4 on the SourceForge original. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Repository/mirrors question 2022-01-22 19:24 ` Bart Schaefer @ 2022-01-23 21:42 ` Jim 2022-01-24 0:14 ` Phil Pennock 0 siblings, 1 reply; 6+ messages in thread From: Jim @ 2022-01-23 21:42 UTC (permalink / raw) To: devs [-- Attachment #1.1: Type: text/plain, Size: 1694 bytes --] On Sat, Jan 22, 2022 at 1:24 PM Bart Schaefer <schaefer@brasslantern.com> wrote: > On Sat, Jan 22, 2022 at 11:16 AM Jim <linux.tech.guy@gmail.com> wrote: > > > > Building a new system and cloned the git repository and(for whatever > reason) one of > > the mirrors(github). I noticed the size of the two was over 15M > difference. > > I think the difference is in the number of forks. There are 341 forks > of the mirror on github and only 4 on the SourceForge original. > Aren't forks just copies of the git repository, once forked then the fork does their own thing. That wouldn't change the repository that the fork was taken from, would it? 'man git' hasn't any references to forks. But it wouldn't be the first time I was wrong. What would concern me more is the object count on gitlab's mirror. git count-objects -v -H ( Take prior to Sunday's commits ) sourceforge: in-pack: 98608 size-pack: 19.53 MiB github: in-pack: 98608 size-pack: 36.25 MiB gitlab: in-pack: 98494 size-pack: 21.13 MiB After Sunday's commits, I did an object comparison between sourceforce's repository and the two mirrors. Ignoring the size difference, the objects in the github mirror match those in sourceforge. But there are differences between the gitlab mirror and sourceforge. There are 124 objects in sourceforge that are not in the gitlab mirror. But more surprising there are 10 objects in the gitlab mirror that are not in sourceforge. Find differences in attachments. Again, I don't know if this is an integrity issue or not. I'll leave that for someone else to determine. I do believe it should be brought to someone's attention in case it is a problem. I appreciate your time, Jim [-- Attachment #1.2: Type: text/html, Size: 2500 bytes --] [-- Attachment #2: diff_gl_sf --] [-- Type: application/octet-stream, Size: 390 bytes --] 231d7cb000b1688b66641bc81d30cbb59a457e 5623335e829cbdfde09ad0beb6a4fa9e4b3861 57a7c5b803794f938b5184abbef6a5fb6dacbf 7f517c62f0e648fd3815edb3072ebb8335809f 94d1af85cab974594fa3c1a758e1a1c53403e0 cd0499762229e6568728c281538017013c49d1 e4f9e81cef4908f159a8e826b1f6a8aba49fd0 eb75bbc67ef0571f832ca97a951e8936ebadff ed333d550ebc71bae245b279821806b5aaf072 f8537764c08f069482bb0731f9f26e90b8ef77 [-- Attachment #3: diff_sf_gl --] [-- Type: application/octet-stream, Size: 4836 bytes --] 00b0261ac12764902bb1cc1077f108426cf9cd 01138d0f2fd80ab8b48c4b85cea4fc4464b9d6 0407ad0e1148df3d38e0ada4312fe6593c710e 047d9e0f2f43d8905402eee6a33957ad4ab0c5 054a32b906c475bf5ff9df16c58220b218499b 0c02283983a61287a013dcb548d0250fc63515 1058f0455e46dcd3519cb681d4e178ab57a2c8 11abe01c1f2247fc078230eec205ba7e6b08d2 11b330b6f101e3a22aaf46d6b43e60ad7f39ef 16cc35596d0f007550d0f3c605562a14b689b8 17342216d8db4c97397ca18c5a3ef43a8ec8f4 1ace54cd1b4e4deaa1e5fe79771c45a940209d 1e8c1f657e7f0c06eda80d6b3fab2218b25ed7 1f4c1e1b3c515dd4cc2e58194bc4321f25a46b 1f82b072f67fdd9c80520ee8c4991879a57d46 20497ffe7a2a4bf836bc6ce4ecb0195b4fe719 21fb7ab8ba9a7d4992a45b4b1033e7ef897154 2533ab86b61bc249fd17ba77a030caf3d7a081 25c543c118c5ade169dfa24a7c8d3076bd3390 2ad7de15aeefb5a90b003f09df68e2b60e5333 2bee09c1f85fc7bdb08b6bdb142a0047ed7ed4 2d6589ba5e088b959657e35fee867d3d72e55b 2f4562c48b915b1ca189c88fac2eadb52cbb1f 3738b53372c1962fbb4e221980026f15011aa1 3866411235405300903fdad4557bf9834d6625 394986d460eaeab6d960bff5f9bc3ba4ad54fa 3c166115841e42c39a5a11fed0a3e79b12d7d1 3c16b9f1a2bfbcc7ce1335eecaea152a3c413f 3dfec62c39d1b1fe6d829445ad3a4d8fe0695e 3f8259d07ca79cf22b3b75e333a64f9dfd0f4d 404daf872d649d6c02af718943d403e003d0fb 4093659ba316e8fb1ba0559457c72ba450ca99 413980a51925aa01ac3091efb3535835d1f32a 46060cb93ceec140de2ae8bc207a5fbebf53fa 47ed968b2c4ad19ed7ec181bc60e85dfca9bd2 4851af43c680d0b72e495fc89fc83fbb210d10 48b9d9608c3d2d45fc6ea524eb242927ac44c5 49bf78dc3096d6ca0c705257a928b1acce3b82 4a6cb2f94c1775cae6b0396d2a0a6c9b583965 4c33b7489f4f814b34b79760d62cfd670949d9 4de57a7e3bdee56c7c3b7f4e0ded7a7d3d7974 4f418c1edb35b7b0d124b9653117abf0919962 50a6161176739e6cc892a47a87a1ac4ff732aa 56faa18c296d972bb05858a2a15a2a1d690160 5aca42a1f058c85206439b6d417d668a4f6795 5cb4504561df0b486e3e140d4c0434516a0f7f 5d543a72260ac9ac6774dd5b2d7505475da4a4 5f59bf710099dcaa8f22729591c2580c052db9 613e4895f7b059558c4ef211189b516dbf903d 6485ece60df85519abe3e9628cf30b82cdfd0d 68e8294a0ed0a24f72d5f6cabb7e24b2960fc1 6ad44859e5550cf006f581a12447295e7b9bc9 6d23e7ce3c8cce5e39df638f5cc01389d89d20 6e1c7ff63c0b59573f5f993e50b63791b7f651 6e64f79ea0942c55152fb0427554208479e454 6eddebad00b1c2603d5b7da9746a6f83d1790e 6ffdfdb2a8d0c9c63de34d537a9cfefcb744dc 72325d2e9e2b44d817161a6a1741434ad8f394 769f3aa3484b6f0df05ca86a024e030a72142a 7afaafa9a0f2d618746c371ee79662ac31061f 7ccb60409c5508410b163e67c1164955724801 7ccd7f1efae7696dc17f6e3e720cd994e90155 8035c93c5bc149e9c3babd48c8c145609e29e0 81e64c002e1b2c1b389e4c2a45684524bdd3e7 8323dadd9dc2dd2246ba01f7d595939d80889f 8374fd003d43d02ca3de8006a832f1b7b5ac4c 86eb982ba89eaf5ae323cff206ae84ad279b9e 87145a4655767cc29dc053be082ede0dfef487 886b5bac75cda19506ba357136200ccf2fe2d4 89e7cefa4d3806f3f469d5ad4b496d130305a6 8d620a97dd7130e40cb6067d0063ff664499ca 8f3e32819d95301a2103fbfa190688879b04d2 8ff68e0e025111088b6ba3977b26a1794a4045 9039499c06d2908572f0e4934ae799aa818bee 915f9d1b95853333e76410d7cd30562385e733 935ae187fef33477a8dfa900bf7a8c59588477 93bcc363d98b74f3ded52fad4ecede930d245a 93ce4e61290d15ee7e53c15a2b8425e20c806c 99e79d25cd1cb1f718a060eadb4bb55080d816 9a1f5e80d018e28cf6760cda7ffe18cc5a2df8 9a3eccf74519effbf494e107744a4d7ebf57bc 9af2bf3c7eba33c1425b2dcd2071a2c5bb1179 9ca4f6e2205e86df0731b46ba2dbffc50c6225 a25afe323abe0183c2a530c5324e7e93426b64 a4945fde48a99974192a9670eae88c5044b983 a6c7ca58c8d7a09126bb5af05b87ddaadf16f3 a7cde2efe7e7d3dfe65661039bc74c7b024496 ae7720ba5e9d659776a3bc03da8f4f4db052d2 b21f218b7f72b0746c9eb5096bc979956d004b b2a40c5b7d7773c7587bfef36b13d2fedafbb5 b661623c9838521ba90d764567532651ed7108 b9289d1277bf71b1b9dea2addbddd36b61f276 ba4cbc7da05ff77a7308d35d10a5d4df4e6678 bd4a197c0df766d84dcaadbd76bc7a8e94a6ce bd99b67f1a2e09dff651f102249c441529d530 be1f7883e4786d506d95d0e80a5188d2c7286f c0078fe2a08b057c4cb635dc8cbb4889a718dd c1125316b07ddc9d51fa032fffc5b09b98f2ac c49011944484608ecbc779c0b20a02274f2db0 c52f0dfb787f2351ce81ec33d3a0db11fc53b7 c534ce950063cb8e606428a7b9b167e366becc c6989a564cfb079b73a1f13a3d2a89ff881bb9 d009a11ac4ae3bf81b1e300d5721f7b48eb883 d0cb2e035f4fa93a2f4851bedb612067ff7f1a d3a85b48fb47034db266728f453eb629b11e1d d4439a9f3a2212d753ece6ef644aed2ec4d333 d4cad44b06062092bf65bd800bf9c75aa4e2d3 d4eb7f7b2e0a3097e01b207131d15fd4b418ce d7e8bf2b2e7f809d4a1c0587efac038ce7a73f dae7f05582c6c8e91501dbea6106636cd76f6c dec73420804cbe74a6cab7aab4ba40fb260757 e394ed33f4af82ee92295da119cd9a5adeadd1 e5c76c0c429f189a1985ff14caa092459f6ab5 ebead7ec64e9c44a679394cadc229732114cee ecce55a3c68b7df2fc4017b65485fe6f07d2fd efb784ff7c14118ef00f605be9ddb284a684c6 efc438c913466747e219618ca768e0d6caba28 f0b641f04f6439e82d768cf916aa68c39ca68e f2a5d0c868e2a2263bbfaf98713f43477ba896 f727a628ea701d9123cf8da0420f3c3fa64972 f74cb2303532c85fc43b098dc85a4965211cd8 f85796ac15c871cf760f6415424f4f96c0a460 fa4c36896e0d89bf4b32388db08fc1a8523dd4 fd3854b4abea8b46a2ad568762d3a710af77d9 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Repository/mirrors question 2022-01-23 21:42 ` Jim @ 2022-01-24 0:14 ` Phil Pennock 2022-01-24 0:32 ` Bart Schaefer 2022-01-24 5:47 ` Daniel Shahaf 0 siblings, 2 replies; 6+ messages in thread From: Phil Pennock @ 2022-01-24 0:14 UTC (permalink / raw) To: linuxtechguy; +Cc: devs On 2022-01-23 at 15:42 -0600, Jim wrote: > Aren't forks just copies of the git repository, once forked then the fork > does > their own thing. That wouldn't change the repository that the fork was taken > from, would it? 'man git' hasn't any references to forks. But it wouldn't > be > the first time I was wrong. What would concern me more is the object count > on gitlab's mirror. I'm going to answer the question and explain a few ways this can happen, but the last section below has the actual explanation of what's happening here: GitLab has one dev branch behind the others. Whoever owns that copy should take a look at why that branch wedged. --- So a fork is not integral to git, it's a feature of the various git forges. There have been a number of interesting stories over the years about the fact that many of the forges did not isolate the content-addressed namespaces (the bits where you go in by object ID), leading to people faking up things to "appear" to be in the original repository. Thus if B forks from A and then adds commits and objects, going via any tree reference or commit in the hosted A won't show those items, and cloning _shouldn't_, normally, but you have been able to use the direct SHA1 of the objects to still reach them via the hosted A. I _think_ GitHub put in some mitigations because they got tired of the games people were playing. That said, often a pull-request, and its objects, _are_ available in the original repository, but not in the normal heads. So a regular `git clone` wont see them, but if you do a `git clone --mirror` then you might see more. Eg, given: [remote "origin"] url = https://github.com/a/b fetch = +refs/heads/*:refs/remotes/origin/* fetch = +refs/pull/*/head:refs/remotes/pr/* then GitHub's extra "pull" area becomes visible under a pseudo-remote of "pr", letting you diff locally and do fancier stuff. Since `--mirror` will set up `fetch = +refs/*:refs/*` you would then see all of the pull-requests from all of the forks show up within the repository. --- The next major difference is in history and what happens as branches get abandoned. `git prune` honors presence in the "reflog" for a certain amount of time, so that you can go back to recently-deleted items. --- And then there's the fact that git transfers "packs" and then the loose objects, and if the pack contains extra items since deleted, then you'll end up with a bunch of now-garbage items in the pack. "git help repack" might be your friend for your own repos, but doesn't help with controlling what the forges are doing. --- In some fresh clones, ReachableObjects CommitGraphCount GitHub 98615 14158 GitLab 98501 14132 SF 98615 14158 So GitLab sees 26 fewer commits in the entire graph than the other two. But is up-to-date. So, we check which branches exist and what they point at, and find one branch discrepancy. The `declarednull` branch is zsh-5.8-401-gf85cb4504 on GH and SF, but zsh-5.8-270-g6bcd04997 on GitLab. So that branch is 131 commits behind, but given the difference in graph commits, I suspect that there has been some merging along the way. You can get a robust API and see where the commits are for each remote using `git ls-remote --heads` for any given remote; that output will be diffable. I cheated and used a porcelain command, which is not guaranteed stable for scripting: git branch -r | grep -v /HEAD | while read B; do printf '%30s\t' $B git describe --tags $B done which had the advantage of letting me see where the commits were relative to tags, so we can see the "401 commits since 5.8" vs the "270 commits since 5.8". The last commit on branch origin/declarednull in the gitlab repo is from 2020-11-28, the last on the other two is from 2021-04-13. -Phil ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Repository/mirrors question 2022-01-24 0:14 ` Phil Pennock @ 2022-01-24 0:32 ` Bart Schaefer 2022-01-24 5:47 ` Daniel Shahaf 1 sibling, 0 replies; 6+ messages in thread From: Bart Schaefer @ 2022-01-24 0:32 UTC (permalink / raw) To: linuxtechguy, devs On Sun, Jan 23, 2022 at 4:15 PM Phil Pennock <zsh-workers+phil.pennock@spodhuis.org> wrote: > > So GitLab sees 26 fewer commits in the entire graph than the other two. > But is up-to-date. So, we check which branches exist and what they > point at, and find one branch discrepancy. > > The `declarednull` branch is zsh-5.8-401-gf85cb4504 on GH and SF, but > zsh-5.8-270-g6bcd04997 on GitLab. > > So that branch is 131 commits behind, but given the difference in graph > commits, I suspect that there has been some merging along the way. Hmm. I created and manipulated that branch on the SF repository, but I do not have accounts on either github or GitLab, so there's nothing I can do about it there. Could this be a side-effect of a forced push? That branch has been abandoned and the relevant changes from it applied to the master branch, so it can probably just be discarded, if that's possible. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Repository/mirrors question 2022-01-24 0:14 ` Phil Pennock 2022-01-24 0:32 ` Bart Schaefer @ 2022-01-24 5:47 ` Daniel Shahaf 1 sibling, 0 replies; 6+ messages in thread From: Daniel Shahaf @ 2022-01-24 5:47 UTC (permalink / raw) To: zsh-workers; +Cc: linuxtechguy Phil Pennock wrote on Sun, Jan 23, 2022 at 19:14:24 -0500: > On 2022-01-23 at 15:42 -0600, Jim wrote: > > Aren't forks just copies of the git repository, once forked then the fork > > does > > their own thing. That wouldn't change the repository that the fork was taken > > from, would it? 'man git' hasn't any references to forks. But it wouldn't > > be > > the first time I was wrong. What would concern me more is the object count > > on gitlab's mirror. > > I'm going to answer the question and explain a few ways this can happen, > but the last section below has the actual explanation of what's > happening here: GitLab has one dev branch behind the others. Whoever > owns that copy should take a look at why that branch wedged. That'd be Oliver and I. When I go to [1] it has a "diverged from upstream" warning label, in red with a ⚠ next to it, and the following mouseover text: "The branch could not be updated automatically because it has diverged from its upstream counterpart.<br>To discard the local changes and overwrite the branch with the upstream version, delete it here and choose 'Update Now' above." That does sound like the fallout of a forced push. Reviewing the branch, everything on it is already on the sf.net copy of the branch. I've done what the mouseover text recommended. The tip of that branch on gitlab is now identical to its tip on sf. Cheers, Daniel [1] https://gitlab.com/zsh-org/zsh/-/branches?state=all&sort=updated_desc&search=declarednull ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-01-24 5:48 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-01-22 19:15 Repository/mirrors question Jim 2022-01-22 19:24 ` Bart Schaefer 2022-01-23 21:42 ` Jim 2022-01-24 0:14 ` Phil Pennock 2022-01-24 0:32 ` Bart Schaefer 2022-01-24 5:47 ` Daniel Shahaf
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).