[ISSUE] Hunt for dead packages & more

[ISSUE] Hunt for dead packages & more

[meator]

[-- Attachment #1: Type: text/plain, Size: 2121 bytes --]

New issue by meator on void-packages repository

https://github.com/void-linux/void-packages/issues/38555

Description:
Hi. I've compiled a list of all distfiles and homepages of all packages in the void-packages GitHub repository. I then tried to test each and every URL Voidlinux has to offer to see whether they are still accessible without problems. Here is a JSON file that represents several days of sending HEAD requests[^1].

[data.json](https://github.com/void-linux/void-packages/files/9290529/data.json.txt)

I've only processed the template files, so packages that were [removed from the repos](https://github.com/void-linux/void-packages/issues/23575) but are still on the mirrors aren't listed in the file. I hope this will help me and others to uncover dead links and other problems with the packages.

The JSON file is a collections of URLs and responses. There are three types of responses:

1. normal response (for example `HTTP/2 200` or `HTTP/1.1 404 Not Found`)
2. `Error`: This means that `curl -Is` had a nonzero exit status (the page probably doesn't exist anymore).
3. `Timed out`: It took more than a minute to get the info.

I will in the near future make a PR that will remove or fix the packages with `Error` or 404 status.

I've also noticed that some `http://` links returned `HTTP/1.1 301 Moved Permanently`. [Repology also doesn't like `http`](https://repology.org/repository/void_x86_64/problems). It might be useful to `https`ify all `http` links which support it to improve security and to make Repology happy. But a bulk `distfiles` and `homepage` change would mean that a lot o packages would have to be rebuilt just to change the metadata of the package. One solution would be to do it but without revbumping the files, but usefulness of this is questionable. What do you think?

[^1]: I've been using `curl -Is | head -1` to do the work so FTP files have `Last-Modified` instead of HTTP response or nothing at all (but that doesn't mean it's bad). The gopher URLs also don't have anything there (there are only two of those URLs: `geomyidae` and `sacc` homepages).

Re: Hunt for dead packages & more

[meator]

[-- Attachment #1: Type: text/plain, Size: 279 bytes --]

New comment by meator on void-packages repository

https://github.com/void-linux/void-packages/issues/38555#issuecomment-1209417495

Comment:
Here is a more `grep` friendly version (beware, CRLF):

[data.txt](https://github.com/void-linux/void-packages/files/9291348/data.txt)

Re: Hunt for dead packages & more

[meator]

[-- Attachment #1: Type: text/plain, Size: 355 bytes --]

New comment by meator on void-packages repository

https://github.com/void-linux/void-packages/issues/38555#issuecomment-1214312738

Comment:
Here is the same data with added `maintainer`:
[data.txt](https://github.com/void-linux/void-packages/files/9332966/data.txt)
[data.json](https://github.com/void-linux/void-packages/files/9332967/data.json.txt)

Re: Hunt for dead packages & more

[classabbyamp]

[-- Attachment #1: Type: text/plain, Size: 247 bytes --]

New comment by classabbyamp on void-packages repository

https://github.com/void-linux/void-packages/issues/38555#issuecomment-1225010238

Comment:
repology has a nice list of similar problems: https://repology.org/repository/void_x86_64/problems

Re: Hunt for dead packages & more

[classabbyamp]

[-- Attachment #1: Type: text/plain, Size: 317 bytes --]

New comment by classabbyamp on void-packages repository

https://github.com/void-linux/void-packages/issues/38555#issuecomment-1225010238

Comment:
repology has a nice list of similar problems: https://repology.org/repository/void_x86_64/problems ([json](https://repology.org/api/v1/repository/void_x86_64/problems))

Re: Hunt for dead packages & more

[meator]

[-- Attachment #1: Type: text/plain, Size: 284 bytes --]

New comment by meator on void-packages repository

https://github.com/void-linux/void-packages/issues/38555#issuecomment-1225327261

Comment:
Repology is great but they (I think) don't include the distfiles which are the main problem. I've included the same link to my first comment.

Re: Hunt for dead packages & more

[github-actions]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

New comment by github-actions[bot] on void-packages repository

https://github.com/void-linux/void-packages/issues/38555#issuecomment-1324457693

Comment:
Issues become stale 90 days after last activity and are closed 14 days after that.  If this issue is still relevant bump it or assign it.

Re: [ISSUE] [CLOSED] Hunt for dead packages & more

[github-actions]

[-- Attachment #1: Type: text/plain, Size: 2124 bytes --]

Closed issue by meator on void-packages repository

https://github.com/void-linux/void-packages/issues/38555

Description:
Hi. I've compiled a list of all distfiles and homepages of all packages in the void-packages GitHub repository. I then tried to test each and every URL Voidlinux has to offer to see whether they are still accessible without problems. Here is a JSON file that represents several days of sending HEAD requests[^1].

[data.json](https://github.com/void-linux/void-packages/files/9290529/data.json.txt)

I've only processed the template files, so packages that were [removed from the repos](https://github.com/void-linux/void-packages/issues/23575) but are still on the mirrors aren't listed in the file. I hope this will help me and others to uncover dead links and other problems with the packages.

The JSON file is a collections of URLs and responses. There are three types of responses:

1. normal response (for example `HTTP/2 200` or `HTTP/1.1 404 Not Found`)
2. `Error`: This means that `curl -Is` had a nonzero exit status (the page probably doesn't exist anymore).
3. `Timed out`: It took more than a minute to get the info.

I will in the near future make a PR that will remove or fix the packages with `Error` or 404 status.

I've also noticed that some `http://` links returned `HTTP/1.1 301 Moved Permanently`. [Repology also doesn't like `http`](https://repology.org/repository/void_x86_64/problems). It might be useful to `https`ify all `http` links which support it to improve security and to make Repology happy. But a bulk `distfiles` and `homepage` change would mean that a lot o packages would have to be rebuilt just to change the metadata of the package. One solution would be to do it but without revbumping the files, but usefulness of this is questionable. What do you think?

[^1]: I've been using `curl -Is | head -1` to do the work so FTP files have `Last-Modified` instead of HTTP response or nothing at all (but that doesn't mean it's bad). The gopher URLs also don't have anything there (there are only two of those URLs: `geomyidae` and `sacc` homepages).