Github messages for voidlinux
 help / color / mirror / Atom feed
* [PR PATCH] xbps-src: Explain python module dependencies in Manual.md
@ 2020-07-11 17:34 ahesford
  2020-07-11 17:52 ` [PR REVIEW] " ericonr
                   ` (11 more replies)
  0 siblings, 12 replies; 13+ messages in thread
From: ahesford @ 2020-07-11 17:34 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 468 bytes --]

There is a new pull request by ahesford against master on the void-packages repository

https://github.com/ahesford/void-packages pydocs
https://github.com/void-linux/void-packages/pull/23525

xbps-src: Explain python module dependencies in Manual.md
Proper handling of python module dependencies in xbps-src needs some clarification. This is a first attempt.

cc: @ericonr 

A patch file from https://github.com/void-linux/void-packages/pull/23525.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-pydocs-23525.patch --]
[-- Type: text/x-diff, Size: 2785 bytes --]

From 3ccdb9cc5bb4d48a86a80efa44fe20c2384dd6aa Mon Sep 17 00:00:00 2001
From: "Andrew J. Hesford" <ajh@sideband.org>
Date: Sat, 11 Jul 2020 13:31:58 -0400
Subject: [PATCH] xbps-src: Explain python module dependencies in Manual.md

---
 Manual.md | 22 +++++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/Manual.md b/Manual.md
index 10082b68da5..a4bfb7dbcab 100644
--- a/Manual.md
+++ b/Manual.md
@@ -1444,9 +1444,25 @@ Python packages should be built with the `python{,2,3}-module` build style, if p
 This sets some environment variables required to allow cross compilation. Support to allow
 building a python module for multiple versions from a single template is also possible.
 
-To allow cross compilation, the `python-devel` package (for python 2.7) must be added
-to `hostmakedepends` and `makedepends`. If any other python version is also supported,
-for example python3.4, those must also be added as host and target build dependencies.
+If a python package includes a compiled extension, the `python-devel` (for python 2.7) or
+`python3-devel` (for python 3.X) packages should be added to `makedepends`. Generally,
+dependencies on `python-*` or `python3-*` packages should be included in `hostmakedepends` because
+the package build process often requires the host interpreter be run with the necessary packages
+available. In most cases, especially those with pure-python dependencies, it is not necessary to
+duplicate these in `makedepends`. Exceptions are dependencies that provide both a python modul and
+libraries or headers against which extensions will be built (*e.g.*, `python3-numpy`). In these
+cases, the packages should typically be listed in both `hostmakedepends` and `makedepends`.
+
+**NB**: Specifying `python-*` or `python3-*` dependencies in `makedepends` rather than
+`hostmakedepends` may lead to non-reproducible packaging rather than obvious failures because the
+python `setuptools` package will attempt to use `pip` or `easy_install` to fetch missing
+dependencies on the host while building the package. There is currently no way to intercept this
+process and force packaging to fail when such dependencies are incorrectly specified. Care should
+be given when creating python package templates to watch the build process for warnings about
+falling back to the deprecated `EasyInstall` for handling dependencies, as this is a sign that
+`setuptools` is attempting to fetch depencies that should be specified in `hostmakedepends`.
+Another sign of missing `hostmakedepends` is the presence of an `.eggs` directory in `$wrksrc`,
+which is where `setuptools` places any dependencies that it fetches.
 
 The following variables may influence how the python packages are built and configured
 at post-install time:

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PR REVIEW] xbps-src: Explain python module dependencies in Manual.md
  2020-07-11 17:34 [PR PATCH] xbps-src: Explain python module dependencies in Manual.md ahesford
@ 2020-07-11 17:52 ` ericonr
  2020-07-11 17:52 ` ericonr
                   ` (10 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: ericonr @ 2020-07-11 17:52 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 162 bytes --]

New review comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/pull/23525#discussion_r453218216

Comment:
"python module"

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PR REVIEW] xbps-src: Explain python module dependencies in Manual.md
  2020-07-11 17:34 [PR PATCH] xbps-src: Explain python module dependencies in Manual.md ahesford
  2020-07-11 17:52 ` [PR REVIEW] " ericonr
@ 2020-07-11 17:52 ` ericonr
  2020-07-11 19:31 ` Chocimier
                   ` (9 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: ericonr @ 2020-07-11 17:52 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 280 bytes --]

New review comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/pull/23525#discussion_r453218398

Comment:
Can we make it clear that these should be the same dependencies that would also be in `depends`? I think that's what's needed, right?

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: xbps-src: Explain python module dependencies in Manual.md
  2020-07-11 17:34 [PR PATCH] xbps-src: Explain python module dependencies in Manual.md ahesford
  2020-07-11 17:52 ` [PR REVIEW] " ericonr
  2020-07-11 17:52 ` ericonr
@ 2020-07-11 19:31 ` Chocimier
  2020-07-11 19:42 ` [PR REVIEW] " ahesford
                   ` (8 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Chocimier @ 2020-07-11 19:31 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 409 bytes --]

New comment by Chocimier on void-packages repository

https://github.com/void-linux/void-packages/pull/23525#issuecomment-657116934

Comment:
Besides mention of pip, this is mainly repetition of "About the many types of depends variable". Mention of pip is too wordy, something like "If you notice build installing modules, add them to aproppriate `depends` variable. This is a case when ..." should suffice.

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PR REVIEW] xbps-src: Explain python module dependencies in Manual.md
  2020-07-11 17:34 [PR PATCH] xbps-src: Explain python module dependencies in Manual.md ahesford
                   ` (2 preceding siblings ...)
  2020-07-11 19:31 ` Chocimier
@ 2020-07-11 19:42 ` ahesford
  2020-07-11 19:54 ` ahesford
                   ` (7 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: ahesford @ 2020-07-11 19:42 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 417 bytes --]

New review comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/23525#discussion_r453227849

Comment:
The `setuptools` installer recognizes `install_requires`, which should generally be placed in `hostmakedepends`, and `requires`, which should be placed in `depends`. I'm going to rewrite some of this based on comments from @Chocimier to simplify and clarify this point.

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PR REVIEW] xbps-src: Explain python module dependencies in Manual.md
  2020-07-11 17:34 [PR PATCH] xbps-src: Explain python module dependencies in Manual.md ahesford
                   ` (3 preceding siblings ...)
  2020-07-11 19:42 ` [PR REVIEW] " ahesford
@ 2020-07-11 19:54 ` ahesford
  2020-07-11 19:55 ` [PR PATCH] [Updated] " ahesford
                   ` (6 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: ahesford @ 2020-07-11 19:54 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 156 bytes --]

New review comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/23525#discussion_r453228856

Comment:
Obsolete

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PR PATCH] [Updated] xbps-src: Explain python module dependencies in Manual.md
  2020-07-11 17:34 [PR PATCH] xbps-src: Explain python module dependencies in Manual.md ahesford
                   ` (4 preceding siblings ...)
  2020-07-11 19:54 ` ahesford
@ 2020-07-11 19:55 ` ahesford
  2020-07-11 19:56 ` ahesford
                   ` (5 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: ahesford @ 2020-07-11 19:55 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 473 bytes --]

There is an updated pull request by ahesford against master on the void-packages repository

https://github.com/ahesford/void-packages pydocs
https://github.com/void-linux/void-packages/pull/23525

xbps-src: Explain python module dependencies in Manual.md
Proper handling of python module dependencies in xbps-src needs some clarification. This is a first attempt.

cc: @ericonr 

A patch file from https://github.com/void-linux/void-packages/pull/23525.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-pydocs-23525.patch --]
[-- Type: text/x-diff, Size: 2005 bytes --]

From e0e84843edd9f8aec540b223e4bae65c24f8b38a Mon Sep 17 00:00:00 2001
From: "Andrew J. Hesford" <ajh@sideband.org>
Date: Sat, 11 Jul 2020 13:31:58 -0400
Subject: [PATCH] xbps-src: Explain python module dependencies in Manual.md

---
 Manual.md | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/Manual.md b/Manual.md
index 10082b68da5..bdfe9dffb81 100644
--- a/Manual.md
+++ b/Manual.md
@@ -1444,9 +1444,18 @@ Python packages should be built with the `python{,2,3}-module` build style, if p
 This sets some environment variables required to allow cross compilation. Support to allow
 building a python module for multiple versions from a single template is also possible.
 
-To allow cross compilation, the `python-devel` package (for python 2.7) must be added
-to `hostmakedepends` and `makedepends`. If any other python version is also supported,
-for example python3.4, those must also be added as host and target build dependencies.
+Python packages that rely on `python-setuptools` or `python3-setuptools` should generally map
+`install_requires` dependencies in `setup.py` to `depends` in the template, and `setup_requires`
+dependencies to `hostmakedepends` in the template. If a python package includes a compiled
+extension, the `python-devel` (for python 2.7) or `python3-devel` (for python 3.X) packages should
+be added to `makedepends`, as should any python packages that also provide native libraries
+against which the extension will be linked (even if that package is also included in
+`hostmakedepends` to satisfy `setuptools`).
+
+**NB**: Python `setuptools` will attempt to use `pip` or `EasyInstall` to fetch any missing
+dependencies at build time. If you notice warnings about `EasyInstall` deprecation or python eggs
+present in `${wrksrc}/.eggs` after building the package, then those packages should be added to
+`hostmakedepends`.
 
 The following variables may influence how the python packages are built and configured
 at post-install time:

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: xbps-src: Explain python module dependencies in Manual.md
  2020-07-11 17:34 [PR PATCH] xbps-src: Explain python module dependencies in Manual.md ahesford
                   ` (5 preceding siblings ...)
  2020-07-11 19:55 ` [PR PATCH] [Updated] " ahesford
@ 2020-07-11 19:56 ` ahesford
  2020-07-11 20:11 ` [PR REVIEW] " Chocimier
                   ` (4 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: ahesford @ 2020-07-11 19:56 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 185 bytes --]

New comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/23525#issuecomment-657120310

Comment:
Substantially shortened in the latest push.

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PR REVIEW] xbps-src: Explain python module dependencies in Manual.md
  2020-07-11 17:34 [PR PATCH] xbps-src: Explain python module dependencies in Manual.md ahesford
                   ` (6 preceding siblings ...)
  2020-07-11 19:56 ` ahesford
@ 2020-07-11 20:11 ` Chocimier
  2020-07-11 20:35 ` [PR PATCH] [Updated] " ahesford
                   ` (3 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Chocimier @ 2020-07-11 20:11 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 248 bytes --]

New review comment by Chocimier on void-packages repository

https://github.com/void-linux/void-packages/pull/23525#discussion_r453230293

Comment:
While you are here, could you add: ".. `depends` in the template, or `python3` if there are none."?

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PR PATCH] [Updated] xbps-src: Explain python module dependencies in Manual.md
  2020-07-11 17:34 [PR PATCH] xbps-src: Explain python module dependencies in Manual.md ahesford
                   ` (7 preceding siblings ...)
  2020-07-11 20:11 ` [PR REVIEW] " Chocimier
@ 2020-07-11 20:35 ` ahesford
  2020-07-11 20:36 ` ahesford
                   ` (2 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: ahesford @ 2020-07-11 20:35 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 473 bytes --]

There is an updated pull request by ahesford against master on the void-packages repository

https://github.com/ahesford/void-packages pydocs
https://github.com/void-linux/void-packages/pull/23525

xbps-src: Explain python module dependencies in Manual.md
Proper handling of python module dependencies in xbps-src needs some clarification. This is a first attempt.

cc: @ericonr 

A patch file from https://github.com/void-linux/void-packages/pull/23525.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-pydocs-23525.patch --]
[-- Type: text/x-diff, Size: 1998 bytes --]

From 4933c887a504f505adf51e33239be7417dcbc4d9 Mon Sep 17 00:00:00 2001
From: "Andrew J. Hesford" <ajh@sideband.org>
Date: Sat, 11 Jul 2020 13:31:58 -0400
Subject: [PATCH] xbps-src: Explain python module dependencies in Manual.md

---
 Manual.md | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/Manual.md b/Manual.md
index 10082b68da5..31babf14d6a 100644
--- a/Manual.md
+++ b/Manual.md
@@ -1444,9 +1444,18 @@ Python packages should be built with the `python{,2,3}-module` build style, if p
 This sets some environment variables required to allow cross compilation. Support to allow
 building a python module for multiple versions from a single template is also possible.
 
-To allow cross compilation, the `python-devel` package (for python 2.7) must be added
-to `hostmakedepends` and `makedepends`. If any other python version is also supported,
-for example python3.4, those must also be added as host and target build dependencies.
+Python packages that rely on `python3-setuptools` should generally map `setup_requires`
+dependencies in `setup.py` to `hostmakedepends` in the template and `install_requires`
+dependencies to `depends` in the template; include `python3` in `depends` if there are no other
+python dependencies. If the package includes a compiled extension, the `python3-devel` packages
+should be added to `makedepends`, as should any python packages that also provide native libraries
+against which the extension will be linked (even if that package is also included in
+`hostmakedepends` to satisfy `setuptools`).
+
+**NB**: Python `setuptools` will attempt to use `pip` or `EasyInstall` to fetch any missing
+dependencies at build time. If you notice warnings about `EasyInstall` deprecation or python eggs
+present in `${wrksrc}/.eggs` after building the package, then those packages should be added to
+`hostmakedepends`.
 
 The following variables may influence how the python packages are built and configured
 at post-install time:

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: xbps-src: Explain python module dependencies in Manual.md
  2020-07-11 17:34 [PR PATCH] xbps-src: Explain python module dependencies in Manual.md ahesford
                   ` (8 preceding siblings ...)
  2020-07-11 20:35 ` [PR PATCH] [Updated] " ahesford
@ 2020-07-11 20:36 ` ahesford
  2020-07-11 21:59 ` [PR PATCH] [Closed]: " ahesford
  2020-07-11 21:59 ` ahesford
  11 siblings, 0 replies; 13+ messages in thread
From: ahesford @ 2020-07-11 20:36 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 243 bytes --]

New comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/23525#issuecomment-657125736

Comment:
Removed the python2 mentions to keep the text a little cleaner and addressed request from @Chocimier.

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: xbps-src: Explain python module dependencies in Manual.md
  2020-07-11 17:34 [PR PATCH] xbps-src: Explain python module dependencies in Manual.md ahesford
                   ` (10 preceding siblings ...)
  2020-07-11 21:59 ` [PR PATCH] [Closed]: " ahesford
@ 2020-07-11 21:59 ` ahesford
  11 siblings, 0 replies; 13+ messages in thread
From: ahesford @ 2020-07-11 21:59 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 192 bytes --]

New comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/23525#issuecomment-657136575

Comment:
Closed in ff61ec45aeba3b19ba25a782037bc80e5a6e2182

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PR PATCH] [Closed]: xbps-src: Explain python module dependencies in Manual.md
  2020-07-11 17:34 [PR PATCH] xbps-src: Explain python module dependencies in Manual.md ahesford
                   ` (9 preceding siblings ...)
  2020-07-11 20:36 ` ahesford
@ 2020-07-11 21:59 ` ahesford
  2020-07-11 21:59 ` ahesford
  11 siblings, 0 replies; 13+ messages in thread
From: ahesford @ 2020-07-11 21:59 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 316 bytes --]

There's a closed pull request on the void-packages repository

xbps-src: Explain python module dependencies in Manual.md
https://github.com/void-linux/void-packages/pull/23525

Description:
Proper handling of python module dependencies in xbps-src needs some clarification. This is a first attempt.

cc: @ericonr 

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2020-07-11 21:59 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-11 17:34 [PR PATCH] xbps-src: Explain python module dependencies in Manual.md ahesford
2020-07-11 17:52 ` [PR REVIEW] " ericonr
2020-07-11 17:52 ` ericonr
2020-07-11 19:31 ` Chocimier
2020-07-11 19:42 ` [PR REVIEW] " ahesford
2020-07-11 19:54 ` ahesford
2020-07-11 19:55 ` [PR PATCH] [Updated] " ahesford
2020-07-11 19:56 ` ahesford
2020-07-11 20:11 ` [PR REVIEW] " Chocimier
2020-07-11 20:35 ` [PR PATCH] [Updated] " ahesford
2020-07-11 20:36 ` ahesford
2020-07-11 21:59 ` [PR PATCH] [Closed]: " ahesford
2020-07-11 21:59 ` ahesford

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).