[Caml-list] [ANN] Win-builds 1.5.0 - fully-bootstrapped free software distribution for Windows

[Caml-list] [ANN] Win-builds 1.5.0 - fully-bootstrapped free software distribution for Windows

[Adrien Nader]

[ This email is posted to several mailing-lists: win-builds', mingw-w64
and the caml-list. ]

Hi,

I am happy to announce the availability of Win-builds 1.5.0 which is the
result of many months of work in order to add new packages, update
available ones and make all operations simpler and more intuitive.

Homepage: http://win-builds.org
Downloads: http://win-builds.org/download.html
Documentation: http://win-builds.org/documentation.html
Support information (IRC, MLs): http://win-builds.org/support.html
Package list (informative): http://win-builds.org/packages.html
Source: http://cgit.notk.org/gitolite/win-builds/

=== Project Description ===

Win-build is best described as a free software distribution for Windows:
the goal isn't to provide dumps of binaries but to build something new,
well-integrated, easy to install and upgrade, focused on stability (but
without being out-of-date either), with a good package coverage and
identical binaries on Windows and Linux.

=== New in 1.5.0 ===

Complaints for the previous version centered around simplicity and
package coverage.
As such, this new version has focused on making it simpler to edit
packages (modifying existing packages or adding new ones) and install
thanks to more automated build steps on Linux and to the re-introduction
of a GUI on Windows.

==== New Packages ====

The package count has risen to 90. This may seem low compared to Linux
distributions but this is enough to build GUI applications using either
the EFLs, Qt and GTK+ (and many more). This makes a lot of sense if you
consider how many packages on Linux make little sense on Windows:
smartd, pciutils, all the X-related ones, KDE and Gnome, motif, mt, ...

==== GUI Package Manager and Installer on Windows ====

The GUI is written using ocaml-efl and is used as the installer on
Windows. The actual executable relies on several dozens of files
(libraries and resources) and is packed into a self-extracting (and
cleaning) executable to make it appear as a single file that can be
readily run[1].
Its build is not documented for this release as a sign that the ocaml
cross-compiler has some quirks[2]. Getting full and upstream support for
cross-compilation is however something I'll be working on
from now on.

==== More Scripted Installation on Linux ====

On the developer side, the component that runs all the build scripts has
been rewritten from shell script to OCaml. This provides a simple
validity check of the build list description and proper ordering of the
build steps, making it safer for people to do their own edits.

To honor the great tradition of OCaml programs taking advantage of
high-level optimizations, doesn't attempt to build several packages in
parallel for a given architecture. It however builds for two
architectures in parallel, thereby providing a reasonable efficiency.

==== Documentation ====

Documentation has been updated and expanded. It finally properly covers
the topic of creating new packages and is therefore now fairly
exhaustive when it comes to the win-builds project.

Over the next few days, the documentation will be expanded again to
cover more topics related to usage on Windows, in particular with IDEs.

=== Post-mortem and Future ===

This release took much longer than planned. Roughly twice as long. This
is due to two factor: adding the GUI for the package manager and
installer which was planned for the subsequent version, and Qt.

The GUI seems to have been worth it with much better download stats
and close to a hundred installations per day (more than +40%), without
additional publicity. [these numbers are guaranteed and not inflated by
anything]

Qt will be dealt with in a dedicated section below.

==== -next repository ====

The work leading to this version, as is most often the case, has made
apparent a number of issues with the development process. The main issue
is probably the time it takes between the addition of a feature request
and the availability of the corresponding package.

This has little to do with the actual packaging or updating work and
almost everything to do with the lack of a repository for the next
version. As such, starting with the first changes after 1.5.0, there
will be a "-next" version that will be regularly updated with packages
built from the latest code and publicly reachable.

==== New contributions: what has been done and what's coming up ====

A number of new contributors and contributions have appeared. Most
changes were fairly late in the development cycle and have therefore not
been merged in the main tree for 1.5.0.  

The code hosting has been moved to a gitolite in order to allow hosting
of branches from others. This is for added convenience and any
publicly-reachable git hosting is fine for merging back to the main
branches.

===== Qt =====

Some packages unfortunately require a lot of work with each new
version. Actually the set of such packages has only one inhabitant: Qt.

I'll be frank: maintaining a Qt that is cross-compiled from Linux for
Windows is probably close to a full-time part-time job, i.e. something
that will take all the time for someone doing part-time on it. My
perception is probably flawed since I've made the initial packaging and
it includes things that have probably never been done before and
definitely deviate from what upstream had in mind (i.e. cross-compile
qmake itself).

It is also unfortunately to be expected that each new version of Qt
breaks something for this packaging (cross-compile to Windows).

As such, additional maintainers are looked for. There is no reason to be
afraid and this is not about abandoning the package unless someone steps
up. This is entirely in order to be able to better track upstream
releases.

==== Security Updates ====

This part has mostly been a failure. The work involved was larger than
expected and was let slip by without bound.

Basically, the work (reading, updating the source, rebuilding, testing,
uploading, announcing) is sometimes too much to do immediately after the
availability of a security fix. With no strict bound and with the need
to spend time on large changes in the build architecture, the updates
have slipped by one day at a time. As usual, Frederick P. Brooks got it
right:
  Q: How does a large software project get to be one year late?
  A: One day at a time.

The larger architecture have been done. There will be others but they
should be less pervasive and shouldn't block working on other tasks for
as much time.

More importantly, at least for this release, the official goal is to
handle most security updates in 3 days at most and let none slip by more
than 8 days. This should be at least as good as most large Linux
distributions. Any security update lagging behind for more than that
should be considered "unnoticed" and a poke warranted.


PS

[1] It obviously takes some time to extract the 25MB or so (8MB
compressed) but it is fast enough and can most probably be made faster.
The main advantage however is that this solves the issue of replacing
files which are in-use: the package manager depends on .dll files and
therefore would have been unable to ever replace these without resorting
to complex tricks.

[2] Perhaps the best proof is the following top-level code:
  let init_count = ref 0
  let init () =
    if !init_count = 0 then (
      do_the_actual_init ();
      incr init_count;
    )
    else
      ()
  let () = init ()

On Windows I had to add an explicit call to the "init" function from
somewhere else in the code.

-- 
Adrien Nader
http://win-builds.org

Re: [Caml-list] [ANN] Win-builds 1.5.0 - fully-bootstrapped free software distribution for Windows

[Christoph Höger]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Interesting!

Maybe this is a stupid question but given that you use OCaml for
implementing and there is still no way to distribute opam packages for
windows, could you not package ocaml/opam for your distribution and
this way import all ocaml packages at once?

Am 28.01.2015 um 11:44 schrieb Adrien Nader:
> [ This email is posted to several mailing-lists: win-builds',
> mingw-w64 and the caml-list. ]
> 
> Hi,
> 
> I am happy to announce the availability of Win-builds 1.5.0 which
> is the result of many months of work in order to add new packages,
> update available ones and make all operations simpler and more
> intuitive.
> 
> Homepage: http://win-builds.org Downloads:
> http://win-builds.org/download.html Documentation:
> http://win-builds.org/documentation.html Support information (IRC,
> MLs): http://win-builds.org/support.html Package list
> (informative): http://win-builds.org/packages.html Source:
> http://cgit.notk.org/gitolite/win-builds/
> 
> === Project Description ===
> 
> Win-build is best described as a free software distribution for
> Windows: the goal isn't to provide dumps of binaries but to build
> something new, well-integrated, easy to install and upgrade,
> focused on stability (but without being out-of-date either), with a
> good package coverage and identical binaries on Windows and Linux.
> 
> === New in 1.5.0 ===
> 
> Complaints for the previous version centered around simplicity and 
> package coverage. As such, this new version has focused on making
> it simpler to edit packages (modifying existing packages or adding
> new ones) and install thanks to more automated build steps on Linux
> and to the re-introduction of a GUI on Windows.
> 
> ==== New Packages ====
> 
> The package count has risen to 90. This may seem low compared to
> Linux distributions but this is enough to build GUI applications
> using either the EFLs, Qt and GTK+ (and many more). This makes a
> lot of sense if you consider how many packages on Linux make little
> sense on Windows: smartd, pciutils, all the X-related ones, KDE and
> Gnome, motif, mt, ...
> 
> ==== GUI Package Manager and Installer on Windows ====
> 
> The GUI is written using ocaml-efl and is used as the installer on 
> Windows. The actual executable relies on several dozens of files 
> (libraries and resources) and is packed into a self-extracting
> (and cleaning) executable to make it appear as a single file that
> can be readily run[1]. Its build is not documented for this release
> as a sign that the ocaml cross-compiler has some quirks[2]. Getting
> full and upstream support for cross-compilation is however
> something I'll be working on from now on.
> 
> ==== More Scripted Installation on Linux ====
> 
> On the developer side, the component that runs all the build
> scripts has been rewritten from shell script to OCaml. This
> provides a simple validity check of the build list description and
> proper ordering of the build steps, making it safer for people to
> do their own edits.
> 
> To honor the great tradition of OCaml programs taking advantage of 
> high-level optimizations, doesn't attempt to build several packages
> in parallel for a given architecture. It however builds for two 
> architectures in parallel, thereby providing a reasonable
> efficiency.
> 
> ==== Documentation ====
> 
> Documentation has been updated and expanded. It finally properly
> covers the topic of creating new packages and is therefore now
> fairly exhaustive when it comes to the win-builds project.
> 
> Over the next few days, the documentation will be expanded again
> to cover more topics related to usage on Windows, in particular
> with IDEs.
> 
> === Post-mortem and Future ===
> 
> This release took much longer than planned. Roughly twice as long.
> This is due to two factor: adding the GUI for the package manager
> and installer which was planned for the subsequent version, and
> Qt.
> 
> The GUI seems to have been worth it with much better download
> stats and close to a hundred installations per day (more than
> +40%), without additional publicity. [these numbers are guaranteed
> and not inflated by anything]
> 
> Qt will be dealt with in a dedicated section below.
> 
> ==== -next repository ====
> 
> The work leading to this version, as is most often the case, has
> made apparent a number of issues with the development process. The
> main issue is probably the time it takes between the addition of a
> feature request and the availability of the corresponding package.
> 
> This has little to do with the actual packaging or updating work
> and almost everything to do with the lack of a repository for the
> next version. As such, starting with the first changes after 1.5.0,
> there will be a "-next" version that will be regularly updated with
> packages built from the latest code and publicly reachable.
> 
> ==== New contributions: what has been done and what's coming up
> ====
> 
> A number of new contributors and contributions have appeared. Most 
> changes were fairly late in the development cycle and have
> therefore not been merged in the main tree for 1.5.0.
> 
> The code hosting has been moved to a gitolite in order to allow
> hosting of branches from others. This is for added convenience and
> any publicly-reachable git hosting is fine for merging back to the
> main branches.
> 
> ===== Qt =====
> 
> Some packages unfortunately require a lot of work with each new 
> version. Actually the set of such packages has only one inhabitant:
> Qt.
> 
> I'll be frank: maintaining a Qt that is cross-compiled from Linux
> for Windows is probably close to a full-time part-time job, i.e.
> something that will take all the time for someone doing part-time
> on it. My perception is probably flawed since I've made the initial
> packaging and it includes things that have probably never been done
> before and definitely deviate from what upstream had in mind (i.e.
> cross-compile qmake itself).
> 
> It is also unfortunately to be expected that each new version of
> Qt breaks something for this packaging (cross-compile to Windows).
> 
> As such, additional maintainers are looked for. There is no reason
> to be afraid and this is not about abandoning the package unless
> someone steps up. This is entirely in order to be able to better
> track upstream releases.
> 
> ==== Security Updates ====
> 
> This part has mostly been a failure. The work involved was larger
> than expected and was let slip by without bound.
> 
> Basically, the work (reading, updating the source, rebuilding,
> testing, uploading, announcing) is sometimes too much to do
> immediately after the availability of a security fix. With no
> strict bound and with the need to spend time on large changes in
> the build architecture, the updates have slipped by one day at a
> time. As usual, Frederick P. Brooks got it right: Q: How does a
> large software project get to be one year late? A: One day at a
> time.
> 
> The larger architecture have been done. There will be others but
> they should be less pervasive and shouldn't block working on other
> tasks for as much time.
> 
> More importantly, at least for this release, the official goal is
> to handle most security updates in 3 days at most and let none slip
> by more than 8 days. This should be at least as good as most large
> Linux distributions. Any security update lagging behind for more
> than that should be considered "unnoticed" and a poke warranted.
> 
> 
> PS
> 
> [1] It obviously takes some time to extract the 25MB or so (8MB 
> compressed) but it is fast enough and can most probably be made
> faster. The main advantage however is that this solves the issue of
> replacing files which are in-use: the package manager depends on
> .dll files and therefore would have been unable to ever replace
> these without resorting to complex tricks.
> 
> [2] Perhaps the best proof is the following top-level code: let
> init_count = ref 0 let init () = if !init_count = 0 then ( 
> do_the_actual_init (); incr init_count; ) else () let () = init ()
> 
> On Windows I had to add an explicit call to the "init" function
> from somewhere else in the code.
> 


- -- 
Christoph Höger

Technische Universität Berlin
Fakultät IV - Elektrotechnik und Informatik
Übersetzerbau und Programmiersprachen

Sekr. TEL12-2, Ernst-Reuter-Platz 7, 10587 Berlin

Tel.: +49 (30) 314-24890
E-Mail: christoph.hoeger@tu-berlin.de
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iEYEARECAAYFAlTIwfgACgkQhMBO4cVSGS/VpwCfbnofOuJM1mvoV6p6X2RtwmeC
92AAoJ7ywnbt6D2QnPPZ06P6EDZk20Lo
=GlOv
-----END PGP SIGNATURE-----

Re: [Caml-list] [ANN] Win-builds 1.5.0 - fully-bootstrapped free software distribution for Windows

[Adrien Nader]

On Wed, Jan 28, 2015, Christoph Höger wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Interesting!
> 
> Maybe this is a stupid question but given that you use OCaml for
> implementing and there is still no way to distribute opam packages for
> windows, could you not package ocaml/opam for your distribution and
> this way import all ocaml packages at once?

I want to make an ocaml cross-compiler actually available in win-builds
for people to use. There is already one in win-builds but it's not built
by default as it has too many (potential) quirks.

In the near future I'll work again on making cross-compilation upstream
for the ocaml compiler. Working on win-builds has also convinced me that
being able to cross-compile a compiler would be useful too. That would
make it possible to have a package that provides an ocaml compiler that
runs on Windows and targets Windows.

As far as opam is concerned, I currently don't intend to do anything
specific for it. The reason is that there's already a lot to do on
things that are not specific to opam and it's not sensible to expand my
TODO list. :) 
Another thing to note is that win-builds provides binary packages and
that doesn't match the usual opam usage. In any case, packages for ocaml
libraries and tools is planned and wished but a realistic schedule would
be in 6 months or more (time to get better ocaml cross-compiler, to test
it and to make more packages).

-- 
Adrien Nader