Re: [Caml-list] [community poll for PR#5312] Do some OCaml Windows users still use the @responsefile feature?

Re: [Caml-list] [community poll for PR#5312] Do some OCaml Windows users still use the @responsefile feature?

[Damien Guichard]

Hello gasche,

Actually i didn't even known about the @responsefile feature.
My ocaml 3.12.1 console programs still run under Windows98se, Win98 can no
more compile yet it can run.
So ocaml windows support is quite long-term.
 
I don't mind whether the @responsefile feature is removed or not.
I won't be affected.  

Thanks for the consideration,

- Damien Guichard (SpiceGuid)


Le 05/03/2012 à 11:46:08, Gabriel Scherer <gabriel.scherer@gmail.com> à écrit
:
>In the process of discussing bug #5312, the caml team would like to
>know if people still have use of the @responsefile feature under
>windows. If not, it could be removed from the runtime -- that is from
>all OCaml programs.
>
>  http://caml.inria.fr/mantis/view.php?id=5312
>
>@responsefile is a feature/convention under Windows to provide files
>containing command-line argument options; when a tool parses
>command-line options and encounters a file name prefixed by a '@'
>character, it expands its contents as if it were part of the
>command-line invocation. This is used to circumvent the historically
>fairly ridiculous limit on command-line length in the old 'cmd.com'.
>
>The OCaml toolchain copes with @responsefile in two places -- as far
>as I know, but I'm not familiar with anything Windows. First, when the
>compiler invokes external tools (linkers, etc.) under Windows, it uses
>a @responsefile if the command-line length exceeds a fixed limit --
>curently 4096, used to be 256 and annoy users.
>
>Second, under Windows only, the OCaml runtime considers @-prefixed
>arguments as responsefile file names, and expands them during its
>initialization phase. This is silently done by the OCaml *runtime*, so
>all OCaml programs are affected; the compilers, but also the user
>programs. Did you know that you shouldn't use '@' in your command-line
>parameters syntax if you want your program to work on Windows?
>
>The first use has been problematic in the past because some of the
>underlying toolchains (Cygwin, mingw...) did not support
>@responsefiles. The second case is now problematic as the @-syntax
>conflicts with the warning-as-error syntax of the compiler: as
>reported by Dmitry Grebeniuk, "-w @a" under windows complains about
>a missing file "a", while it really should mark all warnings as
>errors -- a very bad idea for future compatibility when new warnings
>are added, by the way; don't use that in released OCaml software.
>
>According to our Windows spies, the command-line restrictions are
>nowadays very reasonable: 8K for cmd.com, and 32K internally. Maybe
>the @responsefile feature has outlived its use, and this bug could be
>fixed by simply removing the @-files expansion phase of the runtime.
>
>This change would however affect all user programs, so it should not
>be taken lightly; it could break your programs.
>
>What do OCaml Windows user think? Do you still rely on @reponsefile?
>Please complain if you do -- or your users do -- and don't hesitate to
>pass the question to off-list OCaml Windows users.
>
>Some links:
>  - previous angry discussions about @responsefile:
>      http://caml.inria.fr/mantis/view.php?id=1877
>     
http://caml.inria.fr/pub/ml-archives/caml-list/2001/04/ba5a929cb6f14c1148929855a9b55765.en.html
>     
http://caml.inria.fr/pub/ml-archives/caml-list/2007/08/a3cee429c9fe0dd9181975bc1d44b777.en.html
>     
http://caml.inria.fr/pub/ml-archives/caml-list/2007/08/2e8f9b99ab8c61568b09ce28b5c27cc1.en.html
>  - documentation about the compiler warning options:
>      http://caml.inria.fr/pub/docs/manual-ocaml/manual022.html
>  - a warning against using "-warn a -warn-error a" -- unrelated, but can't
hurt
>     
http://caml.inria.fr/pub/ml-archives/caml-list/2009/11/91883440c8a0481a4233758946e5c3bf.en.html
>
>-- 
>Caml-list mailing list.  Subscription management and archives:
>https://sympa-roc.inria.fr/wws/info/caml-list
>Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[Caml-list] [community poll for PR#5312] Do some OCaml Windows users still use the @responsefile feature?

[Gabriel Scherer]

In the process of discussing bug #5312, the caml team would like to
know if people still have use of the @responsefile feature under
windows. If not, it could be removed from the runtime -- that is from
all OCaml programs.

  http://caml.inria.fr/mantis/view.php?id=5312

@responsefile is a feature/convention under Windows to provide files
containing command-line argument options; when a tool parses
command-line options and encounters a file name prefixed by a '@'
character, it expands its contents as if it were part of the
command-line invocation. This is used to circumvent the historically
fairly ridiculous limit on command-line length in the old 'cmd.com'.

The OCaml toolchain copes with @responsefile in two places -- as far
as I know, but I'm not familiar with anything Windows. First, when the
compiler invokes external tools (linkers, etc.) under Windows, it uses
a @responsefile if the command-line length exceeds a fixed limit --
curently 4096, used to be 256 and annoy users.

Second, under Windows only, the OCaml runtime considers @-prefixed
arguments as responsefile file names, and expands them during its
initialization phase. This is silently done by the OCaml *runtime*, so
all OCaml programs are affected; the compilers, but also the user
programs. Did you know that you shouldn't use '@' in your command-line
parameters syntax if you want your program to work on Windows?

The first use has been problematic in the past because some of the
underlying toolchains (Cygwin, mingw...) did not support
@responsefiles. The second case is now problematic as the @-syntax
conflicts with the warning-as-error syntax of the compiler: as
reported by Dmitry Grebeniuk, "-w @a" under windows complains about
a missing file "a", while it really should mark all warnings as
errors -- a very bad idea for future compatibility when new warnings
are added, by the way; don't use that in released OCaml software.

According to our Windows spies, the command-line restrictions are
nowadays very reasonable: 8K for cmd.com, and 32K internally. Maybe
the @responsefile feature has outlived its use, and this bug could be
fixed by simply removing the @-files expansion phase of the runtime.

This change would however affect all user programs, so it should not
be taken lightly; it could break your programs.

What do OCaml Windows user think? Do you still rely on @reponsefile?
Please complain if you do -- or your users do -- and don't hesitate to
pass the question to off-list OCaml Windows users.

Some links:
  - previous angry discussions about @responsefile:
      http://caml.inria.fr/mantis/view.php?id=1877
      http://caml.inria.fr/pub/ml-archives/caml-list/2001/04/ba5a929cb6f14c1148929855a9b55765.en.html
      http://caml.inria.fr/pub/ml-archives/caml-list/2007/08/a3cee429c9fe0dd9181975bc1d44b777.en.html
      http://caml.inria.fr/pub/ml-archives/caml-list/2007/08/2e8f9b99ab8c61568b09ce28b5c27cc1.en.html
  - documentation about the compiler warning options:
      http://caml.inria.fr/pub/docs/manual-ocaml/manual022.html
  - a warning against using "-warn a -warn-error a" -- unrelated, but can't hurt
      http://caml.inria.fr/pub/ml-archives/caml-list/2009/11/91883440c8a0481a4233758946e5c3bf.en.html

RE: [Caml-list] [community poll for PR#5312] Do some OCaml Windows users still use the @responsefile feature?

[David Allsopp]

Gabriel Scherer wrote:
> In the process of discussing bug #5312, the caml team would like to know
> if people still have use of the @responsefile feature under windows. If
> not, it could be removed from the runtime -- that is from all OCaml
> programs.
> 
>   http://caml.inria.fr/mantis/view.php?id=5312

<snip>

> What do OCaml Windows user think? Do you still rely on @reponsefile?
> Please complain if you do -- or your users do -- and don't hesitate to
> pass the question to off-list OCaml Windows users.

Wasn't aware it existed (in the runtime, I mean). It would seem to be an obsolete feature now - personally, I'd expect to have to implement @responsefile in a program of my own if I had some weird need for a Windows command line utility that accepted arguments over 8K.


David

Re: [Caml-list] [community poll for PR#5312] Do some OCaml Windows users still use the @responsefile feature?

[Romain Bardou]

> Wasn't aware it existed (in the runtime, I mean). It would seem to be an obsolete feature now - personally, I'd expect to have to implement @responsefile in a program of my own if I had some weird need for a Windows command line utility that accepted arguments over 8K.

Same here, I was not aware of its existence.

-- 
Romain Bardou

Re: [Caml-list] [community poll for PR#5312] Do some OCaml Windows users still use the @responsefile feature?

[Adrien]

On 05/03/2012, Gabriel Scherer <gabriel.scherer@gmail.com> wrote:
> In the process of discussing bug #5312, the caml team would like to
> know if people still have use of the @responsefile feature under
> windows. If not, it could be removed from the runtime -- that is from
> all OCaml programs.
>
>   http://caml.inria.fr/mantis/view.php?id=5312
>
> @responsefile is a feature/convention under Windows to provide files
> containing command-line argument options; when a tool parses
> command-line options and encounters a file name prefixed by a '@'
> character, it expands its contents as if it were part of the
> command-line invocation. This is used to circumvent the historically
> fairly ridiculous limit on command-line length in the old 'cmd.com'.
>
> The OCaml toolchain copes with @responsefile in two places -- as far
> as I know, but I'm not familiar with anything Windows. First, when the
> compiler invokes external tools (linkers, etc.) under Windows, it uses
> a @responsefile if the command-line length exceeds a fixed limit --
> curently 4096, used to be 256 and annoy users.
>
> Second, under Windows only, the OCaml runtime considers @-prefixed
> arguments as responsefile file names, and expands them during its
> initialization phase. This is silently done by the OCaml *runtime*, so
> all OCaml programs are affected; the compilers, but also the user
> programs. Did you know that you shouldn't use '@' in your command-line
> parameters syntax if you want your program to work on Windows?
>
> The first use has been problematic in the past because some of the
> underlying toolchains (Cygwin, mingw...) did not support
> @responsefiles. The second case is now problematic as the @-syntax
> conflicts with the warning-as-error syntax of the compiler: as
> reported by Dmitry Grebeniuk, "-w @a" under windows complains about
> a missing file "a", while it really should mark all warnings as
> errors -- a very bad idea for future compatibility when new warnings
> are added, by the way; don't use that in released OCaml software.
>
> According to our Windows spies, the command-line restrictions are
> nowadays very reasonable: 8K for cmd.com, and 32K internally. Maybe
> the @responsefile feature has outlived its use, and this bug could be
> fixed by simply removing the @-files expansion phase of the runtime.

As far as I can see on msdn, 8K for cmd.exe has started with Windows
XP. It was 2047 bytes on Windows 2000.

In other words, do we still want to build on 2000 or 9x/Me? I don't
think that was event supposed to be supported, even without this
@responsefile issue and I think I remember some issues running ocaml
programs on win9x. Moreover, noone is going to be able to support
that; i.e. win9x support has probably already bitrot a lot.

By the way, I had only known of reponse files a pretty long time ago
because Cygwin and/or MSYS didn't handle it.

I guess the only way to make sure noone uses it is to break response
files and wait until someone shouts.

Regards,
Adrien Nader

Re: [Caml-list] [community poll for PR#5312] Do some OCaml Windows users still use the @responsefile feature?

[Jeff Schultz]

On Mon, Mar 05, 2012 at 11:46:08AM +0100, Gabriel Scherer wrote:
> According to our Windows spies, the command-line restrictions are
> nowadays very reasonable: 8K for cmd.com, and 32K internally. Maybe
> the @responsefile feature has outlived its use, and this bug could be
> fixed by simply removing the @-files expansion phase of the runtime.

8K is not much.

> This change would however affect all user programs, so it should not
> be taken lightly; it could break your programs.

> What do OCaml Windows user think? Do you still rely on @reponsefile?
> Please complain if you do -- or your users do -- and don't hesitate to
> pass the question to off-list OCaml Windows users.

I stopped using OCaml for new development *because* of the pain it
caused on MSWIN, so my opinion is probably not very valuable.

It looks to me like you could simply disable the feature for arguments
following -w.  However, if you want to remove the feature, I suggest
that you do so in three stages:

    1.  Disable the @responsefile feature and provide a command line
	flag to reenable it.  Warn if this conflicts with the -w @a
	style of options.
    2.  In a later release, remove the code, but make the flag print
	an appropriate diagnostic.
    3.  In an even later release, remove the flag completely.

The time between the first two steps should be determined by the
response to the first step.

This is the standard evolution: make people notice that you are taking
a feature away, but let them keep using it, followed by taking it
away, but telling them it's gone, followed by pretending it never
existed.

It's good that you're adding the desirable preliminary of asking
first.


    Jeff Schultz

Re: [Caml-list] [community poll for PR#5312] Do some OCaml Windows users still use the @responsefile feature?

[Dmitry Grebeniuk]

Hello.

>    1.  Disable the @responsefile feature and provide a command line
>        flag to reenable it.

  Nice transition scheme, but I'd prefer a flag in
OCAMLRUNPARAM environment variable, if/when
such scheme will be approved/implemented.


  btw, still no problems with disabled @responsefile by the
home-made patch, 8 months.

Re: [Caml-list] [community poll for PR#5312] Do some OCaml Windows users still use the @responsefile feature?

[Gabriel Scherer]

> It looks to me like you could simply disable the feature for arguments
> following -w.

If I understand the situation correctly, this approach is not
currently an option, as the @responsefile expansion is done by the
OCaml runtime -- affects all programs, before they process their
arguments -- while the "-w" exception would only make sense for the
compiler toolchains; there is no reason why an user program (that
currently can use the @responsefile feature) would suddenly stop
expanding @-files after a "-w", an option which may have completely
different semantics in this program command-line interface.

On Mon, Mar 5, 2012 at 2:46 PM, Jeff Schultz <jws@csse.unimelb.edu.au> wrote:
> On Mon, Mar 05, 2012 at 11:46:08AM +0100, Gabriel Scherer wrote:
>> According to our Windows spies, the command-line restrictions are
>> nowadays very reasonable: 8K for cmd.com, and 32K internally. Maybe
>> the @responsefile feature has outlived its use, and this bug could be
>> fixed by simply removing the @-files expansion phase of the runtime.
>
> 8K is not much.
>
>> This change would however affect all user programs, so it should not
>> be taken lightly; it could break your programs.
>
>> What do OCaml Windows user think? Do you still rely on @reponsefile?
>> Please complain if you do -- or your users do -- and don't hesitate to
>> pass the question to off-list OCaml Windows users.
>
> I stopped using OCaml for new development *because* of the pain it
> caused on MSWIN, so my opinion is probably not very valuable.
>
> It looks to me like you could simply disable the feature for arguments
> following -w.  However, if you want to remove the feature, I suggest
> that you do so in three stages:
>
>    1.  Disable the @responsefile feature and provide a command line
>        flag to reenable it.  Warn if this conflicts with the -w @a
>        style of options.
>    2.  In a later release, remove the code, but make the flag print
>        an appropriate diagnostic.
>    3.  In an even later release, remove the flag completely.
>
> The time between the first two steps should be determined by the
> response to the first step.
>
> This is the standard evolution: make people notice that you are taking
> a feature away, but let them keep using it, followed by taking it
> away, but telling them it's gone, followed by pretending it never
> existed.
>
> It's good that you're adding the desirable preliminary of asking
> first.
>
>
>    Jeff Schultz
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa-roc.inria.fr/wws/info/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

Re: [Caml-list] [community poll for PR#5312] Do some OCaml Windows users still use the @responsefile feature?

[Gabriel Scherer]

> One could probably make it work, but it would
> involve modifying the compiler programs to say "turn @-files
> processing off, I'll process them myself" and it seems that it's not
> worth the effort?

I think that could only be done if there was a way to disable @-files
expansion in the runtime (not only for the compiler programs), so in
any case one would need to implement your "step 1".

> In any case, does the staged feature-abandonment proposal make sense?

I'm not the qualified person to answer -- I just wrote and sent this poll
request to lighten the OCaml maintainers work -- but it looks
reasonable as a perfectionist scenario. Due to the lack of workforce,
we may end up with a "reduced effort" solution such as simply removing
the feature and duly noting it in the Changelog. We'll see, and it depends
on what other feedback there is.

By looking at the code (byterun/main.c and asmrun/main.c), it looks
simpler to enable/disable the @-expansion based on CAMLRUNPARAM as
Dimtry suggested, but that's a detail.

Thanks for your feedback!

Regarding your larger remark on Ocaml on Windows, the situation is
unfortunate -- due mostly to the lack of Windows-informed people ready
to step up and help fixing the issues -- but there have been efforts
of people trying to make the situation better. For example, Jonathan
Protzenko has a Windows installer for 3.12 that should work out of the
box -- for the compilers only, of course, this doesn't fix the larger
issue of third-party OCaml software. I'm sure he, and other people
working on OCaml/Windows, would welcome precise feedback on things
that don't work; the bugtracker is the right place to report issues on
the ocaml compilers -- there is a specific "ocaml windows" category --
and for third-party programs you should not hesitate to contact the
authors, or ask the caml-list.
  http://protz.github.com/ocaml-installer/
  http://caml.inria.fr/mantis/my_view_page.php

On Mon, Mar 5, 2012 at 4:10 PM, Jeff Schultz <jws@csse.unimelb.edu.au> wrote:
> On Mon, Mar 05, 2012 at 03:31:55PM +0100, Gabriel Scherer wrote:
>> > It looks to me like you could simply disable the feature for arguments
>> > following -w.
>>
>> If I understand the situation correctly, this approach is not
>> currently an option, as the @responsefile expansion is done by the
>> OCaml runtime -- affects all programs, before they process their
>> arguments -- while the "-w" exception would only make sense for the
>> compiler toolchains; there is no reason why an user program (that
>> currently can use the @responsefile feature) would suddenly stop
>> expanding @-files after a "-w", an option which may have completely
>> different semantics in this program command-line interface.
>
> Oh, that's a pity.  One could probably make it work, but it would
> involve modifying the compiler programs to say "turn @-files
> processing off, I'll process them myself" and it seems that it's not
> worth the effort?
>
> In any case, does the staged feature-abandonment proposal make sense?
>
>
>    Jeff Schultz

Re: [Caml-list] [community poll for PR#5312] Do some OCaml Windows users still use the @responsefile feature?

[Gerd Stolpmann]

> On Mon, Mar 05, 2012 at 11:46:08AM +0100, Gabriel Scherer wrote:
>> According to our Windows spies, the command-line restrictions are
>> nowadays very reasonable: 8K for cmd.com, and 32K internally. Maybe
>> the @responsefile feature has outlived its use, and this bug could be
>> fixed by simply removing the @-files expansion phase of the runtime.
>
> 8K is not much.

I second here, although I never hit the limit on Windows. However,
command-line args of several megabytes are not uncommon for big projects,
especially when you call ocamlc with tons of libraries. Well, I never did
that large projects under Windows, only under Linux.

>> This change would however affect all user programs, so it should not
>> be taken lightly; it could break your programs.
>
>> What do OCaml Windows user think? Do you still rely on @reponsefile?
>> Please complain if you do -- or your users do -- and don't hesitate to
>> pass the question to off-list OCaml Windows users.
>
> I stopped using OCaml for new development *because* of the pain it
> caused on MSWIN, so my opinion is probably not very valuable.
>
> It looks to me like you could simply disable the feature for arguments
> following -w.  However, if you want to remove the feature, I suggest
> that you do so in three stages:

I think a very first shot could be to require that the responsefile must
be given as an absolute path (with drive letter, or as network path). This
will syntactically disambiguate it from most other uses. Normally it
points to the temp folder anyway, given as absolute path, so this
additional requirement will hardly break programs.

A cleaner solution would probably to move the responsefile implementation
up to the Arg module, so that the user (a) sees it, and (b) can enable or
disable it as needed.

Gerd

>
>     1.  Disable the @responsefile feature and provide a command line
> 	flag to reenable it.  Warn if this conflicts with the -w @a
> 	style of options.
>     2.  In a later release, remove the code, but make the flag print
> 	an appropriate diagnostic.
>     3.  In an even later release, remove the flag completely.
>
> The time between the first two steps should be determined by the
> response to the first step.
>
> This is the standard evolution: make people notice that you are taking
> a feature away, but let them keep using it, followed by taking it
> away, but telling them it's gone, followed by pretending it never
> existed.
>
> It's good that you're adding the desirable preliminary of asking
> first.
>
>
>     Jeff Schultz
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa-roc.inria.fr/wws/info/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>
>


-- 
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
*** Searching for new projects! Need consulting for system
*** programming in Ocaml? Gerd Stolpmann can help you.