clang completion

clang completion

[Eitan Adler]

Hey all,

I just discovered this feature of clang but unfortunately lack the
time to properly sort this out:

http://blog.llvm.org/2017/09/clang-bash-better-auto-completion-is.html'

∴clang --autocomplete='-tr'
-traditional-cpp Enable some traditional CPP emulation
-trigraphs Process trigraph sequences

∴clang --autocomplete='-std=,c++0'
c++03
c++0x
and so on

It seems easy enough to use. Maybe its time to give clang its own completer?
gcc might also be thinking of doing something similar
https://gcc.gnu.org/ml/gcc/2018-04/msg00148.html ?


-- 
Eitan Adler

Re: clang completion

[Matthew Martin]

On Sat, Jun 30, 2018 at 01:34:02AM -0700, Eitan Adler wrote:
> Hey all,
> 
> I just discovered this feature of clang but unfortunately lack the
> time to properly sort this out:
> 
> http://blog.llvm.org/2017/09/clang-bash-better-auto-completion-is.html'
> 
> ∴clang --autocomplete='-tr'
> -traditional-cpp Enable some traditional CPP emulation
> -trigraphs Process trigraph sequences
> 
> ∴clang --autocomplete='-std=,c++0'
> c++03
> c++0x
> and so on
> 
> It seems easy enough to use. Maybe its time to give clang its own completer?
> gcc might also be thinking of doing something similar
> https://gcc.gnu.org/ml/gcc/2018-04/msg00148.html ?

Proof of concept completer:

#compdef clang

local -a options=(${${${(f)"$(_call_program clang clang --autocomplete=${words[CURRENT]/=/\=,})"}//:/\\:}/$'\t'/:})
compset -P '*='
_describe options options

The issue is I'm not sure what format --autocomplete expects. It seems
to want commas after the first = as in the -std example, but what about
if there's more =s or if there's a comma in an argument?  I didn't find
docs with a quick search nor did I get a response from #llvm. If anyone
knows where --autocomplete docs are, those would be most helpful. I've
avoided reading the bash completer to avoid GPL taint.

I believe there were also concerns that clang cannot be split from gcc
and cc since clang is cc sometimes (for instance on OpenBSD for some
architectures).

- Matthew Martin

Re: clang completion

[Daniel Shahaf]

Matthew Martin wrote on Sat, Jul 07, 2018 at 15:57:51 -0500:
> local -a options=(${${${(f)"$(_call_program clang clang --autocomplete=${words[CURRENT]/=/\=,})"}//:/\\:}/$'\t'/:})

Note that it's not trivial to support _approximate or _correct here, because
the --autocomplete contract is "give me a prefix", not "give me all possible
matches for me to sort out".  That is: clang wants to do the matching itself,
and it doesn't honour zstyle's.

Debian's bts(1) has the same issue in its 'listcachedbugs' subcommand.
I suppose that as an ecosystem we should be trying to get upstreams not to
implement the clang/bts semantics but a "dump all possible values for this
word" semantics.

Cheers,

Daniel
(still hoping for a world in which all commands have
a --output-my-grammar-in-a-well-known,-machine-parseable-format flag)

> compset -P '*='
> _describe options options

Re: clang completion

[Oliver Kiddle]

Matthew wrote:

> I believe there were also concerns that clang cannot be split from gcc
> and cc since clang is cc sometimes (for instance on OpenBSD for some
> architectures).

I don't see a problem there.
cc can be any one of a variety of different C compilers. We just need to
have an _cc that probes cc to see what it is and either dispatches to
_gcc, _clang or falls back on only completing traditional options. Same
goes for completing CFLAGS. We'll also need to try to detect the linker.

Daniel wrote:

> Note that it's not trivial to support _approximate or _correct here, because
> the --autocomplete contract is "give me a prefix", not "give me all possible
> matches for me to sort out".  That is: clang wants to do the matching itself,
> and it doesn't honour zstyle's.

This sort of design tends to follow from the way bash programmable
completion is designed. I'd far prefer to have clear options for listing
all warnings, languages, options etc. Those potentially have wider uses.

We likely can at least extract useful information using this interface.
We still need our own rules for where and how to split up the
command-line. A proof of concept using some basic patterns is below.
There will be further patterns that would be useful to cover along with
much of the other functionality already in _gcc.

> (still hoping for a world in which all commands have
> a --output-my-grammar-in-a-well-known,-machine-parseable-format flag)

I can see the appeal of such a format. But I also have my reservations.
If you look back at compctl and tcsh completions you'll see some of the
limitations that led to taking the simple and pragmatic approach of just
using the scripting language that is already built into the shell. A
formal grammar also tends to be finicky which for a command-line that by
definition is expected to be incomplete is not always helpful. You can
see this if you use _regex_arguments for a non-trivial completion – it
is harder to make it forgiving because you need to account for whatever
may be in the command-line and not just split on key tokens. The grammar
style approach of _regex_arguments may lend itself to certain use cases
but if you review some of our limited uses of it, I think you'll agree
that they aren't especially readable or simple to follow. It is also the
case that most commands follow a fairly familiar pattern in terms of
their grammar as covered by _arguments. _arguments specs can be fairly
concise but it is far from being comprehensive.

I've wondered whether writing bashcompinit wasn't perhaps a mistake and
that the opposite mightn't have been better. Zsh's interface is more
extensible to begin with.

Completion definitions being included with upstream projects also makes
sense in many ways. Except I now have the additional frustration of
numerous outstanding and ignored github pull requests.

Oliver

#compdef clang clang++

local tag pre filt
local -a mat eq

for tag pre filt in \
  directories      '--*-(dir|path)=' '^*'      \
  values           '-*='     '^*'              \
  warnings         '-Wno-'   '^*'              \
  warnings         '-W'      $'-W(\t|?,|no-)*' \
  target-options   '-mno-'   '^*'              \
  target-options   '-m'      '-mno-*'          \
  debug-options    '-g'      '^*'              \
  target-options   '-fno-'   '^*'              \
  language-options '-f'      '-fno-*'          \
  options          '-'       '-[fgmWX]?*'     \
  files            ''        '^*'
do
  if [[ -prefix $~pre ]]; then
    case $tag in
      directories)
	compset -P 1 '*='
	_directories && return
      ;;
      files)
        _files -g "*.([cCmisSoak]|cc|cpp|cxx|ii|k[ih])(-.)" && return
      ;;
      *)
	compset -P 1 '*=' && pre="${IPREFIX},"
	mat=( ${${${${(f)"$(_call_program $tag $words[1] --autocomplete=$pre)"}//:/\\:}%%[[:blank:]]#}:#$~filt} )
	eq=( ${${(M)mat:#[^[:blank:]]##=*}/=$'\t'/:} )
	mat=( ${${mat:#[^[:blank:]]##=*}/$'\t'/:} )
	(( $#mat + $#eq )) && _describe -t $tag ${${tag//-/ }%s} mat -- eq -qS= && return
      ;;
    esac
  fi
done

return 1

Re: clang completion

[Eric Cook]

On 07/08/2018 08:40 AM, Oliver Kiddle wrote:
> Daniel wrote:
> 
>> Note that it's not trivial to support _approximate or _correct here, because
>> the --autocomplete contract is "give me a prefix", not "give me all possible
>> matches for me to sort out".  That is: clang wants to do the matching itself,
>> and it doesn't honour zstyle's.
> 
> This sort of design tends to follow from the way bash programmable
> completion is designed. I'd far prefer to have clear options for listing
> all warnings, languages, options etc. Those potentially have wider uses.
> 

Do you mind further expanding upon this? so people considering this option
can be directed to a location of why this may not be desired for other shells.

Re: clang completion

[Oliver Kiddle]

Eric Cook wrote:
> > This sort of design tends to follow from the way bash programmable
> > completion is designed. I'd far prefer to have clear options for listing
> > all warnings, languages, options etc. Those potentially have wider uses.
> > 
>
> Do you mind further expanding upon this? so people considering this option
> can be directed to a location of why this may not be desired for other shells.

For a useful link to which we can refer people, some tidying and editing
work might be needed.

In bash, a function adds matches to the COMP_REPLY array. You can get
bash to do matching with compgen -W but clang has saved you the trouble:
clang --autocomplete='-fv' will only list options starting with -fv.
Firstly, we don't want matching done for us that way.

gcc has various forms of --help that provide a good example of the sort
of thing I was suggesting, e.g. gcc --help=target, --help=params and
--help=warnings. Perhaps an additional option could force these into a
more parsable match:description format with no headers/footers or word
wrapping.

Where it is useful for a command to parse the command-line for us, I'd
be happy with getting back a list of tags and offsets for how much of
the prefix and suffix should be ignored. So after -std= it might for
example, return standards:5:0 to tell you to complete standards after
stripping 5 characters of prefix and 0 of suffix. For the general case,
it might return more than one of these. We'd want all the tags
documented and they might be something like "files" that we already
complete some other way. The interface for that should not do weird
stuff with commas like clang. One idea would be to pass the word offset
followed by the character offset of the cursor followed by all words
(prefix and suffix). But, I'm not sure how well something like that
would really work in practice.

Oliver

Re: clang completion

[Daniel Shahaf]

Oliver Kiddle wrote on Sun, 08 Jul 2018 23:34 +0200:
> Eric Cook wrote:
> > > This sort of design tends to follow from the way bash programmable
> > > completion is designed. I'd far prefer to have clear options for listing
> > > all warnings, languages, options etc. Those potentially have wider uses.
> > > 
> >
> > Do you mind further expanding upon this? so people considering this option
> > can be directed to a location of why this may not be desired for other shells.
> 
> For a useful link to which we can refer people, some tidying and editing
> work might be needed.
> 
> In bash, a function adds matches to the COMP_REPLY array. You can get
> bash to do matching with compgen -W but clang has saved you the trouble:
> clang --autocomplete='-fv' will only list options starting with -fv.
> Firstly, we don't want matching done for us that way.

... because we (zsh) let the user configure how to do matching.  The
user can decide whether completion would be case-sensitive or not.  The
number of knobs available for the user to tweak is large.  The user may
even implement his own matching process as a special shell function.
Thus, it is not feasible for the command run to do matching for zsh.

> gcc has various forms of --help that provide a good example of the sort
> of thing I was suggesting, e.g. gcc --help=target, --help=params and
> --help=warnings. Perhaps an additional option could force these into a
> more parsable match:description format with no headers/footers or word
> wrapping.
> 
> Where it is useful for a command to parse the command-line for us, I'd
> be happy with getting back a list of tags and offsets for how much of
> the prefix and suffix should be ignored. So after -std= it might for
> example, return standards:5:0 to tell you to complete standards after
> stripping 5 characters of prefix and 0 of suffix. For the general case,
> it might return more than one of these. We'd want all the tags
> documented and they might be something like "files" that we already
> complete some other way. The interface for that should not do weird
> stuff with commas like clang. One idea would be to pass the word offset
> followed by the character offset of the cursor followed by all words
> (prefix and suffix). But, I'm not sure how well something like that
> would really work in practice.

Cheers,

Daniel
(Oliver, thanks for your reply elsethread; I have yet to fully digest it. :))