Vim yodl syntax highlighting

Vim yodl syntax highlighting

[Daniel Shahaf]

[-- Attachment #1: Type: text/plain, Size: 956 bytes --]

I've sketched a Vim syntax file for zsh yodl files.  Not perfect yet but
better than no highlighting at all.

At this time there are two known issues:

- Parentheses that aren't part of a yodl macro confuse the parsing; example:
    chapter(foo (bar) baz)

- Macros nested within an underlined thing don't get underlined; example:
    sitem(foo)(bar tt(baz))

---

To test this, source it after opening a yodl file.

To install it permanently, save the attachment as ~/.vim/syntax/zyodl.vim
and put
    autocmd! BufRead,BufNewFile **/Doc/Zsh/*.yo	setfiletype zyodl
somewhere.  Putting it in vimrc is good enough, see ':help new-filetype'
for the official way.

---

I think all these macros are zsh specific (even 'chapter' and 'bf' are
defined in ztexi.yo), as opposed to yodl builtins.

Should this live in the zsh repository?  If it's useful to people it
should live alongside the code, but if it's just me, I'll throw it on my
personal github account.

[-- Attachment #2: zyodl.vim --]
[-- Type: text/plain, Size: 2335 bytes --]


"" Test case:
" if 0
" i
"   texinode()()()()
"   chapter(foo)
"   chapter(foo (foo) foo)
"   vindex(foo) 
"   foo tt(foo) var(foo) bf(foo) em(foo) foo
"   xitem(foo)
"   item(foo)(foo)
"   sitem(foo)(foo tt(foo) foo)
"   example(print *.c+LPAR()#q:s/#%+LPAR()#b+RPAR()s+LPAR()*+RPAR().c/'S${match[1]}.C'/+RPAR())
"   ifzman(zmanref(zshmisc))ifnzman(noderef(Redirection))
"   LPAR()foo 42 foo+RPAR()
" .
" endif

"" Syntax groups:
syn clear
syn cluster zyodlInline contains=zyodlTt,zyodlVar,zyodlBold,zyodlEmph,zyodlCond
syn region zyodlTt      start="\<tt("      end=")" contains=zyodlSpecial
syn region zyodlVar     start="\<var("     end=")" contains=zyodlSpecial
syn region zyodlBold    start="\<bf("      end=")" contains=zyodlSpecial
syn region zyodlEmph    start="\<em("      end=")" contains=zyodlSpecial
syn region zyodlIndex   start="\<.index("  end=")" contains=zyodlSpecial
syn match  zyodlSpecial "+\?\<\(LPAR\|RPAR\|PLUS\)()"
syn match  zyodlNumber  "\d\+"
syn region zyodlItem    start="\<xitem(" end=")" contains=zyodlSpecial,@zyodlInline
syn region zyodlItem    start="\<item("  end=")" contains=zyodlSpecial,@zyodlInline
syn region zyodlExample start="\<example(" end=")" contains=zyodlSpecial
syn region zyodlTitle   start="\<\(chapter\|subsect\|sect\)(" end=")" contains=zyodlSpecial,@zyodlInline
syn match  zyodlTitle   "^texinode\%x28.*$"

syn region zyodlCond    start="\<\(ifzman\|ifnzman\)(" end=")" contains=zyodlRef,zyodlSpecial,@zyodlInline
syn region zyodlRef     start="\<\(zmanref\|noderef\)(" end=")"

syn keyword zyodlKeyword sitem nextgroup=zyodlSItemArg1
syn region zyodlSItemArg1 oneline start="(" end=")" contains=zyodlSpecial,@zyodlInline nextgroup=zyodlSItemArg2 contained
syn region zyodlSItemArg2 start="(" end=")" contains=zyodlSpecial,@zyodlInline contained

"" Highlight groups:
hi def link zyodlTt Constant
hi def link zyodlVar Identifier
" Not ':hi def link zyodlBold Bold' since there's no such group.
hi def zyodlBold gui=bold cterm=bold
hi def link zyodlEmph Type
hi def link zyodlIndex Comment
hi def link zyodlSpecial Special
hi def link zyodlNumber Number
hi def link zyodlItem Keyword
hi def link zyodlExample String
hi def link zyodlTitle Title
hi def link zyodlCond Conditional
hi def link zyodlRef Include
hi def link zyodlSItemArg1 Macro
hi def link zyodlSItemArg2 Underlined

Re: Vim yodl syntax highlighting

[Jason Spiro]

On Mon, Jul 6, 2015 at 10:12 AM, Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
> I've sketched a Vim syntax file for zsh yodl files.  Not perfect yet but
> better than no highlighting at all.
>
> At this time there are two known issues:
>
> [...]
>
> Should this live in the zsh repository?  If it's useful to people it
> should live alongside the code, but if it's just me, I'll throw it on my
> personal github account.

Thank you for creating this!

Perhaps it'd be wisest to get your new syntax file included with Vim?
That way, nobody will ever have to install it; it will be already
installed for them.

Ordinary Vim (but not vim-tiny) already includes an enormous number of
syntax files.

You can submit your file to:  https://code.google.com/p/vim/issues/entry

-- 
Jason Spiro:  computer consultant.
Improve your kids' or your workers' productivity.  If you'd like
an Internet filter installed in your home or workplace, contact me today.
+1 (416) 992-3445 / <http://www.jspiro.com/>.

Re: Vim yodl syntax highlighting

[Jason Spiro]

On Mon, Jul 6, 2015 at 2:46 PM, Jason Spiro <jasonspiro4@gmail.com> wrote:
> On Mon, Jul 6, 2015 at 10:12 AM, Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
>> I've sketched a Vim syntax file for zsh yodl files.  Not perfect yet but
>> better than no highlighting at all.
>> [...]
> You can submit your file to:  https://code.google.com/p/vim/issues/entry

Oh.  I just reread what you wrote, and now I actually understand it.
These are the _zsh_ flavor of yodl files.

Please ignore my previous email.  My mistake.

Re: Vim yodl syntax highlighting

[Daniel Shahaf]

Jason Spiro wrote on Mon, Jul 06, 2015 at 14:46:24 -0400:
> On Mon, Jul 6, 2015 at 10:12 AM, Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
> > I've sketched a Vim syntax file for zsh yodl files.  Not perfect yet but
> > better than no highlighting at all.
> >
> > At this time there are two known issues:
> >
> > [...]
> >
> > Should this live in the zsh repository?  If it's useful to people it
> > should live alongside the code, but if it's just me, I'll throw it on my
> > personal github account.
> 
> Thank you for creating this!
> 

Welcome!

> > You can submit your file to:  https://code.google.com/p/vim/issues/entry
> 
> Oh.  I just reread what you wrote, and now I actually understand it.
> These are the _zsh_ flavor of yodl files.

Yes, that's correct.  The syntax file only handles the subset of yodl
used by Doc/Zsh/*.yo.  It doesn't attempt to handle a "generally useful"
subset of yodl (a quick check says there are many yodl macros that the
zsh manual doesn't use).

Perhaps I should advertise the zyodl.vim's existence to the yodl-users@
list so that someone might pick it up and build a generic yodl.vim on
top of it.  The problem with this plan is, there's no such list ☹

Cheers,

Daniel

Re: Vim yodl syntax highlighting

[Bart Schaefer]

On Jul 6,  2:12pm, Daniel Shahaf wrote:
}
} Should this live in the zsh repository?  If it's useful to people it
} should live alongside the code, but if it's just me, I'll throw it on my
} personal github account.

There are presently

    .editorconfig
    Src{,/{Modules,Builtins,Zle}}/.exrc

already in the distribution, so there would be some precedent for putting
this in Doc/ if there were a reasonably portable way to make it load (or
not) as required/supported by the local vi-alike.

I have an ancient but still functional yodl-mode.el around, somewhere.

Re: Vim yodl syntax highlighting

[Daniel Shahaf]

Bart Schaefer wrote on Mon, Jul 06, 2015 at 15:31:58 -0700:
> On Jul 6,  2:12pm, Daniel Shahaf wrote:
> }
> } Should this live in the zsh repository?  If it's useful to people it
> } should live alongside the code, but if it's just me, I'll throw it on my
> } personal github account.
> 
> There are presently
> 
>     .editorconfig
>     Src{,/{Modules,Builtins,Zle}}/.exrc
> 
> already in the distribution, so there would be some precedent for putting
> this in Doc/ if there were a reasonably portable way to make it load (or
> not) as required/supported by the local vi-alike.

What's your concern — portability (not causing errors for people who use
other vi-like editors) or usability (automatically enabling the syntax
highlighting)?

Compatibility with vi isn't an issue: these are Vim scripts, vi won't
see them and won't try to execute them in the first place.

Compatibility with Vim builds that don't support syntax highlighting...
I think this can be made to work.

Automatically enabling syntax highlighting: I don't believe Vim has
built-in support for letting a directory tree specify its own syntax
highlighting, so I assume _some_ opt-in step would be needed.  We could
make this as frictionless as possible, for example, by putting the file in
    Doc/editorconfig/vim/syntax/zyodl.vim
and having a
    Doc/editorconfig/vim/filetype.vim
that installs filetype-detection for the zyodl filetype; this way,
people will be able to get syntax highlighting simply by adding
Doc/editorconfig/vim/ to their 'runtimepath' setting (aka, a one-line
~/.vimrc change).

Perhaps there's a better way; I'm open to suggestions from the audience.

> I have an ancient but still functional yodl-mode.el around, somewhere.

Obviously we can't have _that_ in the repository :-)

Re: Vim yodl syntax highlighting

[Bart Schaefer]

On Jul 10,  6:14pm, Daniel Shahaf wrote:
}
} > [...] there would be some precedent for putting
} > this in Doc/ if there were a reasonably portable way to make it load
} > (or not) as required/supported by the local vi-alike.
} 
} What's your concern - portability (not causing errors for people who use
} other vi-like editors) or usability (automatically enabling the syntax
} highlighting)?

Portability, to use your parenthetical definition thereof.

} Automatically enabling syntax highlighting: I don't believe Vim has
} built-in support for letting a directory tree specify its own syntax
} highlighting, so I assume _some_ opt-in step would be needed.  We could
} make this as frictionless as possible, for example, by putting the file in
}     Doc/editorconfig/vim/syntax/zyodl.vim

In case not clear, it's not Doc/editorconfig/, it's a Doc/.editorconfig
file intended to be loaded by editors that use that common configuration
method.  I'm not in favor of adding a (non-hidden, as in not a dotfile)
tree of stuff under Doc/ that is not part of the actual documentation.

(That's a lot of nots.)

If you can't jam it all in a .vimrc or the like, it should probably go
in Util/ rather than Doc/.

[PATCH] Re: Vim yodl syntax highlighting

[Daniel Shahaf]

[-- Attachment #1: Type: text/plain, Size: 346 bytes --]

Bart Schaefer wrote on Fri, Jul 10, 2015 at 17:01:34 -0700:
> If you can't jam it all in a .vimrc or the like, it should probably go
> in Util/ rather than Doc/.

Attached.  It's autoloading for those people who auto-source
$PWD/.vimrc, and there are instructions for everyone else.
(The "install permanently" part could be further streamlined.)

[-- Attachment #2: 0001-New-zsh-specific-yodl-vim-syntax-highlighting.patch --]
[-- Type: text/x-patch, Size: 4720 bytes --]

>From 4d90d8bf5a8ca51879fe3a6f2ca53969668cf349 Mon Sep 17 00:00:00 2001
From: Daniel Shahaf <d.s@daniel.shahaf.name>
Date: Sat, 11 Jul 2015 09:18:20 +0000
Subject: [PATCH] New zsh-specific yodl vim syntax highlighting.

To try this, see the instructions at the top of Util/zyodl.vim (in the comment).

* Util/zyodl.vim: New syntax highlighting file.
* Doc/Zsh/.vimrc: New file, hooks up zyodl.vim for automatic loading.
* Etc/zsh-development-guide: Namedrop zyodl.vim for discoverability.
---
 Doc/Zsh/.vimrc            |  4 +++
 Etc/zsh-development-guide |  3 ++
 Util/zyodl.vim            | 74 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 81 insertions(+)
 create mode 100644 Doc/Zsh/.vimrc
 create mode 100644 Util/zyodl.vim

diff --git a/Doc/Zsh/.vimrc b/Doc/Zsh/.vimrc
new file mode 100644
index 0000000..6544cd4
--- /dev/null
+++ b/Doc/Zsh/.vimrc
@@ -0,0 +1,4 @@
+augroup filetypedetect
+  autocmd BufRead,BufNewFile **/Doc/Zsh/*.yo set ft=zyodl | source <sfile>:h:h:h/Util/zyodl.vim
+  "autocmd BufRead,BufNewFile **/Etc/FAQ.yo  set ft=zyodl | source <sfile>:h:h:h/Util/zyodl.vim
+augroup END
diff --git a/Etc/zsh-development-guide b/Etc/zsh-development-guide
index d92b724..4d6cefd 100644
--- a/Etc/zsh-development-guide
+++ b/Etc/zsh-development-guide
@@ -920,6 +920,9 @@ Documentation
   surrounding text.  No extra newlines after the opening or before the
   closing parenthesis are required.
 
+A syntax highlighting file for Vim is included, just source tt(Doc/Zsh/.vimrc)
+before editing one of the Doc/Zsh/*.yo files.
+
 Module names
 ------------
 
diff --git a/Util/zyodl.vim b/Util/zyodl.vim
new file mode 100644
index 0000000..db2a599
--- /dev/null
+++ b/Util/zyodl.vim
@@ -0,0 +1,74 @@
+
+"" A Vim syntax highlighting file for Doc/Zsh/*.yo
+
+" To try this, run:
+"     cd Doc/Zsh && vim --cmd "source ./.vimrc" zle.yo
+" (This sources the file <Doc/Zsh/.vimrc>.)
+"
+" To install this permanently:
+" 1. Copy this file to ~/.vim/syntax/zyodl.vim
+" 2. Create ~/.vim/filetype.vim as explained in ":help new-filetype" case C.
+" 3. Add the following command to ~/.vim/filetype.vim:
+"      autocmd BufRead,BufNewFile **/Doc/Zsh/*.yo setfiletype zyodl
+
+"" Test case:
+"   texinode()()()()
+"   chapter(foo)
+"   vindex(foo) 
+"   foo tt(foo) var(foo) bf(foo) em(foo) foo
+"   xitem(foo)
+"   item(foo)(foo)
+"   sitem(foo)(foo foo)
+"   example(print *.c+LPAR()#q:s/#%+LPAR()#b+RPAR()s+LPAR()*+RPAR().c/'S${match[1]}.C'/+RPAR())
+"   ifzman(zmanref(zshmisc))ifnzman(noderef(Redirection))
+"   LPAR()foo 42 foo+RPAR()
+"
+"   chapter(foo (foo) foo) # nested parentheses
+"   sitem(foo)(foo tt(foo) foo) # nested underline
+
+if exists("b:current_syntax")
+  finish
+endif
+
+"" Syntax groups:
+syn clear
+syn cluster zyodlInline contains=zyodlTt,zyodlVar,zyodlBold,zyodlEmph,zyodlCond
+syn region zyodlTt      start="\<tt("      end=")" contains=zyodlSpecial
+syn region zyodlVar     start="\<var("     end=")" contains=zyodlSpecial
+syn region zyodlBold    start="\<bf("      end=")" contains=zyodlSpecial
+syn region zyodlEmph    start="\<em("      end=")" contains=zyodlSpecial
+syn region zyodlIndex   start="\<.index("  end=")" contains=zyodlSpecial
+syn match  zyodlSpecial "+\?\<\(LPAR\|RPAR\|PLUS\)()"
+syn match  zyodlNumber  "\d\+"
+syn region zyodlItem    start="\<xitem(" end=")" contains=zyodlSpecial,@zyodlInline
+syn region zyodlItem    start="\<item("  end=")" contains=zyodlSpecial,@zyodlInline
+syn region zyodlExample start="\<example(" end=")" contains=zyodlSpecial
+syn region zyodlTitle   start="\<\(chapter\|subsect\|sect\)(" end=")" contains=zyodlSpecial,@zyodlInline
+syn match  zyodlTitle   "^texinode\%x28.*$"
+
+syn region zyodlCond    start="\<\(ifzman\|ifnzman\)(" end=")" contains=zyodlRef,zyodlSpecial,@zyodlInline
+syn region zyodlRef     start="\<\(zmanref\|noderef\)(" end=")"
+
+syn keyword zyodlKeyword sitem nextgroup=zyodlSItemArg1
+syn region zyodlSItemArg1 oneline start="(" end=")" contains=zyodlSpecial,@zyodlInline nextgroup=zyodlSItemArg2 contained
+syn region zyodlSItemArg2 start="(" end=")" contains=zyodlSpecial,@zyodlInline contained
+
+"" Highlight groups:
+hi def link zyodlTt Constant
+hi def link zyodlVar Identifier
+" Not ':hi def link zyodlBold Bold' since there's no such group.
+hi def zyodlBold gui=bold cterm=bold
+hi def link zyodlEmph Type
+hi def link zyodlIndex Comment
+hi def link zyodlSpecial Special
+hi def link zyodlNumber Number
+hi def link zyodlItem Keyword
+hi def link zyodlExample String
+hi def link zyodlTitle Title
+hi def link zyodlCond Conditional
+hi def link zyodlRef Include
+hi def link zyodlSItemArg1 Macro
+hi def link zyodlSItemArg2 Underlined
+
+let b:current_syntax = "zyodl"
+
-- 
1.9.1

Re: [PATCH] Re: Vim yodl syntax highlighting

[Bart Schaefer]

On Jul 12,  6:51am, Daniel Shahaf wrote:
}
} Bart Schaefer wrote on Fri, Jul 10, 2015 at 17:01:34 -0700:
} > If you can't jam it all in a .vimrc or the like, it should probably go
} > in Util/ rather than Doc/.
} 
} Attached.  It's autoloading for those people who auto-source
} $PWD/.vimrc, and there are instructions for everyone else.
} (The "install permanently" part could be further streamlined.)

Hmm, maybe I just have a really old vim, but I get

Error detected while processing Util/zyodl.vim:
line   47:
E71: Invalid character after \%
E475: Invalid argument: zyodlTitle   "^texinode\%x28.*$"

Re: [PATCH] Re: Vim yodl syntax highlighting

[Daniel Shahaf]

Bart Schaefer wrote on Mon, Jul 13, 2015 at 08:08:44 -0700:
> On Jul 12,  6:51am, Daniel Shahaf wrote:
> }
> } Bart Schaefer wrote on Fri, Jul 10, 2015 at 17:01:34 -0700:
> } > If you can't jam it all in a .vimrc or the like, it should probably go
> } > in Util/ rather than Doc/.
> } 
> } Attached.  It's autoloading for those people who auto-source
> } $PWD/.vimrc, and there are instructions for everyone else.
> } (The "install permanently" part could be further streamlined.)
> 
> Hmm, maybe I just have a really old vim, but I get
> 

Looks like you run vim 6.x or older, i.e., over 9 years old
(see ':help version7 | /\\%x').

> Error detected while processing Util/zyodl.vim:
> line   47:
> E71: Invalid character after \%
> E475: Invalid argument: zyodlTitle   "^texinode\%x28.*$"

After changing «\%x28» to «(», the syntax file works under Vim 6.4.
Will push with this change.

Thanks for testing!