syntax-highlighting.py

syntax-highlighting.py

[stefan]

Hi there,
I created a python script for the syntax-highlighting thing in cgit
which solves a few problems with the shipped script in cgit [1]. I use
it in production since half a year.

- It uses python2 because there are still a lot of problems with 
  python3 and pygments out there. [2], [3]
- I used guess_lexer_for_filename for a better detecting of the used    
  languages (e.g. mixed HTML and PHP, HTML (Django)...). 
  guess_lexer_for_filename does not work with python3 yet.
- The script looks whether there is a shebang line if it cannot   
  determine the language.
- It maps cmakelists.txt and pkgbuild (arch linux package system) files 
  to the correct lexer. Any other filenames can be added easily. 
- The CSS has to be defined in the CSS file to avoid inline CSS.
- `sys.stdin.read().decode(encoding='utf-8', errors='ignore')` is used
  to ensure the functionality even for corrupted file encodings.

If you like my implementation I would be happy to create a patchfile. :)

Stefan

Links:
[1]:
https://github.com/statschner/cgit_stuff/blob/4f847ddcd5aa7de0948bdb6b200a966b6389d94a/cgit_pygments.py
[2]:
https://bitbucket.org/birkenfeld/pygments-main/issue/901/problems-with-python3
[3]:
https://bitbucket.org/birkenfeld/pygments-main/issue/847/test-failures-with-python-33

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: This is a digitally signed message part
URL: <http://lists.zx2c4.com/pipermail/cgit/attachments/20131104/1fae4a14/attachment.asc>

RESEND: syntax-highlighting.py

[stefan]

Am 04.11.2013 21:46, schrieb Stefan Tatschner:
Hi there,
I created a python script for the syntax-highlighting thing in cgit
which solves a few problems with the shipped script in cgit [1]. I use
it in production since half a year.

- It uses python2 because there are still a lot of problems with
  python3 and pygments out there. [2], [3]
- I used guess_lexer_for_filename for a better detecting of the used
  languages (e.g. mixed HTML and PHP, HTML (Django)...).
  guess_lexer_for_filename does not work with python3 yet.
- The script looks whether there is a shebang line if it cannot
  determine the language.
- It maps cmakelists.txt and pkgbuild (arch linux package system) files
  to the correct lexer. Any other filenames can be added easily.
- The CSS has to be defined in the CSS file to avoid inline CSS.
- `sys.stdin.read().decode(encoding='utf-8', errors='ignore')` is used
  to ensure the functionality even for corrupted file encodings.

If you like my implementation I would be happy to create a patchfile. :)

Stefan

Links:
[1]:
http://cgit.sevenbyte.org/cgit_stuff/tree/cgit_pygments.py?id=4f847ddcd5aa7de0948bdb6b200a966b6389d94a
[2]:
https://bitbucket.org/birkenfeld/pygments-main/issue/901/problems-with-python3
[3]:
https://bitbucket.org/birkenfeld/pygments-main/issue/847/test-failures-with-python-33


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: OpenPGP digital signature
URL: <http://lists.zx2c4.com/pipermail/cgit/attachments/20140108/8f6e0a2b/attachment.asc>

RESEND: syntax-highlighting.py

[Jason]

Hi Stefan,

We ship with this script already:
http://git.zx2c4.com/cgit/tree/filters/syntax-highlighting.py

It's pretty similar. Yours might have some fixes or bells and whistles
that mine does not? If so, would you mind submitting a patch that
improves the one we already have?

Thanks,
Jason

RESEND: syntax-highlighting.py

[stefan]

Am 08.01.2014 16:19, schrieb Jason A. Donenfeld:
> Okay reading this closer, it seems what the one in-tree could benefit from is:
> 
> - Expanded list of filename mappings, made more generic than what you
> have in your script, but basically the same idea. -- { "pkgbuild":
> "bashlexer", "cmakelists.txt", "cmakelexer" }. Is there a way to do
> this that ties directly into pygment's guess_lexer_for_filename? If
> not, could you submit a patch upstream?

I have created an upstream pull request and it got merged after 30
seconds [1]. Moreover it seems like the cmakelists.txt thing was also
fixed [2].

Should I remove the filename mapping thing in my patch according to
upstream or should I keep this until the next pygments release?


[1]
https://bitbucket.org/birkenfeld/pygments-main/commits/7cc1e6d0143445302445360fb8b963a7644f1912
[2]
https://bitbucket.org/birkenfeld/pygments-main/src/8fda165f5da04bdf5040fc7b8d3e1589d8fc1b4f/pygments/lexers/text.py?at=default#cl-1609

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: OpenPGP digital signature
URL: <http://lists.zx2c4.com/pipermail/cgit/attachments/20140110/eb45187a/attachment.asc>

RESEND: syntax-highlighting.py

[Jason]

On Fri, Jan 10, 2014 at 2:07 PM, Stefan Tatschner <stefan at sevenbyte.org> wrote:
> I have created an upstream pull request and it got merged after 30
> seconds [1]. Moreover it seems like the cmakelists.txt thing was also
> fixed [2].

Nice! Great!

>
> Should I remove the filename mapping thing in my patch according to
> upstream or should I keep this until the next pygments release?

Yea, go ahead and remove that, and resubmit. When pygments releases,
it'll automatically have that improved functionality.