Re: Way to have markdown ==highlighting== show up as highlighting in .docx or .odt files?

[BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]

[-- Attachment #1.1: Type: text/plain, Size: 9789 bytes --]

Ack, I made a mistake in the documentation. The long names of the --db and
--sb options are of course --tex-math-double-backslash and
--tex-math-single-backslash, not ~~*-backticks~~. That's what you get for
copy-and-pasting too much! Sorry!!

Den tors 23 juni 2022 21:06BPJ <bpj-J3H7GcXPSITLoDKTGw+V6w@public.gmane.org> skrev:

> It would be possible but it would be rather fragile and finicky because
> you would have to
>
> 1.  traverse lists of inline elements,
> 2.  locate string elements which contain "==",
> 3.  split that strings into the bit before and after "==",
> 4.  insert the right raw markup for the output format in place of "=="
> 5.  collect elements up to the next string element which contains "==",
> 6.  Redo #3 and #4 with that string,
> 7.  Throw an error if #5 fails!
>
> You are probably better off replacing the `==...==` in your existing files
> using the attached Perl script. It is a modification of a script which I
> have used to convert `_..._` and the like to spans. It uses regexes, but is
> smart enough to leave block and inline code and math as well as "==" in
> contexts were it probably isn't a delimiter alone. Make sure to check out
> the -h and -m options for documentation
>
>
>
> Den tors 23 juni 2022 13:15Emiliano <gattulli.emiliano-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:
>
>> BPJ, is it possible to create a lua filter that does the same thing but
>> it converts Obsidian syntax '== ==' into a highlighted text? I have tons of
>> notes written in Obsidian syntax and it would be an enormous task to modify
>> all of them with the 'new' syntax. By the way, your lua filter works
>> perfectly!
>>
>> Il giorno mercoledì 22 giugno 2022 alle 19:45:07 UTC+2 BPJ ha scritto:
>>
>>> According to the principle that it's better to find out what you can do
>>> with the tools you have you can use a span with a class, like `[text]{.hl}`
>>> and use a simple filter to convert that to Obsidian's syntax when
>>> processing with Obsidian, by choosing `markdown` as output format, or
>>> insert the necessary LaTeX markup when producing PDF (or arrange for the
>>> necessary CSS to be loaded if producing PDF via HTML.)
>>>
>>> ``````lua
>>> local eq_hl = pandoc.RawInline('markdown', '==')
>>>
>>> local highlight = {
>>>   markdown = { start = eq_hl, stop = eq_hl },
>>>   latex = {
>>>     start = pandoc.RawInline('latex', '\\colorbox[named]{yellow}{'),
>>>     stop = pandoc.RawInline('latex', '}'),
>>>   },
>>> }
>>>
>>> local hl = highlight[FORMAT]
>>>
>>> function Span (s)
>>>   if s.classes:includes('hl') then
>>>     if hl then
>>>       rv = s.content
>>>       rv:insert(1, hl.start)
>>>       rv:insert(hl.stop)
>>>       return rv
>>>     end
>>>   end
>>>   return nil
>>> end
>>> ``````
>>>
>>> I'm not sure that the default LaTeX template always loads the xcolor
>>> package. You may need a modifier template.
>>>
>>> I can imagine you lose some in-editor preview, but you get reasonable
>>> output.
>>>
>>> HTH,
>>>
>>> /bpj
>>>
>>> Den ons 22 juni 2022 16:11Emiliano <gattulli...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:
>>>
>>>> Well, if you export in PDF through Obsidian the highlighted text is
>>>> rendered correctly but not if you use Pandoc. I do not export in PDF
>>>> through Obsidian because then I would be bound to the style of the active
>>>> theme, namely, I would see the PDF file with a black background (I use the
>>>> Dark Mode), font size, spacing, margins, etc. of Obsidian's active theme.
>>>>
>>>> Il giorno martedì 21 giugno 2022 alle 18:44:42 UTC+2
>>>> paulschi...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org ha scritto:
>>>>
>>>>> Good question! Thanks for reminding me of this. But exporting to PDF
>>>>> in Obsidian with highlights should work automatically, no?
>>>>>
>>>>> On Tuesday, June 21, 2022 at 3:21:03 p.m. UTC+2 Emiliano wrote:
>>>>>
>>>>>> Any news about this feature for Pandoc? I use a lot the highlight
>>>>>> syntax ('== ==') in Obsidian and it would be great if I could render my
>>>>>> highlighted text in PDF (also in DOCX and ODT).
>>>>>>
>>>>>> Il giorno domenica 2 gennaio 2022 alle 17:52:44 UTC+1 Alx Nbl ha
>>>>>> scritto:
>>>>>>
>>>>>>> My use case is different from paulschi, in my case i am trying to
>>>>>>> convert docx into markdown and generating '== ==' syntax when there is
>>>>>>> higlighted text in the docx file.
>>>>>>>
>>>>>>> On Sunday, January 2, 2022 at 3:09:42 PM UTC+1 Alx Nbl wrote:
>>>>>>>
>>>>>>>> Hi all. The '== ==' syntax is also used by Joplin app. I would also
>>>>>>>> be very interested by such a feature.
>>>>>>>>
>>>>>>>> On Thursday, December 9, 2021 at 6:29:51 PM UTC+1 John MacFarlane
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On CriticMarkup, see
>>>>>>>>>
>>>>>>>>> https://github.com/jgm/pandoc/issues/2873
>>>>>>>>> https://github.com/jgm/pandoc/issues/5430
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Joseph Reagle <josep...-T1oY19WcHSwdnm+yROfE0A@public.gmane.org> writes:
>>>>>>>>>
>>>>>>>>> > BTW: If CommonMark or pandoc were to support highlight, I would
>>>>>>>>> then wonder why not support all of CriticMarkup, which supports highlight
>>>>>>>>> as `{== ==}` or `{>> <<}`. (It's a shame that we have two different
>>>>>>>>> syntaxes emerging for highlight.)
>>>>>>>>> >
>>>>>>>>> > On 21-12-09 11:10, John MacFarlane wrote:
>>>>>>>>> >>
>>>>>>>>> >> If this is a syntax that is becoming common, we could consider
>>>>>>>>> >> adding a markdown extension for it. You could open an issue on
>>>>>>>>> >> our issue tracker.
>>>>>>>>> >>
>>>>>>>>> >> Joseph Reagle <josep...-T1oY19WcHSwdnm+yROfE0A@public.gmane.org> writes:
>>>>>>>>> >>
>>>>>>>>> >>> This is the first time I've encountered [this syntax][1] and
>>>>>>>>> it is not natively supported by pandoc. Or am I wrong and you are saying
>>>>>>>>> pandoc handles it when using the latex/PDF writer? (Or, are you saying
>>>>>>>>> Obsidian can export to PDF, but not Word?)
>>>>>>>>> >>>
>>>>>>>>> >>> I see there's been some discussion on the [CommonMark
>>>>>>>>> forum][2], but it doesn't look like you'd find an immediate solution.
>>>>>>>>> >>>
>>>>>>>>> >>> Using a filter or hacking something that converts `==foo==` to
>>>>>>>>> [foo]{.highlight} that is properly rendered in Word might be options.
>>>>>>>>> >>>
>>>>>>>>> >>> [1]: https://www.markdownguide.org/extended-syntax/#highlight
>>>>>>>>> >>> [2]:
>>>>>>>>> https://talk.commonmark.org/t/highlighting-text-with-the-mark-element/840
>>>>>>>>> >>>
>>>>>>>>> >>> On 21-12-09 08:29, Paul wrote:
>>>>>>>>> >>>> I use a lot of highlighting in my markdown editor Obsidian,
>>>>>>>>> but I was wondering if there's a way to have that highlighting show up in
>>>>>>>>> the Word or Libreoffice Writer files?
>>>>>>>>> >>>>
>>>>>>>>> >>>> Bold and italics work fine, as far as I can tell, and when
>>>>>>>>> converting to a pdf the highlighting transfers great. I gather, however,
>>>>>>>>> that the ==highlighting== is not standard in all markdown so is that the
>>>>>>>>> issue?
>>>>>>>>> >>>
>>>>>>>>> >>> --
>>>>>>>>> >>> You received this message because you are subscribed to the
>>>>>>>>> Google Groups "pandoc-discuss" group.
>>>>>>>>> >>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>> it, send an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>>>>>>>>> >>> To view this discussion on the web visit
>>>>>>>>> https://groups.google.com/d/msgid/pandoc-discuss/9995ee8a-295e-1836-5645-9bb5ff76445d%40reagle.org.
>>>>>>>>>
>>>>>>>>> >>
>>>>>>>>> >
>>>>>>>>> > --
>>>>>>>>> > You received this message because you are subscribed to the
>>>>>>>>> Google Groups "pandoc-discuss" group.
>>>>>>>>> > To unsubscribe from this group and stop receiving emails from
>>>>>>>>> it, send an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>>>>>>>>> > To view this discussion on the web visit
>>>>>>>>> https://groups.google.com/d/msgid/pandoc-discuss/9d89679a-94dc-2459-822f-93dbe4cbca57%40reagle.org.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "pandoc-discuss" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>>>>
>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/pandoc-discuss/ba18ff15-897d-4a7f-bbd4-3735da206f1dn%40googlegroups.com
>>>> <https://groups.google.com/d/msgid/pandoc-discuss/ba18ff15-897d-4a7f-bbd4-3735da206f1dn%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "pandoc-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/pandoc-discuss/3316a007-a142-4d3d-a2f8-40befafb4249n%40googlegroups.com
>> <https://groups.google.com/d/msgid/pandoc-discuss/3316a007-a142-4d3d-a2f8-40befafb4249n%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhAq4vmgvNP7VFduvLQ-EAPeGry%2B-gNcuFYJFpnDbZ02Bw%40mail.gmail.com.

[-- Attachment #1.2: Type: text/html, Size: 14621 bytes --]

[-- Attachment #2: highlight-eq2span.pl --]
[-- Type: text/x-perl, Size: 6371 bytes --]

#!/usr/bin/env perl

use 5.010001;
use utf8;
# use utf8::all;
use strict;
use warnings;
use warnings FATAL => 'utf8';
use autodie;

use open qw[ :utf8 :std ];

use Getopt::Long qw[GetOptions
  :config bundling no_auto_abbrev no_ignore_case];
use Pod::Usage qw[pod2usage];
use Text::Balanced qw[extract_multiple];

my %opt = (
  attributes => '.hl',
  check_word_chars => 1,
  check_whitespace => 1,
  backslash_escapes => 1,
  backticks_code => 1,
  tilde_code_blocks => 1,
  tex_math_dollars => 1,
  tex_math_double_backslash => 0,
  tex_math_single_backslash => 0,
);

my @opts = grep { /_/ } keys %opt;

sub all {
  $opt{$_} = 1 for @opts;
}

sub none {
  $opt{$_} = 0 for @opts;
}

sub neg_opt {
  my($name) = @_;
  $name =~ s/^no_//;
  $opt{$name} = 0;
}

GetOptions(
  \%opt,
  'attributes|a=s',
  'check_whitespace|check-whitespace|s',
  'no_check_whitespace|no-check-whitespace|S' => \&neg_opt,
  'check_word_chars|check-word-chars|w',
  'no_check_word_chars|no-check-word-chars|W' => \&neg_opt,
  'backslash_escapes|backslash-escapes|b',
  'no_backslash_escapes|no-backslash-escapes|B' => \&neg_opt,
  'backticks_code|backticks-code|c',
  'no_backticks_code|no-backticks-code|C' => \&neg_opt,
  'tilde_code_blocks|tilde-code-blocks|t',
  'no_tilde_code_blocks|no-tilde-code-blocks|T' => \&neg_opt,
  'tex_math_dollars|tex-math-dollars|d',
  'no_tex_math_dollars|no-tex-math-dollars|D' => \&neg_opt,
  'tex_math_double_backslash|tex-math-double-backslash|db',
  'no_tex_math_double_backslash|no-tex-math-double-backslash|DB' => \&neg_opt,
  'tex_math_single_backslash|tex-math-single-backslash|sb',
  'no_tex_math_single_backslash|no-tex-math-single-backslash|SB' => \&neg_opt,
  'none|n' => \&none,
  'all|N|A' => \&all,
  'help|h' => sub { pod2usage(1) },
  'man|m' => sub { pod2usage( -verbose => 2) },
);

my $span_start = '[';
my $span_stop  = "]{$opt{attributes}}";

my @extractors;

if ( $opt{tex_math_double_backslash} ) 	{
  push @extractors, (
    qr{ \\\\ \( .+? \\\\ \) }msx,
    qr{ \\\\ \[ .+? \\\\ \] }msx,
  );
}
if ( $opt{tex_math_single_backslash} ) 	{
  push @extractors, (
    qr{ \\ \( .+? \\ \) }msx,
    qr{ \\ \[ .+? \\ \] }msx,
  );
}
push @extractors, qr{ \\. }msx if $opt{backslash_escapes};
push @extractors, qr[ ( ( \~{3,} ) .+? \g{-1} ) ]msx if $opt{tilde_code_blocks};
push @extractors, qr[ ( ( \`+ ) .+? \g{-1} ) ]msx if $opt{backticks_code};
if ( $opt{tex_math_dollars} ) {
  push @extractors, (
    qr{ \$\$ (?: [^\n] | (?<! \n ) \n (?! \n ) )+? \$\$ }msx,
    qr{ \$ (?! \s ) .+? (?<! \s ) \$ (?! \d ) }msx,
  );
}

{
  my $highlight = qr{
    #w (?<! [\pL\pN\p{Mn}] )
    \=\=
    #s (?! \s )
    ( .+? )
    #s (?<! \s )
    \=\=
    #w (?! [\pL\pN\p{Mn}] )
  }msx;
  if ( $opt{check_whitespace} ) {
    $highlight =~ s/#s//g;
  }
  if ( $opt{check_word_chars} ) {
    $highlight =~ s/#w//g;
  }
  push @extractors, +{ highlight => qr/$highlight/msx };
}

# Slurp stdin
my $text = do { local $/; <>; };

# Process the text
my @chunks = extract_multiple $text, \@extractors;
for my $chunk ( @chunks ) {
  if ( ref $chunk ) {
    $chunk = $span_start . $$chunk . $span_stop;
  }
}

print join "", @chunks;
    
__END__

=encoding UTF-8

=head1 NAME

highlight-eq2span.pl -- Replace Obsidian higlight runs with Pandoc spans

=head1 VERSION

This documentation describes version 0.001 of highlight-eq2span.pl

=head1 SYNOPSIS

    perl highlight-eq2span.pl [OPTIONS] <input.md >output.md

=head1 DESCRIPTION

highlight-eq2span.pl replaces C<==HIGHLIGHTED==> as understood
by Obsidian with Pandoc spans like C<[HIGHLIGHTED]{.hl}>.

This script is a regex-based text filter, with far simpler parsing
capabilities than Pandoc.
However it by default tries to leave B<==> sequences which are unlikely
to be highlighting markup alone. There are some command line
options to control this.

=head1 OPTIONS

=over

=item -a, --attributes STR

Use STR as attributes for Pandoc spans.

Default value: C<.hl>

=item -s, --check-whitespace 

Assume that opening C<==> delimiters are not followed by whitespace,
and that closing C<==> delimiters are not preceded by whitespace.

Default value: true

=item -S --no-check-whitespace

Set the -s option just above to false.

=item -w, --check-word-chars 

Assume that opening C<==> delimiters are not preceded by word-chars,
and that closing C<==> delimiters are not followed by word-chars.

Default value: true

=item -W --no-check-word-chars

Set the -w option just above to false.

=item -b, --backslash-escapes

Skip characters preceded by a backslash.
This notably includes C<\=>.

Default value: true

Note that the B<--db> and B<--sb> option below affect this option!

=item -B --no-backslash-escapes

Set the -b option just above to false.

=item -c, --backticks-code 

Skip chunks of text which look like block or inline
backticks-delimited code.

Default value: true

=item -C --no-backticks-code

Set the -c option just above to false.

=item -t, --tilde-code-blocks

Skip chunks of text which look like tilde-delimited code blocks.

Default value: true

=item -T --no-tilde-code-blocks

Set the -t option just above to false.

=item -d, --tex-math-dollars 

Skip chunks of text which look like block or inline $ delimited math.

Default value: true


=item -D --no-tex-math-dollars

Set the -d option just above to false.

=item --db, --tex-math-double-backslash 

Skip chunks of text which look like C<\\(...\\)> or C<\\[...\\]>
delimited math.

Default value: false

=item --DB --no-tex-math-double-backslash

Set the --db option just above to false.

=item --sb, --tex-math-single-backticks 

Skip chunks of text which look like C<\(...\)> or C<\[...\]>
delimited math.

Default value: false

=item --SB --no-tex-math-single-backticks

Set the --sb option just above to false.

=item -n, --none

Disable all switches.

=item -A, -N, --all

Enable all switches.

=item -h --help

Print usage help and exit.

=item -m, --man

Print full documentation and exit.

=head1 LICENSE

This software is copyright (c) 2022 by Benct Philip Jonsson.

This is free software; you can redistribute it and/or modify it under
the same terms as the Perl 5 programming language system itself.

http://dev.perl.org/licenses/

=head1 AUTHOR

Benct Philip Jonsson E<lt>bpjonsson@gmail.comE<gt>

=cut

# Vim: set ft=pod et ts=4 sts=4 sw=4 tw=72 cc=72:


# Vim: set ft=pod et ts=4 sts=4 sw=4 tw=72 cc=72: