From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.text.pandoc/26784 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?Gabriel_N=C3=BCtzi?= Newsgroups: gmane.text.pandoc Subject: File transclusion with no-inline (on steroids) Date: Wed, 11 Nov 2020 10:52:52 +0100 Message-ID: <786591B8-17BD-4B09-90A2-4EC1446F54C9@gmail.com> References: <8c50d176-a9cb-422a-b78f-1e8fd36dce52n@googlegroups.com> <87h7pws4a6.fsf@zeitkraut.de> <8c42cf82-8531-4a70-bbb1-f56de736235fn@googlegroups.com> Reply-To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3601.0.10\)) Content-Type: multipart/alternative; boundary="Apple-Mail=_8899F296-2140-4D99-A533-8CD625D2CD86" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="37520"; mail-complaints-to="usenet@ciao.gmane.io" To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Original-X-From: pandoc-discuss+bncBDRZVNGQEQIPR2FO7UCRUBHMFX4PK-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Wed Nov 11 10:53:00 2020 Return-path: Envelope-to: gtp-pandoc-discuss@m.gmane-mx.org Original-Received: from mail-wr1-f59.google.com ([209.85.221.59]) by ciao.gmane.io with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1kcmoG-0009ci-0T for gtp-pandoc-discuss@m.gmane-mx.org; Wed, 11 Nov 2020 10:53:00 +0100 Original-Received: by mail-wr1-f59.google.com with SMTP id x16sf410583wrg.7 for ; Wed, 11 Nov 2020 01:52:59 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1605088379; cv=pass; d=google.com; s=arc-20160816; b=aEg3ohpg6OhjCCZDBcUhAqWt2B0VUCb9hZdh4yR/tSn4rzuk6uvx/3FhWf67Vw3bl+ lpFoDSBZdPKLMDTOSOyXCqWhP5yhs02wXuN2tTl6Nk7dUN6LNXolysjA4lrL7tkSQhJC j/ZEh+W5GiTpFyGTZvGsNMmhLFIKXB2hZ+79rzA6X2UdJvELRqULrCXygVehh6d8DwT1 0O9ULgDHjaBTX6Y4kMFnDix72pbTlIdsn5Npr+wG3MudQS7FbLowgyFOL8fLDrRzdjmq QmGtsR2aqMbbODAEs9kBb54MqkcnEmAwQ1mVWUYafdNjwai7f26BL9nZVQVrhlP98OKf x5hA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:references:to:in-reply-to :subject:date:mime-version:message-id:from:sender:dkim-signature :dkim-signature; bh=XdADdYj7N8mj7MMVSrxE+dF0w4SPURRF1h3LQ0kAmek=; b=G/YAemR2dYdfvEThmuF7lOJZ5YgCJQSdw8Kdw7TZT7tj6KRUxGpXDv5UvBiUBYs1BZ cELVRlhoqQ//JbMsRFfWY5mIXZ4ap6i4kH2mN06bpZxU0LskSX2YOkVCths5fiw+fIuI AcmEWp1Of40KaxretBjTjPE6yjxx6myd3OH7Gkc4BOijiY8QS7T4McmzQM6Xu40x3RgO DMzPzp6X2ZaihUu3GOQR27NmJLoKPvIi+UGNa5wBmQOI0gpl7JXOGbz4eImyveybKALd llhIpk1dfSgEUXHI/7u+lvfAEEXWSNJWvrO73J4tP5ESujTofgZ1zl1j99FZkDqhSfJc gCnQ== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=KJgXU8pc; spf=pass (google.com: domain of gnuetzi-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org designates 2a00:1450:4864:20::633 as permitted sender) smtp.mailfrom=gnuetzi-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20161025; h=sender:from:message-id:mime-version:date:subject:in-reply-to:to :references:x-original-sender:x-original-authentication-results :reply-to:precedence:mailing-list:list-id:list-post:list-help :list-archive:list-subscribe:list-unsubscribe; bh=XdADdYj7N8mj7MMVSrxE+dF0w4SPURRF1h3LQ0kAmek=; b=Welo6Xm8/0xvSXDlXhqi6oSPaUF7eyaNgtQMOyRsZdVHSXXlhtO/xeLuhmRNHXwvwN f3TTw9U9YoYHVKXKYJoYr7ieuFJIMHu6j0y5EIgGQh+EMHOiHfehafxHQz1bWQRmVPXV WbT13uLk1GlVaHrgVK7e+P1v1dg+CykjLoZOKnqiFMxWGT9t+eN+kWoVImrH/pA/y/99 O864DkfM4qDekiGQYkLrxnP5HFnBEBMJV5XiDO+j+Ve32STnH1u3Hzx20BAFcHujbZgn up68V/Q0fngRHcyhi8aJIW15IbW95X6jaNdODc76iZFdYtVNzAW2OApv4BxN6+tIxhFX gEBw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:message-id:mime-version:date:subject:in-reply-to:to:references :x-original-sender:x-original-authentication-results:reply-to :precedence:mailing-list:list-id:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=XdADdYj7N8mj7MMVSrxE+dF0w4SPURRF1h3LQ0kAmek=; b=nZKWWxJy2Zh7zFgUV6X4juDIM0xnK4idsuCLaNdEoZAkTuNwj7DpNA2tS2RFS0IBtZ n4LY/PSPxpeIyB9MieTVvn5twjh6FShnIU5S+BvcarzA7hCPizUBGomyPBugbt6nGCdQ UfIQ4sc1g7CtHb/blkhMEveMW4riC1FYn4OlGxJrc0MwVr5xWcZ+MZ3SAp9JKZ8D1dP5 gikH+1pC2qT3HEvWA83wwJL5hU72zpU4it4GfWrZhIrgpJyoYPx+77VU6SrjVgpdzQrf teHnhu9FkHzCFGRWjSpuuUdRbzIXzN6Gg8qR/fHIhHuwVFYofkdh/TuAjuysCgUsbXQY PPGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=sender:x-gm-message-state:from:message-id:mime-version:date:subject :in-reply-to:to:references:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=XdADdYj7N8mj7MMVSrxE+dF0w4SPURRF1h3LQ0kAmek=; b=k1sF7htPWPGaNin4UhXHXxszjRE/gtwzdUgwMsVROK7Yyw4MpKObt21w0tGiRBYu2n oNTrov42n2jVQJmjheZlVa1xMU4cA9g6eYLwza0cMGtvPvY0zxkeftHCT+jzyIQflcCU KtegVhcygvz+HLj/BKNuadnZoHLKTYsqK96b/pL+6Ao+V7DLBnoE4QJSlmcrNdzrBjxm XWJ3zUB/aPfPXO0i1kiqRMJXzq76M+AqjuMTY+3arQYMM4jlYen7L4/yzoGmlFtRX7M6 63EEmCN7t+PaWN/7bZYxIGDvZwE8uY4Hz8ETZSLF4ihiuuR/GgOdccZ1eSit0k2w1p8J KtCA== Original-Sender: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org X-Gm-Message-State: AOAM530xFfVRqoSlNxX4TNaWVJ0rSQfACj2io87BWuw0CzKUfrJgxYva /8sBgW/TSjnOCCjvVpVOELY= X-Google-Smtp-Source: ABdhPJwzNr/pyhb8sOn8zv/3BNi1IthS5Oo+3HoDidkd5IeWfb2/b1DqdvnwSs/+NwFsEAXlyPv3Hw== X-Received: by 2002:adf:e350:: with SMTP id n16mr29851796wrj.419.1605088379791; Wed, 11 Nov 2020 01:52:59 -0800 (PST) X-BeenThere: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Original-Received: by 2002:adf:aa87:: with SMTP id h7ls2020634wrc.2.gmail; Wed, 11 Nov 2020 01:52:55 -0800 (PST) X-Received: by 2002:a05:6000:1088:: with SMTP id y8mr25950257wrw.207.1605088375638; Wed, 11 Nov 2020 01:52:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1605088375; cv=none; d=google.com; s=arc-20160816; b=D+qv9oEbXywssWA0FNWKQtwza67If2Ba1HipGqGSfsp4kKjdwRDRoqL+kM6CJVZ5Y/ E/CFxEEhiP5XJWkWENZBK9E2KGPeAr41ltQv8xRW5lzzoU0lKx1dxgwIR8QP5t3JQzew xeqRkml4FDsvXI/rw7jw0/AFLX7xhAiOifDAliU4T2NxvQUQ+gzIqKAJbMDRdvHBofn+ d4L65y4MobbTvCl+nhDI8ZeEBldHfyvU1KuWnOjC6+0V+dalDE0BIAceIvXdC8a+QWFP +P5rhmNLTtpXKsMVhmaAfnbqMUxGWuZmIDPvH5ng4xcooTcwWXzDrw3fLeD8zQAfdmih smFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:to:in-reply-to:subject:date:mime-version:message-id:from :dkim-signature; bh=lpzl6Gii7noZ3tKLDWCU4Zy4Yx9xCF6hTeuSPF80AWs=; b=imcK+Y0Hs9I96cB5UDxLoBKf2f4VhM4iC50aRU0MUIh+vxzWEsaqX/oc84gEZv7Qna ACPCKOL4Peugx6uEa/Zjvetso/HvDDRb3l3akCpYmViI87YJ48JlxWtAHRHk7fL9HGWp SRcyQiw8YSwx0TmeCJvFULfT0PHyQ3pKEcMribpXG7SHZoT+kZQXCntCZrGDUftb86V2 sYE7kcWt1cIoyqAgcx+LhS70+T97RCS+UPEvJtE/4cNKz3Psro50iJa5eKrfHm1GKWKl kyr62KM0KAGjsBB1mFK3uBNpT//sFXw7rRrX7wgo1Qb3tR6roA0LK7SdZIsVZvY6jEwI VVFg== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=KJgXU8pc; spf=pass (google.com: domain of gnuetzi-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org designates 2a00:1450:4864:20::633 as permitted sender) smtp.mailfrom=gnuetzi-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Original-Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com. [2a00:1450:4864:20::633]) by gmr-mx.google.com with ESMTPS id z83si330305wmc.3.2020.11.11.01.52.55 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 11 Nov 2020 01:52:55 -0800 (PST) Received-SPF: pass (google.com: domain of gnuetzi-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org designates 2a00:1450:4864:20::633 as permitted sender) client-ip=2a00:1450:4864:20::633; Original-Received: by mail-ej1-x633.google.com with SMTP id o23so1882882ejn.11 for ; Wed, 11 Nov 2020 01:52:55 -0800 (PST) X-Received: by 2002:a17:906:d7a3:: with SMTP id pk3mr23738375ejb.208.1605088375098; Wed, 11 Nov 2020 01:52:55 -0800 (PST) Original-Received: from gabyxs-mbp.fritz.box ([109.70.117.49]) by smtp.gmail.com with ESMTPSA id qx6sm651430ejb.10.2020.11.11.01.52.54 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Nov 2020 01:52:54 -0800 (PST) In-Reply-To: <8c42cf82-8531-4a70-bbb1-f56de736235fn-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> X-Mailer: Apple Mail (2.3601.0.10) X-Original-Sender: gnuetzi-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=KJgXU8pc; spf=pass (google.com: domain of gnuetzi-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org designates 2a00:1450:4864:20::633 as permitted sender) smtp.mailfrom=gnuetzi-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Precedence: list Mailing-list: list pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org; contact pandoc-discuss+owners-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org List-ID: X-Google-Group-Id: 1007024079513 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , Xref: news.gmane.io gmane.text.pandoc:26784 Archived-At: --Apple-Mail=_8899F296-2140-4D99-A533-8CD625D2CD86 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" Hi all, I am currently useing the recursive file transclusion with `include-files.l= ua` filter https://github.com/pandoc/lua-filters/tree/master/include-files = in full capacity (with `=E2=80=94citeproc` and `pandoc-crossref` afterwards= !). It works great. However: When converting to Latex we mostly end up with a monolithic huge f= ile which is really hard to look at for further processing... I wanted to know what options we would have to end up with an output folder= with the same structure but all includes are replaced by e.g. /input{=E2= =80=A6} in the resulting .tex file: ```{.include no-inline} chapter/A.md chapter/B.md ``` would transform when the format is `latex` to something like (note: no-inli= ne does not exist) \input{/chapter/A.tex} \input{/chapter/B.tex} Solution 1 (External): 1. Convert with pandoc and all filters including `include-files.lua` to a m= onolithic `main.tex` file=20 2. Split the main.tex by any means (python) into their parts by parsing sta= rt/end comment tags introduced by `include-files.lua`. I think a feature to= start/end annotate each included path would be a handy thing. @albert.krew= inkel: What do you think? This is a nice solution as it completely decouples from pandoc processing Solution 2 (Filter) 1 Adapt filter include-files.lua such that it parses (walk_block) the file = and directly produces the output (how?) and then stores the output in :=3D / where is the same path but wit = replace extension. (.tex). (attribute: 'no-inline') 2. Inserts RawInline(tex, "\inputt{}=E2=80=9D =E2=80=A6 to repla= ce the include code block. Maybe another handy attribute: 'no-parse' which skips step 1. and uses the = path directly as .=20 Two questions: 1. Can we produce output in the lua filter? Is there such functionality or = do we need to call pandoc executable from Lua again which is possible troub= le some as we need which flags, filters and other stuff needs to be set to = produce correct output=E2=80=A6? 2. Does such a solution (if possible) work with citeproc/crossref, doesnt i= t need the whole file? Is there any other solution and what are the caveats which one needs to dea= l with when splitting up files as in Solution 2. Thanks a lot for any input on that matter. Hopefully it will find a way to = incorporate and give back the knowledge to the community in the project: ht= tps://github.com/gabyx/TechnicalMarkdown BR Gabriel --=20 You received this message because you are subscribed to the Google Groups "= pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/= pandoc-discuss/786591B8-17BD-4B09-90A2-4EC1446F54C9%40gmail.com. --Apple-Mail=_8899F296-2140-4D99-A533-8CD625D2CD86 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="UTF-8"
Hi all,

I am currently useing the recursive file transclusion with `include-files.= lua` filter https://github.com/pandoc/lua-filters/tree/mas= ter/include-files
in full capacity (with `=E2=80= =94citeproc` and `pandoc-crossref` afterwards!). It works= great.

However: = When converting to Latex we mostly end up with a monolithic h= uge file which is really hard to look at for further processing...

I wanted to know what= options we would have to end up with an output folder with the same struct= ure but all includes are replaced by e.g. /input{=E2=80=A6} in the resultin= g .tex file:

```{= .include no-inline}
chapter/A.md
= chapter/B.md
```
would transform = when the format is `latex` to something like (note: no-inline does not exis= t)

\input{<out= put-dir>/chapter/A.tex}
\input{<= output-dir>/chapter/B.tex}

Solution 1 (External):

1. Convert with pandoc and all f= ilters including `include-files.lua` to a monolithic `main.tex` file <= /div>
2. Split the main.tex by any means (python) into their= parts by parsing start/end comment tags introduced by `include-files.lua`.= I think a feature to start/end annotate each included path would be a hand= y thing. @albert.krewinkel: What do you think?

This is a nice solution as it completely deco= uples from pandoc processing

Solution 2 (Filter)

1 Adapt filter include-files.lua such that= it parses (walk_block) the file and directly produces the output (how?) an= d then stores the output in
<new-path> :=3D <= output-dir>/<path>  where <path> is the same path but w= it replace extension. (.tex). (attribute: 'no-inline')
2. Inserts RawInline(tex, "\inputt{<new-path>}=E2=80=9D  =E2=80= =A6 to replace the include code block.

=
Maybe another handy attribute: 'no-pa= rse' which skips step 1. and uses the path directly as <new-path>.&nb= sp;
Two questions:
1. Can we produce output in the lua filter? Is there such functiona= lity or do we need to call pandoc executable from Lua again which is possib= le trouble some as we need which flags, filters and other stuff needs to be= set to produce correct output=E2=80=A6?
2. Does such = a solution (if possible) work with citeproc/crossref, doesnt it need the wh= ole file?

I= s there any other solution and what are the caveats which one needs to deal= with when splitting up files as in Solution 2.

Thanks a lot for any input on that matter. H= opefully it will find a way to incorporate and give back the knowledge to t= he community in the project: https://github.com/gabyx/TechnicalMarkdown

BR Gabriel



--
You received this message because you are subscribed to the Google Groups &= quot;pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to pand= oc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To view this discussion on the web visit https://groups.google.com/d/msgid/p= andoc-discuss/786591B8-17BD-4B09-90A2-4EC1446F54C9%40gmail.com.
--Apple-Mail=_8899F296-2140-4D99-A533-8CD625D2CD86--