From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.text.pandoc/20248 Path: news.gmane.org!.POSTED!not-for-mail From: pandoc-only-Mmb7MZpHnFY@public.gmane.org Newsgroups: gmane.text.pandoc Subject: docx (Word) reader and complex fields Date: Thu, 24 May 2018 14:06:34 -0700 (PDT) Message-ID: Reply-To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_3322_2069849957.1527195994683" X-Trace: blaine.gmane.org 1527195874 20897 195.159.176.226 (24 May 2018 21:04:34 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 24 May 2018 21:04:34 +0000 (UTC) To: pandoc-discuss Original-X-From: pandoc-discuss+bncBDDNZVXG5MJRBW6STTMAKGQEMVPRL7A-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Thu May 24 23:04:30 2018 Return-path: Envelope-to: gtp-pandoc-discuss@m.gmane.org Original-Received: from mail-oi0-f60.google.com ([209.85.218.60]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fLxPN-0005G9-RO for gtp-pandoc-discuss@m.gmane.org; Thu, 24 May 2018 23:04:26 +0200 Original-Received: by mail-oi0-f60.google.com with SMTP id x195-v6sf1486393oix.18 for ; Thu, 24 May 2018 14:06:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20161025; h=sender:date:from:to:message-id:subject:mime-version :x-original-sender:reply-to:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-subscribe:list-unsubscribe; bh=VYU3m2Glhrz7vIKRUzq0FbUKuk02ZQatcZ+ftSpgtJY=; b=HYepFifCGsR2utVHZTh0aeXNs14jkfE7jriZtS+sKySlh+uN66nNyOssRoNh6yQAMe +6cYEW2fZpnctQJNQCg6GUwkEzwrYRb34fRoCqekQJfcpnNp0DJ6UJe5Q/C+HLiZqj6b NEgWV+gRu8Rx92JdAG8EayeqxgylxIQTpau5eNHFqz/Q57vqm6bOD/umaoc175gQzwzX YsTHFPKYOpheGEo9/hh+CDlLeH0yc1ksMbjSnTSknKz+63nOCnOaT9/W9vKCCqv4Qh76 jSWwZm8BUxOHOGkuY+wJHm1mUAPj42hkyzD0kWq7dCN+vUbDger/umE7HzXeooZGeD+N Ww/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=sender:x-gm-message-state:date:from:to:message-id:subject :mime-version:x-original-sender:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=VYU3m2Glhrz7vIKRUzq0FbUKuk02ZQatcZ+ftSpgtJY=; b=TsWsLhgf/033J6YdDvGKBy/vitmWDVm/9/nIJ6H+4e2uq8FJlwosRC5TU1qwYtFlkz YvDtwt1BKTBcsYrxrONbpYbQJcsjDjhCfUk+Oqea0OJxzvc4kmtxhxCpd7Y1ZP9jf8cC aUEbrb2GkTIXKgkgk5LmWZM+Ry5s8UInjk8cFi1LYsvm2L5vOJPWoJSCgp4TBMIRmvDP ZM8Q5RJhjKyNXLMKz4ympCLMAjqGpRrl3wvIahPLkv5UnL0b6faBSrF/JwbBbky98e6a pJSWMoR7hq2+E0HnyM1A85yQ+j2m9FkPfKJ3LU6m60wpexjYgYhT+Xo+fZcG+HHzoj71 Sp0A== Original-Sender: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org X-Gm-Message-State: ALKqPwdy337fFyzGhOuJ8SLzM8TIKOczLkGJaOkebnLPgBtH4Lx9Mrbt y964zrcULQgrHbBzEkOx2N8= X-Google-Smtp-Source: ADUXVKLOEQRvew8JRo9hogPkfyblTxLRNdSgkmxK1LaQDYLER98hz9GKrg42PEaJkbHNYr+vZM5/5g== X-Received: by 2002:a9d:6183:: with SMTP id g3-v6mr15795otk.3.1527195996747; Thu, 24 May 2018 14:06:36 -0700 (PDT) X-BeenThere: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Original-Received: by 2002:a9d:4418:: with SMTP id u24-v6ls11758392ote.22.gmail; Thu, 24 May 2018 14:06:35 -0700 (PDT) X-Received: by 2002:a9d:4795:: with SMTP id b21-v6mr216274otf.10.1527195995243; Thu, 24 May 2018 14:06:35 -0700 (PDT) X-Original-Sender: pandoc-only-Mmb7MZpHnFY@public.gmane.org Precedence: list Mailing-list: list pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org; contact pandoc-discuss+owners-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org List-ID: X-Spam-Checked-In-Group: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org X-Google-Group-Id: 1007024079513 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , Xref: news.gmane.org gmane.text.pandoc:20248 Archived-At: ------=_Part_3322_2069849957.1527195994683 Content-Type: multipart/alternative; boundary="----=_Part_3323_1486660115.1527195994683" ------=_Part_3323_1486660115.1527195994683 Content-Type: text/plain; charset="UTF-8" Text/Pandoc/Reader/Docx/Parse.hs handles complex fields as if the "separate" field char and subsequent runs were obligatory, but they are optional. While working on other things, I came across this snippet of docx XML (simplified here): SEQ CHAPTER \h \r 1 ECMA-376-1:2016, 17.16.2 (p. 1165), however, marks these parts as optional. Parsing this had the FldCharState transition from Closed -begin-> Open -instr-> FieldInfo, and then ignoring the end. Only when a later field contained a separator, it would continue to -sebarate-> CharContent -end-> Closed. As a fix, I added a case when end is encountered in the FieldInfo state: Fixed a bug in complex field handling: separator fields and rendition runs are optional. --- src/Text/Pandoc/Readers/Docx/Parse.hs | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/Text/Pandoc/Readers/Docx/Parse.hs b/src/Text/Pandoc/Readers/Docx/Parse.hs index 221260f42..b5226a95a 100644 --- a/src/Text/Pandoc/Readers/Docx/Parse.hs +++ b/src/Text/Pandoc/Readers/Docx/Parse.hs @@ -830,9 +830,12 @@ elemToParPart ns element FldCharClosed | fldCharType == "begin" -> do modify $ \st -> st {stateFldCharState = FldCharOpen} return NullParPart - FldCharFieldInfo info | fldCharType == "separate" -> do + FldCharFieldInfo info | fldCharType == "separate" -> do -- optional separator before rendition modify $ \st -> st {stateFldCharState = FldCharContent info []} return NullParPart + FldCharFieldInfo info | fldCharType == "end" -> do -- direct end, without rendition + modify $ \st -> st {stateFldCharState = FldCharClosed} + return $ Field info [] FldCharContent info runs | fldCharType == "end" -> do -- fxg: End in same par modify $ \st -> st {stateFldCharState = FldCharClosed} return $ Field info $ reverse runs -- 2.11.0 I tested that with the current pandoc git master HEAD, versioned 2.2.1 -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/fe8c2cd9-1aca-430a-92de-c20fa268a223%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. ------=_Part_3323_1486660115.1527195994683 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Text/Pandoc/Reader/Docx/Parse.hs handles complex fields as= if the "separate" field char and subsequent runs were obligatory= , but they are optional.

While working on other things, = I came across this snippet of docx XML (simplified here):
<w:r><w:fldChar w:fldCharType=3D"begin"/></w:r>
<= div class=3D"subprettyprint"><w:r>
=C2=A0 <w:instrText xml:space=3D"preserve"> SEQ CHAPTER \h= \r 1</w:instrText>
</w:r>
<w:r><w:fldChar w:fldCharType=3D"end"/= ></= w:r>

ECMA-376-1:2016, 17.16.2 = (p. 1165), however, marks these parts as optional.

Parsing this had the FldCharState transition from Closed -begin-> Open = -instr-> FieldInfo, and then ignoring the end.=C2=A0 Only when a later f= ield contained a separator, it would continue to -sebarate-> CharContent= -end-> Closed.

As a fix, I added a case when e= nd is encountered in the FieldInfo state:
<= code class=3D"prettyprint">
Fixed a bug in complex field handling: = separator fields and rendition runs are optional.

---
=C2=A0src/Text/Pandoc/Readers/Docx/Parse.hs= | 5 ++++-
=C2=A01 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/src/Text/Pandoc/Re= aders/Docx/Parse.hs b/src/Text/Pandoc/Readers/Docx/Parse.hs
index 221260f42..b5226a9= 5a 100644
--- a/src/Text/Pandoc/Readers/Docx/Parse.hs
+++ b/src/Text/Pandoc/Readers/Docx/Pars= e.hs
@@ -= 830,9 +830,12 @@ elemToParPart ns element
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0FldCharC= losed | fldCharType =3D=3D "begin" -> do
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0modify $ \st -> st {stateFldCharState =3D FldCharOpen}
=C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0return NullParPart
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 FldCharF= ieldInfo info | fldCharType =3D=3D "separate" -> do
+=C2=A0 =C2=A0 =C2= =A0 =C2=A0 FldCharFieldInfo info | fldCharType =3D=3D "separate" = -> do -- optional separator before rendition
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0modify $ \st -> st {stateFldCharState =3D FldCharContent info []}<= /font>
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return NullParPart
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Fl= dCharFieldInfo info | fldCharType =3D=3D "end" -> do -- direct= end, without rendition
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 modify $ \st -> st {s= tateFldCharState =3D FldCharClosed}
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return $ Fie= ld info []
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0FldCharContent info runs | fldCharType = =3D=3D "end" -> do -- fxg: End in same par
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0modify $ \st -> st {stateFldCharState =3D FldCharClosed}
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return $ Field info $ reverse runs
--=C2=A0
2.11.0

I tested that with the current pandoc gi= t master HEAD, versioned 2.2.1

--
You received this message because you are subscribed to the Google Groups &= quot;pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to pand= oc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To view this discussion on the web visit https://groups.google.com/d/= msgid/pandoc-discuss/fe8c2cd9-1aca-430a-92de-c20fa268a223%40googlegroups.co= m.
For more options, visit http= s://groups.google.com/d/optout.
------=_Part_3323_1486660115.1527195994683-- ------=_Part_3322_2069849957.1527195994683--