List for cgit developers and users
 help / color / mirror / Atom feed
From: john at keeping.me.uk (John Keeping)
Subject: Rendering of README.md inline with inner tree view dirs
Date: Sat, 16 Jun 2018 18:35:15 +0100	[thread overview]
Message-ID: <20180616173515.GK1922@john.keeping.me.uk> (raw)
In-Reply-To: <20180616141209.GQ1922@john.keeping.me.uk>

On Sat, Jun 16, 2018 at 03:12:09PM +0100, John Keeping wrote:
> > >> 3) You can see on the top level of the tree, the README.md references
> > >>
> > >> <img alt="lws-overview" src="./doc-assets/lws-overview.png">
> > >>
> > >> This url format works in github.  In the cgit About view, this resolves to
> > >>
> > >> /git/libwebsockets/about/doc-assets/lws-overview.png
> > >>
> > >> which also serves the right mimetype and content.  So that kind of URL
> > >> format is useful.  But when we render the same markup and relative path
> > >> via /tree/, it tries to show an html page containing the content.
> > >> That's why the picture is missing in the /tree/ view... other pictures
> > >> in that markup are coming with absolute URLs outside of cgit and are
> > >> working.
> > >>
> > >> I can have the direct content from cgit generally, but either the markup
> > >> needs fixing up to
> > >>
> > >> /git/libwebsockets/plain/doc-assets/lws-overview.png
> > >>
> > >> or /tree/ needs to learn to do what /about/ does.
> > >>
> > >> I'm wondering whether mmd2html might grow an environment var to know the
> > >> base part for URLS that want to direct-render from cgit.  Or if better
> > >> to follow what /about/ did in /tree/.
> > > 
> > > Making tree do this will break the normal use of tree unless we add some
> > > extra query parameter or path element.  Given that, I think teaching the
> > > renderer to use a path to /about/ is the right thing to do.
> > 
> > OK.  Unfortunately I don't know python very well.  It looks like the 
> > markdown python library is able to be told to use extensions that are 
> > capable to do this
> > 
> > https://python-markdown.github.io/extensions/api/
> > 
> > from the md2html wrapper.  But I don't know enough python to do it.
> > 
> > It's a shame, because in-tree assets correctly follow the ref context 
> > being viewed, eg, if you look at a v2 branch you see v2 pngs, master you 
> > see master pngs etc.
> > 
> > I'll "solve" this part for now by changing the README to use external URLs.
> 
> Yeah, I think we have to solve it by having the filter apply a mapping.
> We have ui-plain which provides the right content for images, but what
> should we do for link targets?
> 
> For the purpose of discussion, consider the following HTML fragment that
> could be generated by rendering a README file:
> 
> 	<img src="dataflow.png">
> 	<p>For more details see <a href="dataflow.html">the dedicated
> 	data flow document.</p>
> 
> If dataflow.html is generated from a source file in a similar way, then
> it doesn't exist in the repository and we can't link to it, so the ideal
> output ends up being something like:
> 
> 	<img src="/repo/plain/dataflow.png">
> 	<p>For more details see <a href="dataflow.txt">the dedicated
> 	data flow document.</p>
> 
> The render filter API isn't finalised yet, so we can change the
> parameters that are passed in order to add more information for the
> renderer to use.  At the very least I think we should add a parameter
> for the asset prefix which is essentially the tree path with /tree/
> replaces with /plain/.
> 
> However, I'm not sure how to handle relative links: do we need to pass
> additional parameters for this?  Or can we rely on a render filter doing
> the right thing?

Modifying md2html to use the extension API is reasonably
straightforward.  Below is a modified version which remaps the "src"
attribute on <img> elements according to a second command line argument,
you can try it out with:

	md2html <README.md README.md /path/to/plain/directory/

The trailing "/" is important.

The differences are the AssetMapping classes at the top and the
extension setup at the bottom; the rest is unchanged from the version in
CGit's source tree.

-- >8 --
#!/usr/bin/env python3
import markdown
import sys
import io
from markdown.util import etree
from pygments.formatters import HtmlFormatter
from urllib.parse import urljoin


class AssetMappingProcessor(markdown.treeprocessors.Treeprocessor):

    def __init__(self, asset_prefix):
        self.asset_prefix = asset_prefix

    def run(self, root):
        asset_prefix = self.asset_prefix
        for img in root.iter('img'):
            src = img.get('src')
            if src is None:
                continue
            img.set('src', urljoin(asset_prefix, src))


class AssetMappingExtension(markdown.extensions.Extension):

    def __init__(self, **kwargs):
        self.config = {'asset_prefix': ['', 'prefix for relative asset URLs']}
        super(AssetMappingExtension, self).__init__(**kwargs)

    def extendMarkdown(self, md, md_globals):
        asset_prefix = self.getConfig('asset_prefix')
        if not asset_prefix:
            return

        md.treeprocessors.add('asset_mapping',
                              AssetMappingProcessor(asset_prefix),
                              '_end')


sys.stdin = io.TextIOWrapper(sys.stdin.buffer, encoding='utf-8')
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8')
sys.stdout.write('''
<style>
.markdown-body {
    font-size: 14px;
    line-height: 1.6;
    overflow: hidden;
}
.markdown-body>*:first-child {
    margin-top: 0 !important;
}
.markdown-body>*:last-child {
    margin-bottom: 0 !important;
}
.markdown-body a.absent {
    color: #c00;
}
.markdown-body a.anchor {
    display: block;
    padding-left: 30px;
    margin-left: -30px;
    cursor: pointer;
    position: absolute;
    top: 0;
    left: 0;
    bottom: 0;
}
.markdown-body h1, .markdown-body h2, .markdown-body h3, .markdown-body h4, .markdown-body h5, .markdown-body h6 {
    margin: 20px 0 10px;
    padding: 0;
    font-weight: bold;
    -webkit-font-smoothing: antialiased;
    cursor: text;
    position: relative;
}
.markdown-body h1 .mini-icon-link, .markdown-body h2 .mini-icon-link, .markdown-body h3 .mini-icon-link, .markdown-body h4 .mini-icon-link, .markdown-body h5 .mini-icon-link, .markdown-body h6 .mini-icon-link {
    display: none;
    color: #000;
}
.markdown-body h1:hover a.anchor, .markdown-body h2:hover a.anchor, .markdown-body h3:hover a.anchor, .markdown-body h4:hover a.anchor, .markdown-body h5:hover a.anchor, .markdown-body h6:hover a.anchor {
    text-decoration: none;
    line-height: 1;
    padding-left: 0;
    margin-left: -22px;
    top: 15%}
.markdown-body h1:hover a.anchor .mini-icon-link, .markdown-body h2:hover a.anchor .mini-icon-link, .markdown-body h3:hover a.anchor .mini-icon-link, .markdown-body h4:hover a.anchor .mini-icon-link, .markdown-body h5:hover a.anchor .mini-icon-link, .markdown-body h6:hover a.anchor .mini-icon-link {
    display: inline-block;
}
.markdown-body h1 tt, .markdown-body h1 code, .markdown-body h2 tt, .markdown-body h2 code, .markdown-body h3 tt, .markdown-body h3 code, .markdown-body h4 tt, .markdown-body h4 code, .markdown-body h5 tt, .markdown-body h5 code, .markdown-body h6 tt, .markdown-body h6 code {
    font-size: inherit;
}
.markdown-body h1 {
    font-size: 28px;
    color: #000;
}
.markdown-body h2 {
    font-size: 24px;
    border-bottom: 1px solid #ccc;
    color: #000;
}
.markdown-body h3 {
    font-size: 18px;
}
.markdown-body h4 {
    font-size: 16px;
}
.markdown-body h5 {
    font-size: 14px;
}
.markdown-body h6 {
    color: #777;
    font-size: 14px;
}
.markdown-body p, .markdown-body blockquote, .markdown-body ul, .markdown-body ol, .markdown-body dl, .markdown-body table, .markdown-body pre {
    margin: 15px 0;
}
.markdown-body hr {
    background: transparent url("/dirty-shade.png") repeat-x 0 0;
    border: 0 none;
    color: #ccc;
    height: 4px;
    padding: 0;
}
.markdown-body>h2:first-child, .markdown-body>h1:first-child, .markdown-body>h1:first-child+h2, .markdown-body>h3:first-child, .markdown-body>h4:first-child, .markdown-body>h5:first-child, .markdown-body>h6:first-child {
    margin-top: 0;
    padding-top: 0;
}
.markdown-body a:first-child h1, .markdown-body a:first-child h2, .markdown-body a:first-child h3, .markdown-body a:first-child h4, .markdown-body a:first-child h5, .markdown-body a:first-child h6 {
    margin-top: 0;
    padding-top: 0;
}
.markdown-body h1+p, .markdown-body h2+p, .markdown-body h3+p, .markdown-body h4+p, .markdown-body h5+p, .markdown-body h6+p {
    margin-top: 0;
}
.markdown-body li p.first {
    display: inline-block;
}
.markdown-body ul, .markdown-body ol {
    padding-left: 30px;
}
.markdown-body ul.no-list, .markdown-body ol.no-list {
    list-style-type: none;
    padding: 0;
}
.markdown-body ul li>:first-child, .markdown-body ul li ul:first-of-type, .markdown-body ul li ol:first-of-type, .markdown-body ol li>:first-child, .markdown-body ol li ul:first-of-type, .markdown-body ol li ol:first-of-type {
    margin-top: 0px;
}
.markdown-body ul li p:last-of-type, .markdown-body ol li p:last-of-type {
    margin-bottom: 0;
}
.markdown-body ul ul, .markdown-body ul ol, .markdown-body ol ol, .markdown-body ol ul {
    margin-bottom: 0;
}
.markdown-body dl {
    padding: 0;
}
.markdown-body dl dt {
    font-size: 14px;
    font-weight: bold;
    font-style: italic;
    padding: 0;
    margin: 15px 0 5px;
}
.markdown-body dl dt:first-child {
    padding: 0;
}
.markdown-body dl dt>:first-child {
    margin-top: 0px;
}
.markdown-body dl dt>:last-child {
    margin-bottom: 0px;
}
.markdown-body dl dd {
    margin: 0 0 15px;
    padding: 0 15px;
}
.markdown-body dl dd>:first-child {
    margin-top: 0px;
}
.markdown-body dl dd>:last-child {
    margin-bottom: 0px;
}
.markdown-body blockquote {
    border-left: 4px solid #DDD;
    padding: 0 15px;
    color: #777;
}
.markdown-body blockquote>:first-child {
    margin-top: 0px;
}
.markdown-body blockquote>:last-child {
    margin-bottom: 0px;
}
.markdown-body table th {
    font-weight: bold;
}
.markdown-body table th, .markdown-body table td {
    border: 1px solid #ccc;
    padding: 6px 13px;
}
.markdown-body table tr {
    border-top: 1px solid #ccc;
    background-color: #fff;
}
.markdown-body table tr:nth-child(2n) {
    background-color: #f8f8f8;
}
.markdown-body img {
    max-width: 100%;
    -moz-box-sizing: border-box;
    box-sizing: border-box;
}
.markdown-body span.frame {
    display: block;
    overflow: hidden;
}
.markdown-body span.frame>span {
    border: 1px solid #ddd;
    display: block;
    float: left;
    overflow: hidden;
    margin: 13px 0 0;
    padding: 7px;
    width: auto;
}
.markdown-body span.frame span img {
    display: block;
    float: left;
}
.markdown-body span.frame span span {
    clear: both;
    color: #333;
    display: block;
    padding: 5px 0 0;
}
.markdown-body span.align-center {
    display: block;
    overflow: hidden;
    clear: both;
}
.markdown-body span.align-center>span {
    display: block;
    overflow: hidden;
    margin: 13px auto 0;
    text-align: center;
}
.markdown-body span.align-center span img {
    margin: 0 auto;
    text-align: center;
}
.markdown-body span.align-right {
    display: block;
    overflow: hidden;
    clear: both;
}
.markdown-body span.align-right>span {
    display: block;
    overflow: hidden;
    margin: 13px 0 0;
    text-align: right;
}
.markdown-body span.align-right span img {
    margin: 0;
    text-align: right;
}
.markdown-body span.float-left {
    display: block;
    margin-right: 13px;
    overflow: hidden;
    float: left;
}
.markdown-body span.float-left span {
    margin: 13px 0 0;
}
.markdown-body span.float-right {
    display: block;
    margin-left: 13px;
    overflow: hidden;
    float: right;
}
.markdown-body span.float-right>span {
    display: block;
    overflow: hidden;
    margin: 13px auto 0;
    text-align: right;
}
.markdown-body code, .markdown-body tt {
    margin: 0 2px;
    padding: 0px 5px;
    border: 1px solid #eaeaea;
    background-color: #f8f8f8;
    border-radius: 3px;
}
.markdown-body code {
    white-space: nowrap;
}
.markdown-body pre>code {
    margin: 0;
    padding: 0;
    white-space: pre;
    border: none;
    background: transparent;
}
.markdown-body .highlight pre, .markdown-body pre {
    background-color: #f8f8f8;
    border: 1px solid #ccc;
    font-size: 13px;
    line-height: 19px;
    overflow: auto;
    padding: 6px 10px;
    border-radius: 3px;
}
.markdown-body pre code, .markdown-body pre tt {
    margin: 0;
    padding: 0;
    background-color: transparent;
    border: none;
}
''')
sys.stdout.write(HtmlFormatter(style='pastie').get_style_defs('.highlight'))
sys.stdout.write('''
</style>   
''')
sys.stdout.write("<div class='markdown-body'>")
sys.stdout.flush()

extensions = [
    "markdown.extensions.fenced_code",
    "markdown.extensions.codehilite",
    "markdown.extensions.tables"
]
extension_configs = {
    "markdown.extensions.codehilite":{"css_class":"highlight"}
}

if len(sys.argv) > 2:
    extensions.append(AssetMappingExtension(asset_prefix=sys.argv[2]))

# Note: you may want to run this through bleach for sanitization
markdown.markdownFromFile(output_format="html5", extensions=extensions, extension_configs=extension_configs)
sys.stdout.write("</div>")


  reply	other threads:[~2018-06-16 17:35 UTC|newest]

Thread overview: 140+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-06-11  7:08 andy
2018-06-11  7:31 ` list
2018-06-11  7:38   ` andy
2018-06-11  7:53     ` list
2018-06-11  8:05       ` andy
2018-06-11 15:38         ` john
2018-06-12  5:53           ` andy
2018-06-12  8:35             ` list
2018-06-12  9:24               ` john
2018-06-12  9:27                 ` andy
2018-06-12 12:07                   ` john
2018-06-12  9:31             ` john
2018-06-13  1:47               ` andy
2018-06-13  2:01                 ` [PATCH 00/11] Render READMEs inline in tree view andy
2018-06-13  2:01                   ` [PATCH 01/11] Use string list strdup_strings for mimetypes andy
2018-06-13  2:01                   ` [PATCH 02/11] Add source page andy
2018-06-13  2:01                   ` [PATCH 03/11] Parse render filters from the config andy
2018-06-13  2:01                   ` [PATCH 04/11] ui-tree: split out buffer printing andy
2018-06-13  2:01                   ` [PATCH 05/11] ui-tree: use render fileters to display content andy
2018-06-16 14:26                     ` john
2018-06-16 23:16                       ` andy
2018-06-13  2:02                   ` [PATCH 06/11] ui-tree: free read_sha1_file() buffer after use andy
2018-06-16 14:24                     ` john
2018-06-13  2:02                   ` [PATCH 07/11] ui-blame: " andy
2018-06-16 14:23                     ` john
2018-06-16 23:17                       ` andy
2018-06-13  2:02                   ` [PATCH 08/11] ui-tree: print_object: add is_inline param andy
2018-06-16 14:38                     ` john
2018-06-13  2:02                   ` [PATCH 09/11] ui-tree: ls_tail: add walk table param andy
2018-06-16 14:38                     ` john
2018-06-13  2:02                   ` [PATCH 10/11] config: add tree-readme list andy
2018-06-16 14:44                     ` john
2018-06-13  2:02                   ` [PATCH 11/11] ui-tree: render any matching README file in tree view andy
2018-06-16 14:58                     ` john
2018-06-14  3:47                   ` [PATCH 00/11] Render READMEs inline " andy
2018-06-16 14:17                     ` john
2018-06-19  9:01                   ` [PATCH v3 00/17] " andy
2018-06-19  9:01                     ` [PATCH v3 01/17] manpage: fix sorting order andy
2018-06-19 21:35                       ` john
2018-06-19  9:01                     ` [PATCH v3 02/17] blame: css: make blame highlight div absolute and at parent top andy
2018-06-19  9:01                     ` [PATCH v3 03/17] Use string list strdup_strings for mimetypes andy
2018-06-19  9:01                     ` [PATCH v3 04/17] Add source page andy
2018-06-19  9:01                     ` [PATCH v3 05/17] Parse render filters from the config andy
2018-06-19 21:37                       ` john
2018-06-19  9:01                     ` [PATCH v3 06/17] ui-tree: split out buffer printing andy
2018-06-19  9:02                     ` [PATCH v3 07/17] ui-tree: use render filters to display content andy
2018-06-19  9:02                     ` [PATCH v3 08/17] ui-blame: free read_sha1_file() buffer after use andy
2018-06-19 21:46                       ` john
2018-06-19  9:02                     ` [PATCH v3 09/17] ui-tree: ls_tail: add walk table param andy
2018-06-19  9:02                     ` [PATCH v3 10/17] config: add global inline-readme list andy
2018-06-19  9:02                     ` [PATCH v3 11/17] config: add repo " andy
2018-06-19  9:02                     ` [PATCH v3 12/17] ui-tree: render any matching README file in tree view andy
2018-06-19 21:49                       ` john
2018-06-20  0:00                         ` andy
2018-06-19  9:02                     ` [PATCH v3 13/17] md2html: add asset mapping andy
2018-06-19  9:02                     ` [PATCH v3 14/17] md2html-add-asset-postfix-arg andy
2018-06-19  9:02                     ` [PATCH v3 15/17] ui-shared: deduplicate some code in repolink andy
2018-06-19 21:48                       ` john
2018-06-19  9:02                     ` [PATCH v3 16/17] ui-shared: add helper for generating non-urlencoded links andy
2018-06-19 21:55                       ` john
2018-06-20  0:07                         ` andy
2018-06-19  9:02                     ` [PATCH v3 17/17] render: adapt for providing extra filter args for plain andy
2018-06-19 21:56                       ` john
2018-06-20 10:11                   ` [PATCH v4 00/16] Render READMEs inline in tree view andy
2018-06-20 10:12                     ` [PATCH v4 01/16] manpage: fix sorting order andy
2018-06-27 17:27                       ` Jason
2018-06-20 10:12                     ` [PATCH v4 02/16] Use string list strdup_strings for mimetypes andy
2018-06-27 17:28                       ` Jason
2018-06-20 10:12                     ` [PATCH v4 03/16] Add source page andy
2018-06-20 10:12                     ` [PATCH v4 04/16] Parse render filters from the config andy
2018-06-20 10:12                     ` [PATCH v4 05/16] ui-tree: split out buffer printing andy
2018-06-20 10:12                     ` [PATCH v4 06/16] ui-tree: use render filters to display content andy
2018-06-20 10:12                     ` [PATCH v4 07/16] ui-tree: ls_tail: add walk table param andy
2018-06-20 10:12                     ` [PATCH v4 08/16] config: add global inline-readme list andy
2018-06-20 10:12                     ` [PATCH v4 09/16] config: add repo " andy
2018-06-20 10:12                     ` [PATCH v4 10/16] ui-tree: render any matching README file in tree view andy
2018-06-20 10:12                     ` [PATCH v4 11/16] md2html: add asset mapping andy
2018-06-27 17:32                       ` Jason
2018-06-27 20:00                         ` john
2018-06-20 10:12                     ` [PATCH v4 12/16] md2html: add asset postfix arg andy
2018-06-20 10:13                     ` [PATCH v4 13/16] ui-shared: deduplicate some code in repolink andy
2018-06-27 17:29                       ` Jason
2018-06-27 17:50                         ` Jason
2018-06-20 10:13                     ` [PATCH v4 14/16] ui-shared: add helper for generating non-urlencoded links andy
2018-06-20 10:13                     ` [PATCH v4 15/16] render: adapt for providing extra filter args for plain andy
2018-06-20 10:41                       ` andy
2018-06-20 10:13                     ` [PATCH v4 16/16] md2html: change css name to not conflict with highlight andy
2018-06-27 17:37                       ` Jason
2018-06-27 21:58                         ` andy
2018-06-28  8:32                           ` john
2018-06-23 11:04                     ` [PATCH v4 00/16] Render READMEs inline in tree view john
2018-06-23 11:10                       ` andy
2018-06-27 17:18                     ` Jason
2018-06-27 17:26                       ` Fancier Source view [Was: Re: [PATCH v4 00/16] Render READMEs inline in tree view] Jason
2018-06-27 20:05                         ` john
2018-06-27 19:51                       ` [PATCH v4 00/16] Render READMEs inline in tree view john
2018-06-27 22:48                         ` andy
2018-06-27 23:22                         ` Jason
2018-06-28  8:28                           ` john
2018-07-03 19:34                             ` Jason
2018-07-03 19:53                               ` john
2018-07-03 19:58                                 ` Jason
2018-06-27 22:36                       ` andy
2018-06-27 22:46                         ` Jason
2018-06-27 23:08                           ` andy
2018-06-16 14:12                 ` Rendering of README.md inline with inner tree view dirs john
2018-06-16 17:35                   ` john [this message]
2018-06-18  2:22                     ` andy
2018-06-18  2:56               ` [PATCH v2 00/15] Render READMEs inline in tree view andy
2018-06-18  2:57                 ` [PATCH v2 01/15] manpage: fix sorting order andy
2018-06-18  2:57                 ` [PATCH v2 02/15] gcc8.1: fix strncat warning andy
2018-07-03 23:45                   ` Jason
2018-07-03 23:47                     ` andy
2018-07-03 23:50                       ` Jason
2018-06-18  2:57                 ` [PATCH v2 03/15] Use string list strdup_strings for mimetypes andy
2018-06-18  2:57                 ` [PATCH v2 04/15] Add source page andy
2018-06-18 19:08                   ` john
2018-06-18 19:27                     ` andy
2018-06-18  2:57                 ` [PATCH v2 05/15] Parse render filters from the config andy
2018-06-18  2:57                 ` [PATCH v2 06/15] ui-tree: split out buffer printing andy
2018-06-18  2:57                 ` [PATCH v2 07/15] ui-tree: use render filters to display content andy
2018-06-18  2:57                 ` [PATCH v2 08/15] ui-blame: free read_sha1_file() buffer after use andy
2018-06-18  2:58                 ` [PATCH v2 09/15] ui-tree: ls_tail: add walk table param andy
2018-06-18  2:58                 ` [PATCH v2 10/15] config: add global inline-readme list andy
2018-06-18 19:32                   ` john
2018-06-18  2:58                 ` [PATCH v2 11/15] config: add repo " andy
2018-06-18 19:30                   ` john
2018-06-18  2:58                 ` [PATCH v2 12/15] ui-tree: render any matching README file in tree view andy
2018-06-18 19:36                   ` john
2018-06-19  1:55                     ` andy
2018-06-19  8:31                       ` john
2018-06-19  8:38                         ` andy
2018-06-18  2:58                 ` [PATCH v2 13/15] md2html: add asset mapping andy
2018-06-18  2:58                 ` [PATCH v2 14/15] md2html-add-asset-postfix-arg andy
2018-06-18 19:21                   ` john
2018-06-19  3:55                     ` andy
2018-06-19  8:34                       ` john
2018-06-18  2:58                 ` [PATCH v2 15/15] render: adapt for providing extra filter args for plain andy
2018-06-18 19:25                   ` john
2018-06-19  3:34                     ` andy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180616173515.GK1922@john.keeping.me.uk \
    --to=cgit@lists.zx2c4.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).