[OT] Research into Generative Typesetting

[OT] Research into Generative Typesetting

[John Haltiwanger]

[-- Attachment #1.1: Type: text/plain, Size: 4006 bytes --]

Hi everyone,

As some of you may know, in 2010 I attained a masters degree in New Media
from the University of Amsterdam. The title of my thesis is "Grammars of
Process: Agency, Collective Becoming, and the Organization of Software".

* [PDF](http://drippingdigital.com/gop/grammars_of_process.pdf)
* [TeX](http://drippingdigital.com/gop/grammars_of_process.tex)
* [HTML](http://drippingdigital.com/gop)
* [thesis-groomer.rb](
https://github.com/ab5tract/new_media/blob/master/thesis/infrastructure/thesis-groomer.rb),
in case you are interested in the glue.

Linked above are the PDF and HTML versions of the thesis, which attempts to
examine generative typesetting (ie, going from one input into multiple
outputs with various properties) through the lens of new media theory and
the 'transductive' cybernetics of Gilbert Simondon. The introduction is
particularly heavy with new media theory, but as the chapters progress I
think it becomes more readable for those unfamiliar with this discourse. The
'Operating Systems' chapter was quite fun to write and hopefully contains
some interesting history and reflections.

One of the things this list might be excited about is the typography itself.
I use many of Robert Bringhurst's suggestions, and I think the output is
much the better for his advice.[^1] The HTML version was never totally
finished (I'd prefer it to have JavaScript interaction and more Web-specific
functionality). In fact, I had quite some plans for it, but as the deadline
of the thesis approached, I necessarily poured my efforts more into content
than presentation, and where presentation was concerned I was much more
preoccupied with the Context version. All in all, I am happy with the thesis
but I also know that it could use some work. If you have any feedback,
please let me know.

[^1]: If anyone is interested, I'm thinking I might make a module that sets
up the environment according to these conventions.

Because I was concerned with both PDF and HTML output, I chose to work in
Markdown with Pandoc as my input format. Through the course of trying to
manage a generative workflow, I had to make some unfortunate concessions: on
the one hand, I had to process some of the input in order to get some basic
things I needed, such as a blockquote environment for Context, handling the
title page layout and abstract, among other things; and on the other hand I
was forced to avoid bibliographic automation and instead had to be very
careful to manually cite all my references. (This was because, at the time,
Pandoc did not have the any capacity to do citation management that would
work within HTML; with its new CiteProc support it seems that this is now
feasible). Tagged PDF support is out of the question due to Pandoc generally
only having MkII support, including lacking newer stuff like
\startchapter..\stopchapter and \hyphenatedURL.

The main conclusion I had about generative typesetting is that we are
missing a crucial "glue layer". I have written some on a system which I call
Subtext, a mutable translation layer where one is in control of both the
syntax and the translation effects from a configuration file. Pandoc is
great, but Subtext approaches the edge cases of generative typesetting
differently by encouraging output-specificities without requiring anyone to
learn Haskell to gain a little more control. Looking over Hans' new
`m-markdown` code, I am beginning to see a clearer vision of how to go about
implementing this.

I've done some presentations on generative typesetting:

* [Generative Typesetting @ Libre Graphics Meeting 2011](
http://river-valley.tv/generative-typesetting-with-context/)
* [Textual Liberation @ The Unbound Book Conference](
http://e-boekenstad.nl/unbound/index.php/john-haltiwanger-generative-typesetting/
)
    * [Sozi notes](
http://drippingdigital.com/conf/unbound-book/textual-liberation.svg)

Anyway, I thought that perhaps someone here might find some bit of this
interesting. Have a great weekend!

Sincerely,
John

[-- Attachment #1.2: Type: text/html, Size: 4711 bytes --]

[-- Attachment #2: Type: text/plain, Size: 485 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [OT] Research into Generative Typesetting

[Christian]

First of all, congratulations on your work!

> Hi everyone,
> 
> As some of you may know, in 2010 I attained a masters degree in New Media
> from the University of Amsterdam. The title of my thesis is "Grammars of
> Process: Agency, Collective Becoming, and the Organization of Software".
> 
> * [PDF](http://drippingdigital.com/gop/grammars_of_process.pdf)
> * [TeX](http://drippingdigital.com/gop/grammars_of_process.tex)
> * [HTML](http://drippingdigital.com/gop)
> * [thesis-

It's always interesting to see what others do and especially how they do it with context.
Really great that you are also linking to the source.

> One of the things this list might be excited about is the typography itself. I
> use many of Robert Bringhurst's suggestions, and I think the output is much
> the better for his advice.[^1] The HTML version was never totally finished (I'd
> prefer it to have JavaScript interaction and more Web-specific functionality).
> In fact, I had quite some plans for it, but as the deadline of the thesis
> approached, I necessarily poured my efforts more into content than
> presentation, and where presentation was concerned I was much more
> preoccupied with the Context version. All in all, I am happy with the thesis
> but I also know that it could use some work. If you have any feedback, please
> let me know.
> 
> [^1]: If anyone is interested, I'm thinking I might make a module that sets up
> the environment according to these conventions.

I like the idea of pre-made styles. I'm not shure if a module would provide the necessary flexibility, though. Maybe a style collection (with commented code and linked sample output PDFs) in the wiki would do the job. Like there is for the biochemistry textbook. From there users could just use it as a whole or adapt it to their needs.

From scrolling through the pages I must say that your design is very appealing. I would change some minor details, though (But I'm sure you've had your reasons :)
-Structure level 2 and 3 are not visually distinct (only by the numbers) see e.g. page 65. Maybe using italics for the 3rd level (like Bringhurst does it) would help to distinguish the titles.
-No spacing after paragraphs if you already use indent. (Afaik the consensus is "indent or space, not both")
-Indenting after a picture or line of code looks odd to me and can cause strange results. (See e.g. page 65. After "rm -rf /*" there is an indent in the middle of the sentence.
-There are some issues with the links in the bibliography (e.g. page 99). Also, bibliographies should be set flushleft imho, due to their brief nature they don't really provide enough words to allow automated paragraph construction. The result is a somewhat uneven feel due to big word spacing.

But who am I to grouch? My thesis currently consists of 5 pages xD so I should really not be pointing fingers here^^
Again, congratulations on your thesis.

All the best
Christian

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [OT] Research into Generative Typesetting

[John Haltiwanger]

[-- Attachment #1.1: Type: text/plain, Size: 3511 bytes --]

On Sat, Jul 23, 2011 at 9:37 AM, Christian <metan0r@gmx.de> wrote:

> > One of the things this list might be excited about is the typography
> itself. I
> > use many of Robert Bringhurst's suggestions, and I think the output is
> much
> > the better for his advice.[^1] The HTML version was never totally
> finished (I'd
> > prefer it to have JavaScript interaction and more Web-specific
> functionality).
> > In fact, I had quite some plans for it, but as the deadline of the thesis
> > approached, I necessarily poured my efforts more into content than
> > presentation, and where presentation was concerned I was much more
> > preoccupied with the Context version. All in all, I am happy with the
> thesis
> > but I also know that it could use some work. If you have any feedback,
> please
> > let me know.
> >
> > [^1]: If anyone is interested, I'm thinking I might make a module that
> sets up
> > the environment according to these conventions.
>
>
> I like the idea of pre-made styles. I'm not shure if a module would provide
> the necessary flexibility, though. Maybe a style collection (with commented
> code and linked sample output PDFs) in the wiki would do the job. Like there
> is for the biochemistry textbook. From there users could just use it as a
> whole or adapt it to their needs.
>
>
That is certainly one approach. Hoewever, I'd like to make it as easy as
possible for people who just want to use defaults to do so. One solution is
to have a script that generates a scaffold environment inside a Context
source file. Then they do not need to keep multiple templates around or have
internet access when creating a document. (I know that for experienced
Context users, having some .env files or source templates around is no
problem, but I'm thinking of users who are less experienced or just want to
typeset Markdown documents without messing with TeX).

./context-style-gen.rb --style=bringhurst --input=myThesis.markdown
myThesis.tex

A style I am _really_ excited about is Tufte's, something along the lines of
the 'tufte-latex' package (but without the dozen or so dependencies it
requres in LaTeX, since we should be able to do all that functionality just
from core Context :)

So if anyone is interested in working on such an environment, please get in
touch with me.


From scrolling through the pages I must say that your design is very
> appealing. I would change some minor details, though (But I'm sure you've
> had your reasons :)
> -Structure level 2 and 3 are not visually distinct (only by the numbers)
> see e.g. page 65. Maybe using italics for the 3rd level (like Bringhurst
> does it) would help to distinguish the titles.
> -No spacing after paragraphs if you already use indent. (Afaik the
> consensus is "indent or space, not both")
> -Indenting after a picture or line of code looks odd to me and can cause
> strange results. (See e.g. page 65. After "rm -rf /*" there is an indent in
> the middle of the sentence.
> -There are some issues with the links in the bibliography (e.g. page 99).
> Also, bibliographies should be set flushleft imho, due to their brief nature
> they don't really provide enough words to allow automated paragraph
> construction. The result is a somewhat uneven feel due to big word spacing.
>

All very good points, thank you!


> But who am I to grouch? My thesis currently consists of 5 pages xD so I
> should really not be pointing fingers here^^
> Again, congratulations on your thesis.
>

And good luck to you on yours!

Cheers,
John

[-- Attachment #1.2: Type: text/html, Size: 4272 bytes --]

[-- Attachment #2: Type: text/plain, Size: 485 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [OT] Research into Generative Typesetting

[Aditya Mahajan]

On Wed, Jul 27, 2011 at 6:30 AM, John Haltiwanger
<john.haltiwanger@gmail.com> wrote:
>
>
> On Sat, Jul 23, 2011 at 9:37 AM, Christian <metan0r@gmx.de> wrote:
>>
>> >
>> > [^1]: If anyone is interested, I'm thinking I might make a module that
>> > sets up
>> > the environment according to these conventions.
>>
>>
>> I like the idea of pre-made styles. I'm not shure if a module would
>> provide the necessary flexibility, though. Maybe a style collection (with
>> commented code and linked sample output PDFs) in the wiki would do the job.
>> Like there is for the biochemistry textbook. From there users could just use
>> it as a whole or adapt it to their needs.
>>
>
> That is certainly one approach. Hoewever, I'd like to make it as easy as
> possible for people who just want to use defaults to do so. One solution is
> to have a script that generates a scaffold environment inside a Context
> source file. Then they do not need to keep multiple templates around or have
> internet access when creating a document. (I know that for experienced
> Context users, having some .env files or source templates around is no
> problem, but I'm thinking of users who are less experienced or just want to
> typeset Markdown documents without messing with TeX).
>
> ./context-style-gen.rb --style=bringhurst --input=myThesis.markdown
> myThesis.tex

Note: The following is untested.

You don't need another script, context can handle this natively.  For
example, you can write a 'process-markdown' module that directly
processes a markdown file: (I haven't looked at the new markdown
module. I am assuming that it provides a macro, \inputmarkdown{...} to
process a markdown file.

\startmodule[process-markdown]

\starttext
\inputmarkdown{\inputfilename}
\stoptext

\stopmodule

Then, you can call ConTeXt using

context --usemodules=tufte,other-style,process-markdown
--result=whatever file.markdown

I followed a similar approach when I was processing markdown files
using the filter module. See

http://randomdeterminism.wordpress.com/2011/01/09/markdowntopdf/

I used modes instead of a separate module and an evinronment instead
of a module because that fit my workflow better.

@Hans: do you think that it is a good idea to include something like
this in the m-markdown module so that a user could say

context --usemodule=markdown --mode=process file.markdown

and get a pdf output.

Aditya
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Processing markdown (was: Research into Generative Typesetting)

[Wolfgang Schuster]

Am 28.07.2011 um 00:21 schrieb Aditya Mahajan:

> @Hans: do you think that it is a good idea to include something like
> this in the m-markdown module so that a user could say
> 
> context --usemodule=markdown --mode=process file.markdown
> 
> and get a pdf output.

It’s already there:

    context --extra=markdown file.markdown

There are limitations in Hans method because you can’t define completely
customized styles with it and the basic markdown tags are not enough to
produce a document title (should be simple with the multimarkdown extensions).

Wolfgang

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Processing markdown (was: Research into Generative Typesetting)

[Aditya Mahajan]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 824 bytes --]

On Thu, 28 Jul 2011, Wolfgang Schuster wrote:

>
> Am 28.07.2011 um 00:21 schrieb Aditya Mahajan:
>
>> @Hans: do you think that it is a good idea to include something like
>> this in the m-markdown module so that a user could say
>>
>> context --usemodule=markdown --mode=process file.markdown
>>
>> and get a pdf output.
>
> It’s already there:
>
>    context --extra=markdown file.markdown

Does not work. `context --extra=markdown filename` just gives a list of 
available extras (including markdown), but the file is not processed.

> There are limitations in Hans method because you can’t define completely
> customized styles with it and the basic markdown tags are not enough to
> produce a document title (should be simple with the multimarkdown extensions).

Or pandoc's extensions.

Aditya

[-- Attachment #2: Type: text/plain, Size: 485 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Processing markdown

[Hans Hagen]

On 29-7-2011 8:08, Aditya Mahajan wrote:
> On Thu, 28 Jul 2011, Wolfgang Schuster wrote:
>
>>
>> Am 28.07.2011 um 00:21 schrieb Aditya Mahajan:
>>
>>> @Hans: do you think that it is a good idea to include something like
>>> this in the m-markdown module so that a user could say
>>>
>>> context --usemodule=markdown --mode=process file.markdown
>>>
>>> and get a pdf output.
>>
>> It’s already there:
>>
>> context --extra=markdown file.markdown
>
> Does not work. `context --extra=markdown filename` just gives a list of
> available extras (including markdown), but the file is not processed.

Is there a mtx-context-markdown.tex file on your system? What happens if 
you do:

context --extra=markdown --help

mtx-context     |
mtx-context     | extra: markdown 
(c:/data/develop/context/sources/mtx-context-markdown.tex)
mtx-context     |
mtx-context     | usage: context --extra=markdown [options] list-of-files
mtx-context     |
mtx-context     | --sort                   : sort filenames first
mtx-context     | --paperoffset=dimension  : left-top-offset
mtx-context     | --duplex                 : doublesided (singlesided is 
default)
mtx-context     | --backspace=dimension    : extra left offset
mtx-context     | --topspace=dimension     : extra top offset
mtx-context     | --bodyfont=specification : additional bodyfont 
specification
mtx-context     | --contents               : add table of contents


>> There are limitations in Hans method because you can’t define completely
>> customized styles with it and the basic markdown tags are not enough to
>> produce a document title (should be simple with the multimarkdown
>> extensions).
>
> Or pandoc's extensions.

In due time we can extend the parser (given that I know the specs). Btw, 
the more extensions you have, the more one can consider other approached 
(context with export or just xml).

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Processing markdown

[Aditya Mahajan]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1364 bytes --]

On Fri, 29 Jul 2011, Hans Hagen wrote:

> On 29-7-2011 8:08, Aditya Mahajan wrote:
>> On Thu, 28 Jul 2011, Wolfgang Schuster wrote:
>> 
>>> 
>>> Am 28.07.2011 um 00:21 schrieb Aditya Mahajan:
>>> 
>>>> @Hans: do you think that it is a good idea to include something like
>>>> this in the m-markdown module so that a user could say
>>>> 
>>>> context --usemodule=markdown --mode=process file.markdown
>>>> 
>>>> and get a pdf output.
>>> 
>>> It’s already there:
>>> 
>>> context --extra=markdown file.markdown
>> 
>> Does not work. `context --extra=markdown filename` just gives a list of
>> available extras (including markdown), but the file is not processed.
>
> Is there a mtx-context-markdown.tex file on your system?

Yes. But I needed to run mtxrun --generate (my update scripts were not 
running this). After that everything works fine.

If I compile a file called 1.md, the output file is called 
context-extra.pdf. I think it will be better if the default output is 
<basename>.pdf.

> In due time we can extend the parser (given that I know the specs). Btw, the 
> more extensions you have, the more one can consider other approached (context 
> with export or just xml).

Agreed. Personally, I have now moved from markdown to TeX markup for big 
projects (although I have thoroughly tested the xml export).

Aditya

[-- Attachment #2: Type: text/plain, Size: 485 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Processing markdown

[Wolfgang Schuster]

Am 30.07.2011 um 04:59 schrieb Aditya Mahajan:

> On Fri, 29 Jul 2011, Hans Hagen wrote:
> 
>> In due time we can extend the parser (given that I know the specs). Btw, the more extensions you have, the more one can consider other approached (context with export or just xml).
> 
> Agreed. Personally, I have now moved from markdown to TeX markup for big projects (although I have thoroughly tested the xml export).

There are a few extension which can be usefull but for complex documents TeX is the preferred input method (especially math).

Wolfgang

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Processing markdown

[Hans Hagen]

On 30-7-2011 4:59, Aditya Mahajan wrote:

> If I compile a file called 1.md, the output file is called
> context-extra.pdf. I think it will be better if the default output is
> <basename>.pdf.

that is problematic with other extras as there we often have temp output 
files (e.g. extra=listing)

one can of course use --result=<basename>

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Processing markdown

[Aditya Mahajan]

> On Fri, 29 Jul 2011, Hans Hagen wrote:
>> In due time we can extend the parser (given that I know the specs).

A small feature is missing in the handling of figures. The markdown syntax[1] states 
that figures are specified as follows:

     ![Alt text](/path/to/img.jpg)

Pandoc, when translating from markdown to latex/context, treats the alt-text as
the figure caption. ConTeXt, just ignores the alt text. I would suggest changing
the c_image function to (not completely sure about this)

     local function c_image(lab,src,tit)
         return {
             "\\placemarkdownfigure{",
              lab,
              "}",
              "{",
             "\\externalfigure[",
             src,
             "]",
             "}"
         }
     end

where \placemarkdownfigure is a new type of float. I have not used any optional
argument to \placemarkdownfigure so that the options can be set using
\setupfloat. The default setup could be

     \setupfloat[markdownfigure][default={here,nonumber}]

What do others think?

Aditya

[1]: http://daringfireball.net/projects/markdown/syntax#img
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Processing markdown

[Aditya Mahajan]

On Sat, 30 Jul 2011, Hans Hagen wrote:

> On 30-7-2011 4:59, Aditya Mahajan wrote:
>
>> If I compile a file called 1.md, the output file is called
>> context-extra.pdf. I think it will be better if the default output is
>> <basename>.pdf.
>
> that is problematic with other extras as there we often have temp output 
> files (e.g. extra=listing)
>
> one can of course use --result=<basename>

OK.

Can the load order of mtx-context-markdown be changed?

Suppose I have an environment file, env.tex that sets fonts, styles, and 
layout. If I use

context --extra=markdown --environment=env file

then the environment file is loaded *before* mtx-context-markdown.tex. 
Therefore, the font setup and layout set in the environment file are 
overwritten by mtx-context-markdown. I am not sure what is the best way to 
avoid this.

Aditya
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________