hz in MkIV

* hz in MkIV
@ 2016-07-04  4:56 Mohammad Hossein Bateni
  2016-07-04  7:54 ` Hans Hagen
  0 siblings, 1 reply; 2+ messages in thread
From: Mohammad Hossein Bateni @ 2016-07-04  4:56 UTC (permalink / raw)
  To: mailing list for ConTeXt users

[-- Attachment #1.1: Type: text/plain, Size: 1925 bytes --]

Hi,

In MkIV, the syntax for using hz seems to have changed from
\setupfonthandling to \setupfontexpansion (similarly for protrusion).  What
is the equivalent of the following?  Is this feature supported in MkIV and
LuaTeX?  I tried changing/adding entries to
fonts.expansions.vectors.quality (and .default) to no apparent avail.

\startfonthandling [hz]
  \defineadjustfactor A .5
  \defineadjustfactor B .7
  \defineadjustfactor C .7
  ...
\stopfonthandling

In any case, I'm happy that hz works for Arabic/Farsi despite some warnings
suggesting the contrary (
https://mailman.ntg.nl/pipermail/ntg-context/2008/029765.html).  It does
make certain Farsi texts look better, however, support for tatweel/kashida
would be ideal.  In fact, I was trying to see if I could somehow get it to
work, building on top of hz.  Hans, do you have any suggestions for this?
Would this be feasible and/or worth trying?

I once converted Vafa Khalighi's XeTeX-based code to Lua but ran into
several setbacks.  (The idea is to insert a sequence of
(ZWJ,\nobreak,stretchable leader,\nobreak,ZWJ) between certain character
pairs that join one another.)  Here are some of the issues I faced:
1) This could not be turned off and on within a paragraph because I was
hooking into processors.before action.
2) The mechanism for some (all?) center alignments (including default
figure captions) interferes with what I was hacking (or, to be more
precise, it's the other way around) and causes unnecessary use of tatweel
in those settings.
3) Later I noticed that the added sequence did not inherit many properties
(like font style or color).
4) The optimum values for the stretches I put in depend on the context,
particularly because some environments manipulate \spaceskip for other
reasons.

Sorry for the mumbling.  I can explain more if anyone is interested in
helping to improve this functionality.  Attached is what I have now.

Thanks,
MHB

[-- Attachment #1.2: Type: text/html, Size: 2352 bytes --]

[-- Attachment #2: p-dabeer-kashida.mkiv --]
[-- Type: application/octet-stream, Size: 10750 bytes --]

% A module to get kashida working in CONTEXT.
% Version: 0.18
% 01/16/2016

%% Copyright 2015, 2016 Mohammad Hossein Bateni
%%
%% This program is free software: you can redistribute it and/or modify
%% it under the terms of the GNU General Public License as published by
%% the Free Software Foundation, either version 3 of the License, or
%% (at your option) any later version.
%%
%% This program is distributed in the hope that it will be useful,
%% but WITHOUT ANY WARRANTY; without even the implied warranty of
%% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
%% GNU General Public License for more details.
%%    
%% You should have received a copy of the GNU General Public License
%% along with this program.  If not, see <http://www.gnu.org/licenses/>.

% todo: add proper copyright
% todo: cleanup
% todo: turn into module?
% todo: option to turn the functionality on or off

%% An include guard
\ifdefined\dabeerkashidapackage\endinput\fi
\def\dabeerkashidapackage{loaded}

%% The main part of the code is in Lua
\startluacode

dabeer          = dabeer         or {}
dabeer.kashida  = dabeer.kashida or {}

local getfont = function (id)
    return fonts.hashes.identifiers[id]
end

-- TODO: use a table to optimize subsequent lookups
local get_kashida_dims = function (id)
    local tfmdata = getfont(id)  

    --- need better error handling and comments
    if not (tfmdata and tfmdata.shared) then
      print('cannot open font')
      return 0, 0
    end

    local descriptions = tfmdata.shared.rawdata.descriptions

---    inspect(tfmdata)

    --- the font needs to have both tatweel and letter 'x'
    --- look at height of 'x' to get 1ex in font units
    --- todo: read off tfmdata["parameters"]["ex"]?
    local glyphdata_ex  = descriptions [120]
    if not glyphdata_ex then
      print('font lacks the glyph for x')
      return 0, 0
    end

    local ex_in_fu      = glyphdata_ex.boundingbox[4]
    if not ex_in_fu then
      return 0, 0
    end

    --- todo: can we read the following off of 
    --- tfmdata["characters"][1600]["height"/"depth"]?
    --- apparently not; height is there but some chars {ZWNJ included}
    --- do not have depth information there.
    --- now get dimensions of tatweel:
    local glyphdata_tatweel = descriptions [1600]
    if not glyphdata_tatweel then
      print('font lacks the glyph for tatweel')
      return 0, 0
    end

    local boundingbox   = glyphdata_tatweel.boundingbox
    local height        = boundingbox [4] or 0
    local depth         = boundingbox [2] or 0

    height = height / ex_in_fu
    depth = -depth  / ex_in_fu

    return height, depth
end

-- TODO: set default id to -- font.current()
dabeer.kashida.write_dims = function(id)
  local height
  local depth
  height, depth = get_kashida_dims(id)
--- TODO: do something else if all zero?
  tex.sprint(string.format("height %f ex depth %f ex", height, depth))   
end

-- gets a pointer to the next node, which should be a glyph
-- and inserts a few nodes signifying kashida before it.
local insert_kashida_before_node = function(follower)
  if follower.id ~= nodes.nodecodes.glyph then
    return
  end

  local fid  = follower.font
  local lang = follower.lang

--  print("SALAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAM", fid, lang)

  local kheight, kdepth
  kheight, kdepth = get_kashida_dims(fid)
  if kheight == 0 and kdepth == 0 then
    return
  end
  kheight = kheight * tex.sp("1ex")
  kdepth  = kdepth  * tex.sp("1ex")

  -- build the sequence of nodes
  -- TODO: do some space/time optimization by reusing
  -- The following sequence is the equivalent of ``KashidaSequence'' defined below,
  -- which in turn was borrowed from XePersian (of Vafa Khalighi's).
  local n_1 = node.new("glyph")
  n_1.font  = fid
  n_1.lang  = lang
  n_1.char  = 0x200D  -- ZWNJ

  local n_2   = node.new("penalty")
  n_2.penalty = 10000
  n_2.subtype = 0  -- userpenalty?

  local n_3   = node.new("glue")
  n_3.subtype = 100  -- leaders?
---  n_3.spec    = node.new("glue_spec")
--- I also tried 6*2^16 and 0
--- I also tried 0.5*2^16 and 3
---  n_3.spec.stretch       = 300 * 2^16  -- why?
---  n_3.spec.stretch_order = 0  -- why? 0, 2
--- zeros may not be necessary
---  n_3.spec.width = 0
---  n_3.spec.shrink = 0
--  n_3.spec.shrink_order = 0
--- new version:  
  node.setglue(n_3, 0, 300 * 2^16, 0, 0, 0)
  n_3.leader  = node.new("rule")
  n_3.leader.depth       = kdepth
  n_3.leader.height      = kheight
  n_3.leader.dir         = "TRT"
  n_3.leader.subtype     = 0  -- stretch?
  n_3.leader.width       = -2^30  -- ?

  local n_4 = node.copy(n_1)

  -- connect these up together
  n_4.prev = n_3
  n_3.prev = n_2
  n_2.prev = n_1
  n_1.next = n_2
  n_2.next = n_3
  n_3.next = n_4

  -- insert it before follower
  local head     = n_1
  local tail     = node.tail(head)  -- ?
  tail.next      = follower
  head.prev      = follower.prev
  tail.next.prev = tail
  head.prev.next = head
end

-- character classes used in kashida insertion
local char_class_dual   = 10  -- dual-joiner class
local char_class_lam    = 11  -- lam
local char_class_right  = 12  -- right-joiner
local char_class_alef   = 13  -- alef
local char_class_other  = 255 -- anything other than the above and below
local char_class_ignore = 256 -- vowel or other combining mark (to be ignored)

local char_classes = char_classes or {}
local class_combinations = class_combinations or {}

local init_char_classes = function()
  for _,ch in pairs({ 0x644 }) do
    char_classes[ch] = char_class_lam
  end
  for _,ch in pairs({ 0x622, 0x623, 0x625, 0x627 }) do
    char_classes[ch] = char_class_alef
  end
  for _,ch in pairs({ 0x624, 0x629, 0x62F, 0x630, 0x631, 0x632, 0x648, 0x698 }) do
    char_classes[ch] = char_class_right
  end
  for _,ch in pairs({ 0x64B, 0x64C, 0x64D, 0x64E, 0x64F, 0x650, 0x651, 0x652 }) do
    char_classes[ch] = char_class_ignore
  end
  for _,ch in pairs({ 0x626, 0x628, 0x62A, 0x62B, 0x62C, 0x62D, 0x62E,
                      0x633, 0x634, 0x635, 0x636, 0x637, 0x638, 0x639, 0x63A,
                      0x640, 0x641, 0x642, 0x643, 0x645, 0x646, 0x647, 0x649,
                      0x64A, 0x67E, 0x686, 0x6A9, 0x6AF, 0x6CC }) do
    char_classes[ch] = char_class_dual
  end
end

local add_to_class_combinations = function(left, right, instruction)
  class_combinations[left] = class_combinations[left] or {}
  class_combinations[left][right] = instruction
end

local init_class_combinations = function()
  -- now we only have 1, which means add a kashida sequence
  -- in the future, we can have different types with various stretch values.
  add_to_class_combinations(char_class_dual, char_class_dual,  1)
  add_to_class_combinations(char_class_lam,  char_class_dual,  1)
  add_to_class_combinations(char_class_dual, char_class_lam,   1)
  add_to_class_combinations(char_class_lam,  char_class_lam,   1)
  add_to_class_combinations(char_class_dual, char_class_right, 1)
  add_to_class_combinations(char_class_dual, char_class_alef,  1)
  add_to_class_combinations(char_class_lam,  char_class_right, 1)
--  add_to_class_combinations(char_class_lam,  char_class_alef,  0)
end

-- initalize the tables
init_char_classes()
init_class_combinations()

-- our list processor and shared variables
local last_class = char_class_other  -- set this before calling the following

local function processnestedlist(head)
     local current, done = head, false
     while current do
         local id = current.id
         if id == nodes.nodecodes.hlist or id == nodes.nodecodes.vlist then
             last_class = char_class_other
             local _, d = processnestedlist(current.head)  --- old: current.list
             last_class = char_class_other
             if d then
                 done = true
             end
         -- is the following necessary?
         elseif id == nodes.nodecodes.hglue_code and current.leader then
             last_class = char_class_other
             local _, d = processnestedlist(current.leader,n,depth+1)
             last_class = char_class_other
             if d then
                 done = true
             end
                 last_class = char_class_other
         elseif id == nodes.nodecodes.glyph then
             -- todo: verify that these are the same font
             local new_class = char_classes[current.char] or char_class_other
             if class_combinations[last_class] and 
                class_combinations[last_class][new_class] == 1 then
                insert_kashida_before_node(current)
             end
             if new_class ~= char_class_ignore then
                last_class = new_class
             end
         else
             last_class = char_class_other
         end
         current = current.next
     end
     return head, done
end

dabeer.kashida.process_nodes = function(head)
     last_class = char_class_other
     local head, done = processnestedlist(head)
     return head, done
end

-- register our processor
nodes.tasks.appendaction ("processors", "before", "dabeer.kashida.process_nodes")

\stopluacode

%% TODO: use CONTEXT way.
%% KashidaOn and KashidaOff macros
%% Warning: These work on paragraphs.
\def\KashidaOff{%
\ctxlua{nodes.tasks.disableaction("processors", "dabeer.kashida.process_nodes")}%
}

\def\KashidaOn{%
\ctxlua{nodes.tasks.enableaction("processors", "dabeer.kashida.process_nodes")}%
}

%% glyph ZWNJ [200D, 8205]
%% penalty sub/0 penalty=10000
%% glue [leader]
%% glyph ZWNJ [200D, 8205]

\unprotect

\chardef\xepersian@zwj="200D % zero-width joiner
\def\KashidaSequence{\xepersian@zwj\nobreak%           
                     \leaders\hrule\ctxlua{dabeer.kashida.write_dims()}%
                     \hskip0pt plus 0.5em\xepersian@zwj}

\protect

\continueifinputfile{kashida.tex}

% For testing
\usemodule[simplefonts]
\setmainfont[HM XNiloofar][features=arabic,range=arabic]
\setupinterlinespace[line=1.5\bodyfontsize]
\setupalign[r2l]
\def\sampletext{سلام}
\def\othertext{موفّقیّت}

\usemodule[hemistich]
\setuphemistich[separator={}]

\starttext
\sampletext\par

\rtlhbox{\sampletext}\par
\rtlhbox to 1cm{\sampletext}\par
{\tfb%
\rtlhbox to 2cm {\sampletext}\par
\rtlhbox to 3cm {\sampletext}\par
}
\rtlhbox to 4cm {\sampletext\space\sampletext}\par

\framed[width=2cm]{\sampletext}\par

\rtlhbox{\othertext}\par
\rtlhbox to 1cm{\othertext}\par
{\tfb%
\rtlhbox to 2cm {\othertext}\par
\rtlhbox to 3cm {\othertext}\par
}

\dorecurse{10}{سلام و درود }

\blank[line]
\startnarrower[2cm]
\hemistiches
{به چه کار آیدت ز گُل طَبَقی}
{از گلستانِ من بِبر وَرقی}
\hemistiches
{گُل همین پنج روز و شِش باشد}
{وین گلستان همیشه خوش باشد}
\stopnarrower
\blank[line]

\stoptext

[-- Attachment #3: Type: text/plain, Size: 485 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

^ permalink raw reply	[flat|nested] 2+ messages in thread