- dev-luatex - NTG Mailing Lists

Improvement for `glyph_info`
by Marcel Fabian Krüger 15 Nov '19

15 Nov '19

Hi, the new `glyph_info` callback is called in two pretty different cases: in `short_display(_n)` where a pretty short representation of the actual output characters is required, and in `print_font_and_char`, which creates the representation for `print_node_list`. Here, more details can be useful, e.g.: Which variant of a character is used, are displacements applied etc. These could be added through `glyph_info`, but then these additional fields are also printed in `short_display`, leading to significantly worse overfull warnings etc. This could be improved if `glyph_info` could return different values in different situations. So I suggest to add an additional parameter to the `glyph_info` callback which states if the current context could handle longer output. A possible implementation is attached. Best regards, Marcel

4 4

require doesn't get recorded
by Marcel Fabian Krüger 08 Nov '19

08 Nov '19

Hi, based on an observation by Karl Berry on the TeX Live list I discovered an inconsistency in LuaTeX's handling of `-recorder`: While LuaTeX does record files loaded through `loadfile`, files loaded through `require` are ignored: For example if the current directory contains two empty files l1.lua and l2.lua, then the document \directlua{ loadfile'l1.lua'() require'l2' } \bye compiled with `luatex -recorder recorded` IMO should write to recorded.fls the entries PWD /some/weird/pwd INPUT /usr/local/texlive/2019/texmf-var/web2c/luatex/luatex.fmt INPUT ./recorded.tex OUTPUT recorded.log INPUT l1.lua INPUT ./l2.lua but it does write PWD /some/weird/pwd INPUT /usr/local/texlive/2019/texmf-var/web2c/luatex/luatex.fmt INPUT ./recorded.tex OUTPUT recorded.log INPUT l1.lua I do not think that this is intentional, so I added a patch to add recorder entries iff the file is loaded successfully (similar to `loadfile`) -- Marcel

3 6

Re: [Dev-luatex] Broken \adjdemerits on Linux due to unspecified enumeration type
by Marcel Fabian Krüger 05 Nov '19

05 Nov '19

Hi, thanks for fixing the \adjdemerits problem. One remark about the fix: The line if ( (fit_class>(fitness(r)+1)) || (fitness(r)>(fit_class+1)) ) is indented by one tab "\t" and four spaces, while all surrounding code is only indented with spaces. (Confusingly, tabs are also used as indentation in line 117-119) This leads to confusing appearence if the file is viewed in an editor which does not use 8 spaces for one tab. Maybe this could be changed to be a bit more consistant? Best regards, Marcel

2 1

TTC handling
by Marcel Fabian Krüger 31 Oct '19

31 Oct '19

Hey, in `writefont0.c` and `writefont2.c`, `ff_get_ttc_index` is called to determine the index of the font in a font collection. This uses the original filename from fd->fm->ff_name and therefore does not respect lookups done using the find_opentype_file etc. callbacks. In writefont0.c, this could be fixed relativly easily by passing using cur_file_name, the name after the callback, instead. But even than there are remianing problems because this still wouldn't respect read_..._file callbacks. So implementing this correctly is quite tricky and it also isn't really necessary in most cases: Especially when using Lua-loaded fonts, the code creating the font needs to know the index anyway. So maybe an approach like in Khaled Hosny's HarfTeX could be used: Allow a Lua-loaded font to explicitly set the index and only use ff_get_ttc_index if this index isn't set. Then backward-compatibility is kept but the actually used index selection can be influenced using Lua. (HarfTeX implemented this in https://github.com/khaledhosny/harftex/commit/caded3e80f106687c84f9613aa318…) Best regards, Marcel

2 1

Broken \adjdemerits on Linux due to unspecified enumeration type
by Marcel Fabian Krüger 30 Oct '19

30 Oct '19

Hi, current versions of LuaTeX from TeXLive for Linux (Windows is not affected) always add `\adjdemerits` when a line is more loose than the previous one, even if it is only one step more loose: Take \tracingparagraphs1 \tracingonline1 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa \bye This gives @firstpass []\tenrm aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @ via @0 b=96 p=0 d=1011236 @@1: line 1.1 t=21236 -> @0 aaaaaa @ via @0 b=83 p=0 d=8649 @@2: line 1.3 t=8649 -> @0 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aa aaaa aaaaaa @ via @1 b=3 p=0 d=169 @@3: line 2.2 t=1011405 -> @1 aaaaaa @ via @2 b=3 p=0 d=1000169 @@4: line 2.2 t=1008818 -> @2 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @\par via @3 b=0 p=-10000 d=100 @\par via @4 b=0 p=-10000 d=100 @@5: line 3.2- t=1008918 -> @4 Here, the relevant part is @@2: line 1.3 t=8649 -> @0 ... @ via @2 b=3 p=0 d=1000169 @@4: line 2.2 t=1008818 -> @2 So breakpoint @2 is .3 aka. "tight", while @4 is .2 aka. "decent". This break gives demerits d=100169=1000000+(10+3)^2. So \adjdemerits are inserted which is wrong because tight and decent lines are "visually compatible" as defined in the TeXbook. The problem is in linebreak.c. There the demerits are inserted by if (abs(fit_class - fitness(r)) > 1) d = d + adj_demerits; Both fit_class and fitness(r) have type fitness_value which is an enum with four entries. These enums have an implementation defined type and the bug is triggered if LuaTeX is compiled with a compiler which uses an unsigned type: Then `fit_class - fitness(r)` is a difference of unsigned values and therefore unsigned. If fitness(r) is bigger than fit_class, the attempt to store a negative difference in a unsigned value wraps around to a high positive value which is bigger than 1. So if fitness(r) is bigger than fit_class, the condition is always true. This can be fixed by explicitly casting fit_class and fitness(r) to a signed type. A possible patch is attached. Best regards, Marcel

3 4

Luatex 1.08.0 announcement
by luigi scarso 28 Oct '19

28 Oct '19

============================================================== LuaTeX 1.08 2018-08-28 ============================================================== (1) This release is a prelude to 1.10, the next stable iteration of LuaTeX after version 1.00. (2) Lua 5.3 is now considered to be default and we might use 5.4 in version 1.10. There are no real functional changed expected. You still need to rename the binary for 5.3! (3) Binary mode is no longer available in MPlib but it is still available in stand alone MetaPost. This simplifies compilation and reduces dependencies. (4) The dependency on Poppler for pdf image inclusion has been removed. We now use a small dedicated library written by Pawel Jakowski. We no longer need c++ compilers. We're in the process of making it behave well on all platforms. It has been tested on intel platforms. (5) We know that there can be some (alignment) issues with the arm platform but these are looked into. Therefore, later this year we will release 1.09. Version 1.10 is planned for TeXlive. We hope that ffi works ok on intel and arm platforms at that point. (6) There have been some extensions to the Lua libraries and some callbacks have been added. Also, a few new primitives have been introduced. The documentation mentions the stable extensions. (7) There are the usual bug fixes and cleanups but there have been no real fundamental changes in the API. The LuaTeX team

2 5

mlist_to_hlist restriction
by Linas Stonys 02 Oct '19

02 Oct '19

Hi luatex team, mlist_to_hlist callback requires to return a hlist. And that is a big problem because there is no possibility to use this callback if some style already used it. If two different styles uses this callback luatex always throws error "! This can't happen (mlist1)." because mlist is already converted to hlist for second callback. I want to offer a new callback "mlist_to_mlist" were one could access math list without conversion to hlist. Usually if one uses this callback he needs mlist and not a hlist. And probably would be even better to have one more callback "mhpack_filter" which could access result of node.milst_to_hlist(math list converted hlist). So mlist_to_hlist callback would be splited into to parts like: m = list_of_mlist_to_mlist_callbacks (h) n = node.mlis_to_hlist(m) x = list_of_mhpack_filter_callbacks (n) return x What do you think? Linas

2 4

used_font always return false
by Marcel Fabian Krüger 21 Sep '19

21 Sep '19

Hi, the node(.direct).uses_font helpers currently always return false for glyph nodes: In the implementation static int lua_nodelib_direct_uses_font(lua_State * L) { halfword n = lua_tointeger(L,1); halfword f = lua_tointeger(L,2); halfword p; if (type(n) == glyph_node) { lua_pushboolean(L,font(n) == f); } else if (type(n) == disc_node) { uses_font_disc(pre_break,p,n); uses_font_disc(post_break,p,n); uses_font_disc(no_break,p,n); } /* todo: other node types */ lua_pushboolean(L,0); return 1; } the actual result is first calculated, but then the last pushboolean pushes a `false` on top of the stack and return this false. It could e.g. be fixed by adding a `return 1;` at the end of the first if block. `disc` nodes are not affected because uses_font_disc is a macro containing `return`. A minimal example in plain LuaTeX: \directlua{ local g = node.new'glyph' g.font = font.current() if not node.uses_font(g, font.current()) then error'This should not happen' end node.free(g) } \bye -- Marcel

4 5

Re: [Dev-luatex] Assignments in runtoks
by Marcel Fabian Krüger 05 Sep '19

05 Sep '19

On Mon, Sep 02, 2019 at 07:13:36PM +0200, luigi scarso wrote: > On Mon, Sep 2, 2019 at 10:59 AM Hans Hagen <j.hagen(a)xs4all.nl> wrote: > > > > > (I will send the patches to Luigi later today.) > > > > > ------------------------------------------------------------------------ > r7188 | luigi.scarso(a)gmail.com | 2019-09-02 19:12:22 +0200 (Mon, 02 Sep > 2019) | 1 line > > some cleanup + extra features runtoks (HH) > ------------------------------------------------------------------------ > This introduced a bug when `tex.runtoks` is called with a function: In ltexlib.c, runtoks function branch, token_info(r) is set twice and token_info(r) is never set. Also the first assignment to `token_info(t)` sets it to `token_val(end_local_code,0);`, but `end_local_code` is not a command code, so this becomes a space token. Also the supplied function is never called because the local call token is no longer inserted. -- Marcel

2 2

Assignments in runtoks
by Marcel Fabian Krüger 01 Sep '19

01 Sep '19

Hello, in current versions of LuaTeX, `runtoks` does not preserve token lists of ongoing assignments. Therefore running `runtoks` inside of `\edef` or similar replace the definition with the defined value of the last inner assignment. For example: \toks0{\def\innermacro{inner}} \edef\outermacro{outer\directlua{tex.runtoks(0)}} \show\outermacro \bye in plain TeX should show > \outermacro=macro: ->outer. but currently does show > \outermacro=macro: ->inner. instead. (As a side effect the reference count isn't updated, so for example \toks0{\def\innermacro{inner}\let\innermacro\undefined} \edef\outermacro{outer\directlua{tex.runtoks(0)}} \show\outermacro \bye starts printing hyphenation patterns on my system.) This can be fixed by saving scanner_status, warning_index and def_ref when entering local_control: diff --git a/source/texk/web2c/luatexdir/tex/maincontrol.c b/source/texk/web2c/luatexdir/tex/maincontrol.c index b01931dd1..284fb0ae3 100644 --- a/source/texk/web2c/luatexdir/tex/maincontrol.c +++ b/source/texk/web2c/luatexdir/tex/maincontrol.c @@ -1055,6 +1055,10 @@ extern void local_control_message(const char *s) void local_control(void) { int ll = local_level; + int save_scanner_status = scanner_status; + halfword save_def_ref = def_ref; + halfword save_warning_index = warning_index; + main_control_state = goto_next; local_level += 1; while (1) { @@ -1077,14 +1081,17 @@ void local_control(void) if (tracing_nesting_par > 2) { local_control_message("leaving due to level change"); } - return ; + break; } else if (main_control_state == goto_return) { if (tracing_nesting_par > 2) { local_control_message("leaving due to triggering"); } - return; + break; } } + scanner_status = save_scanner_status; + def_ref = save_def_ref; + warning_index = save_warning_index; return; }

3 5