Re: [NTG-pdftex] vf font expansion bug

On 3/24/2020 10:29 PM, Karl Berry wrote:

Hello Thanh (please ack :), and everyone else - Robert (cc'd) recently reported an "interesting" pdftex bug related to font expansion of vf files on the tex-live list. For the record, his original report is here: https://tug.org/pipermail/tex-live/2020-March/045099.html but I'll quote as needed to make this email self-contained. Sorry, it's unavoidably long.

The test for nullfont looks ok to me and is in itself harmless (no side effects i can foresee). That said, I wonder why it's needed. I'm not familiar with the deep down pdftex code but as the luatex code at some point came from that I assume it's similar. Ok, we removed the pdffontexpand feature because it was fragile anyway. At some point Hartmut rewrote part of the expansion (backend) code. Instead of real instances, scaling was used instead. However, pdftex still needed the instances because it has to get the metrics from someplace. I'm not sure if these are real copies but if the (older) luatex code is the same, then these are not even copied. Each time a width of a glyph is needed for calculation (in the linebreak and packaging routines) that width is calculated from the expansion parameters in place at that time. So, a kind of hidden virtuality. There is no new instance. (\pdffontexpand just creates a copy with different parameters set; it could even nor make a copy but just have different parameters but that would demand adapting the font storage routines as well as maybe some accessors). In the backend (again assuming that old luatex and pdftex are doing the same) horizontal scaling is used: basically you'll now find the 1.020 and 0.980 values in the transform matrix (same /F1 etc as normal but different scaling). This is way more efficient than an copy-of-font approach. In your comment you say that "you're right that those fonts will never be used" and so it looks to me like this whole call to vf_expand_local_fonts is some left over that has no purpose. It makes me wonder if there is code in vf_expand_local_fonts that can be removed or maybe even the whole call is kind of bogus but that's for Thanh to decide. After all, if there really had to be made internal instances of a font, then \pdffontexpand\1 100 100 5 would have to trigger 2 * 20 instances. Because the frontend never looks at virtual properties those 'copied instances of the vf font list' don't even matter, and the backend should just scale the original entry in the vf font list (which is why you don't see the extra defined fonts used). (ok, this might be different for bitmaps but that was dropped i think) Summary: to me it looks like there is some old unused code, but of course only Thanh knows the real truth.

I have a proposed fix (at the end) and am hoping for some kind of confirmation that it makes sense. (I am definitely not planning to make any change here for TL20. It's not that urgent, and testing is needed.)

So, Robert found a case where pdftex mixes up two expanded fonts, when virtual fonts are involved. Here is a test file vfexp.tex, slightly modified from his original so it will run with -ini:

----------------------------------------------------------------------------- \catcode`\{=1 \catcode`\}=2 \pdfoutput=1 \nonstopmode \hbadness=10000 \vsize=20pc \hsize=30pt \parfillskip=0pt plus1fil \font\1=cmr10 \font\2=bchr8t %\font\2=bchr8r \pdfmapfile{} \pdfmapline{+cmr10 CMR10

\2 This is a paragraph with font expansion. \end -----------------------------------------------------------------------------

bchr8t is a virtual font (otherwise the problem does not happen, per the commented-out \font\2 above), and we are not using autoexpand here (bug also goes away with autoexpand), and so Robert has actual tfm files for bchr8[rt][+-]20.tfm. (I'll also attach an archive with all files to reproduce.)

Here are the errors from running (avoiding useless mktextfm calls) env TEXFONTS=.: MKTEXTFM=0 pdftex -ini -file-line-error vfexp.tex now: ./vfexp.tex:12:! Font \csname\endcsname=cmr10+100 at 10.0pt not loadable: Metric (TFM) file no ./vfexp.tex:12:! Font \csname\endcsname=cmr10-100 at 10.0pt not loadable: Metric (TFM) file no ./vfexp.tex:17:! Font \csname\endcsname=bchr8r+20-100 at 10.0pt not loadable: Metric (TFM) fil ./vfexp.tex:17:! Font \csname\endcsname=bchr8r-20-100 at 10.0pt not loadable: Metric (TFM) fil

The first two, about failing to make cmr10+100 and cmr10-100 at the first \pdffontexpand, are expected. (Robert, to answer one of your questions: although you're right that those fonts will never be used, and the doc is not exactly clear on this point [I'll try to improve it], the reality is that pdftex immediately tries to make fonts for the stretch/shrink limit when it processes \pdffontexpand. I don't think this could/should be changed.)

At the second \pdffontexpand line, it finds bchr8t.vf, and hence bchr8r.tfm, and and the +20/-20.tfm files. All that is fine.

But then, when pdftex is generating the output (pdf_hlist_out, etc.), it needs to use the expanded bchr8r. The necessary tfms all exist (included in the attached) but it erroneously uses the max shrink (-100) from the cmr10 font \1, and tries to make the nonsensical bchr8r+20-100.

I don't have any particular familiarity with this part of the code, but tracing through the execution in the debugger, it seems the bug is in vf_def_font, which calls set_expand_params (which makes the fonts) with this call: set_expand_params(k, pdf_font_auto_expand[f], pdf_font_expand_ratio[pdf_font_stretch[f]], -pdf_font_expand_ratio[pdf_font_shrink[f]], pdf_font_step[f], pdf_font_expand_ratio[f]);

The problem is that pdf_font_stretch[f] (and _shrink[f]) are both zero at this point. These are references to another font, i.e., null_font. But the zeroth entry of pdf_font_expand_ratio is not the zero for null_font, but the -100 for the specified cmr10 (which doesn't make sense to me, but I took it as given).

FWIW, here are the various arrays at the critical point: (gdb) ppool fontname[0] $91 = "nullfont" (gdb) ppool fontname[1] $92 = "cmr10" (gdb) ppool fontname[2] $93 = "bchr8t" (gdb) ppool fontname[3] $94 = "bchr8t+20" (gdb) ppool fontname[4] $95 = "bchr8t-20" (gdb) p pdffontstretch[0]@5 $96 = {0, 0, 3, 0, 0} (gdb) p pdffontshrink[0]@5 $97 = {0, 0, 4, 0, 0} (gdb) p pdffontexpandratio[0]@5 $98 = {-100, 0, 0, 20, -20}

In other places in the code, e.g., check_expand_pars, there is an explicit check for pdf_font_stretch and _shrink being null_font (= 0), so I added the same explicit checks to vf_def_font, which seemed to solve the problem. pdffonts reports the expanded/shrunken fonts used: $ pdffonts vfexp.pdf name type emb sub uni object ID ------------------------------------ ----------------- --- --- --- --------- QBBONQ+CMR10 Type 1 yes yes no 4 0 DCAQPT+CharterBT-Roman-Extend_1020 Type 1 yes yes no 5 0 OXEYKB+CharterBT-Roman-Extend_980 Type 1 yes yes no 6 0

The same call to set_expand_params is in vf_expand_local_fonts, so if the above is right, the same fix would be needed there. I could not easily figure out how to exercise that code, though. Something like a vf that refers to another vf?

I'll append the full patch I have. Thanh, anyone else, confirm/deny/comments?

Robert, if you're willing, I could send you a new binary (x86_64-linux?) for you to try. I presume your microtype work has more tests that could go some way toward ensuring there's no unexpected side effect.

Thanks, Karl

--- pdftex.web (revision 822) +++ pdftex.web (working copy) @@ -17519,13 +17519,21 @@ procedure vf_expand_local_fonts(f: internal_font_number); var lf: internal_font_number; k: integer; + shrink_limit, stretch_limit: integer; begin pdfassert(pdf_font_type[f] = virtual_font_type); for k := 0 to vf_local_font_num[f] - 1 do begin lf := vf_i_fnts[vf_default_font[f] + k]; + {Find stretch and shrink values, if any.} + if pdf_font_stretch[f] = null_font then stretch_limit := 0 + else stretch_limit := pdf_font_expand_ratio[pdf_font_stretch[f]]; + {Ditto for shrink.} + if pdf_font_shrink[f] = null_font then shrink_limit := 0 + else shrink_limit := pdf_font_expand_ratio[pdf_font_stretch[f]]; + {Now make the font(s) for |lf|.} set_expand_params(lf, pdf_font_auto_expand[f], - pdf_font_expand_ratio[pdf_font_stretch[f]], - -pdf_font_expand_ratio[pdf_font_shrink[f]], + stretch_limit, + -shrink_limit, pdf_font_step[f], pdf_font_expand_ratio[f]); if pdf_font_type[lf] = virtual_font_type then vf_expand_local_fonts(lf); @@ -17962,6 +17970,7 @@ vf_def_font s: str_number; ds, fs: scaled; cs: four_quarters; + stretch_limit, shrink_limit: integer; begin cs.b0 := vf_byte; cs.b1 := vf_byte; cs.b2 := vf_byte; cs.b3 := vf_byte; fs := store_scaled_f(vf_read_signed(4), font_size[f]); @@ -17992,9 +18001,16 @@ vf_local_font_warning(f, k, "design size mismatch"); end; if (pdf_font_step[f] <> 0) then + {Find stretch and shrink values, if any.} + if pdf_font_stretch[f] = null_font then stretch_limit := 0 + else stretch_limit := pdf_font_expand_ratio[pdf_font_stretch[f]]; + {Ditto for shrink.} + if pdf_font_shrink[f] = null_font then shrink_limit := 0 + else shrink_limit := pdf_font_expand_ratio[pdf_font_stretch[f]]; + {Now make the font(s) for |k|.} set_expand_params(k, pdf_font_auto_expand[f], - pdf_font_expand_ratio[pdf_font_stretch[f]], - -pdf_font_expand_ratio[pdf_font_shrink[f]], + stretch_limit, + -shrink_limit, pdf_font_step[f], pdf_font_expand_ratio[f]); vf_def_font := k; end;

_______________________________________________ ntg-pdftex mailing list ntg-pdftex@ntg.nl https://mailman.ntg.nl/mailman/listinfo/ntg-pdftex

-- ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------