On Thursday 30 March 2017 23:26:24 Karl Berry wrote:

Subject: [PATCH] pdftex - Add new \pdfpkscalable primitive

Why?

As I wrote at the end of my message (and you didn't reply to it), I see no point in retaining the concepts of scalable Type3 / PGC fonts. As far as I can see all that's needed now is to change the documentation.

I appreciate the effort of trying to retain existing behavior, but in this case, the existing behavior is 100% unused. So I'm not enthusiastic about adding yet more complexity around unused features.

As Karl suggested

Sorry about the apparent confusion. I only suggested making this boolean in the event that there was no other fix. But you already made another fix. So I think we are done (without this patch).

There is no fix for supporting non-scalable PK fonts with enc file. Therefore I introduced this \pdfpkscalable boolean register. Once PK font with enc file is defined in mapline it is used as scalable. You can check this fact by generating PDF file that it contains mapping for ffi ligature. Also check pdftex output that mapline was not ignored. Because if mapline is ignored, then pdftex automatically fallback to non-scalable PK fonts, but without enc file. And when pdftex ignore mapline it just show non-fatal warning and produce PDF file without errors. I was already hit by this behaviour that warnings from mapfile are non-fatal... -- Pali Rohár pali.rohar@gmail.com

I repeat: the current behavior seems correct and desirable to me.

In the following case, as in the previous pdftex, only cmr10.600pk is used, and cmr10.3600pk is not used. Is it ok? \input glyphtounicode \pdfgentounicode=1 \pdfmapline{} % disable all Type1 fonts \pdfmapline{cmr10 <8r.enc} \font\ff=cmr10 \font\fg=cmr10 at 60pt \ff Test \smallskip \fg Test \bye Best, Akira

Hi Karl,

It isn't ok -- we want .3600pk to be used -- but that's not the behavior I am seeing. That is, the behavior *before* the latest patch (which you installed one part of, but my binary is from before that).

Confirmed! I agree that the behavior *before* the latest patch is good. Best, Akira

On Friday 31 March 2017 09:34:02 Pali Rohár wrote:

On Friday 31 March 2017 08:24:14 Akira Kakuto wrote:

...

Hi Karl,

...

It isn't ok -- we want .3600pk to be used -- but that's not the behavior I am seeing. That is, the behavior *before* the latest patch (which you installed one part of, but my binary is from before that).

Confirmed! I agree that the behavior *before* the latest patch is good.

Do you mean without patch "+ } else if (is_std_t1font(fm))" ?

Here is test input:

$ cat test.tex \input glyphtounicode \pdfgentounicode=1 \pdfmapline{} % disable all Type1 fonts \pdfmapline{cmr10 < 8r.enc} \font\ff=cmr10 \font\fg=cmr10 at 60pt \ff Test \smallskip \fg Test \bye

No it is not correct, as wrote in previous email, look at pdftex output. There are non-fatal warnings!

$ pdftex test.tex This is pdfTeX, Version 3.14159265-2.6-1.40.17 (TeX Live 2017/dev) (preloaded format=pdftex) restricted \write18 enabled. entering extended mode (./test.tex (/usr/share/texlive/texmf-dist/tex/generic/pdftex/glyphtounicode.tex)

pdfTeX warning: pdftex: invalid entry for `cmr10': both ps_name and font file missing

[1] ) Output written on test.pdf (1 page, 5627 bytes). Transcript written on test.log.

There is warning:

pdfTeX warning: pdftex: invalid entry for `cmr10': both ps_name and font file missing

which means that \pdfmapline from test.tex was ignored and so no enc file was used. So there is no /ToUnicode entry. You can verify it:

$ pdffonts test.pdf name type emb sub uni object ID ------------------------------------ ----------------- --- --- --- --------- [none] Type 3 yes no no 4 0 [none] Type 3 yes no no 5 0

"uni" is set to no.

What happened is that \pdfmapline for cmr10 was ignored, so pdftex fallbacks to PK fonts (by default) which is automatically non-scalable and without any encoding file.

Some more clarification about current state (svn rev 43645 in texlive): 1 Every font specified in \pdfmapline or \pdfmapfile is scalable. This was truth also before and also with my patches up to svn rev 43645. 2 It supports specifying PK/Type3 font in \pdfmapline or \pdfmapline. Due to 1), such font is scalable and so included in output PDF only once. 3 Entry in ChangeLog file is wrong and misleading https://www.tug.org/svn/texlive/trunk/Build/source/texk/web2c/pdftexdir/ChangeLog?view=markup&pathrev=43645 "This way, each bitmap font is embedded separately, thus rasterized by MF instead of geometrically scaling up." This is not truth and in specified link is "enabling scalable PK fonts": https://mailman.ntg.nl/pipermail/ntg-pdftex/2017-March/004125.html When font is "scalable" then it is included in PDF file only once. When font is not "scalable" then it is generated for more different resolutions and so included in PDF file more times. Probably this is reason for misunderstanding and together with not-so-visible non-fatal warning message "both ps_name and font file missing". I hope now with example in previous email with above clarification it is more clear what happened. -- Pali Rohár pali.rohar@gmail.com

On Sunday 02 April 2017 15:10:43 Akira Kakuto wrote:

Dear Pali,

...

--- writet3.c +++ writet3.c

Installed in r43680. Now I can see uni = yes in pdffonts test.pdf. Many thanks. I have added a small change in mapfile.c to warn if only a tfm name is specified in a map for type3.

Best, Akira

Ok, so finally enc files works and PDF files seems to be correctly generated. Many thanks! -- Pali Rohár pali.rohar@gmail.com

I'm sorry, but I am not comfortable with releasing these changes for the Type 3 fonts + encoding + Unicode. I realize the features now work in your cases, but I feel much more testing and understanding is needed, before inflicting changes to these low-level routines on the world. I can't undertake this for TL17. The proximate cause is that, in some further testing, I discovered a case that crashed for me now, but did not crash before: \pdfmapline{<} Also, it's unclear to when the new warning "invalid entry for `%s': encoding file for type3 missing" is intended to be given. I could not find any input that caused it. I'm sure that it would not be hard to remedy these particular things, one way or another, but the general principle remains -- I'm simply too uneasy to release these changes, which have grown and grown. As I said before, the primary criterion for pdftex nowadays must be stability. Unfortunately, in that testing, I also discovered other erroneous mapline cases that were already unhandled (before your patches). I know I have not found them all yet. Sigh. If Thanh or Martin wants to include these changes in the upcoming release, and can help with testing/code review, then that is fine. Otherwise, it will have to wait for another time. For the present, I have reverted the changes in the TL repository. (Hopefully correctly.) At least you have written the code now, so can compile your own pdftex and use it for your own work ... --sorry, karl.

On Saturday 15 April 2017 05:49:23 Pali Rohár wrote:

On Saturday 15 April 2017 01:18:44 Karl Berry wrote:

...

I'm sorry, but I am not comfortable with releasing these changes for the Type 3 fonts + encoding + Unicode. I realize the features now work in your cases, but I feel much more testing and understanding is needed, before inflicting changes to these low-level routines on the world. I can't undertake this for TL17.

The proximate cause is that, in some further testing, I discovered a case that crashed for me now, but did not crash before: \pdfmapline{<}

That is because in mapfile.c check_fm_entry() is missing check if tfm_name is not NULL.

Such thing is possible to hit also without my patches... And is not related to my patches.

Try:

\pdfmapline{

and even on old TeX Live 2012 it crashed too:

../../../texk/web2c/pdftexdir/subfont.c:181: handle_subfont_fm: Assertion `fm->tfm_name != ((void *)0)' failed.

So it is there for a long time.

...

Also, it's unclear to when the new warning "invalid entry for `%s': encoding file for type3 missing" is intended to be given. I could not find any input that caused it.

That was added by Akira.

...

I'm sure that it would not be hard to remedy these particular things, one way or another, but the general principle remains -- I'm simply too uneasy to release these changes, which have grown and grown. As I said before, the primary criterion for pdftex nowadays must be stability.

Unfortunately, in that testing, I also discovered other erroneous mapline cases that were already unhandled (before your patches). I know I have not found them all yet. Sigh.

If Thanh or Martin wants to include these changes in the upcoming release, and can help with testing/code review, then that is fine. Otherwise, it will have to wait for another time.

For the present, I have reverted the changes in the TL repository. (Hopefully correctly.)

At least you have written the code now, so can compile your own pdftex and use it for your own work ... --sorry, karl.

So what is needed for inclusion? More testing?

And do we have a case where pdftex behave incorrect (or crash) which is related to those toUnicode patches for PK fonts? Because I have not found any problem in current code yet and above crash (which you found) is there for a long time.

Do you want a fix for above crash?

Just to note that I sent patch for PK encfile support in Aug 2016... So still I would like to hear what is the problem with last version and how can I fix it or improve if needed. And question about pdftex crash when tfmfile is not specified still remind: Should I look at it and propose fix? -- Pali Rohár pali.rohar@gmail.com

Pali - I regretfully backed out the new feature long ago, as I said. I cannot contemplate this further for TL17. Sorry. I did my best to fix possible crashes in cases of pathological input in the current code (in the TL repository for now). --best, karl.

Ok, then for next year release? I suppose. I need to ask you to redo your patch against the current source (still only in TeX Live for now), with ChangeLog entries. It would also greatly help me if you provided as-simple-as-possible source files, along the lines of what is in pdftex/tests/*, exhibiting the intended changes in behavior. Because that's what I need to be confident in the changes. Not as critical, but still quite helpful, would be a draft patch for the manual. --best, karl.