June 2005 - ntg-pdftex - NTG Mailing Lists

pdftex and xpdf (and its security bugs): What about a libxpdf?
by frank＠kuesterei.ch 24 Jan '06

24 Jan '06

Dear pdftex developers, during the last months security support for software containing xpdf code was a nightmare - not only that there were many security issues discovered, moreover everybody used a different patch, it was unclear to what extent older versions were affected, and so forth. This was also a problem for us, the teTeX maintainer in Debian, which currently has three versions of pdftex with three different versions of the xpdf code. What would you think about creating a shared library, "libxpdf", and using that for pdftex, either linked dynamically (for Linux/Unix distributions) or statically (for TeX-live and friends)? There is already such a library, libpoppler, a fork of the xpdf code (http://poppler.freedesktop.org/) However, I do not know whether the xpdf author, Derek B. Noonburg, is generally unwilling to create a shared library, or whether people simply didn't ask him. Personally I would prefer to have as little a number of versions of the same code, and thus to get a shared library from xpdf proper. If you agree, I would be oblidged if one of you would contact Derek Noonburg (derekn(a)foolabs.com) and ask him. He seems to be a little unresponsive at times, and has not answered a similar question from the maintainer of the xpdf package in Debian for a couple of weeks now. Maybe we can convince him that this is a good idea if he is approached from different sides. Alternatively, if he is unwilling to make this change, do you think using libpoppler would be feasible? Many thanks in advance, Frank Küster -- Frank Küster Inst. f. Biochemie der Univ. Zürich Debian Developer

2 3

1.30.0-beta-20050615
by Martin Schröder 27 Jul '05

27 Jul '05

src/pdftex-1.30.0-beta-20050615.zip should be on sarovar. Here's the proposed NEWS: ---------------------------------------------------------------- ------------------------------------------------- pdfTeX 3.141592-1.30.0 was released on 2005-06-?? ------------------------------------------------- - Some primites have been renamed; the old names will still work, but are deprecated and give a warning that they are obsolete: \pdfoptionpdfminorversion -> \pdfminorversion \pdfoptionalwaysusepdfpagebox -> \pdfforcepagebox \pdfoptionpdfinclusionerrorlevel -> \pdfinclusionerrorlevel - pdfxtex (introduced with 1.20a) is gone; all extensions are now in pdftex. - pdftex nows two new units: - new Didot (1nd=0.375mm) - new Cicero (1nc=12nd) - New primitive \pdfpkmode, which is a token register that sets the MF mode for pixel font generation. The contents of this register is dumped into the format, so one can (optionally) preset it e.g. in part of pdftexconfig.tex. - New primitives: pdfTeX now has an interface to the precise system time (if available) as an aid in benchmarking TeX macro code. Two new primitives are provided: - \elapsedtime is a read-only integer that (initially) returns the amount of time passed since the start of this run. This amount is given in `scaled seconds': the value 65536 counts as one second. If more time has passed than 32767 seconds, (2^31)-1 will be returned. - \resettime updates the internal timer, such that subsequent calls to \elapsedtime will restart from 0. - New primitive \pdfxformname <object number>, which expands to the number in the corresponding form XObject name. - New primitives \leftmarginkern <box number> and \rightmarginkern <box number> for specifying the margin kern at the left and right side of the horizontal list stored in box <box number>. - Inclusion of pngs has been greatly improved: It now does - alpha channel and transparency (only if %!PDF >=1.4) - 16-bit colour (only if %!PDF >=1.5); this must be enabled by setting \pdfimagehicolor to 1. - gamma correction; this must be enabled by setting \pdfimageapplygamma to 1. The device gamma is taken from \pdfgamma. If no embedded value can be found in the png image, then the value of \pdfimagegamma is used instead. - The effects of font expansion can now be logged: There is now a new integer variable \pdftracingfonts. It is zero by default, then we get a log (with fontexpansion) like this ...\tenrm t ...\tenrm (+20) e Without font expansion, this default should be compatible with TeX's original log output. If \pdftracingfonts is set to 1 (or greater), we get a more verbose log: ...\xivtt (cmtt10(a)14.0pt) t ...\xivtt (cmtt10+20(a)14.0pt) e See also bug 304. - The primitive \pdfmovechar has been removed (bug 294) - Bugfix: empty strings are not supressed anymore (bug 335) - Bugfix: \unhboxing and character protusion didn't work well together (bug 292) - Bugfix: --jobname didn't work if there was no filename at all (bug 302) - Bugfix: Some problem with fonts on solaris (bug 315) - Bugfix: \pdfxform didn't work with box>255 (bug 310) - Bugfix: protruding can lead to "This can't happen(paragraph)." (bug 292) ---------------------------------------------------------------- Thanh: I changed only NEWS, README, pdftex.ch and pdftex.mk (you forgot pdftex2.ch). I didn't include texk/dvipsk as it's not needed for pdftex; we'll have to figure out a way to pass that upstream. Best Martin -- http://www.tm.oneiros.de

8 21

changes in forthcomming release
by The Thanh Han 22 Jul '05

22 Jul '05

Hi, I have applied most of patches and suggestions from Hartmut. There are still a few pending issues: 1) while we are renaming some promitives to be consistent, would it be better to change the following as well: pdfoptionpdfminorversion -> pdfminorversion pdfoptionalwaysusepdfpagebox -> pdfforcepagebox pdfoptionpdfinclusionerrorlevel -> ??? 2) what about moving everything from pdfxtex to pdftex and remove pdfxtex? 3) should the pkmode and microinterval patches be in as well? Martin, can you please wait for a few days, I want to make another snapshot this week. Thanh

5 7

Extensions: \pdf$foo or \$foo?
by Martin Schröder 21 Jul '05

21 Jul '05

Hi, at LinuxTag in Karlsruhe I met with Heiko and Hartmut and we talked about pdfTeX. One point discussed was namespace: Till lately most new primitives of pdfTeX startet with \pdf (\efcode, \lpcode and \rpcode are the only exceptions), but now we have some patches that add primitives that don't start with \pdf (\leftmarginkern, \rightmarginkern, \elapsedtime, \resettime, \random). I'm reluctant to use the non-\pdf-namespace and think it better to stay in \pdf. Comments? Best regards Martin PS: One can always \let \pdf$foo to \$foo in *.ini -- http://www.tm.oneiros.de

7 35

[ pdftex-Bugs-359 ] pdftex generates errors when it does not have to - eg \pdfinfo
by noreply＠sarovar.org 15 Jul '05

15 Jul '05

Bugs item #359, was opened at 2005-05-30 19:51 You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=493&aid=359&group_id=106 Category: None Group: None Status: Open Resolution: None Priority: 3 Submitted By: Taco Hoekwater (taco) Assigned to: Nobody (None) Summary: pdftex generates errors when it does not have to - eg \pdfinfo Initial Comment: The following submitted on behalf of Jonathan Fine: pdftex generates errors when it does not have to. This fault prevents the build of Python docs on some platforms. The fix may be to remove the error-generating code· For more details, see my posting to comp.text.tex. Google link: <http://groups-beta.google.com/group/comp.text.tex/msg/4b4842202ce88cda?hl=en> Copy and paste ============== There is, in my opinion, a problem with pdftex. (This is not to say that the macros used could not be improved.) pdftex exits on \pdfinfo error. TeX does not exit on error. == $ pdftex This is pdfTeX, Version 3.14159-1.10b (Web2C 7.4.5) **\pdfoutput 0 {/usr/share/texmf/pdftex/config/pdftex.cfg} *\pdfinfo pdfTeX error (ext1): \pdfinfo used while \pdfoutput is not set. <*> \pdfinfo No pages of output. Transcript written on texput.log. == The present semantics of \pdfinfo are confused. For example, I _can_ use \pdfinfo and generate dvi! I just have to be a bit sneaky. == $ pdftex This is pdfTeX, Version 3.14159-1.10b (Web2C 7.4.5) **\pdfoutput 1 {/usr/share/texmf/pdftex/config/pdftex.cfg} *\pdfinfo {} *\pdfoutput 0 *\shipout\hbox{} [1] *\end Output written on texput.dvi (1 page, 128 bytes). == Here, if pdf is not being generated, pdfinfo is ignored. Suppose the semantics of \pdfinfo were something like: \pdfinfo{ ... } Stores information that is written to the PDF file, if pdftex generates a PDF file. Otherwise ignored. Then the error that prompted this thread would not have occurred. -- ---------------------------------------------------------------------- >Comment By: Taco Hoekwater (taco) Date: 2005-05-30 20:05 Message: Logged In: YES user_id=1608 Relevant bit of code: procedure check_pdfoutput(s: str_number); begin if pdf_output <= 0 then begin print_nl("pdfTeX error (ext1): "); print(s); print(" used while \pdfoutput is not set"); succumb; end; end The fastest resolution, of course, is to change the test to "pdf_output < 0" so that the error is also generated in dvi processing mode. But Jonathan's proposal is also valid, at least for mode-independant commands like \pdfinfo. Or maybe generate a normal TeX error message? ---------------------------------------------------------------------- You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=493&aid=359&group_id=106

4 5

kerning
by Hans Hagen 11 Jul '05

11 Jul '05

Hi, Given those future font capabilities of pdftex (thanh: what happened with your construct fonts on the fly feature?), i wonder if it is possible to add something \kernfont\somefont <char code 1> <charcode 2> <dimen> Take for instance: http:\\www.pdftex.org here the \\ may come out pretty spaced, and space can be saved by something: \def\url#1{\begingroup\kernfont\font `\ `\ -.2em\relax#1\endgroup} (i soppose that something like this woudl be possible with thanhs new runtime font feature anyway) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

3 6

Patch EscapeAndOther
by Heiko Oberdiek 04 Jul '05

04 Jul '05

Hello, I have uploaded the patch EscapeAndOther, #375. The patch is based on pdftex-1.30.0-rc1. It supersedes patch EscapeDivers.tar.bz2, #371. Inherited from patch EscapeDivers: Expandable: \pdfstrcmp, \pdfescapestring, \pdfescapename Added: \pdfescapehex, \pdfunescapehex Fix: \pdfescapename: '%' is also a delimiter that needs escaping. New in patch EscapeAndOther: * \pdfcreationdate * \pdffilemoddate * \pdffilesize * \pdfmdfivesum * \pdffiledump * \pdfshellescape * \pdfmatch and \pdflastmatch * Start with bug fix: quotes are legal in other operating systems than windows. Syntax: %% expandable commands: \pdfstrcmp <general text> <general text> \pdfescapestring <general text> \pdfescapename <general text> \pdfescapehex <general text> \pdfunescapehex <general text> \pdfcreationdate \pdffilemoddate <general text> \pdffilesize <general text> \pdfmdfivesum <file spec> | <general text> \pdfmatch <match options> <general text> <general text> <match options> := [icase] [subcount <number>] \pdflastmatch <number> %% read-only integers \pdfshellescape Common for the following primitives: * The result string is given by characters with catcode "other" (12), only the space has catcode "space" (10). The commands follow the tradition of \meaning, \string, ... * The argument <general text> is expanded before use. It follows the tradition of \special, \message, \pdfobj, ... Description: \pdfstrcmp{<a>}{} Compares two strings and returns the strings "0" if <a> equals "-1" if <a> is less than "1" if <a> is greater than Use example: \ifcase\pdfstrcmp{abc}{def}\relax \message{abc = def} \or \message{abc > def} \else \message{abc < def} \fi Alternative: Implementing as read-only integer, then the use in \ifcase, \ifnum, ... is more save: \ifcase\pdfstrcmp{xyz}{abc}1\or 2\else 3\fi expands to 3, as read-only integer it expands to the expected 2. \pdfescapestring{<a>} Escapes the string <a> that it can be used as PDF string. '(', ')', '\' are escaped along with the control and 8-bit characters. Use example: \pdfinfo{/Title (\pdfescapestring{...})} \special{ps: [ /Title (\pdfescapestring{...}) /DOCINFO pdfmark} Alternative: Perhaps 8-bit characters don't need escaping. Whitespace, especially newlines should be escaped, because of the use for latex/dvips to avoid recoding problems (<LF> -> <CR><LF>). \pdfescapename{<a>} Escapes the string <a> that it can be used as PDF string. Whitespace, delimiters, '#' are escaped along with the control and 8-bit characters, recommended by the spec. Use example: \pdfobj stream attr{/Type/EmbeddedFile% /Subtype/\pdfescapename{text/plain; charset=iso-8859-1}% ... } file {...} \pdfescapehex{<a>}, \pdfunescapehex{} String <a> is converted to uppercase hexadecimal representation, is converted back. Use example: \pdfescapehex{Hello} is converted to 48656C6C6F \pdfinfo{/Title <\pdfescapehex{Hello}>} Also it can be used to write strings in auxiliary files and later reread without worrying about catcodes, unmached curly braces. \pdfcreationdate It expands to the date string that pdfTeX uses in the info dict as default. Rationale: * It provides seconds and especially the time zone. * Setting of /M date for annotations. * Because of the complicate change file structure of the sources it is not easy to synchronize the creation date with \year, \month, \day and \time. Thus \pdfcreationdate can be used to set these registers to the same values. Example: \pdfcreationdate expands to D:20050625015605+02'00' \pdfannot ...{/Subtype /FileAttachment \M (\pdfcreationdate) ...} \pdffilemoddate{<file>} It expands to the modification date of <file> in the same format as \pdfcreationdate (PDF date format). On error it returns the empty string. Rationale: * File embedding: the date is shown in the attachment tab. * "Make feature": files can be compared, it can be checked, which file is newer. Example: pdfTeX does not support EPS files, epstopdf.sty converts them to PDF either always or, if the PDF variants do not exists. Both ways are not satifactory, either the time penalty is large or pdf files are embedded that are out of date. \pdffilemoddate solves this problem. Error handling, see \pdffilesize I don't have implemented a \pdfcreationdate because of portability issue: The "ctime" field of struct "stat" is interpreted differently among operating systems: * creation date, e.g. win * inode change time, e.g. unix \pdffilesize{<file>} It expands to the size of <file> as string. On error it returns the empty string. Rationale: * File embedding: the size is shown in the attachment tab. * Sometimes it is useful to know if a file has size "0" (failed conversions, ...). Error handling: Currently the empty string is silently returned. Alternatives: * Stop with error message. But what can the user do? Error recovery is easy: no information available, thus return nothing. * Warning message. But the primitives could be used for checks on file existence. * Return status in \pdfretval. I doubt a little whether this is really necessary. It is very easy to implement. But the documentation grows by a large list of error codes with its problems: Much to explain to the users. How they are assigned? \pdfmdfivesum{<abc>} or \pdfmdfivesum file {<file>} It calculates the md5 sum and converts it to uppercase hexadecimal format (same as \pdfescapehex). The syntax is a simplified \pdfobj: Either the data is given directly or in a file. Rationale: * File embedding: providing /CheckSum. * Also the md5 sums of auxiliary files could be stored and compared in order to display a rerun warning. (Of course, it can be possible that different files have the same checksum, but the same file does not have different checksums.) \pdfshellescape It is a read-only integer that is 1 if \write18 is enabled, 0 otherwise. Rationale: * It thought that \ifeof18 with \pdftexversion to implement a safe test for the \write18 feature. But I had to learn that I was wrong, see thread in comp.text.tex: "Confused about pstricks and pdftricks": mikTeX's pdfTeX does not implement \ifeof18. For implementing the test, only a obscure way via \pdftexbanner remains, but this way is not very reliable, the contents of \pdftexbanner is not well defined, it could be anything. Quotes in file names: * File name handling is quite chaotic. Quotes are removed by the scanner for \input, \open*. This behaviour is schizophrene: to solve a problem with spaces, quotes are now forbidden. * Inconsistencies: The syntax of \pdfobj and \pdfximage would allow any file name, but \pdfobj removes quotes, only \pdfximage uses the more intelligent way, it removes quotes for windows only. Thus the patch fixes: * append_to_name (used in pack_file_name): quotes are removed for windows only. This fixes \pdfobj. * utils.c: new function "makecfilename", used by \pdfximage (readimage), \pdffilesize, ... \pdfmatch [icase] [subcount <number>}] {<pattern>}{<string>} Implements pattern matching using the POSIX regex (a standard library at least in my linux). It returns the same values as \pdfstrcmp, but with the following semantics: -1: error case (invalid pattern, ...) 0: no match 1: match found Options: * icase: case insensitive matching * subcount: it sets the table size for found subpatterns. A number "-1" resets the table size to the start default. See the manual page regex.3 and regex.7. The implementation shows a possible interface to pattern matching in TeX. Therefore only the basics is implemented. Flags: * REG_EXTENDED is set in the implementation. * REG_ICASE: can be set by user. * other: not implemented. \pdflastmatch <number> The result of \pdfmatch is stored in an array. The entry "0" contains the match, the following entries submatches. The positions of the matches are also available. They are encoded in the following manner to avoid another primitive: <position> "->" <match string> "->" is used as separator in the tradition of \meaning. There exists macros for parsing the output of \meaning (e.g. in LaTeX: \strip@prefix). The position "-1" with an empty string indicates that this entry is not set. Example: \def\msg#{\immediate\write16 } \msg{\pdfmatch{(l+)o (W(o))}{Hello World}} \msg{\pdflastmatch0} \msg{\pdflastmatch1} \msg{\pdflastmatch2} \msg{\pdflastmatch3} \msg{\pdflastmatch4} Result: 1 2->llo Wo 2->ll 6->Wo 7->o -1-> Alternative: PCRE (Perl-compatible regular expressions) is far more powerful. More options, named subpattern, ... License for 0.4 was GPL compatible, since 0.5 it is BSD, current version is 0.6. The TeX interface could be changed in the following way: * Addition: \pdflastmatchbyname <general text> It extracts matches for named subpattern. * Options can be given by the same name as in the PCRE description: \pdfmatch anchored caseless ... {}{} For easier/faster scanning the options could be restricted to be given in sorted order. * Or options can be given by letters in any order in an additional argument: \pdfmatch{<pattern>}{<options>}{<string>} \pdfmatch{l+}{ai}{Hello World} The implementation could then use strchr to check, whether an option is set. Patch instructions for testing are given in the patch description at sarovar. Have fun Heiko <oberdiek(a)uni-freiburg.de> --

4 13

input
by Hans Hagen 04 Jul '05

04 Jul '05

Hi, how about \pdfinput {filename} \pdfopenin {filename} \pdfopenout {filename} and maybe even one that accepts a hex encoded string \pdfinput hex {filename} \pdfopenin hex {filename} \pdfopenout hex {filename} Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

4 11

[ pdftex-Patches-376 ] Uniform and Normal Distribution Pseudo-random numbers patch
by noreply＠sarovar.org 03 Jul '05

03 Jul '05

Patches item #376, was opened at 2005-06-27 16:45 You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=376&group_id=106 Category: Primitives Group: v1.30.0 Status: Open Resolution: None Priority: 5 Submitted By: Taco Hoekwater (taco) Assigned to: Nobody (None) Summary: Uniform and Normal Distribution Pseudo-random numbers patch Initial Comment: See the patch file itself or the mailing list for actual usage and comments. Requested by Pawel. It is intented to be applied after tex.pch (as usual) ---------------------------------------------------------------------- You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=376&group_id=106

4 4

pdftex patches
by The Thanh Han 30 Jun '05

30 Jun '05

Hi, I was away for a few days and was very impressed by the patches you guys made during that period. This *is* cool, I am quite glad that pdftex is not depending on 1 ror 2 people but there is a rather stable group of developers. Martin, what is the schedule for a next release where all the pending patches should be in? Thanh

3 2