- ntg-pdftex - NTG Mailing Lists

[ pdftex-Patches-368 ] make \pdfstrcmp expandable (experimental patch)
by noreply＠sarovar.org 20 Jun '05

20 Jun '05

Patches item #368, was opened at 2005-06-18 00:29 You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=368&group_id=106 Category: Primitives Group: v1.30.0 >Status: Closed Resolution: None Priority: 5 Submitted By: Hartmut Henkel (hhenkel) Assigned to: Nobody (None) Summary: make \pdfstrcmp expandable (experimental patch) Initial Comment: just moved \pdfstrcmp over to procedure conv_toks, so that the result -1/0/1 is inserted into the input token list. Does this make sense? ---------------------------------------------------------------------- >Comment By: Hartmut Henkel (hhenkel) Date: 2005-06-20 22:09 Message: Logged In: YES user_id=929 Patch 368 can be closed as an improved version is now included in patch 371 by Heiko. Many thanks to him and Taco for debugging! Regards, Hartmut ---------------------------------------------------------------------- Comment By: Heiko Oberdiek (oberdiek) Date: 2005-06-20 17:02 Message: Logged In: YES user_id=1276 Hello, Taco I agree with your analysis, last weekend I found the same variables that need saving, too. Also I have a patch that also make \pdfescape* expandable. Thus \pdflastescaped* are not needed anymore. And I could remove the dependency on a limited buffer by copying the destination strings directly in the string pool above the source string. Third I added \pdfescapehex and \pdfunescapehex that adds encoding/decoding for hexadecimal strings: * Strings can also be given as hex strings in angle brackets. * Safe method to store and read string data to auxiliary files without worrying about catcodes, unmachted curly braces, ... * Help in handling UTF-8/Unicode or other hex coding stuff. I hope, I can attach the patch here in this thread (upload part is missing?. Otherwise I will make an new patch. Patch is applied by patch <EscapeDivers.patch in directory src/texk/web2c/pdftexdir The Makefile must be modified. After the lines $(srcdir)/pdftexdir/tex.pch the line $(srcdir)/pdftexdir/pdfstrcmp.ch is inserted. This is done two times for pdftex and pdfetex. Yours sincerely Heiko <oberdiek(a)uni-freiburg.de> ---------------------------------------------------------------------- Comment By: Taco Hoekwater (taco) Date: 2005-06-20 16:33 Message: Logged In: YES user_id=1608 It makes sense, but it is not 100% safe as-is. In an expanding context, lots of stuff can go one at the same time. Esp., scan_toks touches some global variables, so compare_strings should save the globals def_ref, scanner_status and warning _index to prevent stack corruption later on. For instance, this crashes: % process by initex \input plain \edef\alpha{\pdfstrcmp{alpha}{alph}} \message{\meaning\alpha} \end % I am attaching a change file that modifies compare_strings() , cf. scan_general_text() in eTeX. ---------------------------------------------------------------------- You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=368&group_id=106

1 0

[ pdftex-Patches-371 ] Expandable \pdfstrcmp, \pdfescape*, added \pdf(de)escapehex
by noreply＠sarovar.org 20 Jun '05

20 Jun '05

Patches item #371, was opened at 2005-06-20 17:11 You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=371&group_id=106 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Heiko Oberdiek (oberdiek) Assigned to: Nobody (None) Summary: Expandable \pdfstrcmp, \pdfescape*, added \pdf(de)escapehex Initial Comment: Hello, the result of last weekend is a patch that makes \pdfstrcmp (based on Harmut's work) and the \pdfescape* commands expandable. Thus \pdflastescaped* are not needed anymore. I could remove the dependency on a limited buffer by copying the destination strings directly in the string pool above the source string. Also the patch adds the fixes of bug 369. Fourth I add \pdfescapehex and \pdfunescapehex that implements encoding and decoding for hexadecimal strings: * Strings can also be given as hex strings in angle brackets. * Safe method to store and read string data to auxiliary files without worrying about catcodes, unmachted curly braces, ... * Help in handling UTF-8/Unicode or other hex coding stuff. Patch is applied by patch <EscapeDivers.patch in directory src/texk/web2c/pdftexdir The Makefile needs modifying. After the lines $(srcdir)/pdftexdir/tex.pch the line $(srcdir)/pdftexdir/pdfstrcmp.ch is inserted. This should be done two times for pdftex and pdfetex. Yours sincerely Heiko <oberdiek(a)uni-freiburg.de> ---------------------------------------------------------------------- You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=371&group_id=106

1 0

[ pdftex-Patches-368 ] make \pdfstrcmp expandable (experimental patch)
by noreply＠sarovar.org 20 Jun '05

20 Jun '05

Patches item #368, was opened at 2005-06-18 00:29 You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=368&group_id=106 Category: Primitives Group: v1.30.0 Status: Open Resolution: None Priority: 5 Submitted By: Hartmut Henkel (hhenkel) Assigned to: Nobody (None) Summary: make \pdfstrcmp expandable (experimental patch) Initial Comment: just moved \pdfstrcmp over to procedure conv_toks, so that the result -1/0/1 is inserted into the input token list. Does this make sense? ---------------------------------------------------------------------- Comment By: Heiko Oberdiek (oberdiek) Date: 2005-06-20 17:02 Message: Logged In: YES user_id=1276 Hello, Taco I agree with your analysis, last weekend I found the same variables that need saving, too. Also I have a patch that also make \pdfescape* expandable. Thus \pdflastescaped* are not needed anymore. And I could remove the dependency on a limited buffer by copying the destination strings directly in the string pool above the source string. Third I added \pdfescapehex and \pdfunescapehex that adds encoding/decoding for hexadecimal strings: * Strings can also be given as hex strings in angle brackets. * Safe method to store and read string data to auxiliary files without worrying about catcodes, unmachted curly braces, ... * Help in handling UTF-8/Unicode or other hex coding stuff. I hope, I can attach the patch here in this thread (upload part is missing?. Otherwise I will make an new patch. Patch is applied by patch <EscapeDivers.patch in directory src/texk/web2c/pdftexdir The Makefile must be modified. After the lines $(srcdir)/pdftexdir/tex.pch the line $(srcdir)/pdftexdir/pdfstrcmp.ch is inserted. This is done two times for pdftex and pdfetex. Yours sincerely Heiko <oberdiek(a)uni-freiburg.de> ---------------------------------------------------------------------- Comment By: Taco Hoekwater (taco) Date: 2005-06-20 16:33 Message: Logged In: YES user_id=1608 It makes sense, but it is not 100% safe as-is. In an expanding context, lots of stuff can go one at the same time. Esp., scan_toks touches some global variables, so compare_strings should save the globals def_ref, scanner_status and warning _index to prevent stack corruption later on. For instance, this crashes: % process by initex \input plain \edef\alpha{\pdfstrcmp{alpha}{alph}} \message{\meaning\alpha} \end % I am attaching a change file that modifies compare_strings() , cf. scan_general_text() in eTeX. ---------------------------------------------------------------------- You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=368&group_id=106

1 0

[ pdftex-Patches-368 ] make \pdfstrcmp expandable (experimental patch)
by noreply＠sarovar.org 20 Jun '05

20 Jun '05

Patches item #368, was opened at 2005-06-18 00:29 You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=368&group_id=106 Category: Primitives Group: v1.30.0 Status: Open Resolution: None Priority: 5 Submitted By: Hartmut Henkel (hhenkel) Assigned to: Nobody (None) Summary: make \pdfstrcmp expandable (experimental patch) Initial Comment: just moved \pdfstrcmp over to procedure conv_toks, so that the result -1/0/1 is inserted into the input token list. Does this make sense? ---------------------------------------------------------------------- >Comment By: Taco Hoekwater (taco) Date: 2005-06-20 16:33 Message: Logged In: YES user_id=1608 It makes sense, but it is not 100% safe as-is. In an expanding context, lots of stuff can go one at the same time. Esp., scan_toks touches some global variables, so compare_strings should save the globals def_ref, scanner_status and warning _index to prevent stack corruption later on. For instance, this crashes: % process by initex \input plain \edef\alpha{\pdfstrcmp{alpha}{alph}} \message{\meaning\alpha} \end % I am attaching a change file that modifies compare_strings() , cf. scan_general_text() in eTeX. ---------------------------------------------------------------------- You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=368&group_id=106

1 0

[ pdftex-Patches-366 ] patch to grow pdf_mem dynamically
by noreply＠sarovar.org 20 Jun '05

20 Jun '05

Patches item #366, was opened at 2005-06-15 23:51 You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=366&group_id=106 Category: Memory allocation Group: v1.30.0 Status: Open Resolution: None Priority: 5 Submitted By: Hartmut Henkel (hhenkel) Assigned to: The Thanh Han (hanthethanh) Summary: patch to grow pdf_mem dynamically Initial Comment: experimental patch, pdf_mem_size from texmf.cnf would be ignored then. 10000 < pdf_mem_size < 10000000 (arbitrary upper limit, was 524288) ---------------------------------------------------------------------- >Comment By: Taco Hoekwater (taco) Date: 2005-06-20 15:10 Message: Logged In: YES user_id=1608 I was not seriously suggesting to change the default value to 100, I just had to do something to get the new code doing stuff. It is not obvious to me what macros I should use to fill up pdf_mem quickly :-) ---------------------------------------------------------------------- Comment By: Nobody (None) Date: 2005-06-20 13:42 Message: Logged In: NO 100 is a bit low, hz etc need it, so maybe 2K makes more sense Hans ---------------------------------------------------------------------- Comment By: Taco Hoekwater (taco) Date: 2005-06-20 11:00 Message: Logged In: YES user_id=1608 This version looks smart. I applied it, but I do not have a lot of documents that need more than a few words of pdf_mem, so I changed the compile-time lower limit to 100. Works fine on my test documents. ---------------------------------------------------------------------- You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=366&group_id=106

1 0

[ pdftex-Patches-366 ] patch to grow pdf_mem dynamically
by noreply＠sarovar.org 20 Jun '05

20 Jun '05

Patches item #366, was opened at 2005-06-16 03:21 You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=366&group_id=106 Category: Memory allocation Group: v1.30.0 Status: Open Resolution: None Priority: 5 Submitted By: Hartmut Henkel (hhenkel) Assigned to: The Thanh Han (hanthethanh) Summary: patch to grow pdf_mem dynamically Initial Comment: experimental patch, pdf_mem_size from texmf.cnf would be ignored then. 10000 < pdf_mem_size < 10000000 (arbitrary upper limit, was 524288) ---------------------------------------------------------------------- Comment By: Nobody (None) Date: 2005-06-20 17:12 Message: Logged In: NO 100 is a bit low, hz etc need it, so maybe 2K makes more sense Hans ---------------------------------------------------------------------- Comment By: Taco Hoekwater (taco) Date: 2005-06-20 14:30 Message: Logged In: YES user_id=1608 This version looks smart. I applied it, but I do not have a lot of documents that need more than a few words of pdf_mem, so I changed the compile-time lower limit to 100. Works fine on my test documents. ---------------------------------------------------------------------- You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=366&group_id=106

1 0

[ pdftex-Patches-366 ] patch to grow pdf_mem dynamically
by noreply＠sarovar.org 20 Jun '05

20 Jun '05

Patches item #366, was opened at 2005-06-15 23:51 You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=366&group_id=106 Category: Memory allocation Group: v1.30.0 Status: Open Resolution: None Priority: 5 Submitted By: Hartmut Henkel (hhenkel) Assigned to: The Thanh Han (hanthethanh) Summary: patch to grow pdf_mem dynamically Initial Comment: experimental patch, pdf_mem_size from texmf.cnf would be ignored then. 10000 < pdf_mem_size < 10000000 (arbitrary upper limit, was 524288) ---------------------------------------------------------------------- >Comment By: Taco Hoekwater (taco) Date: 2005-06-20 11:00 Message: Logged In: YES user_id=1608 This version looks smart. I applied it, but I do not have a lot of documents that need more than a few words of pdf_mem, so I changed the compile-time lower limit to 100. Works fine on my test documents. ---------------------------------------------------------------------- You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=366&group_id=106

1 0

[ pdftex-Bugs-369 ] Bug fix for \pdfescapename and \pdfescapestring
by noreply＠sarovar.org 18 Jun '05

18 Jun '05

Bugs item #369, was opened at 2005-06-18 19:00 You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=493&aid=369&group_id=106 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Heiko Oberdiek (oberdiek) Assigned to: Nobody (None) Summary: Bug fix for \pdfescapename and \pdfescapestring Initial Comment: Hello, the attached patch is based on pdftex-1.30.0-rc1. It fixes the following problems with \pdfescapename and \pdfescapestring: * In both cases zero data bytes result in truncated strings, because the length is measured by "strlen". This is fixed by adding a length parameter to the string conversion functions, see utils.c: escapestr() and escapename(). * \pdfescapename: * delimiter character must be escaped, too. * zero byte is ignored, see specification. * documentation: specification added for convertStringToPDFName() in utils.c Patched files: * utils.c main part of the fix, especially function convertStringToPDFName(). * pdftoepdf.cc, epdf.h, ptexlib.h because of changed signature of convertStringToPDFString() Patch is applied by patch <EscapeNameString.patch in directory src/texk/web2c/pdftexdir Test files that test the conversion of all characters: test-escapename.tex test-escapestring.tex Yours sincerely Heiko <oberdiek(a)uni-freiburg.de> ---------------------------------------------------------------------- You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=493&aid=369&group_id=106

1 0

\pdfescapestring/convertStringToPDFString
by Heiko Oberdiek 18 Jun '05

18 Jun '05

Hello, what is the purpose of \pdfescapestring? It calls convertStringToPDFString of utils.c with a useless semantics: char *convertStringToPDFString (char *in) { static char pstrbuf[MAX_PSTRING_LEN]; char *out = pstrbuf; int lin = strlen (in); int i, j; char buf[4]; j = 0; for (i = 0; i < lin; i++) { check_buf(j + sizeof(buf), MAX_PSTRING_LEN); if ((unsigned char)in[i] < ' ') { /* convert control characters into hex */ sprintf (buf, "#%02x", (unsigned int)(unsigned char)in[i]); out[j++] = buf[0]; out[j++] = buf[1]; out[j++] = buf[2]; } else if ((in[i] == '(') || (in[i] == ')')) { /* escape paranthesis */ out[j++] = '\\'; out[j++] = in[i]; } else if (in[i] == '\\') { /* escape backslash */ out[j++] = '\\'; out[j++] = '\\'; } else { /* copy char :-) */ out[j++] = in[i]; } } out[j] = '\0'; return pstrbuf; } It mixes escaping for PDF *strings* with escaping for PDF *names*. '(', ')', '\\' are escaped for PDF strings, control characters are escaped for PDF names. Thus I suggest to change the control character escaping from hexadecimal #xx to octal \ooo: sprintf (buf, "\\%03o", (unsigned int)(unsigned char)in[i]); out[j++] = buf[0]; out[j++] = buf[1]; out[j++] = buf[2]; out[j++] = buf[3]; Then \pdfescapestring could be used for conversions to pdf strings. Also a \pdfescapename / convertStringToPDFName is useful. Here especially the delimiter characters must be escaped in the #xx manner. Yours sincerely Heiko <oberdiek(a)uni-freiburg.de> --

4 11

[ pdftex-Patches-368 ] make \pdfstrcmp expandable (experimental patch)
by noreply＠sarovar.org 18 Jun '05

18 Jun '05

Patches item #368, was opened at 2005-06-18 00:29 You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=368&group_id=106 Category: Primitives Group: v1.30.0 Status: Open Resolution: None Priority: 5 Submitted By: Hartmut Henkel (hhenkel) Assigned to: Nobody (None) Summary: make \pdfstrcmp expandable (experimental patch) Initial Comment: just moved \pdfstrcmp over to procedure conv_toks, so that the result -1/0/1 is inserted into the input token list. Does this make sense? ---------------------------------------------------------------------- You can respond by visiting: http://sarovar.org/tracker/?func=detail&atid=495&aid=368&group_id=106

1 0