Hi, It seems that the buffer size of pdftex is maximized to 300K while in practice much larger files show up (at least here); for instance xml files can be 1 meg one-liners. Is it possible to set the max to (say) 10.000.000? (tex.web) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Hans Hagen
It seems that the buffer size of pdftex is maximized to 300K while in practice much larger files show up (at least here); for instance xml files can be 1 meg one-liners.
Is it possible to set the max to (say) 10.000.000? (tex.web)
I think that is crazy. It would make much more sense not to require the input buffer to actually contain beginning and end of line at the same time, but rather take it as a buffer into the input line. It may require some web2c support to do this since Pascal is pretty much line-based IIRC, but setting aside 10MB for the input line buffer is just absurd. Actually, setting aside 300k is ridiculous already. We should reduce this to something like 4k or so and change the code to cater with incomplete lines, or this madness will go on with ever bigger sizes and discussions. Perhaps one should bring up this issue on the TeX-implementors list? -- David Kastrup
David Kastrup wrote:
line-based IIRC, but setting aside 10MB for the input line buffer is just absurd. Actually, setting aside 300k is ridiculous already.
Who talks about setting aside? Hans' proposal is about alleviating an artificial limit to the (supposedly dynamic) texmf.cnf setting for input_buffer. That limit really serves no purpose that I know of. Why should it matter how large Hans (or anyone else) wants the buffer to be, if the limit is artificial anyway?
We should reduce this to something like 4k or so and change the code to cater with incomplete lines, or this madness will go on with ever bigger sizes and discussions.
I agree, but I have no time to write such code. Changing a constant value is much faster. If you write a change file, I expect it will be incorporated asap. Best wishes, Taco
Taco Hoekwater
David Kastrup wrote:
line-based IIRC, but setting aside 10MB for the input line buffer is just absurd. Actually, setting aside 300k is ridiculous already.
Who talks about setting aside? Hans' proposal is about alleviating an artificial limit to the (supposedly dynamic) texmf.cnf setting for input_buffer. That limit really serves no purpose that I know of.
Why should it matter how large Hans (or anyone else) wants the buffer to be, if the limit is artificial anyway?
We should reduce this to something like 4k or so and change the code to cater with incomplete lines, or this madness will go on with ever bigger sizes and discussions.
I agree, but I have no time to write such code. Changing a constant value is much faster. If you write a change file, I expect it will be incorporated asap.
Change file, or would be a patch to luatex.web (respective to the latest SVN version) be ok? In other words: if I do the code, will someone else volunteer for turning this into some change file form or other that would make it propagate elsewhere (stable PDFTeX? I suppose it would be somewhat pointless to get it incorporated into eTeX, since eTeX does no longer seem to make it into binaries on its own, but exists mostly just as a changeset). Or even submit it to Don, since he is supposed to deal with the next batch of bug reports at the end of this year? Hey, I can dream. -- David Kastrup
David Kastrup wrote:
Change file, or would be a patch to luatex.web (respective to the latest SVN version) be ok? In other words: if I do the code, will
change file for pdftex .. since that's now one source; from there it will find its way into luatex (as all other patches to pdftex do) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
David Kastrup wrote:
Change file, or would be a patch to luatex.web (respective to the latest SVN version) be ok? In other words: if I do the code, will someone else volunteer for turning this into some change file form or other that would make it propagate elsewhere (stable PDFTeX? I suppose it would be somewhat pointless to get it incorporated into eTeX, since eTeX does no longer seem to make it into binaries on its own, but exists mostly just as a changeset).
Personally, I believe it is better to base yourself on pdftex. I have a system set up where migrating stuff from pdftex to luatex is easy for me. When it works, I will port to luatex, no problem. Be warned though, the input buffer is not used *only* for file reading. Good luck, Taco
Taco Hoekwater
David Kastrup wrote:
Change file, or would be a patch to luatex.web (respective to the latest SVN version) be ok? In other words: if I do the code, will someone else volunteer for turning this into some change file form or other that would make it propagate elsewhere (stable PDFTeX? I suppose it would be somewhat pointless to get it incorporated into eTeX, since eTeX does no longer seem to make it into binaries on its own, but exists mostly just as a changeset).
Personally, I believe it is better to base yourself on pdftex.
Sigh. Yet another source to check out.
I have a system set up where migrating stuff from pdftex to luatex is easy for me. When it works, I will port to luatex, no problem. Be warned though, the input buffer is not used *only* for file reading.
Yes, thanks. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum
Is it possible to set the max to (say) 10.000.000? (tex.web) Actually the sup_ and inf_ stuff is part of tex.ch rather than tex.web (or the new pdftex.web), as far as I can tell. For regular old tex, ie, TL's Build/source/texk/web2c/tex.ch, I tacked on a couple of zeros to sup_buf_size: @!sup_buf_size = 30000000; Martin, can you make the analogous change for pdftex please? (Assuming you have no objection.) Jonathan, if buf_size is still relevant in xetex-land, you may as well do it too? Last year I recall that we discussed increasing all those sup_* values to their maximum allowed (no reason not to), but no one (including me, sigh) ever actually did it. Would anyone like to prepare a patch for that? Thanks, Karl
"Karl" == Karl Berry
writes:
Last year I recall that we discussed increasing all those sup_* values to their maximum allowed (no reason not to), but no one (including me, sigh) ever actually did it. Would anyone like to prepare a patch for that?
I fear that I'm the culprit. Someone asked for increasing the size of a certain buffer. I proposed to increase this buffer to the maximum allowed value instead of just to double it. The idea was that if the buffer is as large as possible there will be no reason to touch the code later again. I did not say that all buffers should be increased immediately. What I proposed is that when someone asks for increasing a buffer it should be increased to the maximum allowed value instead of by a small amount. The sole reason is to prevent developers to be bothered by such requests again and again. Regards, Reinhard -- ---------------------------------------------------------------------------- Reinhard Kotucha Phone: +49-511-4592165 Marschnerstr. 25 D-30167 Hannover mailto:reinhard.kotucha@web.de ---------------------------------------------------------------------------- Microsoft isn't the answer. Microsoft is the question, and the answer is NO. ----------------------------------------------------------------------------
2007/3/31, Karl Berry
For regular old tex, ie, TL's Build/source/texk/web2c/tex.ch, I tacked on a couple of zeros to sup_buf_size: @!sup_buf_size = 30000000;
Martin, can you make the analogous change for pdftex please? (Assuming you have no objection.)
We still use _that_ file (for web2c), so we get the change automatically. :-) Best Martin
participants (6)
-
David Kastrup
-
Hans Hagen
-
karl@freefriends.org
-
Martin Schröder
-
Reinhard Kotucha
-
Taco Hoekwater