Re: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

newer
[Context] ConTeXt minimals behind...

older
[Context] strange: luatex still at...

Mojca Miklavec

17 Jul 2009 17 Jul '09

4:44 p.m.

On Fri, Jul 17, 2009 at 12:31, Taco Hoekwater wrote:

...

Hi,

I have just uploaded the archives for a new luatex release, 0.42.0.

NOTE: This snapshot is just for testing the new sources and the updated build process, do NOT trust this binary for production work!

Independent of that (considering the fact that luatex should not be trusted for production work anyway :) :) :) :) ... We should probably build it for the minimals, right? Mojca

Show replies by date

Taco Hoekwater

17 Jul 17 Jul

4:47 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Mojca Miklavec wrote:

...

On Fri, Jul 17, 2009 at 12:31, Taco Hoekwater wrote:

...
Hi,

I have just uploaded the archives for a new luatex release, 0.42.0.

NOTE: This snapshot is just for testing the new sources and the updated build process, do NOT trust this binary for production work!

Independent of that (considering the fact that luatex should not be trusted for production work anyway :) :) :) :) ...

We should probably build it for the minimals, right?

Yes, I would think so. There'll be a followup next week, probably. Best wishes, Taco

Mojca Miklavec

9:15 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

On Fri, Jul 17, 2009 at 18:47, Taco Hoekwater wrote:

...

Mojca Miklavec wrote:

...
We should probably build it for the minimals, right?

Yes, I would think so. There'll be a followup next week, probably.

Uoooops ... I just noticed that we filter the version with svn list $LUATEX_SVN/tags | grep beta | tail -1 | tr -d '/' but this version is named 0.42.0, no beta prefix, so the sources didn't get updated yet (I was away in the meantime). Should we change the naming pattern? I have fixed it manually now, so the sources should be ready for building in a few minutes. And I guess that we need to unfreeze beta as well. Mojca

Taco Hoekwater

9:58 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Mojca Miklavec wrote:

...

On Fri, Jul 17, 2009 at 18:47, Taco Hoekwater wrote:

...
Mojca Miklavec wrote:

...
We should probably build it for the minimals, right? Yes, I would think so. There'll be a followup next week, probably.

Uoooops ... I just noticed that we filter the version with svn list $LUATEX_SVN/tags | grep beta | tail -1 | tr -d '/' but this version is named 0.42.0, no beta prefix, so the sources didn't get updated yet (I was away in the meantime).

no problem, I just now created an extra tag _with_ the prefix, and will try to think about that from now on.

...

And I guess that we need to unfreeze beta as well.

definately. Best wishes, Taco

Mojca Miklavec

18 Jul 18 Jul

8:25 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

On Fri, Jul 17, 2009 at 23:58, Taco Hoekwater wrote:

...

Mojca Miklavec wrote:

...
Uoooops ... I just noticed that we filter the version with svn list $LUATEX_SVN/tags | grep beta | tail -1 | tr -d '/' but this version is named 0.42.0, no beta prefix, so the sources didn't get updated yet (I was away in the meantime).

no problem, I just now created an extra tag _with_ the prefix, and will try to think about that from now on.

I wasn't sure. It could have been on purpose (you'll probably want to drop the beta tag one day and I thought that maybe that day has come already :). I'll wait at least for Akira's binary (linux, linux-64, mac and freebsd binaries have been commited by now (thanks guys); only freebsd-amd64, linux-ppc and solaris are missing, but that should be OK) and then I'll unfreeze the beta. Or maybe I should just take the svn version (2888; beta was 2881/2882). Thanks, Mojca

Hans Hagen

9:34 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Mojca Miklavec wrote:

...

I'll wait at least for Akira's binary (linux, linux-64, mac and

fyi: i've been doing some speed test yesterday and was puzzled by the speed of luatex on linux ... as it is a 64 server, i just tried the new bins from the garden the run went down from 1.7 seconds (32 bit) to 1.1 seconds (64 bit) ... quite a difference! Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Taco Hoekwater

10:23 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Hans Hagen wrote:

...

Mojca Miklavec wrote:

...
I'll wait at least for Akira's binary (linux, linux-64, mac and

fyi: i've been doing some speed test yesterday and was puzzled by the speed of luatex on linux ... as it is a 64 server, i just tried the new bins from the garden

the run went down from 1.7 seconds (32 bit) to 1.1 seconds (64 bit) ... quite a difference!

This is a fairly typical effect of the AMD64 architecture's extra cpu registers in 64-bit mode (although the effect is not often as big as it is in tex) Best wishes, Taco

Yue Wang

11:35 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Hi, right. btw, maybe we can try to build universal binaries (x86_64 + i386) for minimals on OS X... Yue Wang On Sat, Jul 18, 2009 at 6:23 PM, Taco Hoekwater wrote:

...

Hans Hagen wrote:

...
Mojca Miklavec wrote:

...
I'll wait at least for Akira's binary (linux, linux-64, mac and

fyi: i've been doing some speed test yesterday and was puzzled by the speed of luatex on linux ... as it is a 64 server, i just tried the new bins from the garden

the run went down from 1.7 seconds (32 bit) to 1.1 seconds (64 bit) ... quite a difference!

This is a fairly typical effect of the AMD64 architecture's extra cpu registers in 64-bit mode (although the effect is not often as big as it is in tex)

Best wishes, Taco

_______________________________________________ Mailing list: https://launchpad.net/~context Post to : context@lists.launchpad.net Unsubscribe : https://launchpad.net/~context More help : https://help.launchpad.net/ListHelp

Taco Hoekwater

11:40 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Yue Wang wrote:

...

Hi,

right.

btw, maybe we can try to build universal binaries (x86_64 + i386) for minimals on OS X...

I do not have time (nor knowledge) to figure that out myself, but patches are more than welcome, of course. Best wishes, Taco

Yue Wang

12:01 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

I think just add two -arch will do the job. I am trying to build luatex universal binaries now and will send the patch to you later... On Sat, Jul 18, 2009 at 7:40 PM, Taco Hoekwater wrote:

...

Yue Wang wrote:

...
Hi,

right.

btw, maybe we can try to build universal binaries (x86_64 + i386) for minimals on OS X...

I do not have time (nor knowledge) to figure that out myself, but patches are more than welcome, of course.

Best wishes, Taco

Mojca Miklavec

12:09 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

On Sat, Jul 18, 2009 at 13:40, Taco Hoekwater wrote:

...

Yue Wang wrote:

...
Hi,

right.

btw, maybe we can try to build universal binaries (x86_64 + i386) for minimals on OS X...

I do not have time (nor knowledge) to figure that out myself, but patches are more than welcome, of course.

The universal binaries are there already. Building directly might be a bit problematic in complex building procedures, so we use lipo to combine binaries together. As a conseqence we need to build all the binaries separately anyway. Minimals could use universal binaries, but that would have a disadvantage of bigger size and no real advantage unless I fail to see it). Oliver has started building some installer and that one would be free to use the universal binaries if it ever gets built. As soon as there will be some apparent advantage, one only needs to patch ConTeXt core to recognize any mac as "osx-universal" instead of "osx-intel".

...

I think just add two -arch will do the job. I am trying to build luatex universal binaries now and will send the patch to you later...

It can even be that this works for LuaTeX/pdftex/metapost, but it fails for XeTeX (for XeTeX I'm not even able to build the binaries for Tiger on Leopard). I figured out it would be far easier to just use lipo instead of using two architectures, but that's pretty much irrelevant question. The real question is: why would we want to use universal binaries? As soon as some plausible reason comes, switching to universal is a matter of: - tiny change in core - users will need to patch "first-setup.sh" (that's going to be renamed/rewritten anyway) and fetch new binaries for osx-universal instead of osx-[intel|ppc] Everything else is probably there already. Mojca

Yue Wang

12:24 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

just a simple test. (works on mac os x 10.5.x, intel mac. on idea on other platforms...) 240:luatex yue$ svn diff Index: build.sh =================================================================== --- build.sh (revision 2888) +++ build.sh (working copy) @@ -59,6 +59,11 @@ if [ `uname` = "Darwin" ] ; then export MACOSX_DEPLOYMENT_TARGET=10.4 + CFLAGS="-arch x86_64 -arch i386 $CFLAGS" + XCFLAGS="-arch x86_64 -arch i386 $XCFLAGS" + CXXFLAGS="-arch x86_64 -arch i386 $CXXFLAGS" + LDFLAGS="-arch x86_64 -arch i386 $LDFLAGS" + export CFLAGS CXXFLAGS LDFLAGS XCFLAGS fi; B=build @@ -165,6 +170,7 @@ --disable-shared \ --disable-largefile \ --disable-ipc \ + --disable-dependency-tracking \ --without-mf-x-toolkit --without-x \ || exit 1 fi 240:luatex yue$ file build/texk/web2c/luatex build/texk/web2c/luatex: Mach-O universal binary with 2 architectures build/texk/web2c/luatex (for architecture x86_64): Mach-O 64-bit executable x86_64 build/texk/web2c/luatex (for architecture i386): Mach-O executable i386 240:luatex yue$ On Sat, Jul 18, 2009 at 7:40 PM, Taco Hoekwater wrote:

...

Yue Wang wrote:

...
Hi,

right.

btw, maybe we can try to build universal binaries (x86_64 + i386) for minimals on OS X...

I do not have time (nor knowledge) to figure that out myself, but patches are more than welcome, of course.

Best wishes, Taco

Mojca Miklavec

1 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

On Sat, Jul 18, 2009 at 14:24, Yue Wang wrote:

...

just a simple test. (works on mac os x 10.5.x, intel mac. on idea on other platforms...)

240:luatex yue$ svn diff Index: build.sh =================================================================== --- build.sh (revision 2888) +++ build.sh (working copy) @@ -59,6 +59,11 @@ if [ `uname` = "Darwin" ] ; then export MACOSX_DEPLOYMENT_TARGET=10.4 + CFLAGS="-arch x86_64 -arch i386 $CFLAGS" + XCFLAGS="-arch x86_64 -arch i386 $XCFLAGS" + CXXFLAGS="-arch x86_64 -arch i386 $CXXFLAGS" + LDFLAGS="-arch x86_64 -arch i386 $LDFLAGS" + export CFLAGS CXXFLAGS LDFLAGS XCFLAGS fi;

I'm sorry, I didn't read careful enough. I thought we were talking about intel vs. ppc. Adding those flags to build.shu nconditionally would mean that anyone trying to compile LuaTeX on ppc won't be able to do so for his own platform, so such flags would need to go to a separate section (if user supplies some special flag to build in such mode). We definitely need to start thinking about 64-bit support, but not unconditionally. Apparently 10.6 won't support ppc any more and Yue says that 64-bit version of luatex runs slower and consumes more memory (+ being twice as big). We should either discuss it a bit or wait until the first user gets bitten by the issue. After all we also depend on TL binaries that are most probably not 64-bit. Mojca

Yue Wang

1:30 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Hi, Taco and Mojca: On Sat, Jul 18, 2009 at 9:00 PM, Mojca Miklavec wrote:

...

On Sat, Jul 18, 2009 at 14:24, Yue Wang wrote:

...
just a simple test. (works on mac os x 10.5.x, intel mac. on idea on other platforms...)

240:luatex yue$ svn diff Index: build.sh =================================================================== --- build.sh (revision 2888) +++ build.sh (working copy) @@ -59,6 +59,11 @@ if [ `uname` = "Darwin" ] ; then export MACOSX_DEPLOYMENT_TARGET=10.4 + CFLAGS="-arch x86_64 -arch i386 $CFLAGS" + XCFLAGS="-arch x86_64 -arch i386 $XCFLAGS" + CXXFLAGS="-arch x86_64 -arch i386 $CXXFLAGS" + LDFLAGS="-arch x86_64 -arch i386 $LDFLAGS" + export CFLAGS CXXFLAGS LDFLAGS XCFLAGS fi;

I'm sorry, I didn't read careful enough. I thought we were talking about intel vs. ppc.

Adding those flags to build.shu nconditionally would mean that anyone trying to compile LuaTeX on ppc won't be able to do so for his own platform, so such flags would need to go to a separate section (if user supplies some special flag to build in such mode).

my fault. there should be a conditional test there. but I have no ppc machine to test as i mentioned in the previous mail.

...

We definitely need to start thinking about 64-bit support, but not unconditionally. Apparently 10.6 won't support ppc any more and Yue says that 64-bit version of luatex runs slower and consumes more memory (+ being twice as big).

Yes, here are the log files when running the x86 and x86_64 binaries using the same test document. x86_64: mkiv lua stats : used config path - /Users/yue/context/tex/texmf/web2c/texmf.cnf mkiv lua stats : used cache path - /Users/yue/context/tex/texmf-cache mkiv lua stats : input load time - 0.053 seconds mkiv lua stats : stored bytecode data - 170 modules, 45 tables, 215 chunks mkiv lua stats : node list callback tasks - 4 unique tasks, 3 created, 810 calls mkiv lua stats : cleaned up reserved nodes - 39 nodes, 10 lists of 1412 mkiv lua stats : node memory usage - 138 attribute, 19 glue_spec, 46 attribute_list, 46 local_par, 2 dir mkiv lua stats : h-node processing time - 0.922 seconds including kernel mkiv lua stats : attribute processing time - 0.257 seconds mkiv lua stats : loaded patterns - en:us:pat:exc:2 mkiv lua stats : startup time - 1.205 seconds (including runtime option file processing) mkiv lua stats : callbacks - direct: 3838, indirect: 5118, total: 8956 mkiv lua stats : loaded fonts - 38 files: adobesongstd-light.otf minionpro-bold.otf minionpro-it.otf minionpro-regular.otf myriadpro-bold.otf myriadpro-regular.otf tektonpro-regular.otf lmmono12-regular.otf lmmono8-regular.otf lmmono9-regular.otf lmroman12-bold.otf lmroman12-regular.otf lmroman7-bold.otf lmroman7-regular.otf lmroman9-bold.otf lmroman9-regular.otf lmsans12-regular.otf lmsans8-regular.otf lmsans9-regular.otf eufb10.tfm eufb7.tfm eufb9.tfm eufm10.tfm eufm7.tfm eufm9.tfm msam10.tfm msam7.tfm msbm10.tfm msbm7.tfm lmex10.tfm lmmi12.tfm lmmi7.tfm lmmi9.tfm lmmib10.tfm lmmib7.tfm lmsy10.tfm lmsy7.tfm lmsy9.tfm mkiv lua stats : fonts load time - 2.163 seconds mkiv lua stats : graphics processing time - 0.098 seconds including tex, n=23 mkiv lua stats : result saved in file - ps.pdf mkiv lua stats : luatex banner - this is luatex, version beta-0.42.0-2009071820 mkiv lua stats : control sequences - 30398 of -443887883 mkiv lua stats : current memory usage - 104 MB (ctx: 107 MB) mkiv lua stats : runtime - 7.127 seconds, 28 processed pages, 28 shipped pages, 3.929 pages/secondMTXrun | total runtime: 7.198 i386: mkiv lua stats : used config path - /Users/yue/context/tex/texmf/web2c/texmf.cnf mkiv lua stats : used cache path - /Users/yue/context/tex/texmf-cache mkiv lua stats : input load time - 0.038 seconds mkiv lua stats : stored bytecode data - 170 modules, 45 tables, 215 chunks mkiv lua stats : node list callback tasks - 4 unique tasks, 3 created, 810 calls mkiv lua stats : cleaned up reserved nodes - 39 nodes, 10 lists of 1412 mkiv lua stats : node memory usage - 138 attribute, 19 glue_spec, 46 attribute_list, 46 local_par, 2 dir mkiv lua stats : h-node processing time - 0.608 seconds including kernel mkiv lua stats : attribute processing time - 0.162 seconds mkiv lua stats : loaded patterns - en:us:pat:exc:2 mkiv lua stats : startup time - 0.898 seconds (including runtime option file processing) mkiv lua stats : callbacks - direct: 3838, indirect: 5118, total: 8956 mkiv lua stats : loaded fonts - 38 files: adobesongstd-light.otf minionpro-bold.otf minionpro-it.otf minionpro-regular.otf myriadpro-bold.otf myriadpro-regular.otf tektonpro-regular.otf lmmono12-regular.otf lmmono8-regular.otf lmmono9-regular.otf lmroman12-bold.otf lmroman12-regular.otf lmroman7-bold.otf lmroman7-regular.otf lmroman9-bold.otf lmroman9-regular.otf lmsans12-regular.otf lmsans8-regular.otf lmsans9-regular.otf eufb10.tfm eufb7.tfm eufb9.tfm eufm10.tfm eufm7.tfm eufm9.tfm msam10.tfm msam7.tfm msbm10.tfm msbm7.tfm lmex10.tfm lmmi12.tfm lmmi7.tfm lmmi9.tfm lmmib10.tfm lmmib7.tfm lmsy10.tfm lmsy7.tfm lmsy9.tfm mkiv lua stats : fonts load time - 1.643 seconds mkiv lua stats : graphics processing time - 0.082 seconds including tex, n=23 mkiv lua stats : result saved in file - ps.pdf mkiv lua stats : luatex banner - this is luatex, version beta-0.42.0-2009071723 mkiv lua stats : control sequences - 30398 of 147189 mkiv lua stats : current memory usage - 69 MB (ctx: 71 MB) mkiv lua stats : runtime - 5.173 seconds, 28 processed pages, 28 shipped pages, 5.413 pages/secondMTXrun | total runtime: 5.221 as you can see, 64 bits is slower, and consume much more memory. (usually 64bits apps should perform faster on math since there are extra registers) In my humble opinion this is because TeX manages memory by it self... and maybe a 64bit memory word of packed data uses twice of memory as 32bits one.... I am not sure.

...

We should either discuss it a bit or wait until the first user gets bitten by the issue. After all we also depend on TL binaries that are most probably not 64-bit.

well, I think we can provide more option to the users... since luatex have been converted to C completely, so no web2c is involved (that is, no yacc and bison anymore....) and now it might be possible to - build luatex using Visual Studio - build under mingw on Windows, (not cross compile) - build on various machine using llvm-clang (clang is the future compiler for Mac OS X, FreeBSD and many other systems. clang compiles faster, generates better code, and it's free under uiuc license (better than gpl). GCC might be dropped out of the base system completely in FreeBSD 9.0)

...

Mojca

_______________________________________________ Mailing list: https://launchpad.net/~context Post to : context@lists.launchpad.net Unsubscribe : https://launchpad.net/~context More help : https://help.launchpad.net/ListHelp

Yue Wang

Taco Hoekwater

3:22 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Yue Wang wrote:

...

In my humble opinion this is because TeX manages memory by it self... and maybe a 64bit memory word of packed data uses twice of memory as 32bits one.... I am not sure.

It could be because web2c by default declares the integer type as "long int" on 64-bit platforms, and that that is what makes everything twice the size, but the --enable-dump-share argument to configure is supposed to prevent it from doing so, so it seems something odd is going on. Can you try to check the sizeof(integer) within a debugger? (the 64bit linux is indeed faster than the 32bit linux on the same hardware) Best wishes, Taco

Diego Depaoli

3:52 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

2009/7/18 Taco Hoekwater :

...

Yue Wang wrote:

...
In my humble opinion this is because TeX manages memory by it self... and maybe a 64bit memory word of packed data uses twice of memory as 32bits one.... I am not sure.

It could be because web2c by default declares the integer type as "long int" on 64-bit platforms, and that that is what makes everything twice the size, but the --enable-dump-share argument to configure is supposed to prevent it from doing so, so it seems something odd is going on. Can you try to check the sizeof(integer) within a debugger?

(the 64bit linux is indeed faster than the 32bit linux on the same hardware) There is anywhere a benchmark file?

Cheers -- Diego Depaoli

Taco Hoekwater

4 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Diego Depaoli wrote:

...

2009/7/18 Taco Hoekwater :

...
Yue Wang wrote:

...
In my humble opinion this is because TeX manages memory by it self... and maybe a 64bit memory word of packed data uses twice of memory as 32bits one.... I am not sure. It could be because web2c by default declares the integer type as "long int" on 64-bit platforms, and that that is what makes everything twice the size, but the --enable-dump-share argument to configure is supposed to prevent it from doing so, so it seems something odd is going on. Can you try to check the sizeof(integer) within a debugger?

(the 64bit linux is indeed faster than the 32bit linux on the same hardware) There is anywhere a benchmark file?

Not really, I just run the luatexref-t.tex document for timing tests. Best wishes, Taco

Diego Depaoli

5:13 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

2009/7/18 Taco Hoekwater :

...

Diego Depaoli wrote:

...
There is anywhere a benchmark file?

Not really, I just run the luatexref-t.tex document for timing tests.

Just tried, I get %context luatexref-t.tex define font | font with name LMRoman10-BoldObliqu is not found define font | unknown font LMRoman10-BoldObliqu, loading aborted define font | unable to define LMRoman10-BoldObliqu as \*modern12ptrmbs* ! Undefined control sequence. <recently read> \ALEPH l.76 \item \ALEPH \ RC4 (from the \TEXLIVE\ repository) %context --version MTXrun | main context file: /home/diego/tex/texmf-context/tex/context/base/context.tex MTXrun | current version: 2009.07.17 17:23 %luatex --version This is LuaTeX, Version beta-0.42.0-2009071808 Cheers -- Diego Depaoli

Taco Hoekwater

5:31 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Diego Depaoli wrote:

...

2009/7/18 Taco Hoekwater :

...
Diego Depaoli wrote:

...
There is anywhere a benchmark file? Not really, I just run the luatexref-t.tex document for timing tests.

Just tried, I get

%context luatexref-t.tex

define font | font with name LMRoman10-BoldObliqu is not found define font | unknown font LMRoman10-BoldObliqu, loading aborted define font | unable to define LMRoman10-BoldObliqu as \*modern12ptrmbs*

this appears to be a typo in type-siz (missing e). Hans?

...

! Undefined control sequence. <recently read> \ALEPH

This logo is defined in s-abr-01.tex, in the same directory as luatexref-t.tex. Is it not found? Best wishes, Taco

Diego Depaoli

6:53 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

2009/7/18 Taco Hoekwater :

...

Diego Depaoli wrote:

...
! Undefined control sequence. <recently read> \ALEPH

This logo is defined in s-abr-01.tex, in the same directory as luatexref-t.tex. Is it not found? Fixed. I ran luatexref in a wrong directory Many thanks

For the record on a FreeBSD i386 mkiv lua stats : fonts load time - 1.163 seconds mkiv lua stats : metapost processing time - 0.184 seconds, loading: 0.058 seconds, execution: 0.100 seconds, n: 163 mkiv lua stats : result saved in file - luatexref-t.pdf mkiv lua stats : luatex banner - this is luatex, version beta-0.42.0-2009071808 mkiv lua stats : control sequences - 31676 of 147189 mkiv lua stats : current memory usage - 124 MB (ctx: 127 MB) mkiv lua stats : runtime - 12.784 seconds, 166 processed pages, 166 shipped pages, 12.985 pages/second MTXrun | total runtime: 36.778 -- Diego Depaoli

Hans Hagen

19 Jul 19 Jul

10:57 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Diego Depaoli wrote:

...

2009/7/18 Taco Hoekwater :

...
Diego Depaoli wrote:

...
! Undefined control sequence. <recently read> \ALEPH This logo is defined in s-abr-01.tex, in the same directory as luatexref-t.tex. Is it not found? Fixed. I ran luatexref in a wrong directory Many thanks

For the record on a FreeBSD i386 mkiv lua stats : fonts load time - 1.163 seconds mkiv lua stats : metapost processing time - 0.184 seconds, loading: 0.058 seconds, execution: 0.100 seconds, n: 163 mkiv lua stats : result saved in file - luatexref-t.pdf mkiv lua stats : luatex banner - this is luatex, version beta-0.42.0-2009071808 mkiv lua stats : control sequences - 31676 of 147189 mkiv lua stats : current memory usage - 124 MB (ctx: 127 MB) mkiv lua stats : runtime - 12.784 seconds, 166 processed pages, 166 shipped pages, 12.985 pages/second MTXrun | total runtime: 36.778

good, so luatex/mkiv performs quite the same as on our machines (i wonder if hartmut will test it on his little nokia machine, as he used to test pdftex on it) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Diego Depaoli

6:41 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

2009/7/18 Diego Depaoli :

...

For the record on a FreeBSD i386 mkiv lua stats : fonts load time - 1.163 seconds mkiv lua stats : metapost processing time - 0.184 seconds, loading: 0.058 seconds, execution: 0.100 seconds, n: 163 mkiv lua stats : result saved in file - luatexref-t.pdf mkiv lua stats : luatex banner - this is luatex, version beta-0.42.0-2009071808 mkiv lua stats : control sequences - 31676 of 147189 mkiv lua stats : current memory usage - 124 MB (ctx: 127 MB) mkiv lua stats : runtime - 12.784 seconds, 166 processed pages, 166 shipped pages, 12.985 pages/second MTXrun | total runtime: 36.778

on a FreeBSD-amd64 I get mkiv lua stats : fonts load time - 1.135 seconds mkiv lua stats : metapost processing time - 0.176 seconds, loading: 0.034 seconds, execution: 0.108 seconds, n: 163 mkiv lua stats : result saved in file - luatexref-t.pdf mkiv lua stats : luatex banner - this is luatex, version beta-0.42.0-2009071800 mkiv lua stats : control sequences - 32785 of 16877400 mkiv lua stats : current memory usage - 178 MB (ctx: 182 MB) mkiv lua stats : runtime - 12.088 seconds, 166 processed pages, 166 shipped pages, 13.733 pages/secondMTXrun | total runtime: 35.553 Cheers -- Diego Depaoli

Hans Hagen

10:59 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Taco Hoekwater wrote:

...

Diego Depaoli wrote:

...
2009/7/18 Taco Hoekwater :

...
Diego Depaoli wrote:

...
There is anywhere a benchmark file? Not really, I just run the luatexref-t.tex document for timing tests.

Just tried, I get

%context luatexref-t.tex

define font | font with name LMRoman10-BoldObliqu is not found define font | unknown font LMRoman10-BoldObliqu, loading aborted define font | unable to define LMRoman10-BoldObliqu as \*modern12ptrmbs*

this appears to be a typo in type-siz (missing e). Hans?

Yue Wang

2:39 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Hi, Taco: On Sat, Jul 18, 2009 at 11:22 PM, Taco Hoekwater wrote:

...

Yue Wang wrote:

...
In my humble opinion this is because TeX manages memory by it self... and maybe a 64bit memory word of packed data uses twice of memory as 32bits one.... I am not sure.

It could be because web2c by default declares the integer type as "long int" on 64-bit platforms, and that that is what makes everything twice the size, but the --enable-dump-share argument to configure is supposed to prevent it from doing so, so it seems something odd is going on. Can you try to check the sizeof(integer) within a debugger?

169:luatex yue$ ls -lah build/texk/web2c/luatex -rwxr-xr-x 1 yue staff 7.2M Jul 19 10:31 build/texk/web2c/luatex 169:luatex yue$ file build/texk/web2c/luatex build/texk/web2c/luatex: Mach-O 64-bit executable x86_64 169:luatex yue$ gdb build/texk/web2c/luatex GNU gdb 6.3.50-20050815 (Apple version gdb-962) (Sat Jul 26 08:14:40 UTC 2008) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries .... done (gdb) p sizeof(integer) $1 = 4 (gdb) Here is my build configuration (a patch to build.sh) 169:luatex yue$ svn diff Index: build.sh =================================================================== --- build.sh (revision 2890) +++ build.sh (working copy) @@ -59,6 +59,11 @@ if [ `uname` = "Darwin" ] ; then export MACOSX_DEPLOYMENT_TARGET=10.4 + CFLAGS="-arch x86_64 -g $CFLAGS" + XCFLAGS="-arch x86_64 -g $XCFLAGS" + CXXFLAGS="-arch x86_64 -g $CXXFLAGS" + LDFLAGS="-arch x86_64 -g $LDFLAGS" + export CFLAGS CXXFLAGS LDFLAGS XCFLAGS fi; B=build Did I miss something here? Yue Wang

...

(the 64bit linux is indeed faster than the 32bit linux on the same hardware)

Best wishes, Taco

Taco Hoekwater

7:44 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Yue Wang wrote:

...

Did I miss something here?

It seems either osx or gcc is outsmarting you, but I have no idea why/how. Here are my timings, on a 64bit linux kernel (AMD Athlon(tm) 64 X2 Dual Core Processor 3800+) 32: mkiv lua stats : used config path - /opt/tex/texmf/web2c/texmf.cnf mkiv lua stats : used cache path - /home/taco/tmp mkiv lua stats : input load time - 0.129 seconds mkiv lua stats : stored bytecode data - 184 modules, 45 tables, 229 chunks mkiv lua stats : node list callback tasks - 4 unique tasks, 4 created, 20995 calls mkiv lua stats : cleaned up reserved nodes - 29 nodes, 10 lists of 1426 mkiv lua stats : node memory usage - 20 glue_spec, 2 dir mkiv lua stats : h-node processing time - 0.456 seconds including kernel mkiv lua stats : attribute processing time - 1.910 seconds mkiv lua stats : used backend - pdf (backend for directly generating pdf output) mkiv lua stats : loaded patterns - en:us:pat:exc:2 mkiv lua stats : jobdata time - 0.113 seconds saving, 0.042 seconds loading mkiv lua stats : callbacks - direct: 74687, indirect: 12527, total: 87214 mkiv lua stats : interactive elements - 179 references, 356 destinations mkiv lua stats : loaded fonts - 43 files: iwona-bold.otf iwona-bolditalic.otf .... mkiv lua stats : fonts load time - 1.480 seconds mkiv lua stats : metapost processing time - 0.252 seconds, loading: 0.025 seconds, execution: 0.131 seconds, n: 163 mkiv lua stats : result saved in file - luatexref-t.pdf mkiv lua stats : luatex banner - this is luatex, version beta-0.42.0-2009071802 mkiv lua stats : control sequences - 31677 of 147189 mkiv lua stats : current memory usage - 131 MB (ctx: 135 MB) mkiv lua stats : runtime - 17.118 seconds, 166 processed pages, 166 shipped pages, 9.697 pages/second 64: mkiv lua stats : used config path - /opt/tex/texmf/web2c/texmf.cnf mkiv lua stats : used cache path - /home/taco/tmp mkiv lua stats : input load time - 0.120 seconds mkiv lua stats : stored bytecode data - 184 modules, 45 tables, 229 chunks mkiv lua stats : node list callback tasks - 4 unique tasks, 4 created, 20995 calls mkiv lua stats : cleaned up reserved nodes - 29 nodes, 10 lists of 1426 mkiv lua stats : node memory usage - 20 glue_spec, 2 dir mkiv lua stats : h-node processing time - 0.464 seconds including kernel mkiv lua stats : attribute processing time - 1.713 seconds mkiv lua stats : used backend - pdf (backend for directly generating pdf output) mkiv lua stats : loaded patterns - en:us:pat:exc:2 mkiv lua stats : jobdata time - 0.135 seconds saving, 0.029 seconds loading mkiv lua stats : callbacks - direct: 74687, indirect: 12527, total: 87214 mkiv lua stats : interactive elements - 179 references, 356 destinations mkiv lua stats : loaded fonts - 43 files: iwona-bold.otf iwona-bolditalic.otf ... mkiv lua stats : fonts load time - 1.485 seconds mkiv lua stats : metapost processing time - 0.316 seconds, loading: 0.023 seconds, execution: 0.116 seconds, n: 163 mkiv lua stats : result saved in file - luatexref-t.pdf mkiv lua stats : luatex banner - this is luatex, version beta-0.42.0-2009071909 mkiv lua stats : control sequences - 31677 of 16877400 mkiv lua stats : current memory usage - 119 MB (ctx: 122 MB) mkiv lua stats : runtime - 15.998 seconds, 166 processed pages, 166 shipped pages, 10.376 pages/second

Taco Hoekwater

7:54 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Taco Hoekwater wrote:

...

64:

mkiv lua stats : input load time - 0.120 seconds mkiv lua stats : h-node processing time - 0.464 seconds including kernel mkiv lua stats : attribute processing time - 1.713 seconds mkiv lua stats : jobdata time - 0.135 seconds saving, 0.029 lo mkiv lua stats : fonts load time - 1.485 seconds mkiv lua stats : metapost processing time - 0.316 seconds, loading: 0.023 seconds, execution: 0.116 seconds, n: 163 mkiv lua stats : runtime - 15.998 seconds, 166 processed pages, 166 shipped pages, 10.376 pages/second

Just for fun (and to show how much luatex depends on optimization) here are the timings for my 'normal' debug-enabled un-optimized un-stripped luatex executable: mkiv lua stats : input load time - 0.186 seconds mkiv lua stats : h-node processing time - 0.865 seconds including mkiv lua stats : attribute processing time - 3.289 seconds mkiv lua stats : jobdata time - 0.408 seconds saving, 0.064 mkiv lua stats : fonts load time - 3.496 seconds mkiv lua stats : metapost processing time - 0.570 seconds, loading: 0.055 seconds, execution: 0.223 seconds, n: 163 mkiv lua stats : runtime - 35.413 seconds, 166 processed pages, 166 shipped pages, 4.688 pages/second Best wishes, Taco

Hans Hagen

10:48 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Taco Hoekwater wrote:

...

mkiv lua stats : input load time - 0.129 seconds mkiv lua stats : h-node processing time - 0.456 seconds including kernel mkiv lua stats : attribute processing time - 1.910 seconds mkiv lua stats : jobdata time - 0.113 seconds saving, 0.042 mkiv lua stats : fonts load time - 1.480 seconds mkiv lua stats : metapost processing time - 0.252 seconds, loading: 0.025 seconds, execution: 0.131 seconds, n: 163 mkiv lua stats : current memory usage - 131 MB (ctx: 135 MB) mkiv lua stats : runtime - 17.118 seconds, 166 processed pages, 166 shipped pages, 9.697 pages/second

64:

mkiv lua stats : input load time - 0.120 seconds mkiv lua stats : h-node processing time - 0.464 seconds including kernel mkiv lua stats : attribute processing time - 1.713 seconds mkiv lua stats : jobdata time - 0.135 seconds saving, 0.029 seconds loading mkiv lua stats : fonts load time - 1.485 seconds mkiv lua stats : metapost processing time - 0.316 seconds, loading: 0.023 seconds, execution: 0.116 seconds, n: 163 mkiv lua stats : current memory usage - 119 MB (ctx: 122 MB) mkiv lua stats : runtime - 15.998 seconds, 166 processed pages, 166 shipped pages, 10.376 pages/second

interesting ... lua is about the same but it's the overal tex side of the processing that is faster; also, mem consumption is definitely not doubled on 64 bit so it looks indeed liks osx itself is creating the problems taco: it would be interesting to see how metafun performs on the 1770 inline graphics, as mplib timings seems a bit different Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Taco Hoekwater

11:57 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Hans Hagen wrote:

...

interesting ... lua is about the same but it's the overal tex side of the processing that is faster; also, mem consumption is definitely not doubled on 64 bit so it looks indeed liks osx itself is creating the problems

taco: it would be interesting to see how metafun performs on the 1770 inline graphics, as mplib timings seems a bit different

Maybe so, but I am not eager to do these comparisons as they need two specific binary builds that I don't normally use, which makes it an extended exercise in patience (that I do not have a lot of). Best wishes, Taco

Hans Hagen

10:43 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Yue Wang wrote:

...

Did I miss something here?

so, in the minimals, will we have an extra texmf-...-64 for osx then? i agree with mojca that we should not mess around too much with binaries in this tex-live-freeze stage Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Mojca Miklavec

11:02 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

On Sun, Jul 19, 2009 at 12:43, Hans Hagen wrote:

...

Yue Wang wrote:

...
Did I miss something here?

so, in the minimals, will we have an extra texmf-...-64 for osx then?

i agree with mojca that we should not mess around too much with binaries in this tex-live-freeze stage

1.) Yue Wang says that the 64-bit binary is bigger, slower and consumes more memory 2.) If we start supporting 64-bit intel, we could equally think about 64-bit PPC 3.) TeX Live has no 64-bit support yet I guess (one might want to ask Dick/Richard Koch) 4.) Maybe just want what differences Snow Leopard will bring 5.) uname shows i386 as platform; in order to figure out that Mac is 64-bit one needs to issue some weird mac-specific command (both in command-line and in mtx[something]); the idea is probably to provide "fat" binaries, but then the question: should fat binaries now contain ppc+i386+x86_64 or just a subset of them? (2) is not to be taken seriously, but as long as (1) is true I see no reason to provide 64-bit binaries at this moment. But definitely we need to consider that at some later point. Mojca

Hans Hagen

11:11 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Mojca Miklavec wrote:

...

(2) is not to be taken seriously, but as long as (1) is true I see no reason to provide 64-bit binaries at this moment. But definitely we need to consider that at some later point.

so let's discuss it at the context meeting garden session (we should make a list of items) ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Hans Hagen

10:23 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Yue Wang wrote:

...

In my humble opinion this is because TeX manages memory by it self... and maybe a 64bit memory word of packed data uses twice of memory as 32bits one.... I am not sure.

well, the fact that linux 64 bit runs much faster contradicts that one thing that we observed is that the lua garbage collected performs different on different platforms / architectures / cpu cache etc Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Taco Hoekwater

10:26 a.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Hans Hagen wrote:

...

Yue Wang wrote:

...
In my humble opinion this is because TeX manages memory by it self... and maybe a 64bit memory word of packed data uses twice of memory as 32bits one.... I am not sure.

well, the fact that linux 64 bit runs much faster contradicts that

one thing that we observed is that the lua garbage collected performs different on different platforms / architectures / cpu cache etc

Another way to get a speed penalty is when the exe 64-bit but some of the libraries it (or the operating system) needs to use are 32-bit. Best wishes, Taco

Hans Hagen

17 Jul 17 Jul

6:17 p.m.

New subject: [Context] [Dev-luatex] Luatex 0.42.0 (a snapshot release)

Mojca Miklavec wrote:

...

On Fri, Jul 17, 2009 at 12:31, Taco Hoekwater wrote:

...
Hi,

I have just uploaded the archives for a new luatex release, 0.42.0.

NOTE: This snapshot is just for testing the new sources and the updated build process, do NOT trust this binary for production work!

Independent of that (considering the fact that luatex should not be trusted for production work anyway :) :) :) :) ...

well, it's not that bad but your thesis is probably way beyond my production work Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

5634

Age (days ago)

5636

Last active (days ago)

List overview

Download

33 comments

5 participants

participants (5)

Diego Depaoli
Hans Hagen
Mojca Miklavec
Taco Hoekwater
Yue Wang