[Context] One more arch (OS X, snow leopard)
Hi, Mojca: Maybe we need one more arch:) Yue:context yue$ uname -a Darwin Yue.local 10.0.0 Darwin Kernel Version 10.0.0: Fri Jul 31 22:46:25 PDT 2009; root:xnu-1456.1.25~1/RELEASE_X86_64 x86_64 Yue:context yue$ sh first-setup.sh Error: your system "Darwin x86_64" is not supported yet. Please report to the ConTeXt mailing-list (ntg-context@ntg.nl) Snow Leopard, boot from the x86_64 kernel. Yue Wang
On Tue, Oct 27, 2009 at 20:16, Yue Wang wrote:
Hi, Mojca:
Maybe we need one more arch:)
Yue:context yue$ uname -a Darwin Yue.local 10.0.0 Darwin Kernel Version 10.0.0: Fri Jul 31 22:46:25 PDT 2009; root:xnu-1456.1.25~1/RELEASE_X86_64 x86_64 Yue:context yue$ sh first-setup.sh Error: your system "Darwin x86_64" is not supported yet. Please report to the ConTeXt mailing-list (ntg-context@ntg.nl)
Snow Leopard, boot from the x86_64 kernel.
First question: do "normal universal binaries" from TL 2009 work OK? I can do it faster, but is something like <waiting time to get my computer> acceptable for you, so that I can test it at the same time? (If anyone wants, feel free to go modifying files & start building the binaries.) Today is too late anyway. Mojca
32bits luatex binary works on x86_64 kernel.
On Tue, Oct 27, 2009 at 5:34 PM, Mojca Miklavec
On Tue, Oct 27, 2009 at 20:16, Yue Wang wrote:
Hi, Mojca:
Maybe we need one more arch:)
Yue:context yue$ uname -a Darwin Yue.local 10.0.0 Darwin Kernel Version 10.0.0: Fri Jul 31 22:46:25 PDT 2009; root:xnu-1456.1.25~1/RELEASE_X86_64 x86_64 Yue:context yue$ sh first-setup.sh Error: your system "Darwin x86_64" is not supported yet. Please report to the ConTeXt mailing-list (ntg-context@ntg.nl)
Snow Leopard, boot from the x86_64 kernel.
First question: do "normal universal binaries" from TL 2009 work OK?
I can do it faster, but is something like <waiting time to get my computer> acceptable for you, so that I can test it at the same time? (If anyone wants, feel free to go modifying files & start building the binaries.) Today is too late anyway.
Mojca
We need several things in this case: 1.) We need a name for Darwin x86_64. Should it be osx-64? Any other suggestions? The name "intel" is not particularly suitable for i386 any more, but well ... 2.) This is for Taco: I would be grateful if we could have a switch similar to --ppc in luatex building script for: a) -arch ppc b) -arch i386 c) -arch x86_64 doing exactly the same thing as -ppc switch, just having a different argument for arch. I would leave it up to you to name it (maybe exactly the same way: -arch=ppc or -macarch=ppc) and I do not care about backward compatibility of the way to call the script. I'm really impressed - it's possible to compile ppc binaries on Snow Leopard, but not possible to execute them without additional software installed. This is what cross-compiling metapost for ppc returns me: checking whether the C compiler works... configure: error: in `/Users/mojca/context/build-binaries/src/metapost/build': configure: error: cannot run C compiled programs. If you meant to cross compile, use `--host'. See `config.log' for more details. but I will try to get over it. I think that I just need to install additional software to support running ppc applications (or if Taco can give me a hint how to trick the building process to use host programs to execute some commands, but that's not of too big importance). I didn't know this would be the case, but I'm very grateful that Snow Leopard supports compiling for Tiger. One problem less (though we'll probably drop support for Tiger once the next version of OS comes out). 3.) We have never really started to use universal binaries, and maybe there's no need to do so. Some people were enthusiastic about universal binaries, but the truth is that Apple already dropped all the ppc stuff out of their OS and now makes i386+x86_64 fat binaries. If we do it (for example for TeX Live packages?), it might make sense to bundle all the three architectures together for the moment. 4.) I'll ask Hans to add x86_64 architecture to some places, I just need to change a few things first. Is the name osx-64 OK? Mojca
Mojca Miklavec wrote:
2.) This is for Taco: I would be grateful if we could have a switch similar to --ppc in luatex building script for: a) -arch ppc b) -arch i386 c) -arch x86_64 doing exactly the same thing as -ppc switch, just having a different argument for arch. I would leave it up to you to name it (maybe exactly the same way: -arch=ppc or -macarch=ppc) and I do not care about backward compatibility of the way to call the script.
No problem at all to add that, but I am not very good in shell script programming so a patch would be very welcome. Best wishes, Taco
On Fri, Oct 30, 2009 at 09:03, Taco Hoekwater wrote:
Mojca Miklavec wrote:
2.) This is for Taco: I would be grateful if we could have a switch similar to --ppc in luatex building script for: a) -arch ppc b) -arch i386 c) -arch x86_64 doing exactly the same thing as -ppc switch, just having a different argument for arch. I would leave it up to you to name it (maybe exactly the same way: -arch=ppc or -macarch=ppc) and I do not care about backward compatibility of the way to call the script.
No problem at all to add that, but I am not very good in shell script programming so a patch would be very welcome.
I can send you the patch, no problem, just one question for you before I continue. The weird thing is that: - I'm able to compile binaries for ppc - I'm not able to run them - ("uname" says the system is "i386", but generates "x86_64" binaries, but let's just ignore that fact for a moment ...) I should probably be able to run the ppc binaries by installing additional software (I didn't try that yet). However, if there exists some not-too-complicated way for cross-compilation, that would be great since it would By crosscompilation I mean: - if I use "gcc hello.c -o hello -arch ppc" this works perfectly well (it compiles, works on old ppc-s, but I'm not able to execute it) - metapost, luatex, pdftex, ... use some intermediate commands (like tangle maybe) that cannot be executed if they are built for ppc Since I know that you are cross-compiling for windows already: what's your estimate about complexity of cross-compilation on Mac? Where should I start looking? If it is too complicated, I'll just install the software, but I'm ready to invest a bit into "less-dependencies" and consequently having a slightly bigger set of volunteers that would be able to compile when I'm away. Mojca PS: thumbs up for the FreeBSD maintainers for the fastest response!
I can send you the patch, no problem, just one question for you before I continue. The weird thing is that: - I'm able to compile binaries for ppc - I'm not able to run them
Mojca, that *is* cross-compilation; the very point of it is to compile binaries that you're not able to run on the host architecture. The situation on Mac OS <= 10.5 was quite special in that you *could* run standalone binaries compiled for the "other" architecture, but that's not the general case when cross-compiling. I understand there are side-effects to it (that the build system wants to run the tangle binary it just compiled), but there must be workarounds for that. I can look into it, but after the week-end. Arthur
Mojca Miklavec wrote:
- I'm able to compile binaries for ppc
That *is* cross compilation, in a way.
- I'm not able to run them
Sure. I can't run win32 exes either, but I can build them easily enough.
- ("uname" says the system is "i386", but generates "x86_64" binaries, but let's just ignore that fact for a moment ...)
That is just a bit odd.
I should probably be able to run the ppc binaries by installing additional software (I didn't try that yet).
Probably. After all, both Tiger and (old) Leopard could run ppc files on intel (just slowly). But the problem is not with the actual compilation (as there was a succesful report already). Instead, it is about the build process.
By crosscompilation I mean: - if I use "gcc hello.c -o hello -arch ppc" this works perfectly well (it compiles, works on old ppc-s, but I'm not able to execute it) - metapost, luatex, pdftex, ... use some intermediate commands (like tangle maybe) that cannot be executed if they are built for ppc
Autoconf is supposed to fix that for you, but that does not work if you do only -arch=ppc. You'll need to have a CONFHOST in build.sh, like I do for windows. That tells 'configure' that it is cross-compiling, so it will set up the build process to fetch pre-installed executables for these commands instead of the ones it is currently building. It is quite straightforward, really, but you need to know the correct string to feed it (something like --host=powerpc-apple-darwin9 is likely) Generally, --host=<whatever> results in execution of <whatever>-gcc, <whatever>-ld, <whatever>-strip etc. Apple's gcc does that --arch thing, but probably the dev tools have also installed small stub scripts with the names configure expects. So, first action is to look for those. There is a tar.gz of those stubs at the bottom of this page, which explains basically what I wrote above (and possibly better): http://r.research.att.com/tools/ Best wishes, Taco
I wish to hereby deny any suspected brain connection because Taco Hoewater and me. Arthur
- ("uname" says the system is "i386", but generates "x86_64" binaries, but let's just ignore that fact for a moment ...)
That is just a bit odd.
This is not odd at all. uname says i386 means you are running snow leopard *kernel* in 32bit mode. but 32bit mode *kernel* can run 64 bit userland. (In tiger, 32 bits kernel can run 64bits unix tools, In Leopard, apple provide 32bits kernel which can execute 64bit user land executables, and 64bits Cocoa. In Snow Leopard, apple ships the kernel in universal binary format with 32bit intel and 64bit intel) You can boot into 64bits kernel, and uname will give you x86_64. Apple does not enable 64bits kernel by default since some 3rd party old kernel extensions might still need 32bits kernel.
I should probably be able to run the ppc binaries by installing additional software (I didn't try that yet).
Probably. After all, both Tiger and (old) Leopard could run ppc files on intel (just slowly). But the problem is not with the actual compilation (as there was a succesful report already). Instead, it is about the build process.
Yes, by Rosetta.
By crosscompilation I mean: - if I use "gcc hello.c -o hello -arch ppc" this works perfectly well (it compiles, works on old ppc-s, but I'm not able to execute it)
You should be able to execute ppc binaries on intel platform using rosetta.
- metapost, luatex, pdftex, ... use some intermediate commands (like tangle maybe) that cannot be executed if they are built for ppc
Autoconf is supposed to fix that for you, but that does not work if you do only -arch=ppc. You'll need to have a CONFHOST in build.sh, like I do for windows. That tells 'configure' that it is cross-compiling, so it will set up the build process to fetch pre-installed executables for these commands instead of the ones it is currently building. It is quite straightforward, really, but you need to know the correct string to feed it (something like --host=powerpc-apple-darwin9 is likely)
Generally, --host=<whatever> results in execution of <whatever>-gcc, <whatever>-ld, <whatever>-strip etc.
Apple's gcc does that --arch thing, but probably the dev tools have also installed small stub scripts with the names configure expects. So, first action is to look for those. There is a tar.gz of those stubs at the bottom of this page, which explains basically what I wrote above (and possibly better):
http://r.research.att.com/tools/
Best wishes, Taco
_______________________________________________ Mailing list: https://launchpad.net/~context Post to : context@lists.launchpad.net Unsubscribe : https://launchpad.net/~context More help : https://help.launchpad.net/ListHelp
Hi, Mojca:
On Fri, Oct 30, 2009 at 2:11 AM, Mojca Miklavec
We need several things in this case:
1.) We need a name for Darwin x86_64. Should it be osx-64? Any other suggestions? The name "intel" is not particularly suitable for i386 any more, but well ...
that's ok.
2.) This is for Taco: I would be grateful if we could have a switch similar to --ppc in luatex building script for: a) -arch ppc b) -arch i386 c) -arch x86_64 doing exactly the same thing as -ppc switch, just having a different argument for arch. I would leave it up to you to name it (maybe exactly the same way: -arch=ppc or -macarch=ppc) and I do not care about backward compatibility of the way to call the script.
I'm really impressed - it's possible to compile ppc binaries on Snow Leopard, but not possible to execute them without additional software installed. This is what cross-compiling metapost for ppc returns me:
Hey, how did you install your Snow Leopard. Default installation didn't install Rosetta. But you can find them in the install DVD. When you run the Optional Install, you can add this package in. After doing that, you should be able to run ppc binaries by simulation.
checking whether the C compiler works... configure: error: in `/Users/mojca/context/build-binaries/src/metapost/build': configure: error: cannot run C compiled programs. If you meant to cross compile, use `--host'. See `config.log' for more details.
but I will try to get over it. I think that I just need to install additional software to support running ppc applications (or if Taco can give me a hint how to trick the building process to use host programs to execute some commands, but that's not of too big importance).
I didn't know this would be the case, but I'm very grateful that Snow Leopard supports compiling for Tiger. One problem less (though we'll probably drop support for Tiger once the next version of OS comes out).
Xcode support the past two versions of OS X. for example, Xcode in 10.5 should support 10.4 and 10.5, and 10.3 in optional install. But in fact that is not a big problem, these backward supports are usually for Cocoa APIs (For example, each version of OS X deprecates a lot of Cocoa APIs, but you are still able to compile the program and run them in old versions since for most of the time they provide binary compatibility). For traditional ANSI or POSIX C programs like LuaTeX, I think it won't be a problem.
3.) We have never really started to use universal binaries, and maybe there's no need to do so. Some people were enthusiastic about universal binaries, but the truth is that Apple already dropped all the ppc stuff out of their OS and now makes i386+x86_64 fat binaries.
I think either way is ok. fat binaries are good, but they are fat. I can compile ppc+i386+x86_64 binaries without any problems.
If we do it (for example for TeX Live packages?), it might make sense to bundle all the three architectures together for the moment.
4.) I'll ask Hans to add x86_64 architecture to some places, I just need to change a few things first. Is the name osx-64 OK?
Mojca
_______________________________________________ Mailing list: https://launchpad.net/~context Post to : context@lists.launchpad.net Unsubscribe : https://launchpad.net/~context More help : https://help.launchpad.net/ListHelp
On Fri, Oct 30, 2009 at 21:04, Yue Wang wrote:
Hey, how did you install your Snow Leopard.
It was preinstalled. But I'm looking for a screw-driver to replace the hard drive, so I'll have to install it anyway.
Default installation didn't install Rosetta.
I know about Rosetta, but I also thought that if we figure out how to "properly" cross-compile, it might be a bit easier to also compile 64-bit binaries on 32-bit machines etc.
Xcode support the past two versions of OS X. for example, Xcode in 10.5 should support 10.4 and 10.5, and 10.3 in optional install.
I didn't know that. I only had support for 10.4 on 10.5, so I was surprised to learn that 10.4 was still available.
But in fact that is not a big problem, these backward supports are usually for Cocoa APIs (For example, each version of OS X deprecates a lot of Cocoa APIs, but you are still able to compile the program and run them in old versions since for most of the time they provide binary compatibility). For traditional ANSI or POSIX C programs like LuaTeX, I think it won't be a problem.
I remember that I compiled TeX binaries on Leopard and people started complaining that they were not able to use them on Tiger. Do you have any idea how to cross-compile XeTeX (with export MACOSX_DEPLOYMENT_TARGET=10.4)?
3.) We have never really started to use universal binaries, and maybe there's no need to do so. Some people were enthusiastic about universal binaries, but the truth is that Apple already dropped all the ppc stuff out of their OS and now makes i386+x86_64 fat binaries.
I think either way is ok. fat binaries are good, but they are fat. I can compile ppc+i386+x86_64 binaries without any problems.
I have already committed the luatex for i386. Feel free to commit the binary for ppc and 64-bit (that need a new directory structure first), but make sure to use export MACOSX_DEPLOYMENT_TARGET=10.4 The building script needs to be adapted anyway. (Some scripts in SVN have been modified already, but not all of them yet.) Mojca
Hi, Mojca:
It was preinstalled. But I'm looking for a screw-driver to replace the hard drive, so I'll have to install it anyway.
No need to reinstall. In 10.6 the installer is rewritten. It's in the Optional Install package. You can just install one package.
Default installation didn't install Rosetta.
I know about Rosetta, but I also thought that if we figure out how to "properly" cross-compile, it might be a bit easier to also compile 64-bit binaries on 32-bit machines etc.
Compile 64bits binaries on 32bits machine/operating system is possible. But, it is not, however, possible to run 64bits binaries on 32bits machine.
I didn't know that. I only had support for 10.4 on 10.5, so I was surprised to learn that 10.4 was still available.
10.3 support is in the 10.5's Xcode's Optional Install. When install Xcode, check that box, and you will have 10.3 support.
I remember that I compiled TeX binaries on Leopard and people started complaining that they were not able to use them on Tiger.
OK. Maybe there are some problems in linking symbols...
Do you have any idea how to cross-compile XeTeX (with export MACOSX_DEPLOYMENT_TARGET=10.4)?
I don't think XeTeX can be cross-compile easily. Since it is difficult to cross-compile ICU. Anyway, I can try that:)
I have already committed the luatex for i386. Feel free to commit the binary for ppc and 64-bit (that need a new directory structure first), but make sure to use export MACOSX_DEPLOYMENT_TARGET=10.4
The building script needs to be adapted anyway. (Some scripts in SVN have been modified already, but not all of them yet.)
I have permission to change FreeBSD binaries, do I have permission to send Mac binaries also? Yue Wang
On Tue, Oct 27, 2009 at 20:16, Yue Wang wrote:
Hi, Mojca:
Maybe we need one more arch:)
Yue:context yue$ uname -a Darwin Yue.local 10.0.0 Darwin Kernel Version 10.0.0: Fri Jul 31 22:46:25 PDT 2009; root:xnu-1456.1.25~1/RELEASE_X86_64 x86_64 Yue:context yue$ sh first-setup.sh Error: your system "Darwin x86_64" is not supported yet. Please report to the ConTeXt mailing-list (ntg-context@ntg.nl)
Snow Leopard, boot from the x86_64 kernel.
I tried to boot into 64-bit kernel and it seems to work OK. Mojca
I tried to boot into 64-bit kernel and it seems to work OK.
And a short benchmark shows that the same document runs 10.6 seconds on average with 32-bit LuaTeX and 5.9 seconds on average with 64-bit LuaTeX (I made sure to make at least 5 runs before measuring). Both tests were performed on 64-bit kernel running with all other binaries but LuaTeX being the same. So that's in contradiction with Yue's observation that 64-bit LuaTeX runs slower. (Though it may be biased. I wouldn't be surprised if Mac OS X tries to run some kind of "virtualizer" for 32-bit binaries.) Mojca
Mojca Miklavec wrote:
So that's in contradiction with Yue's observation that 64-bit LuaTeX runs slower. (Though it may be biased. I wouldn't be surprised if Mac OS X tries to run some kind of "virtualizer" for 32-bit binaries.)
on linux 64 bit binaries also run faster than 32 bit ones Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Hi, Mojca:
Last time I said that 64 bits luatex was slower, Snow Leopard hadn't
been released.
On Mac OS X 10.5.8, I run the same test using the two LuaTeX binaries
compiled on that platform, and conclude that 64 bits luatex is slower.
See: https://lists.launchpad.net/context/msg00168.html
Today, I find the old document that I used at that time and compile it
again. i386 runtime is 6.635, and x86_64 runtime is 3.493. So I think
this is due to Apple's improvement on their 64 bit support in Snow
Leopard (kernel, library and complier).
And here is a longer document compilation result (many images, math,
and complicated styles):
64bits:
mkiv lua stats : current memory usage - 136 MB (ctx: 139 MB)
mkiv lua stats : runtime - 6.243 seconds, 71
processed pages, 72 shipped pages, 11.533 pages/second
MTXrun | total runtime: 6.294
32bits
mkiv lua stats : current memory usage - 85 MB (ctx: 87 MB)
mkiv lua stats : runtime - 12.132 seconds, 71
processed pages, 72 shipped pages, 5.935 pages/second
MTXrun | total runtime: 12.183
It is also possible I did something wrong at that time. 64bits
compilation is not native in Leopard. gcc produce 32bit binaries by
default. In order to get 64bits binaries on Leopard that time, I added
a few modifications to the build script. Maybe some of the
configurations are wrong. On Snow Leopard, this is not needed since
gcc generates 64bits binaries by default.
To conclude: in Snow Leopard, 64bit binary is faster, but uses 2x memory.
So, we should encourage our users to use 64 bits version on Snow
Leopard, as long as their machines supports that. Since most of the
time, users do not enable 64bits kernel, they will fetch 32bits luatex
by default. We should change that behavior in first-setup.sh.
Yue Wang
On Thu, Nov 5, 2009 at 3:52 PM, Mojca Miklavec
I tried to boot into 64-bit kernel and it seems to work OK.
And a short benchmark shows that the same document runs 10.6 seconds on average with 32-bit LuaTeX and 5.9 seconds on average with 64-bit LuaTeX (I made sure to make at least 5 runs before measuring).
Both tests were performed on 64-bit kernel running with all other binaries but LuaTeX being the same.
So that's in contradiction with Yue's observation that 64-bit LuaTeX runs slower. (Though it may be biased. I wouldn't be surprised if Mac OS X tries to run some kind of "virtualizer" for 32-bit binaries.)
Mojca
_______________________________________________ Mailing list: https://launchpad.net/~context Post to : context@lists.launchpad.net Unsubscribe : https://launchpad.net/~context More help : https://help.launchpad.net/ListHelp
participants (5)
-
Arthur Reutenauer
-
Hans Hagen
-
Mojca Miklavec
-
Taco Hoekwater
-
Yue Wang