Hi, I wanted to use my minimal installation on arch linux this morning and received this error: Binaries for platform 'linuxmusl-64' are missing. (There is no folder "/mnt/shared/context/tex/texmf-linuxmusl-64/bin") provide a proper tex root (like '. setuptex /something/tex') Looking into setuptex, I see indeed a test for "libc=musl" or "libc=glibc," so it appears that we now have two sets of binaries for linux-64? Is this supposed to be a long-term thing or just in the preparation for texlive 2018? (I share my minimal directory between several linux installations, so I assume I would need both sets?) Thank you, and all best Thomas
On Sat, Mar 24, 2018 at 11:59 AM, Thomas A. Schmitz
Hi,
I wanted to use my minimal installation on arch linux this morning and received this error:
Binaries for platform 'linuxmusl-64' are missing. (There is no folder "/mnt/shared/context/tex/texmf-linuxmusl-64/bin") provide a proper tex root (like '. setuptex /something/tex') hm, what does command -v ldd >/dev/null && ldd --version 2>&1 say ?
Looking into setuptex, I see indeed a test for "libc=musl" or "libc=glibc," so it appears that we now have two sets of binaries for linux-64? Is this supposed to be a long-term thing or just in the preparation for texlive 2018? (I share my minimal directory between several linux installations, so I assume I would need both sets?) it's a "long term", ie until it's in texlive . If not in texlive, it can be dropped, but not necessarily.
-- luigi
On 03/24/2018 12:14 PM, luigi scarso wrote:
hm, what does command -v ldd >/dev/null && ldd --version 2>&1 say ?
ldd (GNU libc) 2.26 Copyright (C) 2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by Roland McGrath and Ulrich Drepper.
it's a "long term", ie until it's in texlive . If not in texlive, it can be dropped, but not necessarily.
Sorry, I don't understand, this sounds contradictory to me. On 03/24/2018 12:15 PM, Hans Hagen wrote:
i have no clue what drives this but i suppose than a specific linux only has one of these two libs
(no sign of musl in my scripts here)
I see this tex/setuptex: if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl then libc=musl else libc=glibc fi
rsync -ptv rsync://contextgarden.net/minimals/setup/first-setup.sh .
it looks like that one has changed
Even with the new first-setup.sh: when I run it, there are no new binaries in context/tex/texmf-linuxmusl-64/bin, so for the time being, context minimals are broken on arch linux (and its derivatives, I guess). Thomas
On 3/24/2018 1:30 PM, Thomas A. Schmitz wrote:
On 03/24/2018 12:14 PM, luigi scarso wrote:
hm, what does command -v ldd >/dev/null && ldd --version 2>&1 say ?
ldd (GNU libc) 2.26 Copyright (C) 2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by Roland McGrath and Ulrich Drepper.
it's a "long term", ie until it's in texlive . If not in texlive, it can be dropped, but not necessarily.
Sorry, I don't understand, this sounds contradictory to me.
On 03/24/2018 12:15 PM, Hans Hagen wrote:
i have no clue what drives this but i suppose than a specific linux only has one of these two libs
(no sign of musl in my scripts here)
I see this tex/setuptex:
if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl then libc=musl else libc=glibc fi
rsync -ptv rsync://contextgarden.net/minimals/setup/first-setup.sh .
it looks like that one has changed
Even with the new first-setup.sh: when I run it, there are no new binaries in context/tex/texmf-linuxmusl-64/bin, so for the time being, context minimals are broken on arch linux (and its derivatives, I guess). ok, another thing you can try:
fix mtx-update.lua ["linux-64"] = "linux-64", ["linux64"] = "linux-64", -- ["linuxmusl-64"] = "linuxmusl-64", the problem then is that it gets overwritten so yoou need to comment a line in first-setup.sh that does that it might get the right bins then ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On 24 March 2018 at 13:30, Thomas A. Schmitz wrote:
On 03/24/2018 12:14 PM, luigi scarso wrote:
hm, what does command -v ldd >/dev/null && ldd --version 2>&1 say ?
ldd (GNU libc) 2.26 Copyright (C) 2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by Roland McGrath and Ulrich Drepper.
So you don't have musl. This means that we have a bug in first-setup.sh and setuptex then. I'm sorry.
it's a "long term", ie until it's in texlive . If not in texlive, it can be dropped, but not necessarily.
Sorry, I don't understand, this sounds contradictory to me.
Well, we just dropped i386-darwin and powerpc-darwin from the minimals, we dropped powerpc-linux some time before that which shows that there's no super strong promise to keep the binaries there forever (oh, and we also had i386-linuxmusl for a couple of days which I deleted a few days after adding it because Karl removed it from TeX Live). Whether that's considered "long term" is left as an exercise to the reader. If getting the binaries gets difficult and there are no users requiring support, the binaries will go.
On 03/24/2018 12:15 PM, Hans Hagen wrote:
i have no clue what drives this but i suppose than a specific linux only has one of these two libs
(no sign of musl in my scripts here)
I see this tex/setuptex:
if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl then libc=musl else libc=glibc fi
rsync -ptv rsync://contextgarden.net/minimals/setup/first-setup.sh .
it looks like that one has changed
Even with the new first-setup.sh: when I run it, there are no new binaries in context/tex/texmf-linuxmusl-64/bin, so for the time being, context minimals are broken on arch linux (and its derivatives, I guess).
I was aware that until Hans implements support in mtxrun the first installation on musl would be broken (I never liked this strange dependency with mtxrun trying to guess the platform). But that should not happen on a libc system, that's a bug. Mojca
I was aware that until Hans implements support in mtxrun the first installation on musl would be broken (I never liked this strange dependency with mtxrun trying to guess the platform). But that should not happen on a libc system, that's a bug.
On 3/24/2018 1:40 PM, Mojca Miklavec wrote: the problem is that we didn't have a linuxmusl-64 entry in mtx-update yet Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On 3/24/2018 11:59 AM, Thomas A. Schmitz wrote:
Hi,
I wanted to use my minimal installation on arch linux this morning and received this error:
Binaries for platform 'linuxmusl-64' are missing. (There is no folder "/mnt/shared/context/tex/texmf-linuxmusl-64/bin") provide a proper tex root (like '. setuptex /something/tex')
Looking into setuptex, I see indeed a test for "libc=musl" or "libc=glibc," so it appears that we now have two sets of binaries for linux-64? Is this supposed to be a long-term thing or just in the preparation for texlive 2018? (I share my minimal directory between several linux installations, so I assume I would need both sets?) i have no clue what drives this but i suppose than a specific linux only has one of these two libs
(no sign of musl in my scripts here) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On 3/24/2018 11:59 AM, Thomas A. Schmitz wrote:
Hi,
I wanted to use my minimal installation on arch linux this morning and received this error:
Binaries for platform 'linuxmusl-64' are missing. (There is no folder "/mnt/shared/context/tex/texmf-linuxmusl-64/bin") provide a proper tex root (like '. setuptex /something/tex')
Looking into setuptex, I see indeed a test for "libc=musl" or "libc=glibc," so it appears that we now have two sets of binaries for linux-64? Is this supposed to be a long-term thing or just in the preparation for texlive 2018? (I share my minimal directory between several linux installations, so I assume I would need both sets?) you also need to update firstsetup.sh
rsync -ptv rsync://contextgarden.net/minimals/setup/first-setup.sh . it looks like that one has changed (i don't know what gets downloaded then but it might help) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On 24 March 2018 at 11:59, Thomas A. Schmitz wrote:
Hi,
I wanted to use my minimal installation on arch linux this morning and received this error:
Binaries for platform 'linuxmusl-64' are missing. (There is no folder "/mnt/shared/context/tex/texmf-linuxmusl-64/bin") provide a proper tex root (like '. setuptex /something/tex')
Looking into setuptex, I see indeed a test for "libc=musl" or "libc=glibc," so it appears that we now have two sets of binaries for linux-64? Is this supposed to be a long-term thing or just in the preparation for texlive 2018? (I share my minimal directory between several linux installations, so I assume I would need both sets?)
The binaries were requested (and provided) by Henri Menke: https://mailman.ntg.nl/pipermail/ntg-context/2018/090611.html
From what I understood (maybe I misunderstood) the regular linux binaries would not even work on machines with musl, but maybe you have both libc and musl installed?
It was not meant to be a short-term thing (unless/until we figure out that there are too many problems with it and nobody uses it), but it was probably never even tested properly and Hans did not yet implement support in mtxrun, so maybe installing out-of-the-box would not yet work, I don't have a machine to test it. All the required binaries should be there though. Mojca
On 3/24/2018 1:32 PM, Mojca Miklavec wrote:
On 24 March 2018 at 11:59, Thomas A. Schmitz wrote:
Hi,
I wanted to use my minimal installation on arch linux this morning and received this error:
Binaries for platform 'linuxmusl-64' are missing. (There is no folder "/mnt/shared/context/tex/texmf-linuxmusl-64/bin") provide a proper tex root (like '. setuptex /something/tex')
Looking into setuptex, I see indeed a test for "libc=musl" or "libc=glibc," so it appears that we now have two sets of binaries for linux-64? Is this supposed to be a long-term thing or just in the preparation for texlive 2018? (I share my minimal directory between several linux installations, so I assume I would need both sets?)
The binaries were requested (and provided) by Henri Menke: https://mailman.ntg.nl/pipermail/ntg-context/2018/090611.html
From what I understood (maybe I misunderstood) the regular linux binaries would not even work on machines with musl, but maybe you have both libc and musl installed?
It was not meant to be a short-term thing (unless/until we figure out that there are too many problems with it and nobody uses it), but it was probably never even tested properly and Hans did not yet implement support in mtxrun, so maybe installing out-of-the-box would not yet work, I don't have a machine to test it.
All the required binaries should be there though. yes but setuptex, first-setup and mtx-update need to be in sync .. i adapted what i have and it will be in the next beta but i cannot test it as i have no such linux
Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On 24.03.2018 13:32, Mojca Miklavec wrote:
The binaries were requested (and provided) by Henri Menke: https://mailman.ntg.nl/pipermail/ntg-context/2018/090611.html
From what I understood (maybe I misunderstood) the regular linux binaries would not even work on machines with musl, but maybe you have both libc and musl installed?
It was not meant to be a short-term thing (unless/until we figure out that there are too many problems with it and nobody uses it), but it was probably never even tested properly and Hans did not yet implement support in mtxrun, so maybe installing out-of-the-box would not yet work, I don't have a machine to test it.
I was just writing this message when more replies arrived. I have never heard of musl and have never installed it; pretty sure all my linux installations are regular ones with libc. So: the problem appears to be with the test in if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl then libc=musl else libc=glibc fi which appears to default to musl even if it is not present. But I don't know enough about shell scripting to debug it - could it also be a problem with the shell used? (Mine is zsh.) Thomas
if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl then libc=musl else libc=glibc fi
which appears to default to musl even if it is not present. But I don't know enough about shell scripting to debug it - could it also be a problem with the shell used? (Mine is zsh.)
Quite possibly the shell makes a difference, but the expression being grepped for really should be protected by quotes. Can you try ldd --version 2>&1 | fgrep -q '^musl' ? Best, Arthur
On 24.03.2018 14:06, Arthur Reutenauer wrote:
Quite possibly the shell makes a difference, but the expression being grepped for really should be protected by quotes. Can you try
ldd --version 2>&1 | fgrep -q '^musl'
?
Best,
Arthur
Arthur, with your command, I get an empty line as return. My question still stands: what is the expected result for this test? If I run the command ldd --version 2>&1 | grep -q ^musl from inside my system, grep will report all the subdirectories with a line grep: XXX: Is a directory So: if the test defaults to "musl" when the return is non-empty, that would explain a lot. And it should be adapted since it puts a lot of confidence into the return of this command. Thomas
Arthur, with your command, I get an empty line as return. My question still stands: what is the expected result for this test?
Sorry, I should have been clearer in my previous email: the relevant part of that command is not its printed output on the terminal, but its return value, that you can test with “echo $?” right after running the command. In that context, 0 means true (yes, it may be counter-intuitive), so you would expect a non-zero result (often 1) since you don’t have musl (and, presumably you do get 0).
If I run the command
ldd --version 2>&1 | grep -q ^musl
from inside my system, grep will report all the subdirectories with a line
grep: XXX: Is a directory
This really does feel like zsh is interpreting the command in some unhelpful way.
So: if the test defaults to "musl" when the return is non-empty, that would explain a lot. And it should be adapted since it puts a lot of confidence into the return of this command.
Agreed. Best, Arthur
I reverted the change for now until someone can come up with a working command. Mojca
On 03/24/2018 05:51 PM, Mojca Miklavec wrote:
I reverted the change for now until someone can come up with a working command.
Arthur's if command -v ldd >/dev/null && ldd --version 2>&1 | fgrep -q '^musl' works for me, but again, there may be other corner cases that we don't see now. I would suggest reversing the logic of this test: default to linux unless you clearly find the string "musl" in the output; don't rely on a zero result, which may be prevented by a number f reasons... Thomas
On Sat, Mar 24, 2018 at 06:05:08PM +0100, Thomas A. Schmitz wrote:
On 03/24/2018 05:51 PM, Mojca Miklavec wrote:
I reverted the change for now until someone can come up with a working command.
Arthur's
if command -v ldd >/dev/null && ldd --version 2>&1 | fgrep -q '^musl'
works for me, but again, there may be other corner cases that we don't see now. I would suggest reversing the logic of this test: default to linux unless you clearly find the string "musl" in the output; don't rely on a zero result, which may be prevented by a number f reasons...
I completely agree. Even in that form, the test can fail if for example there is no ldd in the path; which is somewhat unlikely on Linux, but still. Reversing the logic seems imperative; as for example in if ! command -v ldd >/dev/null || ! ldd --version 2>&1 | grep -E -q '^musl'; then libc=glic else libc=musl fi But do test thoroughly before installing, please. By the way, Thomas, what you wrote is slightly different from what I suggested earlier (by one character), and it would actually make the test fail even if you had a musl libc: fgrep tests for the presence of the literal string “^musl” and will thus return 1. You meant, of course, egrep for grepping regular expressions (alias of grep -E). That’s what I’m using in the test above, which will thus also fail if the default grep doesn’t support the -E switch, but with my suggestion it won’t affect the vast majority of users, since they don’t have musl. Best, Arthur
On Sat, Mar 24, 2018 at 06:05:08PM +0100, Thomas A. Schmitz wrote:
Arthur's
if command -v ldd >/dev/null && ldd --version 2>&1 | fgrep -q '^musl'
works for me, but again, there may be other corner cases that we don't see now. I would suggest reversing the logic of this test: default to linux unless you clearly find the string "musl" in the output; don't rely on a zero result, which may be prevented by a number f reasons...
Yes, well. About that ... If you think about what “default” means, you’ll realise that the bit of code you quote does actually make glibc the default: indeed, if [some complex condition]; then libc=musl else libc=glic fi or, equivalently, Henri’s suggested alternative libc=glibc if [some complex condition]; then libc=musl fi means exactly “set libc to glibc, unless [some complex condition] is met”. The trick is that in your case, the test unexpectedly *succeeds* (which confused me yesterday). How can that be? The exact test is command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl which in your case boils down to grep -q ^musl [output of ldd --version] meaning that grep will search for the regexp ^musl in the output of ldd --version. Or will it? As I explained, the string ^musl can in zsh (if EXTENDED_GLOB is set) be interpreted as “all file names in the current directory but musl”, which means that it is actually expanded as grep -q [filename 1] [filename 2] [filename 3] ... [output of ldd --version] and grep will thus look for the first file name in that list (whose exact order is hard to predict and depends in particular on your locale). Every file in the current directory (and the output of ldd --version) will be searched for that name, so if it’s simple enough there’s a high chance that it will be found. Amusingly, in my case, the first file name is the rather unlikely 20170802093441240.pdf, but it is found anyway, because I sent it as an email attachment one day, so the line Content-Disposition: attachment; filename="20170802093441240.pdf" is found in my sent messages (which is a plain text file in my home directory). That is not all! As you experienced, grep does actually report an error for each directory name, and should thus return a non-zero value; which indeed it does in a number of experiments I conducted. But! In the original test, grep is called with the -q switch, that has the following specification in POSIX: Quiet. Nothing shall be written to the standard output, regardless of matching lines. Exit with zero status if an input line is selected. And here we are. For a number of unlikely reasons, the condition, that should only hold true in a very specific situation, ends up being satisfied. Best, Arthur
On 24 March 2018 at 18:36, luigi scarso
On Sat, Mar 24, 2018 at 5:51 PM, Mojca Miklavec wrote:
I reverted the change for now until someone can come up with a working command. can you send me offlist the relevant *lua *sh script ? I cannot reproduce the error now with linux/zsh.
echo "Henri's test:" if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl then echo "This is musl" else echo "This is libc" fi # not sure about the exact form echo "Arthur's test:" if command -v ldd --version 2>&1 | fgrep -q '^musl' then echo "This is musl" else echo "This is libc" fi Mojca
On Sat, Mar 24, 2018 at 6:43 PM, Mojca Miklavec
On 24 March 2018 at 18:36, luigi scarso
wrote: On Sat, Mar 24, 2018 at 5:51 PM, Mojca Miklavec wrote:
I reverted the change for now until someone can come up with a working command. can you send me offlist the relevant *lua *sh script ? I cannot reproduce the error now with linux/zsh.
echo "Henri's test:" if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl then echo "This is musl" else echo "This is libc" fi
# not sure about the exact form echo "Arthur's test:" if command -v ldd --version 2>&1 | fgrep -q '^musl' then echo "This is musl" else echo "This is libc" fi
Mojca ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net ___________________________________________________________________________________
% cat ./test-musl.zsh if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q "^musl" then echo "1 libc=musl" else echo "1 libc=glibc" fi if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q '^musl' then echo "2 libc=musl" else echo "2 libc=glibc" fi if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl then echo "3 libc=musl" else echo "3 libc=glibc" fi % zsh ./test-musl.zsh 1 libc=glibc 2 libc=glibc 3 libc=glibc -- luigi
I am not the author of this check, I simply took it from config.guess https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob;f=config.guess;h=25... (Ctrl+F musl) However, I was the one who requested the musl detection in config.guess and the maintainer implemented the check like this for reasons of portability. Perhaps the problem is the else case in this statement. Maybe it should be: LIBC="glibc" if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl then LIBC="musl" fi On 03/25/2018 06:47 AM, luigi scarso wrote:
On Sat, Mar 24, 2018 at 6:43 PM, Mojca Miklavec
wrote: On 24 March 2018 at 18:36, luigi scarso
wrote: On Sat, Mar 24, 2018 at 5:51 PM, Mojca Miklavec wrote:
I reverted the change for now until someone can come up with a working command. can you send me offlist the relevant *lua *sh script ? I cannot reproduce the error now with linux/zsh.
echo "Henri's test:" if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl then echo "This is musl" else echo "This is libc" fi
# not sure about the exact form echo "Arthur's test:" if command -v ldd --version 2>&1 | fgrep -q '^musl' then echo "This is musl" else echo "This is libc" fi
Mojca ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net ___________________________________________________________________________________
% cat ./test-musl.zsh if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q "^musl" then echo "1 libc=musl" else echo "1 libc=glibc" fi
if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q '^musl' then echo "2 libc=musl" else echo "2 libc=glibc" fi
if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl then echo "3 libc=musl" else echo "3 libc=glibc" fi
% zsh ./test-musl.zsh 1 libc=glibc 2 libc=glibc 3 libc=glibc
However, I was the one who requested the musl detection in config.guess and the maintainer implemented the check like this for reasons of portability.
Without escaping the caret it’s not as portable as it could be. By the way, I checked in the mean time, and the problem with unescaped ^musl in zsh is that it may be expanded to “any file name that is not musl” which explains the output Thomas was getting. The grep command is thus expanded to something like “grep -q dir1 dir2 file1 [piped output]”, generating an error message for each directory. Mojca, if you want to reproduce that, you need the extended glob option (setopt EXTENDED_GLOB). See man zshexpn for details.
Perhaps the problem is the else case in this statement. Maybe it should be:
LIBC="glibc" if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl then LIBC="musl" fi
That won’t help; it is logically equivalent to if command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl then LIBC=musl else LIBC=glibc fi and doesn’t address any of problems. Best, Arthur
participants (6)
-
Arthur Reutenauer
-
Hans Hagen
-
Henri Menke
-
luigi scarso
-
Mojca Miklavec
-
Thomas A. Schmitz