Problem with encoding, XML and writetolist...
Hi all, I have problem with chars encoding now; I have updated ConTeXt sometimes in January; before the time I was using old (year) version. In this prehistorical version the problem was not present. What is the stuff? The following code generates content with titles without spaces -- accented characters are replaced with \char XXX variant, so during reading TUI line TeX gobbles a space after XXX as a part of <number> syntax rule. Can I do something? With encoding? With XML? Thank you very much, have a nice day, Martin Code: \defineXMLenvironment [chapter] {\defineXMLsave[title]} {\expanded {\chapter\XMLflush{title}}} \starttext \startbuffer <chapter> <title>Ahojá hojč kuna</title> </chapter> \stopbuffer \processXMLbuffer \completecontent[criterium=all] \stoptext
� wrote:
Hi all,
I have problem with chars encoding now; I have updated ConTeXt sometimes in January; before the time I was using old (year) version. In this prehistorical version the problem was not present.
What is the stuff? The following code generates content with titles without spaces -- accented characters are replaced with \char XXX variant, so during reading TUI line TeX gobbles a space after XXX as a part of <number> syntax rule.
can you send me a zip with the test file so that the 8 bit chars are ok? Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Here the file with 8bit chars is, Martin -----Original Message----- From: Hans Hagen [mailto:pragma@wxs.nl] Sent: Tuesday, February 28, 2006 10:16 AM To: martin.kolarik@email.cz; mailing list for ConTeXt users Subject: Re: [NTG-context] Problem with encoding, XML and writetolist... � wrote:
Hi all,
I have problem with chars encoding now; I have updated ConTeXt sometimes in January; before the time I was using old (year) version. In this prehistorical version the problem was not present.
What is the stuff? The following code generates content with titles without spaces -- accented characters are replaced with \char XXX variant, so during reading TUI line TeX gobbles a space after XXX as a part of <number> syntax rule.
can you send me a zip with the test file so that the 8 bit chars are ok? Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Hi Hans, isn't there some news with the stuff? Thank you and have a nice day, Martin -----Original Message----- From: ntg-context-bounces@ntg.nl [mailto:ntg-context-bounces@ntg.nl]On Behalf Of Martin Kolařík Sent: Tuesday, February 28, 2006 10:33 AM To: mailing list for ConTeXt users Subject: Re: [NTG-context] Problem with encoding, XML and writetolist... Here the file with 8bit chars is, Martin -----Original Message----- From: Hans Hagen [mailto:pragma@wxs.nl] Sent: Tuesday, February 28, 2006 10:16 AM To: martin.kolarik@email.cz; mailing list for ConTeXt users Subject: Re: [NTG-context] Problem with encoding, XML and writetolist... � wrote:
Hi all,
I have problem with chars encoding now; I have updated ConTeXt sometimes in January; before the time I was using old (year) version. In this prehistorical version the problem was not present.
What is the stuff? The following code generates content with titles without spaces -- accented characters are replaced with \char XXX variant, so during reading TUI line TeX gobbles a space after XXX as a part of <number> syntax rule.
can you send me a zip with the test file so that the 8 bit chars are ok? Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Martin Kolařík wrote:
Hi Hans,
isn't there some news with the stuff?
small test file needed to refresh my memory Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Hi Hans, the test file is attached, thanks, Martin
-----Original Message----- From: ntg-context-bounces@ntg.nl [mailto:ntg-context-bounces@ntg.nl]On Behalf Of Hans Hagen Sent: Friday, March 31, 2006 7:01 PM To: martin.kolarik@email.cz; mailing list for ConTeXt users Subject: Re: [NTG-context] Problem with encoding, XML and writetolist...
Martin Kolařík wrote:
Hi Hans,
isn't there some news with the stuff?
small test file needed to refresh my memory
Hans
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
_______________________________________________ ntg-context mailing list ntg-context@ntg.nl http://www.ntg.nl/mailman/listinfo/ntg-context
Martin Kolařík wrote:
Hi Hans,
the test file is attached,
what input encoding is used here? i get the same chapter title as toc entry is your tex configured for 8 bit (natural.ctx) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Aha, here maybe will be the solution :-) I'am using standalone Win32 distribution from Pragma and I'm running texexec with --translate=cp1252cs. Typescripts use il2, which corresponds to fonts, of course. If I run the code without XML (if I transcribe XML to \chapter etc.) I get toc entry correctly too. Where should I look for more? What is natural.ctx? Thank you, Martin -----Original Message----- From: Hans Hagen [mailto:pragma@wxs.nl] Sent: Monday, April 03, 2006 10:36 AM To: martin@mii.cz; mailing list for ConTeXt users Subject: Re: [NTG-context] Problem with encoding, XML and writetolist... Martin Kolařík wrote:
Hi Hans,
the test file is attached,
what input encoding is used here? i get the same chapter title as toc entry is your tex configured for 8 bit (natural.ctx) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Martin Kolařík wrote:
Aha, here maybe will be the solution :-)
I'am using standalone Win32 distribution from Pragma and I'm running texexec with --translate=cp1252cs. Typescripts use il2, which corresponds to fonts, of course. If I run the code without XML (if I transcribe XML to \chapter etc.) I get toc entry correctly too.
Where should I look for more? What is natural.ctx?
this is a one-to-one translation vector, and the one standard used by context; forget about all those other mappings, they can be done using regimes; you can use an il2 input regime, and any (resonable) font encoding, Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
I was confused with typo (ctx=tcx :-) but I still did not succeed. I now installed last cont-tfm, I tried 8bit.tex (from the example) and nothing :-( I am using now directly "texexec 8bit.tex", which leads to natural.tcx and english interface, without anything more. I tried to change regimes in 8bit.tex too, and the characters were changing. So regimes are OK, result not. Texexec still reports: chapter : 1 ÄluŁou\char 163 c\char 237 kon\char 165 \char 250 p\char 237 which is not good. And I am not able, when not using cp1250.tcx, recode win1250 (default for win) to latin2. There I am able to write regime for myself (cp1250), but I suppose, that whenever characters will not expand to \char XXX form, cp1250.tcx still will be working and usable. Cannot be set somewhere something badly in my local context configuration? Thank you, Martin -----Original Message----- From: Hans Hagen [mailto:pragma@wxs.nl] Sent: Monday, April 03, 2006 11:57 AM To: martin.kolarik@email.cz; mailing list for ConTeXt users Subject: Re: [NTG-context] Problem with encoding, XML and writetolist... Martin Kolařík wrote:
Aha, here maybe will be the solution :-)
I'am using standalone Win32 distribution from Pragma and I'm running texexec with --translate=cp1252cs. Typescripts use il2, which corresponds to fonts, of course. If I run the code without XML (if I transcribe XML to \chapter etc.) I get toc entry correctly too.
Where should I look for more? What is natural.ctx?
this is a one-to-one translation vector, and the one standard used by context; forget about all those other mappings, they can be done using regimes; you can use an il2 input regime, and any (resonable) font encoding, Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
On 4/3/06, Martin Kolařík wrote:
which is not good. And I am not able, when not using cp1250.tcx, recode win1250 (default for win) to latin2. There I am able to write regime for myself (cp1250), but I suppose, that whenever characters will not expand to \char XXX form, cp1250.tcx still will be working and usable.
Hans, I would guess that more than 50% of Slavic Windows users (ie. Slovenian, Croatian, Chech, Slovak, ...) use the cp1250 encoding (including me). Would it be possible to add at least this file http://pub.mojca.org/tex/enco/contextbase/regi-cp1250.tex to the default ConTeXt distribution? (with "synonyms" windows-1250 and cp1250). This has probably nothing to do with the problem described here, but I believe that many users would appreciate to have that "encoding" (called regime in ConTeXt) available.
when I added \mainlanguage[cz] before \enableregime[windows] in 8bit.tex and run texexec with --translate=cp1250cs, I got:
\enableregime[windows] is cp1252 (Western European Windows) not cp1250 (Central European Windows), --translate=cp1250cs is cp1250 (supposing that you don't use \enableregime[windows], but the default one): everything will be mixed up if you use both simultaneously. If you download the file mentioned above (place it into the current directory or somewhere in the TeX tree and refresh the filename database) or if Hans adds it to the distribution, you can use \input regi-cp1250 \enableregime[cp1250] This probably won't solve your problem with missing spaces (didn't try it), but you'll have the proper encoding. Mojca
Thanks Mojca, I downloaded your regime file; now I will not need write it myself :-) Martin -----Original Message----- From: ntg-context-bounces@ntg.nl [mailto:ntg-context-bounces@ntg.nl]On Behalf Of Mojca Miklavec Sent: Monday, April 03, 2006 2:33 PM To: martin.kolarik@email.cz; mailing list for ConTeXt users; Hans Hagen Subject: Re: [NTG-context] Problem with encoding, XML and writetolist... On 4/3/06, Martin Kolařík wrote:
which is not good. And I am not able, when not using cp1250.tcx, recode win1250 (default for win) to latin2. There I am able to write regime for myself (cp1250), but I suppose, that whenever characters will not expand to \char XXX form, cp1250.tcx still will be working and usable.
Hans, I would guess that more than 50% of Slavic Windows users (ie. Slovenian, Croatian, Chech, Slovak, ...) use the cp1250 encoding (including me). Would it be possible to add at least this file http://pub.mojca.org/tex/enco/contextbase/regi-cp1250.tex to the default ConTeXt distribution? (with "synonyms" windows-1250 and cp1250). This has probably nothing to do with the problem described here, but I believe that many users would appreciate to have that "encoding" (called regime in ConTeXt) available.
when I added \mainlanguage[cz] before \enableregime[windows] in 8bit.tex and run texexec with --translate=cp1250cs, I got:
\enableregime[windows] is cp1252 (Western European Windows) not cp1250 (Central European Windows), --translate=cp1250cs is cp1250 (supposing that you don't use \enableregime[windows], but the default one): everything will be mixed up if you use both simultaneously. If you download the file mentioned above (place it into the current directory or somewhere in the TeX tree and refresh the filename database) or if Hans adds it to the distribution, you can use \input regi-cp1250 \enableregime[cp1250] This probably won't solve your problem with missing spaces (didn't try it), but you'll have the proper encoding. Mojca _______________________________________________ ntg-context mailing list ntg-context@ntg.nl http://www.ntg.nl/mailman/listinfo/ntg-context
Hans Hagen wrote:
Martin Kolařík wrote:
Aha, here maybe will be the solution :-)
I'am using standalone Win32 distribution from Pragma and I'm running texexec with --translate=cp1252cs. Typescripts use il2, which corresponds to fonts, of course. If I run the code without XML (if I transcribe XML to \chapter etc.) I get toc entry correctly too.
Where should I look for more? What is natural.ctx?
this is a one-to-one translation vector, and the one standard used by context; forget about all those other mappings, they can be done using regimes; you can use an il2 input regime, and any (resonable) font encoding,
Hans
specifically for you: \mainlanguage[cz] \enableregime[windows] % or \enableregime[latin2] for il2 is enough for Latin Modern and Czech. Vit
Thanks Vit, your solution perfectly works (I tried it :-); but not for me -- in my case (and in another mine cases too) I use XML code, which remaps some pieces to \chapter. And during the expansion accented characters are replaced with "\char xxx", this is written to tui/tuo file and read back without spaces -- the spaces are gobbled by TeX's number syntactic rule... But, Hans, when I added \mainlanguage[cz] before \enableregime[windows] in 8bit.tex and run texexec with --translate=cp1250cs, I got: ! Illegal parameter number in definition of \@@expanded. <to be read again> \crcr \oalign ...z@skip \lineskip .25ex\ialign {##\crcr #1\crcr }} \encircled ...\tx #1}}\hfil \crcr \mathhexbox 20D} } \XML:dat:title ->Ä luŁouŔcÝ koný ˙pÝ \expanded ...\long \xdef \@@expanded {\noexpand #1 }\@@expanded l.3 </chapter> Thank you, Martin -----Original Message----- From: ntg-context-bounces@ntg.nl [mailto:ntg-context-bounces@ntg.nl]On Behalf Of Vit Zyka Sent: Monday, April 03, 2006 1:18 PM To: mailing list for ConTeXt users Subject: Re: [NTG-context] Problem with encoding, XML and writetolist... Hans Hagen wrote:
Martin Kolařík wrote:
Aha, here maybe will be the solution :-)
I'am using standalone Win32 distribution from Pragma and I'm running texexec with --translate=cp1252cs. Typescripts use il2, which corresponds to fonts, of course. If I run the code without XML (if I transcribe XML to \chapter etc.) I get toc entry correctly too.
Where should I look for more? What is natural.ctx?
this is a one-to-one translation vector, and the one standard used by context; forget about all those other mappings, they can be done using regimes; you can use an il2 input regime, and any (resonable) font encoding,
Hans
specifically for you: \mainlanguage[cz] \enableregime[windows] % or \enableregime[latin2] for il2 is enough for Latin Modern and Czech. Vit _______________________________________________ ntg-context mailing list ntg-context@ntg.nl http://www.ntg.nl/mailman/listinfo/ntg-context
Hi Hans, I've analyzed the situation using different combinations of regimes and encodings, including switching off handling (\donothandleactivecharacter) and it seems to me, that every text with chars > 127, which is fully expanded (as in 8bit.tex), must cause difficulties, because TOC entry read back from TUO is scanned independently and thus spaces are gobbled. Now I have solved it using not-stable solution -- I am using triple sequence of \expandafter to expand \XMLflush after \chapter instead \expanded. The solution is not stable as it is ConTeXt implementation dependent. On the other side, now I do not know, how to partially expand code \chaper{\XMLflush{XMLTag}} to obtain sequence of simple unexpanded chars. Simple \expandafter is not sufficient, more \expandafters is dirty trick and full expansion (\expanded) leads to TOC entry problem. In TeX source this cannot occur (or can occur in strange situations :-), in XML source this is almost rule. Maybe it would be useful to have in XML engine macro, which expands XMLdata exactly to get read (original) content (like mine triple \expandafter). Maybe such macro already exists and I am a real fool :-) Thank you for help, have a nice day, Martin
-----Original Message----- From: ntg-context-bounces@ntg.nl [mailto:ntg-context-bounces@ntg.nl]On Behalf Of Hans Hagen Sent: Monday, April 03, 2006 11:57 AM To: martin.kolarik@email.cz; mailing list for ConTeXt users Subject: Re: [NTG-context] Problem with encoding, XML and writetolist...
Martin Kolařík wrote:
Aha, here maybe will be the solution :-)
I'am using standalone Win32 distribution from Pragma and I'm running texexec with --translate=cp1252cs. Typescripts use il2, which corresponds to fonts, of course. If I run the code without XML (if I transcribe XML to \chapter etc.) I get toc entry correctly too.
Where should I look for more? What is natural.ctx?
this is a one-to-one translation vector, and the one standard used by context; forget about all those other mappings, they can be done using regimes; you can use an il2 input regime, and any (resonable) font encoding,
Hans
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
_______________________________________________ ntg-context mailing list ntg-context@ntg.nl http://www.ntg.nl/mailman/listinfo/ntg-context
Hi Hans, I've analyzed the situation using different combinations of regimes and encodings, including switching off handling (\donothandleactivecharacter) and it seems to me, that every text with chars > 127, which is fully expanded (as in 8bit.tex), must cause difficulties, because TOC entry read back from TUO is scanned independently and thus spaces are gobbled. Now I have solved it using not-stable solution -- I am using triple sequence of \expandafter to expand \XMLflush after \chapter instead \expanded. The solution is not stable as it is ConTeXt implementation dependent. On the other side, now I do not know, how to partially expand code \chaper{\XMLflush{XMLTag}} to obtain sequence of simple unexpanded chars. Simple \expandafter is not sufficient, more \expandafters is dirty trick and full expansion (\expanded) leads to TOC entry problem. In TeX source this cannot occur (or can occur in strange situations :-), in XML source this is almost rule. Maybe it would be useful to have in XML engine macro, which expands XMLdata exactly to get read (original) content (like mine triple \expandafter). Maybe such macro already exists and I am a real fool :-) Thank you for help, have a nice day, Martin
-----Original Message----- From: ntg-context-bounces@ntg.nl [mailto:ntg-context-bounces@ntg.nl]On Behalf Of Hans Hagen Sent: Monday, April 03, 2006 11:57 AM To: martin.kolarik@email.cz; mailing list for ConTeXt users Subject: Re: [NTG-context] Problem with encoding, XML and writetolist...
Martin Kolařík wrote:
Aha, here maybe will be the solution :-)
I'am using standalone Win32 distribution from Pragma and I'm running texexec with --translate=cp1252cs. Typescripts use il2, which corresponds to fonts, of course. If I run the code without XML (if I transcribe XML to \chapter etc.) I get toc entry correctly too.
Where should I look for more? What is natural.ctx?
this is a one-to-one translation vector, and the one standard used by context; forget about all those other mappings, they can be done using regimes; you can use an il2 input regime, and any (resonable) font encoding,
Hans
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
_______________________________________________ ntg-context mailing list ntg-context@ntg.nl http://www.ntg.nl/mailman/listinfo/ntg-context
Martin Kolařík wrote:
Hi Hans,
I've analyzed the situation using different combinations of regimes and encodings, including switching off handling (\donothandleactivecharacter) and it seems to me, that every text with chars > 127, which is fully expanded (as in 8bit.tex), must cause difficulties, because TOC entry read back from TUO is scanned independently and thus spaces are gobbled.
Now I have solved it using not-stable solution -- I am using triple sequence of \expandafter to expand \XMLflush after \chapter instead \expanded. The solution is not stable as it is ConTeXt implementation dependent.
On the other side, now I do not know, how to partially expand code \chaper{\XMLflush{XMLTag}} to obtain sequence of simple unexpanded chars. Simple \expandafter is not sufficient, more \expandafters is dirty trick and full expansion (\expanded) leads to TOC entry problem. In TeX source this cannot occur (or can occur in strange situations :-), in XML source this is almost rule.
Maybe it would be useful to have in XML engine macro, which expands XMLdata exactly to get read (original) content (like mine triple \expandafter). Maybe such macro already exists and I am a real fool :-)
Thank you for help, have a nice day,
some commands indeed have keys like: expansion=no|yes|utf|xml|command and alike the problem is that when one does something \expanded{...} in a style, it bypasses this mechanism can you make me a (as small as possible) example of a doc? i can then test it here with my version Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Martin Kolařík wrote:
Hi Hans,
I've analyzed the situation using different combinations of regimes and encodings, including switching off handling (\donothandleactivecharacter) and it seems to me, that every text with chars > 127, which is fully expanded (as in 8bit.tex), must cause difficulties, because TOC entry read back from TUO is scanned independently and thus spaces are gobbled.
Hi Martin, we have discussed this problem a year ago. In that time I solve it just writing '{}' in each title where was needed, like \chapter{Dobré{} ráno} Now I have cooked an automatic solution. It is not optimal (effective) but working. The only needs is to redefine \def\numcharacter#1{\char#1{}} \let\dochar\numcharacter and to load appropriate encoding file once more. Since it is banned a new macro \reuseencoding was introduce: For Hans: It would be nice if this feature was enabled in a standard distro. Perhaps you will find some more optimal solution... ------------------------------------------------------- \unprotected \def\doreuseencoding#1% {\letvalue{\c!file\f!encodingprefix#1}\empty \makeshortfilename[\f!encodingprefix#1]% \startreadingfile \readsysfile\shortfilename {\showmessage\m!encodings2{#1}} {\showmessage\m!encodings3{#1}}% \stopreadingfile} \def\reuseencoding[#1]% {\processcommalist[#1]\doreuseencoding} \def\numcharacter#1{\char#1{}} \let\dochar\numcharacter \reuseencoding[ec] \enableregime[latin2] \starttext \writetolist[chapter]{1}{Dobré ráno} \expanded{\writetolist[chapter]{2}{Dobré odpoledne}} \placecontent \stoptext -------------------------------------------------------
Now I have solved it using not-stable solution -- I am using triple sequence of \expandafter to expand \XMLflush after \chapter instead \expanded. The solution is not stable as it is ConTeXt implementation dependent.
On the other side, now I do not know, how to partially expand code \chaper{\XMLflush{XMLTag}} to obtain sequence of simple unexpanded chars. Simple \expandafter is not sufficient, more \expandafters is dirty trick and full expansion (\expanded) leads to TOC entry problem. In TeX source this cannot occur (or can occur in strange situations :-), in XML source this is almost rule.
Maybe it would be useful to have in XML engine macro, which expands XMLdata exactly to get read (original) content (like mine triple \expandafter). Maybe such macro already exists and I am a real fool :-)
Thank you for help, have a nice day,
Martin
-----Original Message----- From: ntg-context-bounces@ntg.nl [mailto:ntg-context-bounces@ntg.nl]On Behalf Of Hans Hagen Sent: Monday, April 03, 2006 11:57 AM To: martin.kolarik@email.cz; mailing list for ConTeXt users Subject: Re: [NTG-context] Problem with encoding, XML and writetolist...
Martin Kolařík wrote:
Aha, here maybe will be the solution :-)
I'am using standalone Win32 distribution from Pragma and I'm
running texexec with --translate=cp1252cs. Typescripts use il2, which corresponds to fonts, of course. If I run the code without XML (if I transcribe XML to \chapter etc.) I get toc entry correctly too.
Where should I look for more? What is natural.ctx?
this is a one-to-one translation vector, and the one standard used by context; forget about all those other mappings, they can be done using regimes; you can use an il2 input regime, and any (resonable) font encoding,
Hans
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
_______________________________________________ ntg-context mailing list ntg-context@ntg.nl http://www.ntg.nl/mailman/listinfo/ntg-context
_______________________________________________ ntg-context mailing list ntg-context@ntg.nl http://www.ntg.nl/mailman/listinfo/ntg-context
-- ======================================================= Ing. Vít Zýka, Ph.D. TYPOkvítek database publishing databazove publikovani data maintaining and typesetting in typographic quality priprava dat a jejich sazba v typograficke kvalite tel.: (+420) 777 198 189 www: http://typokvitek.com =======================================================
Vit Zyka wrote:
Martin Kolařík wrote:
Hi Hans,
I've analyzed the situation using different combinations of regimes and encodings, including switching off handling (\donothandleactivecharacter) and it seems to me, that every text with chars > 127, which is fully expanded (as in 8bit.tex), must cause difficulties, because TOC entry read back from TUO is scanned independently and thus spaces are gobbled.
Hi Martin,
we have discussed this problem a year ago. In that time I solve it just writing '{}' in each title where was needed, like
\chapter{Dobré{} ráno}
Now I have cooked an automatic solution. It is not optimal (effective) but working. The only needs is to redefine
\def\numcharacter#1{\char#1{}} \let\dochar\numcharacter
and to load appropriate encoding file once more. Since it is banned a new macro \reuseencoding was introduce:
For Hans: It would be nice if this feature was enabled in a standard distro. Perhaps you will find some more optimal solution...
------------------------------------------------------- \unprotected \def\doreuseencoding#1% {\letvalue{\c!file\f!encodingprefix#1}\empty \makeshortfilename[\f!encodingprefix#1]% \startreadingfile \readsysfile\shortfilename {\showmessage\m!encodings2{#1}} {\showmessage\m!encodings3{#1}}% \stopreadingfile} \def\reuseencoding[#1]% {\processcommalist[#1]\doreuseencoding}
\def\numcharacter#1{\char#1{}} \let\dochar\numcharacter
\reuseencoding[ec]
\enableregime[latin2]
\starttext \writetolist[chapter]{1}{Dobré ráno} \expanded{\writetolist[chapter]{2}{Dobré odpoledne}}
\placecontent \stoptext
the {} may break liguature building (and maybe kerning) ... taco knows ... I got a brainwave ... \enableregime[latin2] \starttext \def\numcharacter#1{\rawcharacter{#1}}% \let\dochar\numcharacter \writetolist[chapter]{1}{Dobré ráno} \expanded{\writetolist[chapter]{2}{Dobré odpoledne}} \placelist[chapter][criterium=text] test \typefile{test.tuo} \stoptext this can be sped up (a bit) with % \dostepwiserecurse{0}{255}{1} % {\setevalue{rchr:\recurselevel}{\rawcharacter\recurselevel}} % \def\rawcharacter#1{\getvalue{rchr:#1}} this means that all chars are round tripped to themselves in output files! raw characters are not active and they are therefore equivalent to \char it's enough then to put \def\numcharacter#1{\rawcharacter{#1}}% \let\dochar\numcharacter into cont-new.tex (i will discuss this in more detail with taco later) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Hans Hagen wrote:
Vit Zyka wrote:
Martin Kolařík wrote:
Hi Hans,
I've analyzed the situation using different combinations of regimes and encodings, including switching off handling (\donothandleactivecharacter) and it seems to me, that every text with chars > 127, which is fully expanded (as in 8bit.tex), must cause difficulties, because TOC entry read back from TUO is scanned independently and thus spaces are gobbled.
Hi Martin,
we have discussed this problem a year ago. In that time I solve it just writing '{}' in each title where was needed, like
\chapter{Dobré{} ráno}
Now I have cooked an automatic solution. It is not optimal (effective) but working. The only needs is to redefine
\def\numcharacter#1{\char#1{}} \let\dochar\numcharacter
and to load appropriate encoding file once more. Since it is banned a new macro \reuseencoding was introduce:
For Hans: It would be nice if this feature was enabled in a standard distro. Perhaps you will find some more optimal solution...
------------------------------------------------------- \unprotected \def\doreuseencoding#1% {\letvalue{\c!file\f!encodingprefix#1}\empty \makeshortfilename[\f!encodingprefix#1]% \startreadingfile \readsysfile\shortfilename {\showmessage\m!encodings2{#1}} {\showmessage\m!encodings3{#1}}% \stopreadingfile} \def\reuseencoding[#1]% {\processcommalist[#1]\doreuseencoding}
\def\numcharacter#1{\char#1{}} \let\dochar\numcharacter
\reuseencoding[ec]
\enableregime[latin2]
\starttext \writetolist[chapter]{1}{Dobré ráno} \expanded{\writetolist[chapter]{2}{Dobré odpoledne}}
\placecontent \stoptext
the {} may break liguature building (and maybe kerning) ... taco knows ...
Hmmm, that's right.
I got a brainwave ...
\enableregime[latin2]
\starttext
\def\numcharacter#1{\rawcharacter{#1}}% \let\dochar\numcharacter
So there is no need to have only 7-bit chars in tui/tuo etc.? All right, I think this will be better also for sorting. Sounds promissing.
\writetolist[chapter]{1}{Dobré ráno}
\expanded{\writetolist[chapter]{2}{Dobré odpoledne}}
\placelist[chapter][criterium=text]
test
\typefile{test.tuo}
\stoptext
this can be sped up (a bit) with
% \dostepwiserecurse{0}{255}{1} % {\setevalue{rchr:\recurselevel}{\rawcharacter\recurselevel}}
% \def\rawcharacter#1{\getvalue{rchr:#1}}
this means that all chars are round tripped to themselves in output files!
raw characters are not active and they are therefore equivalent to \char
it's enough then to put
\def\numcharacter#1{\rawcharacter{#1}}% \let\dochar\numcharacter
into cont-new.tex
(i will discuss this in more detail with taco later)
All right, thanks Hans, Vit
Vit Zyka wrote:
So there is no need to have only 7-bit chars in tui/tuo etc.? All right, I think this will be better also for sorting. Sounds promissing.
if the regime is the same for a whole document ... indeed Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
On 2/28/06, Martin Kolařík wrote:
Hi all,
I have problem with chars encoding now; I have updated ConTeXt sometimes in January; before the time I was using old (year) version. In this prehistorical version the problem was not present.
What is the stuff? The following code generates content with titles without spaces -- accented characters are replaced with \char XXX variant, so during reading TUI line TeX gobbles a space after XXX as a part of <number> syntax rule.
Can I do something? With encoding? With XML?
Encoding has "nothing" to do with it. I don't really understand the \[un]expanded commands in TeX, but this part could theoretically be problematic: {\expanded {\section{\XMLflush{title}}}} Consider the following (encoding-independent) minimal example, just in case that this can help any of the gurus to understand the problem: \defineXMLenvironment [chapter] {\defineXMLsave[title]} {\expanded {\section{\XMLflush{title}}}} \starttext \startbuffer <chapter> <title>$a+b$</title> </chapter> \stopbuffer \processXMLbuffer \placecontent[criterium=all] \stoptext The entry in TOC is different from the title (one is in math mode and the other one writen with dollars - literally). Mojca
Thank you very much, have a nice day, Martin
Code:
\defineXMLenvironment [chapter] {\defineXMLsave[title]} {\expanded {\chapter\XMLflush{title}}}
\starttext
\startbuffer
<chapter> <title>Ahojá hojč kuna</title> </chapter>
\stopbuffer
\processXMLbuffer
\completecontent[criterium=all]
\stoptext
OK, expansion is not encoding, but expansion in this place here is not a problem (I suppose). The section head is good, but the TOC entry not. So the inner expansion (or conversion or detokenization or so) around writetolist is probably affected by something, what now I cannot handle. And what is strange -- Hans did not reproduce the behaviour (as he wrote), so I am asking a trying to find the problem on my side (or lucky thougth :-)... Thank for your ideas and help, Martin -----Original Message----- From: ntg-context-bounces@ntg.nl [mailto:ntg-context-bounces@ntg.nl]On Behalf Of Mojca Miklavec Sent: Monday, April 03, 2006 4:25 PM To: martin.kolarik@email.cz; mailing list for ConTeXt users Subject: Re: [NTG-context] Problem with encoding, XML and writetolist... On 2/28/06, Martin Kolařík wrote:
Hi all,
I have problem with chars encoding now; I have updated ConTeXt sometimes in January; before the time I was using old (year) version. In this prehistorical version the problem was not present.
What is the stuff? The following code generates content with titles without spaces -- accented characters are replaced with \char XXX variant, so during reading TUI line TeX gobbles a space after XXX as a part of <number> syntax rule.
Can I do something? With encoding? With XML?
Encoding has "nothing" to do with it. I don't really understand the \[un]expanded commands in TeX, but this part could theoretically be problematic: {\expanded {\section{\XMLflush{title}}}} Consider the following (encoding-independent) minimal example, just in case that this can help any of the gurus to understand the problem: \defineXMLenvironment [chapter] {\defineXMLsave[title]} {\expanded {\section{\XMLflush{title}}}} \starttext \startbuffer <chapter> <title>$a+b$</title> </chapter> \stopbuffer \processXMLbuffer \placecontent[criterium=all] \stoptext The entry in TOC is different from the title (one is in math mode and the other one writen with dollars - literally). Mojca
Thank you very much, have a nice day, Martin
Code:
\defineXMLenvironment [chapter] {\defineXMLsave[title]} {\expanded {\chapter\XMLflush{title}}}
\starttext
\startbuffer
<chapter> <title>Ahojá hojč kuna</title> </chapter>
\stopbuffer
\processXMLbuffer
\completecontent[criterium=all]
\stoptext
ntg-context mailing list ntg-context@ntg.nl http://www.ntg.nl/mailman/listinfo/ntg-context
participants (6)
-
Hans Hagen -
Martin Kolařík -
Martin Kolařík -
Martin Kolařík -
Mojca Miklavec -
Vit Zyka