Utf-8 encoding issue with foriegn language in non-mac machine

Mon Jul 16 02:31:47 PDT 2012

chandrashekar wrote:

>> > chandrashekar wrote:
>> >
>> > >> chandrashekar wrote:
>> > >>
>> > >> >> chandrashekar wrote:
>> > >> >>
>> > >> >> >> chandrashekar wrote:
>> > >> >> >>
>> > >> >> >> > Hi,
>> > >> >> >> >
>> > >> >> >> > I am trying to print utf-8 encoded text file with korean
>> > >> >> >> > fonts in it, I am using Generic-PostScript.ppd which is
>> > >> >> >> > globalized with korean translation. I am trying to print a
>> > >> >> >> > text file but instead of korean font its displaying boxes.
>> > >> >> >> > Please provide me some insight of how to achieve
>> > >> >> >> > localization.
>> > >> >> >> >
>> > >> >> >> > Changes done:
>> > >> >> >> >
>> > >> >> >> > Message catalog present in: share/locale/ko folder
>> > >> >> >> > Globalized PPD: etc/cups/ppd folder
>> > >> >> >> > Cups version: 1.5.2
>> > >> >> >> >
>> > >> >> >> >
>> > >> >>
>> > 
http://www.opensource.apple.com/source/cups/cups-136/cups/ppd/generic.ppd
>> > >> >> >> >
>> > >> >> >> > Still no luck, need help in this regards,
>> > >> >> >>
>> > >> >> >> Well, the Monospace font family distributed with CUPS does not
>> > >> >> >> contain a korean script (set of glyphs) as far as I can see,
>> > >> >> >> and GNU FreeMono doesn't either.
>> > >> >> >> You'll need to find a suitable font (see CJK fonts) and
>> > >> >> >> possibly merge that into Monospace or FreeMono and use that
>> > >> >> >> instead. As for font merging, this can be done e.g. using the
>> > >> >> >> free FontForge; you may ask me off the list if you need
>> > >> >> >> assistance.
>> > >> >> >>
>> > >> >> >> Helge
>> > >> >> >>
>> > >> >> > Thanks for replying, I need your assistance in merging chinese,
>> > >> >> > chinese(Taiwan), japanese and korean fonts with Monospace.
>> > >> >> >
>> > >> >> > Under CJK fonts there are 3 varieties of typeface ie Ming,
>> > >> >> > sans-serif and script. which wont would suite for merging to
>> > >> >> > Monospace and please let me know some insight on same.
>> > >> >>
>> > >> >> I found
>> > >> >> http://en.wikipedia.org/wiki/List_of_CJK_fonts
>> > >> >> as a first place for further investigation. Especially see the
>> > >> >> link to the list of free Korean fonts there.
>> > >> >>
>> > >> >> If you need technical support for merging fonts, you may ask me
>> > >> >> (even off the list), but I'm not the person to make proposals
>> > >> >> which font to use.
>> > >> >>
>> > >> >> Helge
>> > >> >>
>> > >> > Thanks Helge,
>> > >> >
>> > >> > I need your technical support for merging fonts.
>> > >> >
>> > >> > From CJK fonts I am choosing Sans-Serif -> UnDotum korean fonts
>> > >> > which is GPL. http://en.fonts2u.com/download/un-dotum.family
>> > >> >
>> > >> > I think 2 changes need to be done.
>> > >> > 1. Merge UnDotum fonts with Monospace, Monospace-Bold,
>> > >> > Monospace-BoldOblique, Monospace-Oblique fonts. 2. Mention unicode
>> > >> > character mapping inside /share/cups/charsets/utf-8
>> > >> >
>> > >> > I used fontforge to open UnDotum Korean font and tried to copy
>> > >> > glyphs but when I tried to paste in Monospace font its not
>> > >> > accepting.
>> > >> >
>> > >> > Please tell me the procedure to do it.
>> > >>
>> > >> As you told you are using CUPS 1.5.2, I need to merge and convert
>> > >> the fonts to Type1 fonts, as the texttops filter needs this font
>> > >> file format. To verify that the font merging works, I'd need a
>> > >> sample file to check if the korean text shows up as expected (as far
>> > >> as I can judge) and not as black or white squares.
>> > >>
>> > >> Helge
>> > >>
>> > >
>> > > Helge,
>> > >
>> > > I sent a sample korean.txt file to ur mail id. I am even pasting the
>> > > file contents here please let me know the procedure to merge and
>> > > convert fonts to type1 fonts.
>> > >
>> > > 중앙일보 - 사건/사회 - 극지탐험　협회결
>> > > 극지탐험　협회결성　체계적　연구
>> > > 지구상의　３대　극지라　불리는　남극·북극·에베레스트를　한번이라
>> > > 도　다녀와야　정회원으로　들어갈　수　있는　한국극지협회가　발족다
>> > > ．윤석순（한·러시아극동협회　상임고문）씨과　홍석하（사람과　산　발
>> > > 행인）씨가　극지탐험과　이곳에서의　학술연구를　체계적　으로　해보자
>> > > 는데　뜻을　같이하고　협회결성에　나섰다．
>> > > 이　협회는　지난　１６일　호텔신라에서　２０여명의　준비위원이　참
>> > > 석한　가운데　준비위원회를　가졌으며　내년　３월　정식　출범한다．
>> > > 이　협회에는　３극오지를　모두　밟은　세계적인　산악인　허영호씨를
>> > > 비롯해　에베레스트를　올랐던　엄홍길·박영석·정승권씨　등　국내의
>> > > 저명한　산악인들이　회원으로　참여할　것으로　보인다．
>> > > 극지협회는　극지탐험가는　물론　학계·경제계인사들도　참가시킬　계
>> > > 획이다．　단순한　탐험차원을　넘어　지구상에　마지막　남은　자원의
>> > > 보고인　극지에서의　연구활동도　하겠다는　의미다．
>> > > 한국극지협회가　발족하면　우리　극지탐험이　체계화될　것으로　기대
>> > > 되고있다．협회는　극지관련자료를　데이터베이스로　축적해　오지탐험가
>> > > 들에게　제공할　계획이다．
>> > > 입력시간
>> > > ⓒ 중앙일보사
>> > >
>> > > -chandrashekar
>> >
>> > The mail you sent me didn't reach me (no idea why), so I did a test
>> > with the pasted variant.
>> > A quick test script revealed that the characters of the pasted text all
>> > are in unicode ranges 0x31.. to ox35.., but the UnDotum font does not
>> > contain these.
>> > Could you please give more hints?
>> >
>> > Helge
>> >
>>
>> Helge,
>>
>> Korean fonts range from { U+AC00–U+D7AF, U+1100–U+11FF, U+3130–U+318F,
>> U+3200–U+32FF, U+A960–U+A97F, U+D7B0–U+D7FF, U+FF00–U+FFEF } this
>> includes both north & south korean fonts. The sample file which i had
>> taken is from google translate. I am attaching few samples to ur email-id
>> "h.blischke at acm.org". Please let me know if you dont receive again.
>>
>> What i observed is korean font(UnDotum.ttf) is 2-byte encoding(ISO
>> 10646-1(Unicode, BMP) where as monospace is encoded with 1-byte(Custom
>> encoding).
>>
>> I am referring to UnDotum font family as well Nanum korean fonts which is
>> used in Ubuntu Linux pc's. which ever resolves our issues.
>>
>> http://en.wikipedia.org/wiki/Hangul
>> http://packages.ubuntu.com/natty/fonts/ttf-nanum
>>
>> Is there any way we could use the Undottum/Nanum fonts along with
>> Monospace instead of merging. I tried to copy individual glyphs from
>> Undottum to Monospace I could succeed in copying, but copying 15k
>> characters manually is very difficult process. I need ur help in knowing
>> the steps to merge the fonts by keeping Monospace encoding standard as it
>> is.
>>
> 
> Helge,
> 
> I tried copying individual glyphs from UnDottum.ttf and pasted on
> Monospace and generated fonts. I am facing below issue. I need your
> assistance in achieving this task.
> 
> 1. Getting .nodef in few glyphs because of this I am not able to give
> korean print. If i consider printing with glyphs after .nodef and before
> .nodef and giving this range in utf-8 character set, then its working
> fine.
> 
> Please help me in merging the foreign language, I need steps to achieve
> this using fontforge.
> 
> -chandrashekar

I successfully converted the said truetype font to a type1 font and verified 
that the glyphs beyond the Adobe glyph list are correctly named like 
"unixxxx" where xxxx denotes the unicode in hex.
But cups' texttops filter (I'm testing with cups 1.3.11) failed to work with 
the font, so now I'm going to set up a virtual machine that uses a newer 
version of cups to test there.

Helge