[cups.bugs] [HIGH] STR #2533: still charset encoding problems

[Jerome Alet]

DO NOT REPLY TO THIS MESSAGE.  INSTEAD, POST ANY RESPONSES TO THE LINK BELOW.

[STR New]

Please consider the following charset encoding bug tested with CUPS v1.2.7.

Here we parse the IPP file for job 1729. This file represents a
print job which title contains an ISO-8859-15 encoded character (the
é) displayed as \xe9 in Python's output. The attributes charset
contained in the file is correctly set to ISO-8859-15, so the encoded
character matches the defined charset :

--- CUT ---
jerome at nordine:~$ python pkipplib/pkipplib/pkipplib.py
/var/spool/cups/c01729
Test OK : parsing original and parsing the output of the dump produce the
same dump !
IPP version : 1.1
IPP operation Id : 0x0002
IPP request Id : 0x00000001
Operation attributes :
  attributes-charset : [('charset', 'iso-8859-15')]
  attributes-natural-language : [('naturalLanguage', 'fr-fr at euro')]
Job attributes :
  printer-uri : [('uri', 'ipp://localhost/printers/HP2100')]
  job-originating-user-name : [('nameWithoutLanguage', 'jerome')]
  job-name : [('nameWithoutLanguage', "Paroles.net - Yves Montand - En
sortant de l'\xe9cole k")]
  document-format : [('mimeMediaType', 'application/postscript')]
  job-sheets : [('nameWithoutLanguage', 'none'), ('nameWithoutLanguage',
'none')]
  job-priority : [('integer', 50L)]
  job-uuid : [('uri', 'urn:uuid:933c9279-afe2-3a69-57f0-8c1d335d5a93')]
  job-originating-host-name : [('nameWithoutLanguage', 'localhost')]
  time-at-creation : [('integer', 1190826863L)]
  time-at-processing : [('integer', 1190826863L)]
  time-at-completed : [('integer', 1190826915L)]
  job-id : [('integer', 1729L)]
  job-state : [('enum', 9L)]
  job-media-sheets-completed : [('integer', 0L)]
  job-printer-uri : [('uri', 'ipp://nordine.ateur:631/printers/HP2100')]
  job-name : [('nameWithoutLanguage', "Paroles.net - Yves Montand - En
sortant de l'\xe9cole k")]
  job-k-octets : [('integer', 81L)]
  job-hold-until : [('keyword', 'no-hold')]
--- CUT ---

Now, we will use the very same Python library, but ask job 1729's
attributes directly to CUPS through http://localhost:631. The
attributes charset value in the request is by default set to
"UTF-8", meaning we want all attributes in the result encoded in the
"UTF-8" charset :


--- CUT ---
$ python
>>> from pkipplib import pkipplib
>>> cups=pkipplib.CUPS()
>>> answer = cups.getJobAttributes(1729)
>>> print answer
IPP version : 1.1
IPP operation Id : 0x0000
IPP request Id : 0x00000001
Operation attributes :
  attributes-charset : [('charset', 'utf-8')]
  attributes-natural-language : [('naturalLanguage', 'en-us')]
Job attributes :
  job-more-info : [('uri', 'ipp://localhost.localdomain:631/jobs/1729')]
  job-preserved : [('boolean', 0)]
  job-printer-up-time : [('integer', 1190842225L)]
  job-state-reasons : [('keyword', 'job-completed-successfully')]
  job-uri : [('uri', 'ipp://localhost.localdomain:631/jobs/1729')]
  printer-uri : [('uri', 'ipp://localhost/printers/HP2100')]
  job-originating-user-name : [('nameWithoutLanguage', 'jerome')]
  job-name : [('nameWithoutLanguage', "Paroles.net - Yves Montand - En
sortant de l'\xe9cole k")]
  document-format : [('mimeMediaType', 'application/postscript')]
  job-sheets : [('nameWithoutLanguage', 'none'), ('nameWithoutLanguage',
'none')]
  job-priority : [('integer', 50L)]
  job-uuid : [('uri', 'urn:uuid:933c9279-afe2-3a69-57f0-8c1d335d5a93')]
  job-originating-host-name : [('nameWithoutLanguage', 'localhost')]
  time-at-creation : [('integer', 1190826863L)]
  time-at-processing : [('integer', 1190826863L)]
  time-at-completed : [('integer', 1190826915L)]
  job-id : [('integer', 1729L)]
  job-state : [('enum', 9L)]
  job-media-sheets-completed : [('integer', 0L)]
  job-printer-uri : [('uri', 'ipp://nordine.ateur:631/printers/HP2100')]
  job-name : [('nameWithoutLanguage', "Paroles.net - Yves Montand - En
sortant de l'\xe9cole k")]
  job-k-octets : [('integer', 81L)]
  job-hold-until : [('keyword', 'no-hold')]
--- CUT ---

This means that despite CUPS answering with the same attributes-charset
and attributes-natural-language values that my cups.getJobAttributes()
method uses by default, the other attributes are not converted
to the charset we want, i.e. UTF-8, but instead are returned as-is.

Is this a bug ?

Is there a fix planned ?

Or is this me who don't understand something or is expecting too much ?

Is this related to #2167 ? (I'm not sure I understand this report
entirely)

FYI the job was submitted from the Galeon web browser using a french
locale and the ISO-8859-15 charset.

The problem is that the charset of the job's title passed on a
backend's command line is mostly unknown to that backend, and that
when trying to retrieve the job-name attribute directly from CUPS
and specifying the encoding we want, we don't get it anyway.

This is highly problematic because job's titles (and other
similar attributes) can't be saved reliably in a database
for later reuse, since the charset can't be known in a sure
way.

Link: http://www.cups.org/str.php?L2533
Version: 1.2.7