Importing iCalendar non-UTF-8 calendars

Bug #79907 reported by Tom Hoffman
18
Affects Status Importance Assigned to Milestone
SchoolTool
Incomplete
Medium
Justas Sadzevičius

Bug Description

A user gets this when importing a medium-sized .ics file from Sunbird:

2005-04-26 11:48:21,625
http://server:7080/persons/jim/calendar.ics
Traceback (most recent call last):
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\Zope3\src\zope\publisher\publish.py",
line 138, in publish
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\Zope3\src\zope\app\publication\http.py",
line 43, in callObject
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\Zope3\src\zope\publisher\publish.py",
line 113, in mapply
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\Zope3\src\zope\publisher\publish.py",
line 119, in debug_call
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\Zope3\src\zope\app\http\put.py",
line 112, in PUT
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\src\schoolbell\app\cal.py",
line 140, in write
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\src\schoolbell\calendar\icalendar.py",
line 314, in read_icalendar
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\src\schoolbell\calendar\icalendar.py",
line 539, in iterEvents
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\src\schoolbell\calendar\icalendar.py",
line 1068, in validate
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\src\schoolbell\calendar\icalendar.py",
line 1099, in _getType
AssertionError

Unfortunately, looking at that leaves me none the
wiser!

Most of the blocks in the log (presumably
corresponding to the several attempts I made to
publish the calendar files) look exactly the same
(except the time); however, there is a 'variation upon
the theme':

2005-04-26 12:12:07,000
http://server:7080/persons/julian/calendar.ics
Traceback (most recent call last):
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\Zope3\src\zope\publisher\publish.py",
line 138, in publish
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\Zope3\src\zope\app\publication\http.py",
line 43, in callObject
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\Zope3\src\zope\publisher\publish.py",
line 113, in mapply
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\Zope3\src\zope\publisher\publish.py",
line 119, in debug_call
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\Zope3\src\zope\app\http\put.py",
line 112, in PUT
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\src\schoolbell\app\cal.py",
line 140, in write
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\src\schoolbell\calendar\icalendar.py",
line 314, in read_icalendar
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\src\schoolbell\calendar\icalendar.py",
line 521, in iterEvents
 File "C:\Shared
Files\Downloads\Diary\schoolbell-1.0\src\schoolbell\calendar\icalendar.py",
line 487, in _iterRow
UnicodeDecodeError: 'utf8' codec can't decode byte
0xa3 in position 21: unexpected code byte

Revision history for this message
Marius Gedminas (mgedmin) wrote :

There are two distinct problems here.

One problem is the assertion error. It is a bug in our iCalendar parser. It
happens when there is a recurring event with more than one exception date. I've
marked the location of the bug with an XXX comment that includes a proposed fix;
it remains to write a regression test for this bug and apply the fix. (See the
checkin for revision 3533).

Another problem is character-set related. The iCalendar spec (RFC 2445) says
that the default charset of iCalendar files is UTF-8, but it can be overriden
with MIME Content-Type headers. The Zope 3 infrastructure that implements HTTP
PUT does not pass the Content-Type request header to our code and we always
assume the data is in UTF-8. It is possible that Sunbird sends data in some
other encoding and specifies it in the header.

Fixing this other problem might require some changes in Zope 3.

Revision history for this message
Tom Hoffman (tom-hoffman) wrote :

On 4/29/05, Marius Gedminas <email address hidden> wrote:

> Another problem is character-set related. The iCalendar spec (RFC 2445) says
> that the default charset of iCalendar files is UTF-8, but it can be overriden
> with MIME Content-Type headers. The Zope 3 infrastructure that implements HTTP
> PUT does not pass the Content-Type request header to our code and we always
> assume the data is in UTF-8. It is possible that Sunbird sends data in some
> other encoding and specifies it in the header.
>
> Fixing this other problem might require some changes in Zope 3.

Should we forward this to Zope3-dev?

--Tom

Revision history for this message
Ignas Mikalajūnas (ignas) wrote :

Well either we write a proposal to zope so they would extend the default put
views so they would try
   queryMultiAdapter((container, request), IFileFactory) and
   queryMultiAdapter((file, request), IWriteFile)
I am attaching a small patch that i hope will ilustrate the idea.

Or do it the way I just did with Timetable Put view - just write a custom view
for PUT.

I guess i'll write a custom view and then write a proposal (is the propsal
needed for such a small change ?)

Revision history for this message
Ignas Mikalajūnas (ignas) wrote :

Ooops there was a bug in the patch.

Revision history for this message
Marius Gedminas (mgedmin) wrote :

Will Zope 3 folks accept a patch without a unit test? I doubt it.

Marius Gedminas
--
I noticed that Open Source proponents using MacOS X have developed highly tuned
excuses, similar to those that smokers have about why cigarettes are good for
you.
 -- Miguel de Icaza,
    http://primates.ximian.com/~miguel/archive/2004/Aug-03.html

Revision history for this message
Gintautas Miliauskas (gintas) wrote :

Ignas' plan was first to find out if people agree with the general idea and then
polish the patch and add unit tests.

Changed in schooltool:
status: In Progress → Confirmed
Revision history for this message
Tom Hoffman (tom-hoffman) wrote :

Any clue about the status of this, ignas?

Changed in schooltool:
assignee: nobody → ignas
status: Confirmed → Incomplete
tags: added: calendar
tags: added: traceback
Changed in schooltool:
assignee: Ignas Mikalajūnas (ignas) → Justas Sadzevičius (justas-pov)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.