python-xml seems to be broken with python-2.6: xpath does not work

As stated in the openerp-server bug, it seems like boolean.so is missing.

But the directory
  /usr/lib/python2.6/dist-packages/oldxml/_xmlplus/utils
on my box contains the missing boolean.so, so i created a symlink in
  /usr/lib/python2.6/dist-packages/_xmlplus/utils

and it seems to be working now.
(Note: I have no idea about python / python modules, just digging around a bit)

I've done a little research about the files which python-xml installs:

# dpkg -L python-xml | grep xmlplus
/usr/lib/python2.5/site-packages/_xmlplus
/usr/lib/python2.5/site-packages/_xmlplus/utils
/usr/lib/python2.5/site-packages/_xmlplus/utils/boolean.so
/usr/lib/python2.6/dist-packages/oldxml/_xmlplus
/usr/lib/python2.6/dist-packages/oldxml/_xmlplus/utils
/usr/lib/python2.6/dist-packages/oldxml/_xmlplus/utils/boolean.so
/usr/share/pyshared/oldxml/_xmlplus
...
/usr/share/pyshared/oldxml/_xmlplus/utils
/usr/share/pyshared/oldxml/_xmlplus/utils/__init__.py
/usr/share/pyshared/oldxml/_xmlplus/utils/characters.py
/usr/share/pyshared/oldxml/_xmlplus/utils/iso8601.py
/usr/share/pyshared/oldxml/_xmlplus/utils/qp_xml.py
...
/usr/share/pyshared/_xmlplus
...
/usr/share/pyshared/_xmlplus/utils
/usr/share/pyshared/_xmlplus/utils/__init__.py
/usr/share/pyshared/_xmlplus/utils/characters.py
/usr/share/pyshared/_xmlplus/utils/iso8601.py
/usr/share/pyshared/_xmlplus/utils/qp_xml.py

It seems to me like most files are installed twice or more (old/new directory structure?) but boolean.so is about the only file installed only once, and I'm guessing in a directory which isn't used anymore ;)

For me it seems that python directory structure has changed, and that xparth or openerp are still looking at the old place...

IThe package python-xml contains :
/usr/lib/python2.5/site-packages/_xmlplus/utils/boolean.so
/usr/lib/python2.6/dist-packages/oldxml/_xmlplus/utils/boolean.so

But after installation, only the 2.6 file is there :
find /usr/lib/ -name boolean.so
/usr/lib/python2.6/dist-packages/oldxml/_xmlplus/utils/boolean.so
/usr/lib/python2.6/dist-packages/_xmlplus/utils/boolean.so

True it does not cp boolean.so into python2.5/site-packages/_xmlplus/utils/.

And it does not cp pyexpat.so and sgmlop.so into python2.5/site-packages/_xmlplus/parsers/.

Or let me say the python2.5/site-packages/_xmlplus directory does not exists at all. The directory python2.5/site-packages/oldxml does. And python2.5/site-packages/oldxml/_xmlplus as well.

it's finally time to remove python-xml. please use the xml modules included in the python core.

ubuntu-archive, while this breaks some packages in universe, these packages should be changed instead to use the libraries provided in the tools or in recent third party xml libarries.

2009-06-08 10:39:21 INFO Removing candidates:
2009-06-08 10:39:21 INFO python-xml 0.8.4-10.1ubuntu2 in karmic
2009-06-08 10:39:21 INFO python-xml 0.8.4-10.1ubuntu2 in karmic amd64
2009-06-08 10:39:21 INFO python-xml 0.8.4-10.1ubuntu2 in karmic armel
2009-06-08 10:39:21 INFO python-xml 0.8.4-10.1ubuntu2 in karmic i386
2009-06-08 10:39:21 INFO python-xml 0.8.4-10.1ubuntu2 in karmic ia64
2009-06-08 10:39:21 INFO python-xml 0.8.4-10.1ubuntu2 in karmic lpia
2009-06-08 10:39:21 INFO python-xml 0.8.4-10.1ubuntu2 in karmic powerpc
2009-06-08 10:39:21 INFO python-xml 0.8.4-10.1ubuntu2 in karmic sparc
2009-06-08 10:39:21 INFO python-xml-dbg 0.8.4-10.1ubuntu2 in karmic amd64
2009-06-08 10:39:21 INFO python-xml-dbg 0.8.4-10.1ubuntu2 in karmic armel
2009-06-08 10:39:21 INFO python-xml-dbg 0.8.4-10.1ubuntu2 in karmic i386
2009-06-08 10:39:21 INFO python-xml-dbg 0.8.4-10.1ubuntu2 in karmic ia64
2009-06-08 10:39:21 INFO python-xml-dbg 0.8.4-10.1ubuntu2 in karmic lpia
2009-06-08 10:39:21 INFO python-xml-dbg 0.8.4-10.1ubuntu2 in karmic powerpc
2009-06-08 10:39:21 INFO python-xml-dbg 0.8.4-10.1ubuntu2 in karmic sparc
2009-06-08 10:39:21 INFO python-xml-doc 0.8.4-10.1ubuntu2 in karmic amd64
2009-06-08 10:39:21 INFO python-xml-doc 0.8.4-10.1ubuntu2 in karmic armel
2009-06-08 10:39:21 INFO python-xml-doc 0.8.4-10.1ubuntu2 in karmic i386
2009-06-08 10:39:21 INFO python-xml-doc 0.8.4-10.1ubuntu2 in karmic ia64
2009-06-08 10:39:21 INFO python-xml-doc 0.8.4-10.1ubuntu2 in karmic lpia
2009-06-08 10:39:21 INFO python-xml-doc 0.8.4-10.1ubuntu2 in karmic powerpc
2009-06-08 10:39:21 INFO python-xml-doc 0.8.4-10.1ubuntu2 in karmic sparc
2009-06-08 10:39:21 INFO xbel 0.8.4-10.1ubuntu2 in karmic amd64
2009-06-08 10:39:21 INFO xbel 0.8.4-10.1ubuntu2 in karmic armel
2009-06-08 10:39:21 INFO xbel 0.8.4-10.1ubuntu2 in karmic i386
2009-06-08 10:39:21 INFO xbel 0.8.4-10.1ubuntu2 in karmic ia64
2009-06-08 10:39:21 INFO xbel 0.8.4-10.1ubuntu2 in karmic lpia
2009-06-08 10:39:21 INFO xbel 0.8.4-10.1ubuntu2 in karmic powerpc
2009-06-08 10:39:21 INFO xbel 0.8.4-10.1ubuntu2 in karmic sparc
2009-06-08 10:39:21 INFO xbel-utils 0.8.4-10.1ubuntu2 in karmic amd64
2009-06-08 10:39:21 INFO xbel-utils 0.8.4-10.1ubuntu2 in karmic armel
2009-06-08 10:39:21 INFO xbel-utils 0.8.4-10.1ubuntu2 in karmic i386
2009-06-08 10:39:21 INFO xbel-utils 0.8.4-10.1ubuntu2 in karmic ia64
2009-06-08 10:39:21 INFO xbel-utils 0.8.4-10.1ubuntu2 in karmic lpia
2009-06-08 10:39:21 INFO xbel-utils 0.8.4-10.1ubuntu2 in karmic powerpc
2009-06-08 10:39:21 INFO xbel-utils 0.8.4-10.1ubuntu2 in karmic sparc
2009-06-08 10:39:21 INFO Removed-by: Steve Langasek
2009-06-08 10:39:21 INFO Comment: obsolete and broken; reverse-deps must be migrated to core python modules. LP: #343242
2009-06-08 10:39:21 INFO 36 packages successfully removed.
2009-06-08 10:39:21 INFO Transaction committed.
2009-06-08 10:39:21 INFO The archive wil...

Streamlining the use of the standard library xml module is probably a good idea. However, the removal of python-xml breaks not only packages in universe, but also third party applications. At my company, various products include code built around PyXML's xpath module. While they run perfectly well on our servers using older Python and PyXML versions, I am currently only able to run test builds on my Ubuntu 9.04 workstation after I used the "symlink booleans.so" hack mentioned above.

Is there a specific reason why PyXML shouldn't be in Ubuntu? Because now that python-xml has been removed completely, I probably won't be able to upgrade to future Ubuntu versions without pulling an outdated (or non-packaged) PyXML back in.

> it's finally time to remove python-xml. please use the xml modules included in the python core.

I sincerely hope you didn't just remove the only functioning Python XPath implementation. I have production code that depends on this library because there doesn't seem to be anything else that provides it.

PyXML is now unmaintained for nearly 10 years. if you need some functionality from the package, please consider starting a new upstream project with just these bits you need (e.g. xpath).

Openerp need python-xml !! And I don't think there will do anything upstream in short future, so if we wan't openerp-server working in karmic, you need to provide python-xml...

PyXML won't come back. The last upstream release was ages/years ago.

Hello,

Apologies for restarting the discussion, but I'm still not convinced about the arguments for removing PyXML. (Like others who have commented on this issue, I have some code that depends on PyXML's XPath capabilities; changing to another implementation would require substantial changes.)

I do realise that PyXML has been inactive for a while (the latest file available for download on the SourceForge page dates back from November 2004, which is almost 5 years ago). However, I wonder if this really is an argument for discontinuing its inclusion. Does a project need to have continuous releases to be included? If something is not broken, why fix it?
A good reason for not including it would be that it breaks with some newer versions of Python and that it hasn't been maintained, but perhaps the reason there haven't been any new releases is simply the fact it doesn't need any urgent maintenance (triggered by external changes). Are there any issues that need maintenance because of recent changes in Python?

It's already been removed.

Could it not be re-included?

Just to answer my previous question. PyXML breaks with Python 2.6 because "as" has become a reserved keyword:
https://bugzilla.redhat.com/show_bug.cgi?id=477783

I'm attaching a patch to fix this based on PyXML-0.8.4.tar.gz (as downloaded from http://sourceforge.net/projects/pyxml/files/ ). I can't remember the details of Debian (and thus Ubuntu) packaging, but I think there is a way to include patches in packages (debian/patches directory). I suppose it shouldn't be a big problem to include this patch (slightly adapted to the actual package).

In theory, it could be, but the Python maintainer had asked its because of
conflicts with the parts that were included in Python 2.4 and later.

In addition to your change, it would also need to be modified to not
duplicate what's already in Python itself, get agreement from the Python
maintainer, and have someone who will keep up with the package in Ubuntu.

So far no one has appeared to do this work.

Sorry, I hadn't realised the entire python-xml package had been removed (I though you were only talking of PyXML). It has indeed been removed from Debian too.

I'm not sure about the other libraries that were in python-xml, but it seems that PyXML works fine (with the patch for "as" keyword) so far. I managed to install it with "python setup.py install" (which can be done via checkinstall to create a package). Perhaps this could be packaged in a package on its own (python-pyxml?).

I found that some features of PyXML were not available via ElementTree 1.2.6 (which is the one package with python-2.6 2.6.2-0ubuntu1 - Jaunty/9.04), in particular, XPath expressions based on attributes ([@attr]). This feature seem only available from version 1.3 of ElementTree (http://effbot.org/zone/element-xpath.htm ). The same version seems to be in the python2.6 package for the upcoming karmic/9.10. I suppose libxml2 might provide this feature.

libxml2 would appear to be the way to do xpath these days. It does require a bit of a rewrite to the code using it and I just filed a bug to address it in openERP https://bugs.launchpad.net/openobject-server/+bug/429519

Quick question as I'm also affected by this. I've made the symlink suggested above, so now this is not urgent. I assume there is a policy or philosophy statement wherein it is OK to break existing code in favor of better infrastructure; I'd like to make a process suggestion about how to get around this sort of issue, but I want to make it in the right place and in the right way. Where could I read that kind of stuff?

@cantorman, in this instance at least Ubuntu is following Debian, where the package was dropped http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=540475 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=468602 basically the functionality was duplicated in the core and in another library.

This dependency hell makes me want to go back to Windows. At least the DLL hell was easier fixable.

Removing python-XML BREAKS PACKAGES that your users want to use.

what packages? have bugs been filed against them? python-xml is broken and isn't going to be fixed, it is a dead end and there is a different way of providing the functionality it provides. If packages depend on it then they need to be fixed so they don't depend on broken stuff.

Going back to Windows just because of this seems a bit radical. If PyXML is what's required, doing the following seems rather straightforward compared with some DLL conflicts:

1. Download PyXML-0.8.4.tar.gz from http://sourceforge.net/projects/pyxml/files/
2. You may need to install python2.6-dev, for example via: sudo apt-get install python2.6-dev
3. Use these commands:
tar xvzf PyXML-0.8.4.tar.gz
cd PyXML-0.8.4
wget http://launchpadlibrarian.net/31786748/0001-Patch-for-Python-2.6.patch
patch -p1 < 0001-Patch-for-Python-2.6.patch
sudo python2.6 setup.py install

I still reckon PyXML should be packaged in its own package (if the other things in python-xml are broken). I think the patch is easy enough for it to be dealt with as part of the packaging process. Libxml2 may be the modern way to do XPath in Python, but having all the legacy applications (whether-or-not packaged by Debian/Ubuntu) that currently rely on PyXML change their code to use libxml2 sounds like a lot of work, possibly unrealistic.

The problem is that if pyxml is installed, the parts of pyxml that were
integrated (and maintained) in Python 2.4 and later get superceded by the
obsolete pyxml code if it's installed, so installing it to support one
package can have negative effects on others.

For people that really want the other pyxml bits, the thing to do is to go
through and remove all the code that was integrated into Python, make a new
release of this subset, and take up maintenance of it.

This also affects CalendarServer, so, Ubuntu-Server cannot act as a iCal server by default.

apt-get install calendarserver Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
  calendarserver: Depends: python-xml but it is not installable
E: Broken packages

I can't seem to install EasyCam without python-xml... any way around this?

easycam2 require this too