Please sync 0.19.2-1 from Debian unstable

I concur.

This also concerns the Wormux game. Currently, we are waiting for distributions to move to this version. Farsi (or Persian), spoken in iran and Afghanistan, is an example of language not supported by 0.10.9.

Unfortunately, even Debian (next stable or sid) doesn't include it either.

Thanks for reporting this bug and any supporting documentation. Since this bug has enough information provided for a developer to begin work, I'm going to mark it as confirmed and let them handle it from here. Thanks for taking the time to make Ubuntu better!

I made libfribidi0 and arabic working with VLC for subtitles using version 0.20 as joined to this bug report. I consider that using this version solves the RTL unjoined chars bug.

Can you please follow https://wiki.ubuntu.com/PackagingGuide/Recipes/PackageUpdate and https://wiki.ubuntu.com/SponsorshipProcess and attach the .diff.gz here?

Actually, there are packages ready at http://mentors.debian.net/cgi-bin/sponsor-pkglist?action=details;package=fribidi and http://tzafrir.org.il/~tzafrir/fribidi/

Salam Ahmed... do you think it wuold make sense to get this into Ubuntu first as the mentoring request in Debian seems to be getting quite old?

On Wed, Jan 06, 2010 at 10:19:14AM -0000, Daniel Holbach wrote:
> Salam Ahmed... do you think it wuold make sense to get this into Ubuntu
> first as the mentoring request in Debian seems to be getting quite old?
---end quoted text---

There is only one issue which is removed symbols discussed on both those
threads:
http://lists.freedesktop.org/archives/fribidi/2009-March/000562.html
http://lists.alioth.debian.org/pipermail/debian-hebrew-package/2009-March/002330.html

I don't get any useful feedback from the Debian Hebrew maintainers:
http://lists.alioth.debian.org/pipermail/debian-hebrew-package/2009-June/002540.html

Also Tzafrir Cohen done some work on the package:
http://lists.alioth.debian.org/pipermail/debian-hebrew-package/2009-July/002550.html

So, do you think that the removed symbols don't require a soname bump as
Behdad suggests ?

--
 ‎أحمد المحمودي (Ahmed El-Mahmoudy)
  Digital design engineer
 GPG KeyID: 0xEDDDA1B7 (@ subkeys.pgp.net)
 GPG Fingerprint: 8206 A196 2084 7E6D 0DF8 B176 BC19 6A94 EDDD A1B7

That's very hard for me to say... it should be an upstream maintainer decision and we should make sure that no package needs those symbols any more.

by upstream maintainer , you mean Debian or Behdad ?

By decision I meant if a soname bump should be introduced or not and that'd be Behdad's call as maintaining a soname bump over upstream is a big delta to carry.

So, it seems like Ahmed is only a packager and may not be able to decide on this matter. Still, I think it is better to include a package that works (and is recommended by the authors) than ship something that doesn't work at all.

Are the missing symbols anything that's used in the distro? Is it functions that were supposed to be exposed at all?

On Mon, Jan 11, 2010 at 08:31:04AM -0000, Daniel Holbach wrote:
> Are the missing symbols anything that's used in the distro? Is it
> functions that were supposed to be exposed at all?
---end quoted text---

Many were symbols that weren't supposed to be exposed in the first place
indeed. How can I check if any of the missing symbols were used in the
distro ?

--
 ‎أحمد المحمودي (Ahmed El-Mahmoudy)
  Digital design engineer
 GPG KeyID: 0xEDDDA1B7 (@ subkeys.pgp.net)
 GPG Fingerprint: 8206 A196 2084 7E6D 0DF8 B176 BC19 6A94 EDDD A1B7

Check the list of rdepends, get the source and grep -r for the function names that were removed between versions.

On Mon, Jan 11, 2010 at 11:18:56AM -0000, Daniel Holbach wrote:
> Check the list of rdepends, get the source and grep -r for the function
> names that were removed between versions.
---end quoted text---

For all packages in Debian unstable (sid) that depend on libfribidi0, I
grep'ed for the symbols removed in 0.19.2. I found that 5 packages do
use some of the removed symbols. I've attached the grep result with this
email.

To summarize:

* bidiv & centerim use: fribidi_iso8859_8_to_unicode_c, fribidi_unicode_to_iso8859_8
* quesoglc uses: nFriBidiMirroredChars, fribidi_prop_to_type
* mailutils uses fribidi_wcwidth provided that HAVE_FRIBIDI_WCWIDTH is
  defined, I grep'ed for HAVE_FRIBIDI_WCWIDTH in both 0.19.2 & 0.10.9
  and I didn't find that
* text-bidi, which is a perl interface for libfribidi, uses many of
  those obsolete symbols of fribidi, yet declares them to be "private".

--
 ‎أحمد المحمودي (Ahmed El-Mahmoudy)
  Digital design engineer
 GPG KeyID: 0xEDDDA1B7 (@ subkeys.pgp.net)
 GPG Fingerprint: 8206 A196 2084 7E6D 0DF8 B176 BC19 6A94 EDDD A1B7

I'm sure that Behdad would be interested in that list. A soname change would make the switch to the new version of fribidi a lot easier and less messy.

On Mon, Jan 11, 2010 at 04:40:02PM -0000, Daniel Holbach wrote:
> I'm sure that Behdad would be interested in that list. A soname change
> would make the switch to the new version of fribidi a lot easier and
> less messy.
---end quoted text---

The fribidi mailing list was CC'ed in the last email.

--
 ‎أحمد المحمودي (Ahmed El-Mahmoudy)
  Digital design engineer
 GPG KeyID: 0xEDDDA1B7 (@ subkeys.pgp.net)
 GPG Fingerprint: 8206 A196 2084 7E6D 0DF8 B176 BC19 6A94 EDDD A1B7

I have a correction and additional info regarding the matter:

The correction, regarding what I mentioned earlier:

> * quesoglc uses: nFriBidiMirroredChars, fribidi_prop_to_type
> [...]
> quesoglc-0.7.2/src/fribidi/fribidi.c: pp->pos, pp->len, fribidi_type_name (pp->type), pp->level);
> quesoglc-0.7.2/src/fribidi/fribidi_mirroring.c: if (pos > nFriBidiMirroredChars - 1)
> quesoglc-0.7.2/src/fribidi/fribidi_mirroring.c: pos = nFriBidiMirroredChars - 1;
> quesoglc-0.7.2/src/fribidi/fribidi_mirroring.c: pos = step = (nFriBidiMirroredChars / 2) + 1;
> quesoglc-0.7.2/src/fribidi/fribidi_tab_char_type_9.i: return fribidi_prop_to_type[(unsigned char)FRIBIDI_GET_TYPE (uch)];
> quesoglc-0.7.2/src/fribidi/fribidi_tab_mirroring.i:const int nFriBidiMirroredChars = 332;
> quesoglc-0.7.2/src/fribidi/fribidi_types.c:const FriBidiCharType fribidi_prop_to_type[] = {
> quesoglc-0.7.2/src/fribidi/fribidi_types.c:fribidi_type_name (FriBidiCharType c)
> quesoglc-0.7.2/src/fribidi/fribidi_types.h: char *fribidi_type_name (FriBidiCharType c);
> quesoglc-0.7.2/src/fribidi/fribidi_types.h: extern const FriBidiCharType fribidi_prop_to_type[];

quesoglc actually defines those symbols in some .h & .i files, so it
does not use fribidi removed symbols.

What I want to add is regarding those:
> * bidiv & centerim use: fribidi_iso8859_8_to_unicode_c, fribidi_unicode_to_iso8859_8
> * mailutils uses fribidi_wcwidth provided that HAVE_FRIBIDI_WCWIDTH is
> defined, I grep'ed for HAVE_FRIBIDI_WCWIDTH in both 0.19.2 & 0.10.9
> and I didn't find that
> * text-bidi, which is a perl interface for libfribidi, uses many of
> those obsolete symbols of fribidi, yet declares them to be "private".

Behdad previously said that fribidi 0.19.x is currently in Fedora
(without any SO bump) and there are no problems there.[1]

I had a look at Fedora source packages, and I found the following:

1) bidiv: the source package in Fedora was patched to be able to compile
with fribidi 0.19.1

2) centerim: fribidi was not enabled in Fedora source package. Hence no
problem appeared.

3) mailutils: this package does not seem to exist in Fedora. Yet I'm not
sure if this package will have a problem in Debian/Ubuntu (please read
the previous email, quoted above)

4) text-bidi: this package does not seem to exist in Fedora, also I am
not really sure if this one is a problem. Since it is a sort of Perl
bindings for fribidi.

[1]
http://lists.alioth.debian.org/pipermail/debian-hebrew-package/2009-March/002330.html

--
 ‎أحمد المحمودي (Ahmed El-Mahmoudy)
  Digital design engineer
 GPG KeyID: 0xEDDDA1B7 (@ subkeys.pgp.net)
 GPG Fingerprint: 8206 A196 2084 7E6D 0DF8 B176 BC19 6A94 EDDD A1B7

It might be worth talking to Behdad about it again and present the evidence that you collected. Maybe there can be a brown-paper bag release?

On Mon, Feb 01, 2010 at 09:32:57PM +0200, أحمد المحمودي wrote:
> 1) bidiv: the source package in Fedora was patched to be able to compile
> with fribidi 0.19.1
>
> 2) centerim: fribidi was not enabled in Fedora source package. Hence no
> problem appeared.
>
> 3) mailutils: this package does not seem to exist in Fedora. Yet I'm not
> sure if this package will have a problem in Debian/Ubuntu (please read
> the previous email, quoted above)
>
> 4) text-bidi: this package does not seem to exist in Fedora, also I am
> not really sure if this one is a problem. Since it is a sort of Perl
> bindings for fribidi.
---end quoted text---

Regarding mailutils, please note that the check for HAVE_FRIBIDI_WCWIDTH
was introduced in the latest release which is currently in Debian
testing, I just compiled mailutils 2.1 against fribidi 0.19.2, and it
compiled. Yet compiling mailutils 2.0 (currently in Debian stable &
Ubuntu karmic) against fribidi 0.19.2 will FTBFS.

So to conclude, I see two solutions:

1) That Debian/Ubuntu do as Fedora, and patch the few apps (probably
only bidiv & centerim will need patching) that use obsolete fribidi
symbols. Note that centerim will also need a small patch very similar to
the one used for bidiv. Also if fribidi is to be backported to older
releases, then mailutils will need to be patched too.

2) Fribidi would SO bump.

--
 ‎أحمد المحمودي (Ahmed El-Mahmoudy)
  Digital design engineer
 GPG KeyID: 0xEDDDA1B7 (@ subkeys.pgp.net)
 GPG Fingerprint: 8206 A196 2084 7E6D 0DF8 B176 BC19 6A94 EDDD A1B7

Thanks Ahmed. My comments below:

On 01/11/2010 11:22 AM, أحمد المحمودي wrote:
> On Mon, Jan 11, 2010 at 11:18:56AM -0000, Daniel Holbach wrote:
>> Check the list of rdepends, get the source and grep -r for the function
>> names that were removed between versions.
> ---end quoted text---
>
> For all packages in Debian unstable (sid) that depend on libfribidi0, I
> grep'ed for the symbols removed in 0.19.2. I found that 5 packages do
> use some of the removed symbols. I've attached the grep result with this
> email.
>
> To summarize:
>
> * bidiv & centerim use: fribidi_iso8859_8_to_unicode_c, fribidi_unicode_to_iso8859_8

Both should be trivial to patch.

> * quesoglc uses: nFriBidiMirroredChars, fribidi_prop_to_type

Apparently quesoglc includes a *copy* of FriBidi in its source tree. So that
doesn't count.

> * mailutils uses fribidi_wcwidth provided that HAVE_FRIBIDI_WCWIDTH is
> defined, I grep'ed for HAVE_FRIBIDI_WCWIDTH in both 0.19.2 & 0.10.9
> and I didn't find that

You later said this one compiles with latest version. So, good?

> * text-bidi, which is a perl interface for libfribidi, uses many of
> those obsolete symbols of fribidi, yet declares them to be "private".

If it's bindings, can simply be updated.

> So to conclude, I see two solutions:
>
> 1) That Debian/Ubuntu do as Fedora, and patch the few apps (probably
> only bidiv & centerim will need patching) that use obsolete fribidi
> symbols. Note that centerim will also need a small patch very similar to
> the one used for bidiv. Also if fribidi is to be backported to older
> releases, then mailutils will need to be patched too.

This is what I suggest.

> 2) Fribidi would SO bump.

Then you have to rebuild all the apps that use the mainstream fribidi API.
The reason I did not bump the .so version was such that the new version is a
drop-in replacement of the old series that adds Arabic shaping. And many
mplayer users appreciate that...

behdad

On Mon, Feb 01, 2010 at 11:07:33PM +0200, أحمد المحمودي wrote:
> 1) That Debian/Ubuntu do as Fedora, and patch the few apps (probably
> only bidiv & centerim will need patching) that use obsolete fribidi
> symbols. Note that centerim will also need a small patch very similar to
> the one used for bidiv. Also if fribidi is to be backported to older
> releases, then mailutils will need to be patched too.
---end quoted text---

The following was done:

1) Filed a bug against bidiv (#568130) with a patch. [1]
2) centerim has a new upstream (which is now in Debian testing) which
   has the issue fixed already, so no patch needed !
3) Sent an email to text-bidi upstream asking about wether text-bidi is
   compatible with fribidi 0.19.x or not. I didn't get a reply yet (just
   sent the email few hours ago), but it seems to me that most probably
   this package isn't a problem.
4) Uploaded fribidi 0.19.2-1 to debian mentors. I hope now the hebrew
   team would take it from there.

So, as it seems to me, it boils down to applying the patch to bidiv is
the only thing needed to be done. (In addition to sponsoring new fribidi
upload)

[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=568130

--
 ‎أحمد المحمودي (Ahmed El-Mahmoudy)
  Digital design engineer
 GPG KeyID: 0xEDDDA1B7 (@ subkeys.pgp.net)
 GPG Fingerprint: 8206 A196 2084 7E6D 0DF8 B176 BC19 6A94 EDDD A1B7

fribidi 0.19.2-1 is on Debian now, please sync

ACKed.

Thanks Ahmed and Behdad for working this out! You guys rock!

[Updating] fribidi (0.10.9-1build1 [Ubuntu] < 0.19.2-1 [Debian])
 * Trying to add fribidi...
  - <fribidi_0.19.2-1.dsc: downloading from http://ftp.debian.org/debian/>
  - <fribidi_0.19.2-1.debian.tar.gz: downloading from http://ftp.debian.org/debian/>
  - <fribidi_0.19.2.orig.tar.gz: downloading from http://ftp.debian.org/debian/>
I: fribidi [main] -> libfribidi0_0.10.9-1build1 [main].
I: fribidi [main] -> libfribidi-dev_0.10.9-1build1 [main].
I: fribidi [main] -> libfribidi0-udeb_0.10.9-1build1 [main].

[Updating] fribidi (0.10.9-1build1 [Ubuntu] < 0.19.2-1 [Debian])
 * Trying to add fribidi...
  - <fribidi_0.19.2-1.dsc: cached>
  - <fribidi_0.19.2-1.debian.tar.gz: cached>
  - <fribidi_0.19.2.orig.tar.gz: cached>
I: fribidi [main] -> libfribidi0_0.10.9-1build1 [main].
I: fribidi [main] -> libfribidi-dev_0.10.9-1build1 [main].
I: fribidi [main] -> libfribidi0-udeb_0.10.9-1build1 [main].