docs make bzr package too big

2009/6/9 Brad Schick <email address hidden>:
> I am deploying bzr 1.15 on Ubuntu Linux systems and noticed that the PPA
> for bzr 1.15 is rather large for a command-line tools (currently just
> over 5MB). Somewhat ironically, I noticed this due to much slower repo
> clones after pushing bzr's .dep into a bzr repository. Looking into this
> a bit, I found that most of the bulk is in the docs:

Hi, thanks for reporting this, you're quite right that it is too big.

I filed <https://bugs.edge.launchpad.net/bzr/+bug/385074>

>
> /usr/share/doc/bzr$ du -h
>
> 288K    ./txt/developers
> 24K    ./txt/en/developer-guide
> 116K    ./txt/en/release-notes
> 4.0K    ./txt/en/mini-tutorial
> 28K    ./txt/en/tutorials
> 172K    ./txt/en/user-reference
> 212K    ./txt/en/user-guide
> 4.0K    ./txt/en/admin-guide
> 560K    ./txt/en
> 4.0K    ./txt/es/mini-tutorial
> 12K    ./txt/es/guia-usuario
> 16K    ./txt/es
> 876K    ./txt
> 1.4M    ./html/developers
> 88K    ./html/en/developer-guide
> 504K    ./html/en/release-notes

> - Many debian/ubuntu packages provide non-essential documentation in
> separate "-doc" packages. This might be a good idea for bzr. In
> particular the developer docs, but I'd actually prefer everythiny except
> man/info pages be extracted.

I agree.

> - Both of those large PNG images can be reduced by 50% or more by
> converting them to indexed (quantized) images. Currently they are 24bit
> RGBs. For example, I can't personally see any different after running
> "convert performance.png +dither -colors 32 performance_indexed.png".
> And the result is 537K smaller!
> - Other smaller PNGs don't convert so well perhaps due to lots of
> anti-aliasing. Although not as important, it seems like those could be
> simplified and indexed as well.

I think performance.png is now obsolete and could just be removed.

--
Martin <http://launchpad.net/~mbp/>

Additional comments from the email thread:

Matt Nordhoff wrote:
>Brad Schick wrote:
> I just noticed that the bzr debian package contains many tests. Seems
> like this should be part of the source distribution and not part of the
> standard install. After install, /usr/share/pyshared/bzrlib/tests is 6.7M.

You can run the test suite with "bzr selftest", and many parts of bzrlib
import bzrlib.tests, so removing them all would be more complicated than
just "rm -rf bzrlib/tests".

Not that it's a bad idea, necessarily... Just sayin' that it wouldn't be
completely trivial to do.

I'll look into creating a separate -doc package when I create the 1.16 ppa packages.

The performance.png file is probably obsolete.

Please try to make the -doc package in a way that can merge back to
ubuntu and debian.

Let's have a separate bug for stripping the test suite.

The PNG can be easily re-compressed in a lossless manner with optipng. "optipng -o7" will try many possible combinations of compression strategies and color-reindexing to get the smallest possible PNG file.

For many of these images, I think you want lossy compression. I played with quick-start-summary.png a bit and got the best results (on Linux) using gimp 78.5% savings. ImageMagick's color reduction seems rather broken to me. It produces very unexpected results, particularly for images with alpha transparency. "optipng -o7 quick-start-summary.png" gave a 24.7% savings.

I found the best looking results using Adobe ImageReady CS. Only 64% smaller, but the results definitely look better than the Linux quantization tools. The best overall result would probably be from using ImageReady to quantize the images and then optipng to compress them optimally.

Probably way more thought than is needed for this ;) But if they will be used, I can perform those steps on all the PNGs in the docs and upload them. Just let me know.

Please attach a better image to this bug, mark it in progress, and
we'll merge it.

johnf, I'm planning on cutting a 1.16rc1 release tomorrow (sometime after Jun 11, 0:00 UTC). Do you need to make any upstream changes to Bazaar to split out a -doc package?

Reduced the overall size from 1.6M to 644K, and they all still look good. That includes performance.png which I understand might not be needed.

I landed a change so that performance.png is no longer built. I'll merge the new images; thanks.

I'll leave this bug open and make the summary more specific to the issue of splitting the docs into a separate package. The other issues are done, and there's a separate bug for separating the test suite.

We also have three versions of the quick reference:

[5] mbp@grace% ls -oh en/quick-reference/
total 740K
-rw-r--r-- 1 mbp 517 2009-06-11 07:49 Makefile
-rw-r--r-- 1 mbp 401K 2009-06-10 15:06 quick-start-summary.pdf
-rw-r--r-- 1 mbp 234K 2009-06-10 14:29 quick-start-summary.png
-rw-r--r-- 1 mbp 82K 2009-06-10 15:06 quick-start-summary.svg

Perhaps just the svg and png would be enough? The pdf is easier for people to print, probably, but I'm not sure we need to ship all of them.

The smaller PNGs are going to be in 1.16rc1. Leaving this bug open since it involves other work.

This seems resolved in bzr 2.0.1. The doc directory contains now only 172KiB of documents.

My understanding is that this is closed. If not, please re-open it.