bzr branch in knit format fails with "File exists" error on Windows

I'm not sure how to reproduce this problem. But it seems there is something related to case-insensitiveness of windows file system. My best guess that for some reason there was directory named 'C1', and bzr fails when it try to create 'c1'. But I don't understand how it might happens. Bzr always creates prefixes in lowercase. It smells like something weird happens with filesystem. I'd rather change status to 'Incomplete'.

a subversion dump posted in the dupe report reproduces this problem 100% of the time for me.

http://buildbot.no-ip.org/~tvo/spring/svndump/spring-r4046.svndmp.bz2 - osrts.info has expired, use this link instead.

Am Mittwoch, den 03.10.2007, 16:49 +0000 schrieb Baczek:
> a subversion dump posted in the dupe report reproduces this problem 100%
> of the time for me.
>
> http://buildbot.no-ip.org/~tvo/spring/svndump/spring-r4046.svndmp.bz2 -
> osrts.info has expired, use this link instead.
On Windows?

Cheers,

Jelmer

--
Jelmer Vernooij <email address hidden> - http://samba.org/~jelmer/
Jabber: <email address hidden>

yes.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jelmer Vernooij пишет:
> Am Mittwoch, den 03.10.2007, 16:49 +0000 schrieb Baczek:
>> a subversion dump posted in the dupe report reproduces this problem 100%
>> of the time for me.
>>
>> http://buildbot.no-ip.org/~tvo/spring/svndump/spring-r4046.svndmp.bz2 -
>> osrts.info has expired, use this link instead.
> On Windows?

Jelmer, IMO it seems to be bug in bzr, but it cannot be reproduced without bzr-svn.
So, I think it's better to assign it to bzr-svn.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHBGrBzYr338mxwCURAiZdAJ9ARckWR5IWZQkyRN1Ra4FZ5duWFACeKiVu
5Uw+Bb9Ztg0YTC4budF/O5k=
=JaXx
-----END PGP SIGNATURE-----

This appears to be very specific to Knits and Windows though. I don't see how this could be a bzr-svn issue.

perhaps the file name specified to _put_non_atomic_helper() is invalid on Windows?

If there is a way to reproduce it without bzr-svn, then I'm happy to say it is a pure bzr bug.
Otherwise it seems that bzr-svn might be influencing something.
(I know we've seen some odd interactions with Alexander's symlink plugin, where you wouldn't really expect to run into anything.)

My bet is that bzr-svn is specifying some file id that happens to be translated to an invalid filename on windows. That isn't really reflected by the exception bzr raises, though.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jelmer Vernooij пишет:
> My bet is that bzr-svn is specifying some file id that happens to be
> translated to an invalid filename on windows. That isn't really
> reflected by the exception bzr raises, though.

Maybe with colon in file name? ':' is invalid character on Windows.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHGPibzYr338mxwCURAsllAJ9xX5dlJVOFnthUVZohgBKyv+H4twCfQj6p
/ZJ4uFgkI5gYZl4GAPHgBtw=
=W2bY
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Alexander Belchenko wrote:
> Jelmer Vernooij пишет:
>> My bet is that bzr-svn is specifying some file id that happens to be
>> translated to an invalid filename on windows. That isn't really
>> reflected by the exception bzr raises, though.
>
> Maybe with colon in file name? ':' is invalid character on Windows.

Most invalid characters should be sanitized by the Store. It has a built-in
ability to escape file-id characters. (For things like :).

bzrlib/store/__init__.py:
    def _escape_file_id(self, file_id):
        """Turn a file id into a filesystem safe string.

        This is similar to a plain urllib.quote, except
        it uses specific safe characters, so that it doesn't
        have to translate a lot of valid file ids.
        """
        if not self._escaped:
            return file_id
        if isinstance(file_id, unicode):
            file_id = file_id.encode('utf-8')
        # @ does not get escaped. This is because it is a valid
        # filesystem character we use all the time, and it looks
        # a lot better than seeing %40 all the time.
        safe = "abcdefghijklmnopqrstuvwxyz0123456789-_@,."
        r = [((c in safe) and c or ('%%%02x' % ord(c)))
             for c in file_id]
        return ''.join(r)

So any character that isn't a-z0-9-_@,. should be replaced with %XX.

The best I can come up with is actually having the path be too long. We've run
into that in the past with names like:
REALLY_LONG_FILENAME_WITH_ALL_CAPITAL_LETTERS

since then every character is expanded to 3 characters.

It could also be possible that there is a readonly issue. (Hence we cannot
mkdir the 'c1' directory.)

Or maybe the exception we are getting isn't one we should be handling. (Though
if that were the case, I would expect it to fail in the general sense, not just
in one specific edge case.)

The best way to test, though, is to just do the conversion of the "always
failing" repository on Linux, and then try to pull that onto a Windows machine.
If the 'bzr branch' fails, then it is definitely a bzr bug.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHGP8yJdeBCYSNAAMRAoDUAKC4398uYcvoG1nHiDzNWyb7t/1+hwCgpZRO
tuMu+ypNs8dkrHkmrcZDDJ8=
=zSOF
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John A Meinel пишет:
>
> The best I can come up with is actually having the path be too long. We've run
> into that in the past with names like:
> REALLY_LONG_FILENAME_WITH_ALL_CAPITAL_LETTERS

Yes, sometimes I forget about this. May be this is the case.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHGQRKzYr338mxwCURAraHAJ4+DD98qD6XYszX9V0GdUvmQmerRwCbB9BZ
W88j507IwOfwLz4S5ZKnIGw=
=M+Fn
-----END PGP SIGNATURE-----

On Fri, 2007-10-19 at 19:02 +0000, John A Meinel wrote:

> So any character that isn't a-z0-9-_@,. should be replaced with %XX.
>
> The best I can come up with is actually having the path be too long. We've run
> into that in the past with names like:
> REALLY_LONG_FILENAME_WITH_ALL_CAPITAL_LETTERS
>
> since then every character is expanded to 3 characters.

Packs will be landing in a day or so - Monday probably. They may well
work better.

-Rob

--
GPG key available at: <http://www.robertcollins.net/keys.txt>.

Packs have landed. Can you please try reproducing this bug with latest bzr.dev, bzr-svn 0.4 and packs?

The real problem in original traceback:
Some code try to create directory `c1` when this directory already exists. On Windows it's error. May be on Linux it's not error, I don't know. Why some code try to create `c1` again -- I don't know. May be it's incorrect usage of bzrlib API from svn plugin, or may be bzrlib API is not completely windows-friendly in this aspect.

On Fri, 2007-11-09 at 08:08 +0000, Alexander Belchenko wrote:
> The real problem in original traceback:
> Some code try to create directory `c1` when this directory already
> exists. On Windows it's error. May be on Linux it's not error, I don't
> know. Why some code try to create `c1` again -- I don't know. May be
> it's incorrect usage of bzrlib API from svn plugin, or may be bzrlib
> API is not completely windows-friendly in this aspect.

It's definately an error on windows too; I think the error code raised
changed in a python version bump on Windows?. Anyhow, packs don't suffer
this...

-Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins пишет:
> On Fri, 2007-11-09 at 08:08 +0000, Alexander Belchenko wrote:
>> The real problem in original traceback:
>> Some code try to create directory `c1` when this directory already
>> exists. On Windows it's error. May be on Linux it's not error, I don't
>> know. Why some code try to create `c1` again -- I don't know. May be
>> it's incorrect usage of bzrlib API from svn plugin, or may be bzrlib
>> API is not completely windows-friendly in this aspect.
>
> It's definately an error on windows too;

s/windows/linux/ ?

> I think the error code raised
> changed in a python version bump on Windows?.

Yes, it's True. Python 2.5 uses windows native error codes instead of unix-like ones where possible.
But as you could see _translate_error() function correctly translate this error to FileExists.

Excerpt from MSDN, page with system error codes:

183 | Cannot create a file when that file already exists. | ERROR_ALREADY_EXISTS

> Anyhow, packs don't suffer
> this...

Then we could close this bug when packs will become standard format?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHNKvbzYr338mxwCURAjg+AJ9+kHQ38O/MiFxDU05kCzmB4dZBHACeOLau
s8y6c+cWgiqzRsPRMpJlZn0=
=w52y
-----END PGP SIGNATURE-----

This only affects a small number of users, and there is now a workaround - use a packs repository rather than a knits repository.

Marking as invalid for bzr-svn as the original poster reproduced it without bzr-svn and there is no indication bzr-svn is using the API incorrectly.

I can't reproduce it.

I have been hit by this bug three times over the last month on Windows 7 64 bit with bzr 2.1.1 and python 2.6.5.

The first two times were with a shared repository version 1.9, tomorrows problem was with version 2a.

I have an automated build system running, which may have been killed during a bzr operation.

I don't think this was the case with the others, but with this one I hit a lock on a branch that I had to remove with bzr break-lock, and after that, I get:

C:\Users\bo\mp\mariadb>bzr branch lp:~maria-captains/maria/5.1-release 5.1-release
bzr: ERROR (ignored): 'file:///C:/Users/bo/mp/mariadb/.bzr/repository/upload/ifyfp443v9nszroaal9x.pack'
bzr: ERROR: File exists: u'C:/Users/bo/mp/mariadb/.bzr/repository/upload/ifyfp443v9nszroaal9x.pack': [Error 183] En fil,
 som allerede findes, kan ikke oprettes

Sorry about the Danish translations. The error 183 says "A file, that already exists, can't be created".

Maybe it's possible to reproduce the bug by checking out the repository multiple times and kill it each time - not ctrl-C it, but kill the process. And it might have to be on Windows.

Deleting the file in .bzr/repository/upload doesn't help. It complains about a new file on each run.

Bo: your problem is actually a different bug. This bug is specifically about a problem with the (now very old) "knit" format. I'm sorry this bug page was unclear about that, I'll update the summary. I've filed <https://bugs.launchpad.net/bzr/+bug/587795> for your problem, please continue discussion of your bug there.

As far as the issue with knits is concerned, I'm just going to mark it as Won't Fix. It seems highly unlikely that anyone cares enough about the knit format to try fix this, given that newer and better formats without this bug have been the default for a long time now.