Comment 5 for bug 407762

Revision history for this message
Shai Tahar (tashai) wrote : Re: ubuntuone-syncdaemon maxes out system resources

Hi Joshua,
i've attached top and part of the log,
i've found the problem:
i've copied one of the files using root, the syncd fails to hash the file sync (permissions)
but it tries over and over...

top - 10:22:23 up 1 day, 15:51, 3 users, load average: 0.60, 0.26, 0.12
Tasks: 175 total, 8 running, 167 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.6%us, 0.9%sy, 0.1%ni, 94.3%id, 0.7%wa, 0.3%hi, 0.0%si, 0.0%st
Mem: 2036092k total, 1402776k used, 633316k free, 21132k buffers
Swap: 4000176k total, 0k used, 4000176k free, 772484k cached

  PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10673 shait 20 0 114m 27m 4564 R 97 1.4 0:57.32 ubuntuone-syncd
 1712 root 20 0 262m 121m 32m S 2 6.1 8:29.13 Xorg
10503 shait 20 0 213m 16m 10m S 2 0.8 0:02.97 gnome-terminal
    1 root 20 0 19444 1784 1188 S 0 0.1 0:00.97 init

2009-11-08 10:21:27,188 - ubuntuone.SyncDaemon.sync - DEBUG - T:NONE:F b72e6d20-d399-419e-ab31-bff18bbded0d [root:b72e6d20-d399-419e-ab31-bff18bbded0d] ''Ubuntu One/file'' | Called new_local_file
2009-11-08 10:21:27,188 - ubuntuone.SyncDaemon.State - DEBUG - READING --[SYS_META_QUEUE_WAITING]--> READING_WITH_METAQ
2009-11-08 10:21:27,188 - ubuntuone.SyncDaemon.EQ - DEBUG - push_event: SYS_STATE_CHANGED, args:(), kw:{'state': <SyncDaemonState READING_WITH_METAQ>}
2009-11-08 10:21:27,188 - ubuntuone.SyncDaemon.EQ - DEBUG - push_event: FS_FILE_CLOSE_WRITE, args:('/home/shait/Ubuntu One/file',), kw:{}
2009-11-08 10:21:27,189 - ubuntuone.SyncDaemon.sync - DEBUG - T:NONE:F b72e6d20-d399-419e-ab31-bff18bbded0d [root:b72e6d20-d399-419e-ab31-bff18bbded0d] ''Ubuntu One/file'' | EVENT: FS_FILE_CLOSE_WRITE:{} with ARGS:()
2009-11-08 10:21:27,189 - ubuntuone.SyncDaemon.sync - DEBUG - T:NONE:F b72e6d20-d399-419e-ab31-bff18bbded0d [root:b72e6d20-d399-419e-ab31-bff18bbded0d] ''Ubuntu One/file'' | Calling calculate_hash
2009-11-08 10:21:27,189 - ubuntuone.SyncDaemon.HQ - DEBUG - HashQueue: inserting path '/home/shait/Ubuntu One/file' mdid b72e6d20-d399-419e-ab31-bff18bbded0d
2009-11-08 10:21:27,190 - ubuntuone.SyncDaemon.sync - DEBUG - T:NONE:F b72e6d20-d399-419e-ab31-bff18bbded0d [root:b72e6d20-d399-419e-ab31-bff18bbded0d] ''Ubuntu One/file'' | Called calculate_hash
2009-11-08 10:21:27,190 - ubuntuone.SyncDaemon.HQ.hasher - DEBUG - Hasher: got file to hash: path '/home/shait/Ubuntu One/file' mdid b72e6d20-d399-419e-ab31-bff18bbded0d
2009-11-08 10:21:27,190 - ubuntuone.SyncDaemon.local_rescan - DEBUG - process next in queue (len 11)
2009-11-08 10:21:27,190 - ubuntuone.SyncDaemon.HQ.hasher - DEBUG - Hasher: hash error [Errno 13] Permission denied: '/home/shait/Ubuntu One/file' (path '/home/shait/Ubuntu One/file' mdid b72e6d20-d399-419e-ab31-bff18bbded0d)
2009-11-08 10:21:27,190 - ubuntuone.SyncDaemon.EQ - DEBUG - push_event: SYS_NET_CONNECTED, args:(), kw:{}
2009-11-08 10:21:27,191 - ubuntuone.SyncDaemon.State - DEBUG - READING_WITH_METAQ --[SYS_NET_CONNECTED]--> READING_WITH_NETWORK_WITH_METAQ
2009-11-08 10:21:27,191 - ubuntuone.SyncDaemon.EQ - DEBUG - push_event: SYS_STATE_CHANGED, args:(), kw:{'state': <SyncDaemonState READING_WITH_NETWORK_WITH_METAQ>}
2009-11-08 10:21:27,191 - ubuntuone.SyncDaemon.EQ - DEBUG - push_event: HQ_HASH_ERROR, args:('b72e6d20-d399-419e-ab31-bff18bbded0d',), kw:{}
2009-11-08 10:21:27,192 - ubuntuone.SyncDaemon.sync - DEBUG - T:NONE:F b72e6d20-d399-419e-ab31-bff18bbded0d [root:b72e6d20-d399-419e-ab31-bff18bbded0d] ''Ubuntu One/file'' | EVENT: HQ_HASH_ERROR:{} with ARGS:()
2009-11-08 10:21:27,192 - ubuntuone.SyncDaemon.sync - DEBUG - T:NONE:F b72e6d20-d399-419e-ab31-bff18bbded0d [root:b72e6d20-d399-419e-ab31-bff18bbded0d] ''Ubuntu One/file'' | Calling calculate_hash
2009-11-08 10:21:27,192 - ubuntuone.SyncDaemon.HQ - DEBUG - HashQueue: inserting path '/home/shait/Ubuntu One/file' mdid b72e6d20-d399-419e-ab31-bff18bbded0d
2009-11-08 10:21:27,192 - ubuntuone.SyncDaemon.sync - DEBUG - T:NONE:F b72e6d20-d399-419e-ab31-bff18bbded0d [root:b72e6d20-d399-419e-ab31-bff18bbded0d] ''Ubuntu One/file'' | Called calculate_hash
2009-11-08 10:21:27,192 - ubuntuone.SyncDaemon.HQ.hasher - DEBUG - Hasher: got file to hash: path '/home/shait/Ubuntu One/file' mdid b72e6d20-d399-419e-ab31-bff18bbded0d
2009-11-08 10:21:27,193 - ubuntuone.SyncDaemon.local_rescan - DEBUG - _scan_tree: share_path: '/home/shait/Ubuntu One' path: '/home/shait/Ubuntu One/Backup/Ofir'
2009-11-08 10:21:27,193 - ubuntuone.SyncDaemon.HQ.hasher - DEBUG - Hasher: hash error [Errno 13] Permission denied: '/home/shait/Ubuntu One/file' (path '/home/shait/Ubuntu One/file' mdid b72e6d20-d399-419e-ab31-bff18bbded0d)
2009-11-08 10:21:27,193 - ubuntuone.SyncDaemon.local_rescan - DEBUG - Adding watch to '/home/shait/Ubuntu One/Backup/Ofir'
2009-11-08 10:21:27,193 - ubuntuone.SyncDaemon.EQ - DEBUG - Adding inotify watch to '/home/shait/Ubuntu One/Backup/Ofir'
2009-11-08 10:21:27,194 - ubuntuone.SyncDaemon.INotifyProcessor - DEBUG - Freeze begin: '/home/shait/Ubuntu One/Backup/Ofir'
2009-11-08 10:21:27,194 - ubuntuone.SyncDaemon.local_rescan - DEBUG - scanning the dir '/home/shait/Ubuntu One/Backup/Ofir'
2009-11-08 10:21:27,194 - ubuntuone.SyncDaemon.local_rescan - DEBUG - comparing directory '/home/shait/Ubuntu One/Backup/Ofir'
2009-11-08 10:21:27,194 - ubuntuone.SyncDaemon.local_rescan - DEBUG - comp yield: file '/home/shait/Ubuntu One/Backup/Ofir/Readme.dongle.important.txt' was here.. stat?
2009-11-08 10:21:27,194 - ubuntuone.SyncDaemon.local_rescan - DEBUG - comp yield STAT prv: posix.stat_result(st_mode=33279, st_ino=76715, st_dev=2051L, st_nlink=1, st_uid=1000, st_gid=1000, st_size=9699, st_atime=1257066765, st_mtime=1256974239, st_ctime=1256974239

Shai