duplicity: OverflowError: signed integer is greater than maximum

[trejkaz@…]

Running duplicity to restore a backup, I reliably get this error at the same file every try, whether I run as a normal user or as root:

bernkastel:Recovered Morden trejkaz$ sudo duplicity restore --no-encryption scp://trejkaz@192.168.1.100//raid/backups/morden ./root

...

Error '[Errno 1] Operation not permitted: 'root/var/lib/nfs/etab'' processing var/lib/nfs/etab
Error '[Errno 1] Operation not permitted: 'root/var/lib/nfs/rmtab'' processing var/lib/nfs/rmtab
Error '[Errno 1] Operation not permitted: 'root/var/lib/nfs/rpc_pipefs'' processing .
Error '[Errno 1] Operation not permitted: 'root/var/lib/nfs/sm/.keep_net-fs_nfs-utils-0'' processing var/lib/nfs/sm/.keep_net-fs_nfs-utils-0
Traceback (most recent call last):
  File "/opt/local/bin/duplicity", line 1236, in <module>
    with_tempdir(main)
  File "/opt/local/bin/duplicity", line 1229, in with_tempdir
    fn()
  File "/opt/local/bin/duplicity", line 1183, in main
    restore(col_stats)
  File "/opt/local/bin/duplicity", line 538, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/duplicity/patchdir.py", line 520, in Write_ROPaths
    ITR(ropath.index, ropath)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/duplicity/lazy.py", line 329, in __call__
    if self.finish_branches(index) is None:
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/duplicity/lazy.py", line 276, in finish_branches
    to_be_finished.call_end_proc()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/duplicity/lazy.py", line 366, in call_end_proc
    robust.check_common_error(self.on_error, self.end_process)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/duplicity/robust.py", line 38, in check_common_error
    return function(*args)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/duplicity/patchdir.py", line 560, in end_process
    self.dir_diff_ropath.copy_attribs(self.dir_new_path)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/duplicity/path.py", line 435, in copy_attribs
    util.maybe_ignore_errors(lambda: os.chown(other.name, self.stat.st_uid, self.stat.st_gid))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/duplicity/util.py", line 62, in maybe_ignore_errors
    return fn()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/duplicity/path.py", line 435, in <lambda>
    util.maybe_ignore_errors(lambda: os.chown(other.name, self.stat.st_uid, self.stat.st_gid))
OverflowError: signed integer is greater than maximum
bernkastel:~ trejkaz$

This effectively prevents restoring backups, making duplicity a write-only backup solution. :-D

I have commented out the call to os.chown as a workaround. There are mentions around the net that there was a Python bug like this, but the version being run is newer than the version in which it was fixed.

[jmroot (Joshua Root)]

Please remember to fill in the Port field and cc the maintainer.

[jmroot (Joshua Root)]

Is this still a problem with 0.6.12?

[trejkaz@…]

nope. problem gone.

[avelo@…]

I get the same traceback with latest duplicity 0.6.18. Apparently, they occurs when I try to restore or fetch some files under /private/var directory. All other directories in HDD restore ok.

At the bottom of this message I've pasted the end of the trace.
Full trace is at: ​http://pastebin.com/YKBxHPSj
Duply exclude file is here (Just in case it could help): ​http://pastebin.com/EBfKuDsJ

Thanks in advance

Making directory /Volumes/RestoreDisk/private/var/db/launchd.db/com.apple.launchd
Writing db/launchd.db/com.apple.launchd/overrides.plist of type reg
Writing db/launchd.db/com.apple.launchd.peruser.0 of type dir
Making directory /Volumes/RestoreDisk/private/var/db/launchd.db/com.apple.launchd.peruser.0
Writing db/launchd.db/com.apple.launchd.peruser.0/overrides.plist of type reg
Writing db/launchd.db/com.apple.launchd.peruser.212 of type dir
Making directory /Volumes/RestoreDisk/private/var/db/launchd.db/com.apple.launchd.peruser.212
Writing db/launchd.db/com.apple.launchd.peruser.212/overrides.plist of type reg
Writing db/launchd.db/com.apple.launchd.peruser.26 of type dir
Making directory /Volumes/RestoreDisk/private/var/db/launchd.db/com.apple.launchd.peruser.26
Writing db/launchd.db/com.apple.launchd.peruser.26/overrides.plist of type reg
Writing db/launchd.db/com.apple.launchd.peruser.4294967294 of type dir
Making directory /Volumes/RestoreDisk/private/var/db/launchd.db/com.apple.launchd.peruser.4294967294
Writing db/launchd.db/com.apple.launchd.peruser.4294967294/overrides.plist of type reg
Removing still remembered temporary file /tmp/duplicity-slm38W-tempdir/mktemp-soPJOo-4
Removing still remembered temporary file /tmp/duplicity-slm38W-tempdir/mktemp-GTvmhs-13
Removing still remembered temporary file /tmp/duplicity-slm38W-tempdir/mkstemp-XO5dl5-1
Removing still remembered temporary file /tmp/duplicity-slm38W-tempdir/mktemp-IqefJe-3
Traceback (most recent call last):
  File "/opt/local/bin/duplicity", line 1391, in <module>
    with_tempdir(main)
  File "/opt/local/bin/duplicity", line 1384, in with_tempdir
    fn()
  File "/opt/local/bin/duplicity", line 1318, in main
    restore(col_stats)
  File "/opt/local/bin/duplicity", line 623, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/duplicity/patchdir.py", line 524, in Write_ROPaths
    ITR( ropath.index, ropath )
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/duplicity/lazy.py", line 335, in __call__
    last_branch.fast_process, args)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/duplicity/robust.py", line 37, in check_common_error
    return function(*args)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/duplicity/patchdir.py", line 577, in fast_process
    ropath.copy( self.base_path.new_index( index ) )
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/duplicity/path.py", line 443, in copy
    self.copy_attribs(other)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/duplicity/path.py", line 448, in copy_attribs
    util.maybe_ignore_errors(lambda: os.chown(other.name, self.stat.st_uid, self.stat.st_gid))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/duplicity/util.py", line 65, in maybe_ignore_errors
    return fn()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/duplicity/path.py", line 448, in <lambda>
    util.maybe_ignore_errors(lambda: os.chown(other.name, self.stat.st_uid, self.stat.st_gid))
OverflowError: signed integer is greater than maximum

[avelo@…]

Crash seems to be related to the owner of file /private/var/db/launchd.db/com.apple.launchd.peruser.4294967294/overrides.plist and its directory. The owner is nobody. Also tried restore with --numeric-owner, but it also crashed with the same error.

Original dir:

# ls -la /private/var/db/launchd.db/com.apple.launchd.peruser.4294967294
total 0
drwx------   3 nobody  wheel  102 Sep 17 12:47 .
drwxr-xr-x  12 root    wheel  408 Feb 28 19:41 ..
-rw-------   1 nobody  wheel    0 Sep 17 12:47 overrides.plist

[avelo@…]

Cc Me!

[jmroot (Joshua Root)]

Is there an upstream bug report?

[ryandesign (Ryan Carsten Schmidt)]

Is this still a problem with 0.6.21?

[neverpanic (Clemens Lang)]

Is this still a problem with 0.7.01 from r133434? If it is, did you file a ticket upstream?

[trejkaz@…]

No idea. I ended up recovering to a Linux system and copying the files over. Once I got it to work, I no longer had a burning need to get the tool working...

[neverpanic (Clemens Lang)]

Let's just close this for now then, to keep our records clean. If anybody still encounters this, he/she can re-open.