Duplicity Changelog

What's new in Duplicity 0.7.07.1

May 7, 2016
  • Fixed bug #1568677 duplicity fails to use existing S3 bucket in boto backend:
  • bug introduced by incomplete fix of bug #1296793
  • simplified setting of bucket locations
  • Fixed bug #1569523 get_bucket unknown keyword location and my_location name error:
  • bug introduced in improper fix of bug #1568677
  • gotta love those inconsistent APIs
  • Fixed bug #1571134 incompatible with python-oauth2client version 2.x and #1558155 PyDrive backend broken, needs update to oauth2client library:
  • used patch from https://bugs.debian.org/820725 but made changes to allow the user to continue using the old version
  • Fixed bug #1570293 duplicity is very slow due to excessive fsync:
  • removed flush() after write.
  • revert to previous version

New in Duplicity 0.7.06 (Mar 3, 2016)

  • Merged in lp:~mnjul/duplicity/s3-infreq-access:
  • This adds support for AWS S3's newly announced Infrequent Access storage class and is intended to implement Blueprint: https://blueprints.launchpad.net/duplicity/+spec/aws-s3-std-ia-class .:
  • A new command line option, --s3-use-ia, is added, and boto backend will automatically use the correct storage class value depending on whether --s3-use-rrs and --s3-use-ia is set. Command line parser will prompt error if both --s3-use-ia and --s3-use-rrs are used together, as they conflict with each other.:
  • The manpage has been updated giving a short explanation on the new option. Its wording derives from Amazon's official announcement: https://aws.amazon.com/about-aws/whats-new/2015/09/announcing-new-\ amazon-s3-storage-class-and-lower-glacier-prices/
  • The ptyprocess module no longer supports Python 2.6, so fix tox.ini to use an older version. Make explicit environs for all tests.
  • Upgrade to newest version of pep8 and pylint. Add three ignores to test_pep8 and one to test_pylint to get the rest to pass. They
  • Applied patch from Alexander Zangerl to update to changes in lockfile API 0.9 and later. Updated README to notify users.
  • Modded tox.ini to use the latest lockfile.
  • Merged in lp:~ed.so/duplicity/setup.shebang:
  • Having the python interpreter searched in the PATH is much more flexible than the /usr/bin/python inserted into our scripts shebang by setuptools. This patch prevents that. don't touch my shebang!
  • Cleanup issues around Launchpad build, mainly lockfile >= 0.9.
  • Merged in lp:~michal-s/duplicity/duplicity:
  • WindowsAzureMissingResourceError and WindowsAzureConflictError changed due to SDK changes. are all valid in our case.
  • Reversed previous changes to lockfile. Now it will take any version extant in the LP build repository. (PyPi is not avail in LP build).
  • Merged in lp:~ed.so/duplicity/tempfile.tempdir:
  • make sure packages using python's tempfile create temp files in duplicity's temp dir
  • Fixed bug #1511308 - Cannot restore no-encryption, no-compression backup:
  • Corrected code to include plain file in write_multivolume():
  • Added PlainWriteFile() to gpg.py
  • Merged in lp:~michal-s/duplicity/duplicity:
  • Fix azurebackend storage class import
  • Merged in lp:~feraudet/duplicity/fix:
  • Fix missing SWIFT_ENDPOINT_TYPE env var, bug 1519694.
  • Fix bug #1520691 - Shell Code Injection in hsi backend:
  • Replace use of os.popen3() with subprocess equivalent.:
  • Added code to expand relative program path to full path.:
  • Fix hisbackend where it expected a list not a string.
  • Merged in lp:~noizyland/duplicity/azurebackend-fixes:
  • Support new version of Azure Storage SDK:
  • Refactor _list method to support containers with >5000 blobs
  • Merged in lp:~matthew-t-bentley/duplicity/b2:
  • Adds a backed for BackBlaze's (currently beta) B2 backup service.:
  • This adds backends/b2backend.py, modifies log.py to add an error code and modifies commandline.py to add the b2:// example to the help text.

New in Duplicity 0.7.02 (Mar 14, 2015)

  • Fixed some tabs/spaces problems that were causing install failures.
  • Fixed variable typo in commandline.py that was causing build fails.
  • Merged in lp:~duplicity-team/duplicity/po-updates
  • Remove 'gs' and 's3+http' from uses_netloc[]. Fixes Bug 1411803.
  • Changed --s3_multipart_max_timeout to --s3-multipart-max-timeout to be consistent with commandline option naming conventions.
  • Applied patch from Adam Reichold to fix bug # 1413792.
  • Fixes for 2to3 issues
  • Fix spelling error in manpage, bug 1419314.
  • Fix _librsyncmodule.c compilation, bug 1416344, thanks to Kari Hautio.
  • Really fix bug 1416344 based on comment #5 by Roman Tereshonkov.
  • Fix for --pydevd debug environment and location under Eclipse.
  • Fix for bug where scp was actually working as scp and not working with rsync.net because of using extraneous test command in restricted shell. Was trying "test -d 'foo' || mkdir -p 'foo'", now only "mkdir -p foo".
  • remove extraneous string format arg in previous scp fix.
  • Fixed bug # 1414418:
  • Aligned commandline.py options and help display contents.
  • Aligned commandline.py options and manpage contents.
  • Merged in lp:~vincegt/duplicity/swift_regionname:
  • Fixes bug #1376628
  • Add mapping of SWIFT_REGIONNAME to select region inside SWIFT when a provider proposes more than one region.
  • Merged in lp:~aaron-whitehouse/duplicity/progress_option_error:
  • Added test_exclude_globbing_filelist_progress_option into functional/test_selection.py, which shows the error reported in Bug #1264744 - that the --exclude-globbing-filelist does not backup the correct files if the --progress option is used. Test is marked as an expected failure so as not to cause the test suite to fail.
  • Merged in lp:~noizyland/duplicity/fix_azurebackend_container_names:
  • Azure Backend examples have underscores in the container names. These are not valid Azure container names. The underscores have been replaced with hypens and a note about valid container names added to the man page.
  • Also corrects a problem where Azure Exceptions were returing unicode strings that were not being handled correctly.
  • Merged in lp:~user3942934/duplicity/pydrive:
  • Currently duplicity uses gdocs backend for Google Drive backups. gdocs uses deprecated API and don't allow backups for managed Google accounts. (see https://bugs.launchpad.net/duplicity/+bug/1315684)
  • Added pydrive backend that solves both of those problems. Published also on https://github.com/westerngateguard/duplicity-pydrive-backend.
  • Merged in lp:~angusgr/duplicity/exclude-older-than:
  • Add "--exclude-older-than" commandline option, that allows you to only back up files with a modification date newer than a particular threshold.
  • Merged in lp:~aaron-whitehouse/duplicity/bug_884371_asterisks_in_includes:
  • Added tests to unit/test_selection.py and funtional/test_selection.py to show the behaviour reported in Bug #884371, i.e. that selection is incorrect when there is a or *on an include line of a filelist or commandline --include.
  • Merged in lp:~aaron-whitehouse/duplicity/bug_932482_trailing_slashes_and_wildcards_error:
  • Added functional and unit tests to show Bug #932482 - that selection does not work correctly when excludes (in a filelist or in a commandline option) contain both a single or double asterisk and a trailing slash.
  • Misc fixes for the following PEP8 issues:
  • E111, E121, E122, E124, E125, E126, E127, E128, E201, E202, E203, E231, E241, E251, E261, E262, E271, E272, E301, E302, E303, E401, E502, E701, E702, E703, E711, E721, W291, W292, W293, W391
  • to run pep8 on duplicity use 'pep8 --ignore=E501'
  • see http://pep8.readthedocs.org

New in Duplicity 0.6.24 (May 10, 2014)

  • ENHANCEMENTS:
  • Applied two patches from mailing list message at https://lists.nongnu.org/archive/html/duplicity-talk/2014-01/msg00030.html
  • "Added command line options to use different prefixes for duplicity files"
  • This resolves https://bugs.launchpad.net/duplicity/+bug/1170161 and provides a workaround for https://bugs.launchpad.net/duplicity/+bug/1170113
  • Merged in lp:~mterry/duplicity/gpg-encode:
  • getpass.getpass(prompt) eventually calls str(prompt). Which is a no go, if the prompt contains unicode. Here's a patch to always pass getpass() a byte string.
  • Our tests didn't catch this because they always set PASSPHRASE. I've added a test that passes the passphrase via stdin.
  • Merged in lp:~mterry/duplicity/pexpect-fix:
  • duplicity has its own copy of pexpect. Use that instead of requiring one from the system.
  • Merged in lp:~prateek/duplicity/s3-glacier:
  • Fixes https://bugs.launchpad.net/duplicity/+bug/1039511 - Adds support to detect when a file is on Glacier and initiates a restore to S3. Also merges overlapping code in the boto backends
  • Fixes https://bugs.launchpad.net/duplicity/+bug/1243246 - Adds a --s3_multipart_max_timeout input option to limit the max execution time of a chunked upload to S3. Also adds debug message to calculate upload speed.
  • Merged in lp:~ed.so/duplicity/fix.dpbx:
  • Fix dpbx backend "NameError: global name 'rest' is not defined"
  • Merged in lp:~prateek/duplicity/botoimportfix:
  • Switches the boto backend back to using lazy imports so there are no complaints during the importing of backends.
  • Merged in lp:~germer/duplicity/par2:
  • This branch adds Par2 recovery files to duplicity. It is a wrapper backend which will create the recovery files and upload them all together with the wrapped backend. Corrupt archives will be detected and repaired (if possible) on the fly during restore.
  • It can be used with url-string par2+webdavs://USER@HOST/PATH
  • Fixes https://bugs.launchpad.net/duplicity/+bug/426282
  • Merged in lp:~fredrik-loch/duplicity/duplicity-S3-SSE:
  • Adds support for server side encryption as requested in Bug #996660
  • Merged in lp:~mterry/duplicity/modern-testing:
  • Enable/use more modern testing tools like nosetests and tox as well as more common setup.py hooks like test and sdist.
  • Specifically:
  • move setup.py to toplevel where most tools and users expect it
  • Move and adjust test files to work when running "nosetests" in toplevel directory. Specifically, do a lot more of the test setup in tests/__init__.py rather than the run-tests scripts
  • Add small tox.ini file for using tox, which is a wrapper for using virtualenv. Only enable for py26 and py27 right now, since modern setuptools dropped support for

New in Duplicity 0.6.23 (Jan 25, 2014)

  • Enhancements:
  • Applied patch from bug 1216921 to fix ignore_missing():
  • merged lp:~mterry/duplicity/ignore-missing to fix patch.
  • Merged in lp:~mterry/duplicity/catch-seq-copy-error:
  • Any exception when running patch_seq2ropath should be ignored (though logged) and duplicity should move on. This covers the two asserts in that function (bug 1155345 and bug 720525) as well as errors that happen during file copying (bug 662442).
  • Merged in lp:~mterry/duplicity/argv:
  • Fix use of argv when calling os.execve
  • Merged in lp:~verb/duplicity/bucket_root_fix:
  • Fix bug that prevents backing up to the root of a bucket with boto backend.
  • Merged in lp:~gliptak/duplicity/415619:
  • Better error message when chown fails
  • Merged in lp:~mterry/duplicity/log-path-type:
  • Any backup browser built on top of duplicity will need to indicate which files in the backup are folders and which are files. The current logging information doesn't provide this detail. So I've added a field to the log.InfoCode.file_list output that includes the path type.
  • Merged in lp:~mterry/duplicity/manifest-oddities:
  • We may accidentally end up with an oddly inconsistent manifest like so: Volume 1 Volume 2 Volume 3 Volume 2 As did get reported recently on the mailing list: http://lists.nongnu.org/archive/html/duplicity-talk/2013-11/msg00009.html
  • One way this can happen (the only way?) is if you back up, then duplicity gets interrupted between writing the manifest and uploading the volume. Then, when restarted, there is no longer enough data to create as many volumes as existed previously.
  • This situation can cause an exception when trying to restart the backup.
  • This branch fixes it by deleting any excess volume information encountered when loading in the manifest. We discard volume with higher numbers than the last one read.
  • Merged in lp:~mterry/duplicity/disappearing-source:
  • When restarting a backup, we may accidentally skip the first chunk of one of the source files. To reproduce this,: 1) interrupt a backup 2) delete the source file it was in the middle of 3) restart the backup
  • When replaying the source iterator to find where to resume from, we can't notice that the file is gone until we've already iterated past where it would be!
  • The solution I came up with is to just let duplicity stuff the data we accidentally read back into the source iterator.
  • This is actually a data loss bug, because it's possible to back up corrupted files (that are missing their first chunk).
  • Merged in lp:~mterry/duplicity/normalize-before-using:
  • Avoid throwing an exception due to a None element in a patch sequence.
  • None elements in a (non-normalized) patch sequence are perfectly normal. With the current code in the patched function, it is certainly possible to hit a crash due a None. See http://lists.nongnu.org/archive/html/duplicity-talk/2013-11/msg00005.html
  • This branch fixes that by normalizing the sequence before using it in the logging code. It's acceptable to bring the normalize_ps() call outside the try/except block because normalize_ps is not expected to throw. It's relatively simple and doesn't really use its objects besides checking if they are None.
  • Applied patch to fix "Access GDrive through gdocs backend failing":
  • see https://lists.nongnu.org/archive/html/duplicity-talk/2013-07/msg00007.html
  • Merged in lp:~jkrauss/duplicity/pyrax:
  • Rackspace has deprecated python-cloudfiles in favor of their pyrax library, which consolidates all Rackspace Cloud API functionality into a single library. Tested it with Duplicity 0.6.21 on both Arch Linux and FreeBSD 8.3.0.
  • Changed to default to pyrax backend rather than cloudfiles backend:
  • To revert to the cloudfiles backend use '--cf-backend=cloudfiles'
  • Merged in lp:~verb/duplicity/boto-min-version:
  • Update documentation and error messages to match the current actual version requirements of boto backend.
  • Merged in lp:~ed.so/duplicity/debian.paramiko.log:
  • upstream debian patch "paramiko logging" http://patch-tracker.debian.org/package/duplicity/0.6.22-2
  • Merged in lp:~ed.so/duplicity/debian.dav.mkdir:
  • upstream debian patch "webdav create folder recursively" http://patch-tracker.debian.org/package/duplicity/0.6.22-2
  • Merged in lp:~mterry/duplicity/encoding:
  • This branch hopefully fixes two filename encoding issues:
  • Users in bug 989496 were noticing a UnicodeEncodeError exception which happens (as far as I can tell) because some backends (like webdav) are returning unicode filenames from list(). When these filenames are combined with the utf8 translations of log messages, either (A) the default ascii encoding can't handle promoting the utf8 bytes or -- if there aren't any utf8 bytes in the translation -- (B) the resulting unicode string raises an error later when log.py tries to upgrade the string again to unicode for printing.
  • This fix is largely implemented by adding a wrapper for backend list() implementations. This wrapper ensures that duplicity internals always see a byte string. (I'd like to eventually use this same wrapping strategy to implement generic retry support without backends having to add any logic, but that's just a thought for the future.)
  • That is, the fix for issue #1 is completely inside backend.py and the changes to backends/*.py.
  • The rest of the invasive changes deal with filenames that may not be valid utf8. This is much rarer, but possible. For proper handling of this, we need to print using unicode, and convert filenames from the system filename encoding to unicode, gracefully handling conversion errors. Some of the filenames we print are remote names. Who knows what encoding they are in; it could be different than the system filename encoding. 99% of the time, everything will be utf8 and we're fine. If we do get conversion errors, the only effect should be some question mark characters in duplicity logging output.
  • I tried to convert as much of the actual codebase to use unicode for printing. But I stopped short of adding an assert in log.py to enforce unicode, because I didn't want to go through all the backend code and manually adjust those bits without being able to test each one.
  • Merged in lp:~louis-bouchard/duplicity/add-allow-concurrency:
  • Implement locking mechanism to avoid concurrent execution under the same cache directory. This is the default behavior.
  • Also implement --alllow-concurrency option to disable the locking if required.
  • This functionality adds a dependency to python-lockfile
  • Restored missing line from patch of gdocsbackend.py
  • Nuke tabs
  • Reverted changes to gdocsbackend.py
  • Restored patch of gdocsbackend.py from original author (thanks ede)
  • Applied patch from bug 1266753: Boto backend removes local cache if connection cannot be made

New in Duplicity 0.6.22 (Sep 2, 2013)

  • ENHANCEMENTS:
  • Applied patches from Laszlo Ersek to rdiffdir to "consume a chain of sigtar files in rdiffdir delta mode" which supports incremental sigtar files.
  • Applied patch from Eric S Raymond to man page to fix markup problems.
  • Merged in lp:~jnoster/duplicity/dpbx-added:
  • Add Dropbox backend
  • NB! In order to use the backend one must: 1. Install Dropbox Python SDK first. 2. Run the duplicity with Dropbox backend (dpbx://) first time interactively to catch and follow the oAuth URL.
  • Merged in lp:~ed.so/duplicity/verify.data:
  • add switch --compare-data, to selectively enable formerly always disabled data comparison on verify runs
  • Merged in lp:~tblue/duplicity/paramiko-1.10.0:
  • This fixes bug #1156746, making the Paramiko backend compatible with Paramiko 1.10.0. It keeps compatibility with older Paramiko versions.
  • Merged in lp:~townsend/duplicity/fix-1161599-2:
  • The fix in revno. 912 didn't take into account that the parameter "body" passed into request() is overloaded, so when it was NULL or of a type other than file, it would fail. This checks if "body" is of type "file" before actually seek()'ing back to the beginning of the file.
  • Merged in lp:~tblue/duplicity/paramiko-fix-delete-retry:
  • This fixes bug #1115715, which is really annoying. Basically it makes using the Paramiko backend with the default settings impossible.
  • Merged in lp:~juan-f/duplicity/progress:
  • From time ago, there are people asking for a progress bar estimation in duplicity. There is even a script that circumvents the issue, getting info from the log so as to estimate the progress status ( https://github.com/quentin/Duplicity-progress ) but does not give enough feedback and the estimation is rather plain.
  • I have developed a set of heuristics that gather information from the deltas and the transfer ratios of the backend so as to forecast % of progress, estimation of remaining time and average speed, for both full and incremental backup uploads.
  • The current implementation works for boto backend, but to port the other backends to use this feature would be quite easy (we can discuss the details if interested).
  • The algorithm is activated by the --progress command line flag, and will perform a first-pass dry-run to collect evidence for all the deltas. Next it will trigger the real upload, while a thread statistically estimates the ratio of changes and compression for the data in/out, and uses these ratios to forecast time remaining and % of completion.
  • The progress data will be logged each 3 seconds, or the --progress-rate flag.
  • Merged in lp:~jnoster/duplicity/dpbx-added:
  • The application key was approved as "production" one after some changes to the code to suit the requirements of Dropbox team (the keys are now obfuscated, for instance).
  • Applied blocksize.patch from https://bugs.launchpad.net/duplicity/+bug/897423:
  • New option --max-blocksize (default 2048) to allow increasing delta blocksize.
  • Applied duplicity-ftps.patch from https://bugs.launchpad.net/duplicity/+bug/1104069:
  • Don't try to delete an empty file list.
  • Merged in lp:~scowcron/duplicity/ftp_password_pexpect:
  • Use common backend.Backend get_password() rather than _ssh_pexpect.py specific code.
  • Merged in lp:~mhu-s/duplicity/swiftbackend:
  • This branch adds support for Swift, the OpenStack Object Storage service. See https://blueprints.launchpad.net/duplicity/+spec/swiftbackend
  • Merged in lp:~verb/duplicity/boto-gcs:
  • These patches add support for Google Cloud Storage via the boto backend.
  • boto has supported GCS in interoperability mode for a few years now. This change adds support by taking advantage of boto's storage_uri abstraction layer.
  • Merged in lp:~ckornacker/duplicity/megacloud:
  • Add support for Mega (mega.co.nz) backend.
  • Merged in lp:~ed.so/duplicity/man.page:
  • update paramiko links
  • add command parameters to synopsis
  • add --compare-data
  • some polishing and several improvements

New in Duplicity 0.6.21 (Jan 28, 2013)

  • Enhancements:
  • Merged in lp:~ed.so/duplicity/24syntaxfix:
  • fix python 2.4 vs 2.5 syntax error
  • Merged in lp:~mterry/duplicity/u1-oauthlib:
  • As the Ubuntu packager for duplicity, I would prefer u1backend.py used oauthlib instead of oauth. oauthlib is well maintained upstream (unlike oauth), has a python3 port (for the future), and is in Ubuntu main (so is oauth right now, but hopefully in the future we can drop it to universe, in which case duplicity can't use it anymore).
  • Merged in lp:~mterry/duplicity/delete-new-sig-in-cache:
  • In duplicity 0.6.20, we fixed bug 1031269. This means that we no longer leave sig files on the remote location. Leaving sig files on the remote location also caused a bug with deleting cache files. Code used to leave remote new-sig but delete the locale cache new-sig; this meant that we would keep downloadoing the new-sig all the time from remote. We had worked around that by just not deleting the new-sig in the cache, which was sort of the wrong side of that problem to tackle. Now that we handle the remote new-sigs better (by deleting them), I don't think we need this code anymore. Patch by [email protected].
  • Merged in lp:~mterry/duplicity/u1-ascii-error:
  • Fix for u1backend unicode error. Patch by Paul Barker.
  • Merged in lp:~satwell/duplicity/caching:
  • Add a cache for password and group lookups. This significantly improves runtime with very large password and group configurations.
  • Merged in lp:~ed.so/duplicity/manpage:
  • more formatting fixes, clarifications in sections EXAMPLES, FILE SELECTION
  • Merged in lp:~ed.so/duplicity/lftp.netrc:
  • Allow .netrc auth for lftp backend
  • Merged in lp:~mterry/duplicity/946988:
  • This fixes bug 946988 by not duplicating the checks for when we should ask for the password (those same checks are done more correctly inside get_passphrase). And add a test to reproduce the bug.
  • Merged in lp:~lenharo-h/duplicity/duplicity:
  • Generate encrypted backups without revealing the user's key id via option --hidden-encrypt-key
  • Merged in lp:~mterry/duplicity/u1-utf8:
  • Make sure u1backend returns filenames as utf8
  • Merged in lp:~carlos-abalde/duplicity/gdocs-backend-gdata-2.0.16.-upgrade:
  • Upgrade of GoogleDocs backend to python gdata lib >= 2.0.15: Stop using get_everything method.
  • Merged in lp:~ed.so/duplicity/webdav.fix-retry:
  • bugfix: webdav retrying broke on ERRORS like "error: [Errno 32] Broken pipe" in socket.pyas reported here https://answers.launchpad.net/duplicity/+question/212966 added a more generalized 'retry_fatal' decorator which makes retrying backend methods even easier
  • Merged in lp:~ed.so/duplicity/manpage:
  • Clear up PASSPHRASE reusage as sign passphrase. Minor fixes.
  • Merged in lp:~ed.so/duplicity/u1_and_manpage:
  • Manpage - document Ubuntu One required python libs - added continuous contributors and backend author notes
  • U1backend - lazily import non standard python libs, fixes http://article.gmane.org/gmane.comp.sysutils.backup.duplicity.general/5753 - fix "not bytearray" prevents PUT with python 2.6 - don't hang after putting in credentials (cause it silently retries in background) but go through with backup
  • Fixed 1091269 Data corruption when resuming with --no-encryption:
  • Patches from Pascual Abellan that make block size consistent and that add -n (no-encryption) option to manual-ctrl-c-test.sh.
  • Modified gpg.py patch to use 64k block size so unit test passes.
  • Merged in lp:~mterry/duplicity/static-corruption:
  • This branch fixes three possible ways a backup could get data-corrupted. Inspired by bug 1091269. A) If resuming after a volume that ended in a one-block file, we would skip the first block of the next file. B) If resuming after a volume that ended in a multi-block file, we would skip the first block of the next file. C) If resuming after a volume that spanned a multi-block file, we would skip some data inside the file.
  • A and B are because when finding the right place in the source files to restart the backup, the iteration loop didn't handle None block numbers very well (which are used to indicate the end of a file).
  • C is what bug 1091269 talks about. This was because data block sizes would get smaller as the difftar file got closer and closer to the volsize. Standard block sizes were 64
  • 1024. But say we were close to the end of the difftar... When resuming, duplicity doesn't know the custom block sizes used by the previous run, so it uses standard block sizes. And it doesn't always match up, as you can imagine. So we would leave chunks of data out of the backed up file.
  • Tests added for these cases.
  • This branch is called 'static-corruption' because all these issues occur even when the source data doesn't change. I still think there are some corruption issues when a file changes in between duplicity runs. I haven't started looking into that yet, but that's next on my list.
  • C only happened without encryption (because the gpg writer function already happened to force a constant data block size). A and B happened with or without encryption.
  • Merged in lp:~ed.so/duplicity/webdav.fix-retry:
  • added ssl certificate verification (see man page)
  • more robust retry routine to survive ssl errors, broken pipe errors
  • added http redirect support
  • Merged in lp:~ed.so/duplicity/webdav.manpage:
  • explanation of webdav changes above
  • Merged in lp:~mterry/duplicity/pygi:
  • Python bindings for the gobject stack (used in the gio backend) have changed from static to dynamically-generated bindings. The old static bindings are deprecated. So here's a branch to change the gio backend from old to new ones.
  • Merged in lp:~mterry/duplicity/py3rsync:
  • This branch lets one build the _librsync module with Python 3. You can't really do anything useful with it, but it's a nicely-isolated piece to add Python 3 support for.
  • The changes are a mix of modernization and #ifdef logic.
  • All tests still pass in Python 2.7 and 2.4. I tested manually that the module worked as expected in Python 3.
  • Merged in lp:~duplicity-team/duplicity/po-updates:
  • Updated translations

New in Duplicity 0.6.20 (Jan 28, 2013)

  • Enhancements:
  • Merged in lp:~ed.so/duplicity/ssh.manpage:
  • added gdocs and rsync REQUIREMENTS
  • added cloudfiles documentation
  • Merged in lp:~ed.so/duplicity/gpginterface:
  • refactor GnuPGInterface to gpginterface.py reasoning can be found in README
  • Merged in lp:~ed.so/duplicity/duplicity.helpfix:
  • fix rare 'TypeError: encode() argument 1 must be string, not None'
  • http://lists.nongnu.org/archive/html/duplicity-talk/2012-09/msg00016.html
  • Merged in lp:~ed.so/duplicity/duplicity.tmpspacefix:
  • use tempfile.TemporaryFile() so unused temp files are deleted automagically
  • propbably solve bug 'Out of space error while restoring a file'
  • https://bugs.launchpad.net/duplicity/+bug/1005901
  • http://lists.gnu.org/archive/html/duplicity-talk/2012-09/msg00000.html
  • Merged in lp:~mterry/duplicity/utf8-po:
  • For some crazy reason, the gettext module defaults to giving you strings in whatever charset the po file happened to define. Which means you never know what string of bytes you're going to get. This module makes sure we always get utf-8 byte strings. So we're at least predictable and reduces one source of UnicodeDecodeErrors (like in bug 989496)
  • Merged in lp:~mterry/duplicity/1031277:
  • ssh: actually delete all the requested files, not just the first one
  • Merged in lp:~mterry/duplicity/leftover-sigtar:
  • So currently, duplicity does not delete signature files when doing a remove-all-but-n operation. Seems wrong, since those signature files are now useless and take up space.
  • This branch does several things: 1) Make remove-all-but-n operate on chains. In practice it did before, since the sets it operated on always came from complete chains (i.e. it never used only some of the sets from a chain) 2) Add a new method to get all signature chains before a certain time. 3) Use this new method to also delete signature chains during remove-all-but operations.
  • And it cleans up the cleanuptest.py file: 1) Removes crufty, unused code 2) Disallows changing the destination folder for the test, which no one would ever want to do and isn't really supported anyway 3) Add some additional checks to the existing test 4) Adds two new methods to test remove-all-but-n and remove-all-inc-of-but-n-full
  • Merged in lp:~ed.so/duplicity/duplicity.manpage:
  • disabled hyphenation and block justification for better readablility of command line examples.
  • reformatted REQUIREMENTS section for hopefully better online rendering
  • minor clarifications
  • Merged in lp:~gregretkowski/duplicity/cf-retry-delete:
  • This will retry cloudfile delete commands. With large numbers of archive files over mediocre links transient network errors will occasionally cause deletes to fail and these should be retried.
  • Merged in lp:~mterry/duplicity/ropath.index:
  • This branch does two main things: 1) Skips base dir entries when compiling the list of deleted delta iters. (this gracefully recovers from the sort of situations that lead to bug 929067). I'm reasonably confident this is an uninvasive change, but please confirm. 2) Overwrites the sigtar file on backup-restart. This is because AFAICT, duplicity will rewrite the entire sigtar each restart. But we were opening the sigtar file as "ab", so we'd just dump the contents on top of the previous contents. Which was causing some confusion in bug 929067. If I'm wrong that we don't always rewrite the entire sigtar each time, this needs some rethink. Please also confirm that.
  • In addition, I added two tests for the above two changes and make some improvements elsewhere in the restarttest.py file while I was at it.
  • Merged in lp:~ed.so/duplicity/ssh-pexpect-msgbug:
  • Fixes 'UnboundLocalError: local variable 'msg' referenced before assignment' in _ssh_pexpect.py
  • Merged in lp:~ed.so/duplicity/gpg.tmp:
  • place gpg.py tempfiles in duplicity's tmp subfolder which is cleaned whatever happens
  • Merged in lp:~mterry/duplicity/u1-402:
  • Switch the code we check for out-of-space in u1backend.
  • Applied patch for #1066625 ubuntu one backend:
  • add delay between retries
  • Bugs closed in this release:
  • remove- commands don't remove signature-files
  • assert len(chain_list) == 2 AssertionError
  • crash during restore: "assert ropath.index == (), ropath.index"
  • doc improvement for --encrypt-key, --sign-key
  • Ubuntu One backend should check for 402 error for out-of-space
  • remove-all-but-n-full broken
  • --exclude-if-present and --exclude-other-filesystems causes crash with inaccessible other fs
  • ubuntu one backend does not work without gnome/dbus/x11 session

New in Duplicity 0.6.19 (Aug 30, 2012)

  • Enhancements:
  • lots of work on the man page to clean up requirements, etc.
  • use empty listbody for enhanced webdav compatibility
  • initial folder creation on backend does not result in a ResponseNotReady anymore
  • add ssh_config support (/etc/ssh/ssh_config~/.ssh/config) to paramiko sshbackend
  • add missing_host_key prompt to new sshbackend similar to ssh procedure
  • added --ssh-backend parameter to switch between paramiko,pexpect
  • allow answering gio mount questions (albeit naively)
  • if the gio backend wants to ask a question during its mount phase, it previously just aborted.
  • a couple more warning error codes that Deja Dup is interested in noticing.
  • ssh paramiko backend respects --num-retries now
  • set retry delay for ssh backends to 10s
  • ssh pexpect backend
  • sftp part does not claim 'Invalid SSH password' although it's only 'Permission denied' now
  • sftp errors are now more talkative
  • gpg.py
  • commented assert which broke otherwise working verify run
  • Bugs closed in this release:
  • 588541: Connection failed, please check your password: Login dialog cancelled

New in Duplicity 0.6.18 (Mar 2, 2012)

  • Enhancements:
  • fix extraneous '.py' in botobackend.py include
  • tests: add delay between backups to avoid assertion error
  • tests: use backup source that is more likely to be larger than 1M compressed
  • tests: make other-filesystem check more robust against certain directories being mounts or not
  • resuming an incremental results in a 'Restarting backup, but current encryption settings do not match original settings' error because curtime is incorrectly set away from previous incremental value
  • added option to not compress the backup, when no encryption is selected
  • always delay a little bit when a backend gives us errors
  • Don't cache TarInfo files. Tests still pass, so I don't believe we need the members cache (and in the old tarfile.py, we didn't cache either).
  • Adding --file-prefix option so different sets of backups can be stored in the same bucket. See blueprint at https://blueprints.launchpad.net/duplicity/+spec/file-prefix-option
  • two changes that help the test suite pass
  • raise log level on backend import failure so it will be visible under default conditions
  • file /etc/motd may not exist in test environment. Use __file__ instead to point to a known plaintext source file.
  • some code/import changes to make the ssh and boto backends compatible with Python 2.4.
  • some changes to make roottest.py compatible with the new dir structure.
  • Bugs closed in this release:
  • 884638 Python 2.5 / boto error
  • 908228 possible memory leak
  • 909031 SSH-Backend: Creating dirs separately causes a permissons-problems
  • 916689 multipart upload fails on python 2.7.2
  • 929465 UnsupportedBackendScheme: scheme not supported in url: scp://[email protected]/foo/
  • 930727 ftpsbackend should respect num_retries for ftp commands
  • 931175 duplicity crashes when PYTHONOPTIMIZE is set

New in Duplicity 0.6.17 (Nov 26, 2011)

  • Enhancements:
  • Added --rsync-options flag to allow user to pass options to rsync at will
  • Added --s3-use-multiprocessing to select the new s3 multiprocessing backend. Default is to use the single processing backend. A helper, filechunkio.py, requires Python 2.6+, so this option is not usable in earlier versions.
  • Bugs closed in this release:
  • Misleading error message: "Invalid SSH password"
  • File ... was corrupted during upload.
  • UnsupportedBackendScheme: scheme not supported in url: s3+http
  • Resuming a backup with a different password should throw an error
  • Bugfix for S3 multipart upload on 0.6.16
  • Duplicity 0.6.16 lists all backup sets as incomplete
  • Duplicity fails to add incremental backups to chain...
  • Merges:
  • lp:~mterry/duplicity/check-passphrase-on-restart
  • lp:~moss-d/duplicity/rsync-options

New in Duplicity 0.6.14 (Jun 20, 2011)

  • Enhancements:
  • 452342 Provide Ubuntu One integration
  • Bugs closed in this release:
  • 433591 AttributeError: FileobjHooked instance has no attribute 'name'
  • 487720 Restore fails with "Invalid data - SHA1 hash mismatch"
  • 507904 Cygwin: Full Backup fails with "IOError: [Errno 13] Permission denied"
  • 512628 --exclude-filelist-stdin and gpg error with/without PASSPHRASE
  • 680425 Endless retype passphrase when typo
  • 705499 "include-filelist-stdin" not implemented on version 0.6.11
  • 739438 [PATCH] Local backend should always try renaming instead of copying
  • 753858 cannot import name S3ResponseError
  • 761688 Difference found: File X has permissions 666, expected 666
  • 777377 collection-status asking for passphrase
  • 778215 ncftpls file delete fails in ftpbackend.py
  • 782294 create tomporary files with sftp
  • 782321 duplicity sftp backend should ignore removing a file which is not there
  • 792704 Webdav(s) url scheme lacks port support
  • 782294 create tomporary files with sftp
  • 782337 sftp backend cannot create new subdirs on new backup
  • 794123 Timeout on sftp command 'ls -1'
  • 797758 Duplicity ignores some FatalErrors
  • 793096 Allow to pass different passwords for --sign-key and --encrypt-key

New in Duplicity 0.6.10 (Sep 20, 2010)

  • Bugs closed in this release:
  • 542482 Offer command to remove old incremental backups from target
  • 578663 Use log codes for common backend errors
  • 589495 duplicity --short-filenames crashes with TypeError
  • 612714 NameError: global name 'parsed_url' is not defined
  • 613448 ftpbackend fails if target directory doesn't exist
  • 615449 Command-line verbosity parsing crash

New in Duplicity 0.6.08b (Mar 16, 2010)

  • Fix bug where encrypted backup without --gpg-options crashes.
  • This was a followup issue to bug 490619 released in 0.6.07.
  • This is attempt #2 -- not sure what happened to the patch, but it did not show up in 0.6-series like it should have.

New in Duplicity 0.6.07 (Mar 5, 2010)

  • tempdir option doesn't override TMPDIR
  • [PATCH] WebDAV backend doesn't work
  • re-add scp backend and make available via command line option
  • Use optparse not getopt
  • 0.6.06, archive dir: cache desynchronization caused by remove*
  • SSHBackend doesn't handle spaces in path
  • "sslerror: The read operation timed out" with S3
  • Don't Warn when there's old backup to delete
  • OSError: [Errno 40] Too many levels of symbolic links
  • Allow renaming paths as they are restored

New in Duplicity 0.6.06 (Nov 27, 2009)

  • Merged in lp:~mterry/duplicity/list-old-chains
  • List/keep old signature chains
  • Applied patches from Kasper Brand that fixed device file handling. http://lists.gnu.org/archive/html/duplicity-talk/2009-09/msg00001.html
  • Merged in lp:~l2g/duplicity/flag-transl-comments which cleared up how translation comments should be passed to the translators cleanly now.
  • Applied 422477; [PATCH] IMAP Backend Error in delete()
  • Merged in lp:~mterry/duplicity/iterate-warnings
  • Add machine codes to various warnings when iterating over source files
  • Fix problems with unittests under Jaunty. It appears that redirection in os.system() has changed for the worse, so a workaround for now.
  • Fix problem in restart where there were no manifest entries and no remote volumes stored. We clean out the partial and restart.
  • Fixed 435975 gpg asks for password in 0.6.05, but not in 0.5.18

New in Duplicity 0.6.04 (Aug 3, 2009)

  • One major and one minor change. The "No such file or directory" error is bad enough that this should be released quickly. For those of you using encryption, this is not a problem, but for those of you that do not use encryption (--no-encryption), then this will manifest itself if the local cache gets out of sync with the remote store.
  • Bugs fixed this release:
  • 405734 duplicity fails to restore files that contain a newline character
  • 403790 Backup error: No such file or directory

New in Duplicity 0.6.03 (Jul 30, 2009)

  • Lots of small changes and some bug fixes:
  • Restart error handling has been smoothed out a great deal and it "does what is right" in order to keep going.
  • Backends are now optional, if they fail an Info message is put out to notify of the failure and why.
  • There was more work on translations and internationalization.
  • Thanks to everyone!
  • Bugs fixed this release:
  • --file-to-restore doesn't work with trailing slash
  • Backend imports should be made optional
  • Deja-dup backup fails with message: "Unable to locate last file"
  • 0.6.2 manpage inconsistent wrt. archive-dir/name
  • Small i18n error
  • duplicity.gpg.gpg_failed() breaks and spews on GnuPG error
  • duplicity public-key-only incompatible with gnupg 2.0.11

New in Duplicity 0.6.02 (Jul 16, 2009)

  • Duplicity will now remove any spurious files left in the cache from a previous run. This will keep the metadata cache in sync with the remote storage metadata.
  • Bugs fixed this release:
  • 394629 Hang on first collection-status
  • 379386 Fix 'list-current-files' with missing archive dir
  • 395826 "No such file or directory" when backing up second time
  • 394627 User-friendly archive dir print
  • 388699 Manifest mismatch error

New in Duplicity 0.5.10 (Mar 3, 2009)

  • The default filename format has changed from W3 style to a long
  • numeric style, YYYYMMDDTHHMMSSZ, with no delimiters, thus is now
  • compatible with Windows/Samba filesystems. The time is UTC, not
  • local, so there will be no timezone or daylight savings time issues.
  • Duplicity still recognizes the old long filename format, and will
  • continue incremental backup chains if found. The old format is
  • still available via the --old-filenames option (pending deprecation).
  • Users of --short-filenames or --time-separator should stop using these options on their next full backup. The new filenames are compatible with your system.
  • The following options are pending deprecation and will be removed in a future release: time-separator, short-filenames, old-filenames
  • bug #19988: Incompatibility to Samba/SMB share
  • https://savannah.nongnu.org/bugs/?19988
  • bug #25097: Allow listing files from any time, not just current time
  • https://savannah.nongnu.org/bugs/?25097
  • bug #25550: Error codes do not propagate from log to exit status
  • https://savannah.nongnu.org/bugs/?25550
  • bug #25308: Signatures orphaned if from another time zone
  • https://savannah.nongnu.org/bugs/?25308
  • Bug #229826: duplicity crashed with ValueError in port()
  • https://bugs.launchpad.net/duplicity/+bug/229826

New in Duplicity 0.5.06 (Jan 29, 2009)

  • Fix to deprecation warnings about sha and md5 modules.
  • Uses hashlib if available, otherwise original module.
  • Added loop to run-all-tests.sh to run all tests against all supported
  • versions of Python if available. Looks for 2.3, 2.4, 2.5, 2.6.
  • Noah Spurrier has given us permission to distribute pexpect.py along
  • with duplicity, so this will no longer be an install requirement.
  • NcFTP version 3.2.0 will not work with duplicity since we require the
  • use of both -f and -C options on ncftpput. 3.1.9, 3.2.1+ work fine.
  • I put in error checks for this situation in the FTP backend code.
  • bug #25230: --include-globbing-filelist only including first entry.
  • https://savannah.nongnu.org/bugs/?25230
  • bug #25239: Error during clean, wrong case in duplcicity
  • https://savannah.nongnu.org/bugs/?25239
  • patch #6709: Report correct number of volumes when restoring
  • https://savannah.nongnu.org/patch/?6709
  • sr #106583: document the need to use the --force option
  • https://savannah.nongnu.org/support/?106583