Posts tagged with: rsync

Linux backup time machine based on Python + Rsync + Mysql

in a corporate network chat:
xxx: I was hoping, that I woudln’t have to say it, but the circumstaces made me.
xxx: gentelmen
xxx: did somebody made a backup?

Yet another backup system for Linux based operation systems. Built on Python, using Rsync. Other backup solutions appeared to be too complicated (like bacula) or not having needed functions (rdiff-backup).

Pros of Linux Time MAchine

  • Works on the principle of MacOs’ TimeMachine, namely: creating incremental copies in separated folders with the ability to fastly recovery any file from any date, or use it in some another way
  • Incremental copies, based on HardLinks, so every folder contains sort of “full copy”, but non-changed files represented by hardlinks to their older versions for previous dates
  • Easy thinning of backup copies, because of hardlinks make any copy self-sufficient
  • Immediate access to files, thanks to files stored in a plain form (unlike rdiff-backup and some others)
  • Built-in functionality to create SQL-dumps of MySQL databases (to backup mysql-tables as files)
  • Automatically resume of interruped backup, from the place, where it were stopped, in case of fault
  • Backup process could be started from PC, where backups are stored, or from PC where source files are.
  • Backup frequency could be configures, so, you can limit maximal frequence, to make Linux Time Machine copy files not too often, store this information in the config file, and not move it to crontab (as it usual happen with sheduled operations like backup)
  • Easy to improve backup process by using the API, for example, add copying to several storage servers
  • Copy whole filesystem or just single folder
  • Posibility to exclude folders, files by names or masks (like *.log)
  • Function to automatically clean old, outdated copies, configured by flexible rules
  • Posibility to configure sending error message to the getSentry system (Error-logging monitor with handy web-interface)

Cons:

  • The data is stored in a clean form, without compression, so, backup could use several times more space, that source data
  • Rsync should be install on both sides (source and destination) for backup to be possible
  • Another requirement is an ssh access to both systems
  • Ssh access is currently working only with key-based authentication
  • When calculating space, consumed by backup copies, single file could be counted several times (because of hardlinks), so space, occupied by it would be measured wrong (but, for example, terminal programm “du” does it the right way)

(more…)