Problems restoring a back-up
You can find many on-line 'tutorials' that simply say 'Map the Pi as a network drive to your PC and drag and drop files' or 'Make an image of the SD card using Win32 Disk Imager (on your PC)'. However, you will soon discover (as I did), that 'drag and dropped' script files won't run on the Pi and the SD card image often CAN'T BE RESTORED to a new 'same size' SD card !
Of course experience has shown that most on-line 'tutorials' are written by people who HAVE NEVER ACTUALLY DONE IT, so this is, perhaps, not surprising. For my 'real world' experiences and advise, read on :-
Problem 1, files copied to a PC won't run when restored to the Pi
If all you need to backup is the content of a few files (text or some other 'non-executable' file(s), such as .jpg / .avi / .mpg etc. from the Pi Camera), the 'simple' way is to install 'samba' onto the Pi and 'share' your Pi file directories with your PC (using 'map network drive').
An alternative is to move the SD card to your PC and just 'read' the Pi file system using a Linux file reader tool.
However after you 'restore' a 'script' (.sh) or other 'runnable' file to the Pi, you will discover that all the files you 'saved' on your PC have lost their 'execute permissions' ! All files copied from PC to Pi will appear to Raspbian as simple 'binary data' files without 'execute' rights.
This means files 'backed-up' to your PC can't be 'seamlessly' restored. For one or two files this is not a big problem = you can log-in to the Pi from the PC (eg. using 'putty'), navigate to the 'share' folder and use 'su chmod' to add 'execute permission' to your shell scripts etc. (or even create a script to do that for you).
However if the files are any part of the core operating system, chances are you won't be able to 'chmod' anything (as the Pi will fail during boot as soon as it runs into that part of it's system code that lacks 'execute' rights) !
Problem 2, your card 'image' will only restore to the original or a 'double size' SD card
In order to back-up the entire Pi SD card (including the 'boot blocks' etc.) most of us will use the most excellent Win32 Imager (freeware) app. on our PC. We expect this to work OK because Win32 Imager is what we used to 'write' our first Pi bootable SDHC card.
But have you ever tried to 'restore' a back-up 'image' to another SDHC card ?
If so, chances are you will have run into the dreaded 'Error - destination media too small' message. At this point most of us will just pop-in a 'double sized' SD card, carry on and forget about it.
Of course if you now have a double sized SD card, why not use all of it ? So we run the 'init-resize.sh' command (which the Pi auto-runs at first boot) and now have the full space available. Of course we then want to back that up .. and one day we will try to 'restore' it ... only to discover the exact same problem 'destination too small' error !
After going from 8Gb to 16Gb to 32Gb to 64 Gb and then 128Gb (and above) we either reach the limits of SDHC card technology or the limits of what we can afford :-)
At this point I have to answer the Question "Why would you ever want/need to 'restore' to a new SDHC card ?". Well, the 'non-volatile' memory in SDHC cards (and USB sticks etc.) have a 'write lifetime limit'. Each byte might well last for millions of 'write operations', however with current version of Raspbian pulling that 'clever' Windows 'virtual memory' trick (saving RAM by writing code to a 'swap file' on the system device), some parts of the SDHC card (like, the directory) might well be getting 'written' thousands of times a second .. So SDHC cards eventually 'wear out' and, sooner rather than later, you are going to be forced to 'recover' your system from the .img back-up to a new SD card
So what's going on ?
As with any disaster, the causes are many .... however the basic problem is that every SD card is of a slightly different size (every one manufactured has a random number of 'bad blocks', all of which are 'mapped out' before shipping, resulting in random exact total size).
A typical '8Gb' SDHC card is NOT 8,000 Mb or even 8,000,000,000 bytes. A typical "8Gb" SDHC I'm looking at right now has only 7,960,788,992 bytes. On 'average' you can expect anything from about 7,900,000,000 bytes (or maybe even lass). As prices get more competitive (and the consumer less caring), you can only expect SDHC cards to get smaller as poor quality cards with more and more 'bad blocks' (all of which are 'mapped out') are shipped. Indeed, over the years, it seems that every 'new' SDHC I purchase comes with fewer usable bytes than the previous.
1) The Pi auto-expands the partitions on a 'new' system SD card (on first boot) to fill all available space.
This is 'not the problem' in itself - even if it didn't expand, the restore problem would still exist - but expanding prevents a simple fix ....
2) Next, when Win32 Imager 'reads' the SDHC card to make your 'backup' image (.img), it reads every last block of the entire SDHC card !
So even if some blocks have not been 'included' in any partition, you end up with an .img that's the total size of the SDHC card (which is why stopping the auto-expansion alone won't fix anything)
3) When you come to 'restore' the .img to a brand-new SD card, Win32 Imager will ONLY write to an exact same or larger sized SD card
Since 'headline sized' cards all have different real sizes, and new cards are typically less than old ones, you will soon discover Win32 Imager refuses to 'Write' with a 'destination card is too small' error. Even if you could 'force' the write, you would end up with a 'corrupted partition' = remember (1), when the Pi expanded the system to fill ALL the space ??
The Win32 Imager software team (at Source Forge) are aware of this limitation and are working to add a 'write anyway' option. The code to support this was submitted on 2014-04-17, however (as of today, 10 Jan 2016) the 'current' build version is still v0.9.5 built on 2014-03-19 (I assume the 'last update 2015-09-22' shown by SourceForge is just some 'advertising' aimed at reassuring users the website is 'current', and has nothing to do with the software version).
Note that, on Windows 7, Win32 Imager has to be both installed by Administrator and RUN AS administrator.
WARNING - Windows 8/8.1 users should be aware that the combination of the 'modern' Microsoft operating system and Win32 Imager means that 'image write' operations may be 'copied' to ANY USB connected device (which is bad news for any USB connected back-up drives, since the chances of 'recovering' a 1Tb device after it's root blocks have been overwritten with a UNIX/FAT32 SDHC card file system are effectively zero). Of course the ancient Windows XP system doesn't do this (so one more reason to keep away from Windows 8/8.1)
How to prevent auto-resize (and make a back-up you can restore)
You might think, after 4+ years of replying to user Forums complaining that their carefully backed up SDHC image can't be 'restored', some-one at the Pi Foundation would have woken up to the problem. Fat chance (these are the guys who built an 'educational' computer with a choice of analogue TV or HD video at a time when the entire educational establishment was (is) using VGA)
1) Prevent the resize (and allow SSH)
Step one is to prevent the Pi performing the stupid auto-resize trick on first boot. This means modifying the cmdline.txt file found in the FAT32 (/boot/) partition after 'burning' the SD card on your PC
Copy cmdline.txt to cmdline.old. Then use any text edit or to edit cmdline.txt. It consists of s SINGLE LINE of text :-
dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=PARTUUID=11eccc69-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet init=/usr/lib/raspi-config/init_resize.sh
Remove the resize command (' init=/usr/lib/raspi-config/init_resize.sh') so the command line now reads :-
dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=PARTUUID=11eccc69-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet
Create a new file named ssh. (NOT ssh.txt :-) ). The file just has to exist,= it can be empty. If the file does not exist, the Pi will not allow SSH (PuTTY) network access. This is a sensible precaution that prevents Internet access to your Pi but also means it can't be controlled from your PC 'by default'.
2) Set resize to 95%
Step two is to plug the SD card into the Pi and let it boot up. You then SSH into the Pi (using PuTTY) from your PC (user pi, p/w raspberry) and edit the /usr/lib/raspi-config/init_resize.sh script (so it expands to 95% of the space and NOT 100%)
You can type 'sudo' at the start of each command or just type :- sudo su Open the /usr/lib/raspi-config/init_resize.sh for edit :- nano /usr/lib/raspi-config/init_resize.sh Find the line :- TARGET_END=$((ROOT_DEV_SIZE - 1)) Modify it to read :- TARGET_END=$((ROOT_DEV_SIZE / 100 \* 95)) Note the spaces and the '\' before the '*'. The spaces are 'separators' (so don't leave them out) and the '\' is used to 'escape' the '*' (which is otherwise interpreted as a 'wild-card') Write out the file and exit nano :- Ctrl o Ctrl x
You can now add the ' init=/usr/lib/raspi-config/init_resize.sh' command back to the cmdline.txt file and reboot the Pi - or run the command manually (the Pi will auto-reboot after the expand). This will result in only 95% of the SDHC card space being used. However next we have to 'backup' only that 95% :-)
3) .img only the used space
You can't use Win32 Imager to make the .img because it will image every last block, which (as we know) often means the .img can't be written back to another 'same sized' SD card.
In future it might be possible to 'force' Win32 Imager to 'write short'. In fact, an earlier version of Win32 Imager is reported to contain a 'bug' that let it do just that (however I've not managed to find this version).
Instead we must use DiskInternals Linux Reader (on my XP PC, this is v2.6). This is a 'read only' utility that is intended for file recovery, however it does a perfect job of generating .img's to the exact size we require (and not just the whole card)
Place the SD card in the PC SDHC reader. Launch DiskInternals Linux Reader - it should automatically 'mount' the SDHC. In the 'Physical Drives' section, right click the SDHC drive and select 'Properties'. At the end of the 'Volume' list, you will see 'Unallocated (at end)'. Make a note of the 'First sector' of Unallocated == this is the end of the 'used' part of the SDHC card and will be the END OF YOUR BACKUP IMAGE
With the SDHC Physical Drive still selected, go to 'Drives' menu and select 'Create Image'
Enter a name and location (note - Win32 Imager won't detect a '.dsk' image, so when entering the file name, make sure to end .img)
In the Create Image window, click 'Selected region' and enter a value a block or two beyond the 'First sector' of 'Unallocated' you noted above (see photo, right)
You now have an .img that will 'restore' to any 'same sized' SDHC card using Win32 Imager !
Backing up an auto-resized Pi
If you already let the Pi 'auto-resize' to 100%, things get a bit more difficult. The 'easy**' way is to resize the partitions 'manually' on the Pi itself and then follow 'step 3' (.img only the used space) above
** in practice, unless your project is very complex, it will actually be significantly easier to start again with a new system burn that you modify by preventing it from auto-resizing before moving the SDHC to the Pi.
Reducing the partition size
Reducing Pi partition sizes can be done using 'any' Linux partition tools (eg GParted), however they won't change anything on a 'mounted' partition (similar to the PC, where most partition tools refuse to touch the running system C: disk)
Of course you could try moving the Pi system SD card to your PC - but then you will discover that most PC partition tools only 'understand' Windows partitions (and don't even recognise the existence of the Pi partition structures)
(+) How do I change the Pi System partition size ?
Backing up with the Linux dd command
Since most of my projects use the Pi zero 'headless', I've adopted the '.imp using a PC' backup approach (detailed in step 3 above), however it's also possible to use the Pi (or any other Linux computer - for example a laptop booted from a 'Live' Umbuto CD/DVD) to 'backup itself'
The Pi system has a built-in 'imaging' command ('dd') that will clone 'disks'. However this won't work on 'mounted partitions', so you can't copy your running 'system' SD. Of course, if you have another Linux computer, you can use 'dd' on that = however 'dd' will copy 'every sector' (just like Win32 Imager), so runs into the same 'short write' = corrupted partition problem.
But 'dd' will actually make the copy, even if it runs out of space. So, if you adjust the partition size first (using GParted) you can then 'dd' from that SD card to a smaller one.
If all you have is a 'backup image' on your PC (created by Win32 Imager), you will have to 'restore' that to a (bigger) SD card first. After checking every 'same sized card' you have is 'too small', you may well end up using a 'double sized' SD card. Later, when 'dd' copies the double sized card down to an original sized card, you can expect a LOT of errors :-)
Since you can't backup a running system, you will need to 'write' a brand-new system SD (and boot to that). You will then need two USB SD card readers (these can be found in the £1 Stores) and a Pi B/B+
If you only have one USB SD card reader, the Pi you are using to perform the backup and restore will need a system SD that has enough space for both it's own system and a full-sized 'backup' image of the one you want to copy (so, to backup a 4Gb Pi SD you will need at least an 8Gb Pi SD, to backup an 8 you need a 16 and so on).
Actual backup/restore
1) Insert the Pi SD card you want to backup into the first SD card reader and use the 'GParted' app. to resize the last (typ. ext4) partition on that card down (so it will 'fit' on a slightly smaller card). Reducing it by 100 or 200 Mb should be enough.
An 'image' of that SD card on your Windows PC would will still contain every sector (including the now unused ones). To 'restore' that image to a 'same sized' SD card you will have to wait for some future next version (1.0) of Win32 Imager or SDimager.
2a) To create a duplicate copy of the 'partition reduced' card in USB reader 1, just pop a new 'same sized' SD card into the second reader and use 'dd' :-
dd if=/dev/sdb of=/dev/sdc
WARNING - above assumes the two SD card readers are 'mapped' as 'sdb' (containing the Pi SD you want to clone) and 'sdc' (a new, empty, SD) = get it wrong and you copy the empty SD over the Pi system SD :-)
WARNING - if you are using a Linux 'LiveCD' on a laptop computer (instead of using a Pi), be aware that your USB SD card reader(s) '/dev/sdb' (and /dev/sdc) could 'map' to almost anything = so watch out you don't overwrite your Windows C: (or some other PC hard drive) !
If you have only one SD card reader :- 2b) use the 'dd' command to copy the whole 'source' SD to an image file (on this Pi's system SD), then swap the 'source' SD for the 'destination' (empty) SD. Finally, copy the image from the system SD to destination using 'dd' :-. Copy the SD card to an image file (eg sdcard.img) :- dd if=/dev/sdb of=/mnt/backup/sdcard.img Swap the Pi SD for an empty one and copy the image file back :- dd if=/mnt/backup/sdcard.img of=/dev/sdb
The dd command in (2) above, which writes the new SD card, will give an error if the new card is smaller, however this can be safely ignored because we know that the Pi file system partition(s) will stop well short of the last block.