Auto-Ripping CD's
Everyone has lots of old CD's sitting on the shelf purchased so we could hear the music of our youth at decent quality (instead of playing the hissy/scratchy old vinyl). Often each CD contains only one or two tracks we would like to hear again, however, in todays high-speed world, we never seem to have enough time to find the right CD, power up the player, insert the CD and navigate to the track (or two) we want to hear
Of course we can always listen to Internet 'Radio' where, between the never ending advertising and moronic DJ chatter we might hear some poor quality modern 'cover' version - or pay through the nose to some 'on-line' music store for an 'almost good quality' mp3 version of exactly what we want - and already have on CD ! What we need is a way to move our music CD's to our own Music 'server' so we can listen to our music 'on demand' and not have to put up with the adverts that fill the 'free' on-line music stations or pay through the nose to the subscription services.
So the goal of my Pi 'auto-ripper' project is to rip my CD's to my own 'server'. To do this, it needs to auto-detect a newly inserted music CD, extract the tracks to .wav (and, perhaps, high quality mp3), find the artist/track name, save them to my music collection and eject the CD. A useful additional feature - which makes the unit useful when not ripping - will be the ability to play back selected tracks from my own collection.
Of course many have done this before - so I used Google to turn up some useful guides - however none are EXACTLY what I want. I found one that's almost what I need, but is based on Python and appears to be both slow and unable to cope with the usual DRM 'tricks' (such as fake track location data and deliberate 'bad sectors' designed to ensure a software Rip either fails to find the track or delivers a wav full of 'clicks') Another one to show promise is RipIt, however this one is Perl based ! On checking the wiki.archlinux.org Ripping list I discovered 'adcde' = which is a BASH command line script, exactly what I wanted (for easy modifications). A quick Google of 'abcde + Raspberry Pi' then led me to this blog on which I've based my own efforts. I use the abcde command line tool" to read the CD and the halevt (Hardware Event) detect daemon (which is part of the Debian system (and thus in Wheezy / Jessie)) to detect CD insertion. For more on using halevt and abcde, see here NB. On my Windows PC I've used EAC for years - the nearest equivalent for *nix systems is cdparanoia. One command line tool that uses this is morituri, however it's not 'fully automatic' and has a long list of requirements. Some efforts have been made to get EAC run under WINE. Getting WINE to run on a Pi B3 is fine, but, I suspect, is another matter on the (somewhat slower) Pi Zero :-)
The hardware
A Raspberry Pi with Jessie Lite, of course. The Pi Zero (£4) is just fine. Add a 3port USB + Ethernet hub for £4 and a pHAT DAC (£12) for audio I2C output and the total is a touch less than the basic 'A+' (which has half the RAM, only 1 USB port and a poor quality 'stereo' output) and is between 1/2 and 1/3rd the price of any B series Pi (all of which also come with poor quality 'stereo')
A 3 port USB + Ethernet hub (rather than a 4 port hub) just makes everything simpler to set up (as the Ethernet cable provides an 'out of the box' LAN link) and gives the finished unit a lot more flexibility. One USB port will be used for the CD drive leaving 2 'spare'. These could be used for things like a multi-Gb drive (for a 'local' music store) a USB Memory stick or even a WiFi dongle (even if you connect to your LAN using WiFi, I still recommend retaining the Ethernet cable option, rather than switching to a 4 port USB hub).
Local playback
For local playback at high quality I recommend the 'pHAT DAC'. This plugs into the I2C i/o port so does not 'use up' a USB port.
Whilst the overall I2C data rate is 'spot on', apparently the actual transfer timing can jump all over the place and (at least in theory) this can be detected by the 'expert ear'. A USB audio 'dongle' would also 'do the job' and these now work well under Jessie (under Wheezy USB audio suffered from the sorts of 'glitches' when other USB devices, such as the CD or Ethernet, were being used). If your sound system has HDMI input, using HDMI is a better choice than the pHAT DAC, especially as you then get the advantage of AC3 (5.1 surround sound) 'pass through'. However HDMI has one major disadvantage = it means you have to reserve at least 100Mb of RAM for the GPU (and you meed that RAM for tmpfs to maintain ripping speed, although I expect it's possible for a clever programmer to work out how to 'switch' RAM from tmpfs to GPU depending on the 'mode' (Rip or playback))
CD drive and case
Of course you will need a USB CD drive. These can be found on eBay for less than £10 (intended for connection to a tablet or modern laptop that comes without a CD/DVD drive). Whilst modern units are 'low power' it's a 'good idea' to get one that can be powered from a separate source = many come with either a second USB socket or some sort of power socket.
To te-use an old laptop CD drive (and maybe even laptop harddrive) you would need to use IDE/SATA to USB 'converters' (although these can cost as much or even more than to totally new USB CD/DCVD drive)
The ideal case would be an ancient 5.25" 'stand-alone' USB (or disk) drive enclosure with space for the ancient 'full height' 5.25" CD drive = so check your loft (or local car boot sale / charity shops !
Using such a case with a modern slim-line CD unit means there's plenty of room inside for Raspberry Pi, pHAT DAC and even for a 2.5" hard drive (with SATA-USB converter) Any built in PSU supporting an old power-hungry 5.25" drives will deliver plenty of 5v power for the Pi (and even some +12v for an audio amp.)
The software
The best command-line script for CD ripping is "abcde" (A Better CD Encoder). The 'correct' way to get this is ro install it from the Pi command-line using sudo apt-get install abcde, however for reference you can download it on your PC from https://abcde.einval.com/download/.
In case it's ever removed, you can download abcde-2.7.2.tar.gz here.
The abcde script is a 'front end' for a number of encoders supporting Ogg/Vorbis, MP3 and FLAC (it also supports FFmpeg). If you install the 'recommended' support packages it will apt-get upto 6Gb of 'extras' including an entire SQL database + web server package (so you can support a local artist/track naming server). I suggest :-
sudo apt-get install abcde lame eject id3 id3v2 eyed3 normalize-audio vorbisgain mkcue mp3gain libdata-dump-perl flac
lame gets you mp3 encoding, eject lets you auto-eject CD's, the 'id3' stuff relates to naming 'tags' in mp3, normalise and gain relates to setting the audio levels, vorbis mk and flac are all data formats you might want to use.
One thing abcde doesn't do 'out of the box' is actually 'output' WAV**.
** The abcde 'works' is to rip tracks from the CD to raw WAV in a temp folder (eg WAVOUTPUTDIR='/tmp'). A separate (background) task then performs 'level adjustment' and encoding (eg into mp3) into a correctly 'named' track in the destination folder, after which the raw WAV is discarded. Note that whilst encoding starts as soon as the first track has been ripped it is quite slow = so encoding frequently continues after the entire CD has been read and ejected. Encoding into 128kbps mp3 is reported to 'load' the CPU to about 16%, which suggests that the 'bottleneck' is not the encoding process itself but the data read/write times to /tmp, from SDHC /tmp folder to encode and then back to the final destination folder (eg OUTPUTDIR='$HOME/Music/'). This is, perhaps, not surprising since the SDHC has to cope with 3 simultaneous transfers (wav to tmp, read of wav by encoder, write of mp3 to destination) as well as supporting the 'system' Fitting the fastest SDHC you can find helps, however the only way to really increase processing speed is to rip the wav into tmpfs (RAM disk) and then process from tmpfs to SDHC. The abcde.conf file includes a parameter (MAXPROCS=n) that sets the number of encoding 'tasks' that can be run at the same time, and this can be really useful on multi-core CPU's. However if the SDHC transfers are the bottleneck, running more than one encode (especially on the single core Pi Zero CPU) is likely to actually slow down the overall processing speed = so set MAXPROCS=1 A typical wav track would be less than 50Mb (and a 256kbps mp3 5Mb) and whilst a standard 640Mb CD (which might actually contain as little as 500Mb of music) won't fit within the amount of RAM available for tmpfs it should still be faster overall (so long as the CD read process can 'pause' to cope with waiting for space to become available for the next track). A Pi zero has 512Kb. A pure 'Rip-engine' with no display need not reserve any of this for the GPU. So a tmpfs of at least 350Mb should thus be quite possible. So long as encoding 'empties' tmpfs faster than 'half speed', it should be possible to read the entire CD (and fill tmpfs) without stopping (at half speed, by the time the CD has reached 700Mb, encode will have processed 350Mb and tmpfs will be full with the remaining 350Mb waiting to be done) Processing is controlled by the abcde 'config' file which, by default, will rip to 128kbps MP3. It's easy enough to change this to 256kbps and add a WAV 'output' option. Whilst the setting "KEEPWAVS=n" might let you keep the WAV after processing, this leaves them in tmpfs with the temp file names '01.wav', '02.wav' etc. Since they are already WAV, all the 'encode' process will need to do is copy and rename the temp files to your specified destination (see below)
Rip to multiple formats
abcde config files are :- /etc/abcde.conf and $HOME/.abcde.conf. By setting appropriate parameters, it's quite possible for abcde to Rip to multiple formats 'at the same time'. See here for some more tips on Ripping to muiltiple formats
You start by defining a 'parent' OUTPUTDIR folder and then the encoder 'types' (OUTPUTTYPE, eg wav,mp3). Finally, in the file name (OUTPUTFORMAT) you must use the '${OUTPUT}' parameter (which will be set to eg is "wav" for WAV and "mp3" for MP3)
OUTPUTDIR="/music"
OUTPUTTYPE=wav,mp3
OUTPUTFORMAT='${OUTPUT}/${ARTISTFILE}-${ALBUMFILE}/${TRACKNUM}.${TRACKFILE}'
The above combination moves the WAV files into /music/wav, and the encoded mp3 files in /music/mp3.
To maintain encoding speed, the WAV files should be 'buffered' to the SDHC card during processing and only transferred to disk storage (local or LAN) after processing is complete. However the MP3 (which is about 10% the size of the WAV) could be sent to your music 'store' across your LAN 'at the same time' as encoding. This means they can be played back as soon as possible after the track has been read
To achieve this, we 'mount' the '/music/mp3' folder into 'tmpfs'. A new background script then 'watches' the '/music/mp3' folder and when it finds a new file it moves that to your music store The only 'clever' bit is to ensure the 'move to store' task waits until the MP3 file has been 'closed' (i.e. the OUTPUT is complete)
File names
Note: You can define a destination sub-folder as part of the file name (as seen above) on a 'per CD' (or even 'per artist') basis
The track names are specified at output time eg :-
# basic naming
OUTPUTFORMAT='${ARTISTFILE}/${ALBUMFILE}/${TRACKNUM}.${TRACKFILE}'
# naming of multi artist albums
VAOUTPUTFORMAT='${ARTISTFILE}/${ARTISTFILE}-${ALBUMFILE}/${ARTISTFILE}-${TRACKNUM}.${TRACKFILE}'
# prevent invalid characters getting into the file name
mungefilename ()
{
echo "$@" | sed s,:,\ -,g | tr \ /\* __+ | tr -d \'\"\?\[:cntrl:\]
}
To discover the Album/Artist/Track names, abcde uses an on-line database. Since many Album, Artist and track names contain 'special' characters, the 'mungefilename ()' flag has to be set and a 'modification process' defined (between the {} brackets). In the above, the modify process uses the 'echo' command and works as follows :-
First the sed command replaces all occurrences of ":" with " -".
Next the tr command converts " " to "_", "\" to "_" and "*" to "+".
Finally the tr command deletes quotation marks, question marks and special control characters.
This allows you to move your music tracks across your LAN and onto USB memory sticks with maximum compatibility (i.e. avoids the use of any non-DOS characters such as " . / \ : < > ? * | )
Detecting a (new) CD insert
In theory udev can 'detect' a CD being inserted, however it seems to be 'focused' on the drive not the 'contents' = so whilst it works just fine spotting USB memory sticks, the 'udevadm monitor' seems incapable of spotting CD's being inserted
However, halevt (which was included with 'Wheezy') works fine 'out of the box' (although if you are using a Pi Zero, you must be using Raspbian Jessie, so may be forced to use udev after all).
If you start the halevt monitor using the command "lshal -m" and then insert an audio CD it will spit out lots of status data about the CD and every USB device you might have plugged in at the time
So the first thing to do is configure the halevt listener/handler (it uses an XML based config file) so it only looks at the CD drive (otherwise it will interfere with all your other USB devices). So start by removing all the default entries, then set up a single Device/Property match to trigger a single Action (run the abcde script)
The default halevt directory for scripts to be launched from halevt events is /usr/lib/hal/scripts/, however it knows the $PATH is /sbin:/usr/sbin:/bin:/usr/bin (where abcde and all its tools are installed) so SHOULD find the abcde script OK. NB. it's a good idea to put the log files somewhere you can find them easily, at least during the initial debug stages (you can dump them to NUL later) So your halevt config file is :-
This looks for the existence of a Device (CD drive = Device match="hal.storage.cdrom.cdr = true") and the presence of a CD in the Device (Property name="hal.storage.removable.media_available">).
When both are true, it Actions the command :-
exec="echo $USER > /home/pi/log/abcde.log ; sleep 10 ; abcde -N >> /home/pi/log/abcde.log 2>&1"
The Action command starts by defining where the event log is kept (/home/pi/log/abcde.log) It then sleeps for 10 seconds = this is needed to let the CD 'settle' before running abcde
Playback
If you are using Music Player Daemon (MPD) and Music Player Client (MPC) to stream music across your LAN, I recommend generating quality (256kbps) mp3 = otherwise you will have the 1.414mbps .wav using up at least 14% of your 100mbps Ethernet (or a larger proportion of your WiFi).
On the other hand, if you are using the Pi to playback the tracks locally, you might as well stick to using wav = the Pi CPU takes ages to 'unpack' the mp3 (without GPU assistance) and the CPU is so slow that chances are it won't actually cope with playing even 256kbps mp3 at the same time as anything else (in the past it would fail when you triad to play 312kbps mp3 even with nothing else running)
MPD runs on the Server (music source) and MPC on the 'client' (Player). Whist 'normally' the Server streams music tracks to the Player across your LAN (or Internet), it's quite OK to have both running on the same Pi = that just means the Pi 'serves' music to itself :-)
Note, if you are using MPD as the playback service, you must add the (just ripped) track to the mpd 'database' (on the Pi) before it can be played.
You will also need a remote control app. to choose tracks etc. - for example 'mpdroid' on your tablet/smart-phone etc. (you can find links to any number of MPD clients here) - I use WinMCC for Windows XP
For more on playback, see my Pi Jukebox project