Best Practices for a Lossless Music Archive? 176
Sparagmei asks: "I'm a big music fan, and I like listening to the music I own on various pieces of digital gear. Right now, my library's at about 20,000 tracks, ripped from CDs to MP3 at 256kbps (enough that I can't tell the difference on my low-end playback gear). However, with the MP3 judgement rippling through the world, I'm interested in perhaps moving to a different compression standard. Before I do that, I'd like to ask a question: what lossless format would you recommend for making a digital 'master library' that could be (relatively) easily down-sampled to a compressed format?"
Important factors would be true losslessness, filesize (smaller than PCM WAV would be nice), embedded metadata (ID3v2-like), existence of automated ripper software, and (to a lesser extent) an open-source implementation of such software. Widespread playback implementation of the lossless codec is not an issue for me; the lossless library would likely be burned to archival DVD media and stored after being down-sampling with the chosen compressor. The reason I ask is this: I've got a 20,000-track re-ripping job ahead of me. I'd like to do that just once, lossless, so that years from now, when I decide to jump from Vorbis to 'komprezzor_2039_1337' or whatever, I don't need to drag out the old plastic discs. Thanks!"
FLAC. (Score:5, Informative)
(all answers below are quoted from wikipedia's FLAC [wikipedia.org] page).
Important factors would be true losslessness,
A digital recording (such as a CD) encoded to FLAC maintains the quality of the audio perfectly.
filesize (smaller than PCM WAV would be nice),
Audio sources encoded to FLAC are typically reduced in size 40 to 50 percent.
embedded metadata (ID3v2-like),
with support for tagging, cover art and fast seeking.
existence of automated ripper software,
Yup, lots [wikipedia.org].
and (to a lesser extent) an open-source implementation of such software.
See above.
Widespread playback implementation of the lossless codec is not an issue for me
Well, bad luck, you're going to get it anyway
Also, consider SHN [wikipedia.org], (although it seems superceded). I'll also mentiuon wave pack [wikipedia.org] - because it uses an interesting approach (splitting the file into a small lossy standalone & a lost bits diff). don't bother with Apple's lossless format [wikipedia.org] - it's going nowhere.
Re:FLAC. (Score:5, Interesting)
I've used FLAC - it just works. Also I like the Application Metadata blocks you can put into the FLAC files. I use this to store the full logging information from cdparanoia. It allows me to perform a quality analysis of the rip and look for jitter, skips etc. If i find a certain pattern which leads to audible artifacts I can just go back through the archive of tracks and perform an automated analysis of anything else which mught show the same problem.
Because of the amount of metadata which we need to store for business reasons (P&C, ISRC, barcodes, etc) I have developed an XML based format for entering the info - you wouldn't need this on a personal system I don't suppose.
For work it's great because I can encode to AAC/MP3/WMA for retailers. At home i use it to export to Ogg because we have an iAudio player, but it's trivial to export to MP3 or AAC instead if we got another device.
I store all the files in a flat system - each track has a unique ID generated when it was ripped - when I export out to the encoded versions I use the tags to create a Artist/Album/Track hierachy which again can be changed at at time fairly trivially.
Periodically rsync the exports out to my gf's machine and i've got the collection whenever I want it
Re: (Score:1)
Re:FLAC. (Score:4, Interesting)
Just one thing... FLAC does not compress to 40-50%. More like 60(rare)-70-80%. That being said, no lossless codec does better than 60% occasionally. There's no point chasing a couple percent, even when we're talking about hundreds of gigs, because if you're archiving this how much would it suck if you went to recover this years from now, Windows XP and Vista was no longer available, Monkey's Audio went out of business in 2008 and never made a Vista version, which is the last "audio path" that's compatible with Windows '84. IOW, you're fucked.
What do I do?
Actually, I started splitting my flacs with SHNsplit and putting in Vorbis tags, but if you're going to archive and never play the list is the way to go.
Re: (Score:2)
There is a huge difference in the number of files you can store if you reduce by 99% vs. 99.99%, but at first glance, it looks like they're almost the same.
It would be better to say reduce by a factor of 100 and factor of 10,000 respectively or reduce to one hundredth and on
Re: (Score:3, Informative)
Re: (Score:2)
I also use FLAC. One point to note is that the compression ratio varies considerably with the type of material compressed. I've posted comparisons of algorithms using linguistic field recordings rather than music here [billposer.org]. For my material, the greatest compression was achieved by Lossless Audio [lossless-audio.com], but the increment is not that great, and the time for both compression and decompression is much greater.
Re: (Score:2)
Unless you want your music to play in iTunes or on your iPod, of course. Personally, I figure that the worst-case scenario is that I have to write an Applescript to convert all my Apple Lossless files to FLAC, which isn't that big a deal.
You're right that FLAC is a better idea otherwise, however.
Re: (Score:2)
Well, he did say he wasn't worried about playback.
Re: (Score:3, Insightful)
Not to mention how you can't take it with you, like you can with an MP3 player, etc.
Re: (Score:2)
Me too (Score:3, Interesting)
I went with FLAC, and ripped 'em all. I'm using media monkey as a filing system, and am transcoding as necessary for portable apps. I'm without media server at the moment, so I can't help with streaming and such, though I'm going to be interested to see what others are doing.
At the risk of asking a stupid question (Score:5, Insightful)
If you're planning on re-converting from these lossless copies, it sounds like you're going to be doing a *lot* of work based on some second-guessing of where you'll be in 5 years time; and things may have changed then.
Re:At the risk of asking a stupid question (Score:5, Insightful)
Re: (Score:2)
(Yes, it takes time, but generally that's just "copy all the
Re:At the risk of asking a stupid question (Score:4, Funny)
We have done the math for you. (We will charge you for this service)
You have ripped 300 CDs and you admit sending music to friends. Well, 300 CDs at about 12 tracks per CD to on average 5 friends means you now owe us well, 5 * 300 * 12 * $0.99 so this comes up to $17,820.
For the hard math, we will charge you $1,000.
Please make out your check to RIAA-R-US.
Thank you for your cooperation.
RIAA
Re:At the risk of asking a stupid question (Score:4, Funny)
5*300*12*$751 = $13,518,000
Plus two thousand now since they had to do the math twice.
Re: (Score:2)
It's not about the space or size, though, but the convenience. Having 1000+CDs is not convenient (presumptive for his 20k tracks), and if you decide to switch formats (for whatever reason) you have to go through a lenghty manual process to re-rip everything. With foobar, yo
Re: (Score:3, Insightful)
Not much size = 50%. That may not be much to you, but it's still a reasonable amount.
Yes, but from that point-of-view alone, you're still having to do a lot of work to get it. And it's not 50%; it's 150%, unless he's binning his original CDs.
With foobar, you can take that collection - or any subset - and do a custom recode. It may take a day or two, but its totally automated.
I dunno; it kind of smacks of the geek tendency to spend a day automating a procedure that would otherwise take 15 minutes or so, and then using it 3 or 4 times before they decide they want things done differently and abandoning it. Can't beat that logic ;-) Maybe I'm wrong, of course, but I'm just getting that vibe; I know, because I've sometimes don
Re: (Score:2)
I'm with you on the automation flaw. Sometimes we do it because we can. It's been good for me though, and the formats seem to really be improving rapidly. There really are 48kb codecs that sound okay now on most music - not all, and not perfect, but good enough for my car. I like the flexibility of the online storage, and it has come in handy a couple of times already.
Re: (Score:2)
one note: CDs scratch. I have several possibly irreplaceable small label disks that have small data layer scratches, I believe in part due to budget pressing. If only I had FLACs
I assume you mean that the upper side has become damaged? If the scratch is on the underside it's normally fixable. Also note that you can rip the raw CD data using (e.g.) cdparanoia; but you have a good point- it hadn't occurred to me because AFAIK all of my CDs are easily replacable and I can't recall ever having one go bad.
Ape (Score:3, Interesting)
Re:Ape (Score:5, Informative)
Ape compresses a few % better than Flac at the expense of much more CPU usage. When compressing lots of CDs at once, the difference is significant. Flac was designed to be light on resources to facilitate portable devices, but it helps with modern computers as well.
Also, Ape is not free software, despite the availability of source for certain versions. It's only officially released for Windows.
So, in line with most others, I'd recommend Flac, but you might also look into WavPack as it also seems to be free software.
Re: (Score:2)
To clarify, Monkey's Audio is a free (as in money) application. Why some people refuse to call free software "free" unless it's FOSS continues to escape my understanding...If you mean FOSS, why not just say "FOSS".
Re: (Score:2)
Re: (Score:3, Insightful)
Re: (Score:2)
Re:Ape (Score:5, Funny)
Re: (Score:2, Offtopic)
Re: (Score:2)
And it doesn't exclusively mean without restriction either. Saying something isn't free, but not specifying under which definition of free, is at best, just confusing.
Re: (Score:2)
Yes, that's what it means. Without restrictions. Precisely what it means. If there are restrictions, it's not free.
Free means that I can come and acquire it and then it is MINE. Mine means my decisions, no restrictions.
If you tell me something is free, but try to claim that it remains yours when I come to get some, you are a liar.
Re: (Score:2)
free
-adjective
1. enjoying personal rights or liberty, as a person who is not in slavery: a land of free people.
2. pertaining to or reserved for those who enjoy personal liberty: They were thankful to be living on free soil.
3. existing under, characteriz
Re: (Score:2)
One meaning. Multiple relevant uses. One meaning.
Re: (Score:2)
Monkey's Audio is free software under at least one definition of the word free. The FSF can't just redefine or limit definitions of free.
Re: (Score:2)
Re: (Score:2)
The word open gets used for a lot of "open" software that isn't free [e.g. MS "shared source"], so I don't see why they continue to push the usage of this word.
In other words, both words are equally problematic; personally I call it "Free Software" (using the capitalization to indicate libre, rather than gratis).
Re: (Score:2)
Re: (Score:2)
If you want to refer to the no-money aspect of software you should use the word "Freeware", no need to trying to hijack the term Free Software for something for which the software world already has a perfectly fitting word.
Re: (Score:2)
Putting the word "free" in front of an object (like beer), by convention, generally means that you don't have to pay for it. This usage of the word free predates "Freeware" and the FSF definition of the term "Free Software" by several centuries. If anybody is hijacking "Free Software"
Re: (Score:2)
So rght now FLAC is the way to go for lossless archives. And if a better codec comes round, you can always rec
Re: (Score:2)
FLAC is asymmetric, meaning that the files take less calculating to decode than to encode, but it was my impression from tests I've seen long ago that it's about the same as "medium APE"
As everyone's said, use FLAC. (Score:2, Insightful)
Hydrogen Audio (Score:3, Informative)
media type (Score:2)
My advice (Score:4, Funny)
Apple Lossless, they just sound sooo much better. But you do need high end Bose equipment and gold
plated Monster cables to really bring out the warmer and fuller mid-range and the increased bass response.
Re: (Score:1)
Dude, I'm a Windows sysadmin, I like Windows and use it exclusively, I have no in depth Nix knowledge (though I can get around okay), and even I know
My suggestion is FLAC, as many have noted here.
Re:My advice - use PKZIP for DOS (Score:4, Funny)
Let me just say...Woooooosh! (Score:3, Informative)
Parent was trying to be funny. He wasn't, but don't go off the deep end.
Overrated is more appropriate, though Troll probably will be the choice of the Mac fanbois for the Apple Lossless dig he included.
Re: (Score:1)
You should just transfer your collection to vinyl if you Care About Music(TM). Although that's kinda hard.
And Apple Lossless sounds so much better than Windows Lossless, you're obviously a troll. And you've forgotten to mention that they should sit on their own hard drive, because putting them on a hard drive used for other things will result in the audio files getting dirty.
Re: (Score:1)
Re:Mod parent troll (Score:5, Funny)
Hardware based FLAC players? (Score:2)
Most hardware devices I've seen either a) can't rip, b) can't convert to FLAC, c) can't burn CDs from FLAC, or d) are just wireless devices to send the audio back to your computer's crappy speakers.
Anyone know of any other great sounding devices that rip, convert, and burn?
Re: (Score:2, Insightful)
Anyone know of any other great sounding devices that rip, convert, and burn?
Dozens of manufacturers have a device that can do that. I believe they're called 'computers'.
Backups (Score:1)
FLAC works for me (Score:2)
I've been ripping all my CDs using cdparanoia and encoding with FLAC. I keep two copies of all the FLAC'd suff on separate hard disks, and verey year or two, when I buy new hard disks, I migrate the data to the new disks in case the old ones fail.
There's no point in burning to DVD for "archival" since DVD is too unreliable. Anecdotally, DVDs seem to last only a few months to a year or two. Perhaps good quality tape archival would be good if you need the security? But really, hard disks are so cheap nowaday
Re: (Score:2)
Re: (Score:2)
I don't do RAID at home since it's too complicated and ropey on Linux(*). The advantage of it being on non-raided separate drives is that they can be spun down when not in use, they can be different sizes and different manufacturers. They can even be on different interfaces i.e. IDE, SCSI and SATA.
* I say that as a Linux fanboi since 1995. I do Linux RAID at work :-( I like to upgrade my kernels at home quite frequently. There is no guaranteeing that a RAID from one kernel version will work on the next. Th
Re: (Score:2)
Actually, maybe I'm being a bit harsh on the Linux RAID stuff. Some of our machines at work have months of uptime.
The thing is, I haven't got the time or money to do RAID on Linux at home.
WTF? (OT) (Score:2)
Re: (Score:2)
Sorry, I work for a dreadful company and it's causing me much bitterness. Please, pay no attention to my rantings.
Re: (Score:2)
There's no point in burning to DVD for "archival" since DVD is too unreliable. Anecdotally, DVDs seem to last only a few months to a year or two. Perhaps good quality tape archival would be good if you need the security? But really, hard disks are so cheap nowadays, it's feasible to have many PeeCees with new, high-qulaity disks in your house and to have multiple copies of the data.
I've been archiving music on DVDs for a couple of years, and just this week I decided to get an external hard drive instead. I figured the price per GB is almost equal; in Finland there's the 'tax' on blank DVDs, but not (yet) on hard drives.
Of course, a hard drive is more versatile and portable. My only worry so far has been that a broken DVD is much less of a loss than a broken HD.
Re: (Score:2)
As we all know, the quality of blank DVD media varies by brand, manufacuring plant, etc. Note that some brands (e.g. Maxell) sell DVDs from different manufacturers (e.g. Mitsubishi Chemical, Taiyo Yuden). Therefore, some Maxell DVD models are very good (manufacured by a good factory) while other Maxells are very poor (cheaply manufactured).
The current "standa
There's only one choice... (Score:2)
It appears to have overtaken all other lossless formats in popularity online, in a short time.
It is the only one seeing significant adoption in multiple brand of hardware (MP3) players.
I'd say it has a brighter future than Vorbis, even though it is at a several year disadvantage.
I've found that just about every audio program I use, has sil
flac (Score:1)
I'd been down the mp3/Ogg Vorbos road but found myself transcoding from one lossy format to another which had to stop. With flac I have the lossless copy to transcode from. I rip on Windows using dBPowerAmp and with AccurateRip I feel that when it says it's accurate, it is. I've seen people start to rate dBPowerAmp as good or better than Exact A
For your iPod (Score:1)
You've already lost, you just don't know it (Score:2, Insightful)
"Best practices" is to IT what the "zero tolerance" concept is to schools - no questioning, no thought required, simply doing whatever the current meme dictates.
Re: (Score:1, Insightful)
what's the point? (Score:2, Insightful)
Re: (Score:3, Insightful)
Maybe because at some point he'd like to upgrade from the low-end gear he said made the difference inaudible? Even on $30 headphones the difference between MP3 and lossless is clear as day.
Or maybe because he wants a backup copy to rebuild from in case the CDs get scratched? Not everyone has an audio CD collection made entirely of titanium and di
Re: (Score:2)
Re: (Score:2, Insightful)
Seriously. What's the point? What are you trying to accomplish? You can't hear the difference at 256kps lossy versus lossless, so why waste you're life converting your already lossess music archive from CD form to harddrive form?
There are so many thing wrong with this statement...
1) Hearing artifacts at 256kbps. I will agree that even with decent speakers you may not be able to hear them. But with good gear, it's *noticable*. I used to have a pair of B&W DM602-S3 speakers as my mains and MP3's were fine. Then I upgraded to the 704's. All of a sudden I had to throw away my entire collection of MP3's: the artifacts just slapped you in the head.
2) Why have your media on hard drive? Why *not* have your media on hard dr
Re: (Score:2)
What's worse than his comment about high bitrate MP3s is that he's planning on transcoding from lossy to lossy. The artifacts get multiplied each time. Disclaimer: I can't tell the difference with MDR-V6 cans or my e3cs (both are decent, though neither are made with snake oil) above ~200kb, and have been stumped by less than 96kb on certain content with advanced mp4 encoding.
*shrug* As I've said - rip to FLAC, transcode t
Re: (Score:2)
Yup, completely agree. CD stands for Crap Digital after listening to vinyl on horns.
> Seeing as a 400GB hard drive is going to cost less than $200
Eh? Where DO you do your shopping??
I just picked up a SATA3 WD 500 Gig for $140 (US) from Newegg.
If you're in Canada, NCIX is the newegg of Canada
Cheers
Re: (Score:2)
I encoded my entire CD collect 7 years ago as 320kbps constant MP3. I'm thinking about re-encoding it, because there's no point in having the files that big. 256kbps variable mp3 is would probably still be more than enough.
Maybe because not everyone uses those $30 speakers that come standard with all Dell computers? I personally rarely use my computer speakers (I have a set of Klipsch 2.1 ProMedia's), but instead have my sound routed to my receiver. No computer speakers can compete with a real stereo.
Conversion question (Score:2)
Re: (Score:2, Informative)
You should try it.
Re: (Score:2)
Judgement? (Score:1)
I'd use FLAC - 20,000 tracks isn't really all that much, and storage is only going to get cheaper. Pulling numbers out of my arse gives me 300 meg or so per CD using FLAC, so assuming you've got 10 tracks per CD that's 600 gigs. That'll fit on a couple of hard disks for £150 or so (UKP), although you might want to back up your stuff onto DVD also. Or you might want to find a backup medium which has some chance of being rea
wav? (Score:2)
Re: (Score:2)
No, because lossless compressing of audio, video, executables and text files require different compression algorithms if you want a decent compression ratio.
Re: (Score:2)
FLAC, but consider ALAC (Score:2)
I'm possibly switching to ALAC because iTunes will play it back natively, and I can use it under FrontRow in a Mac OS X media center frontend. The o
Re: (Score:2)
I wouldn't be personally considering the decision if I couldn't easily migrate off of ALAC. Given the open source decoder and open source AtomicParsley tagging utility, I could very quickly write a perl script that would migrate me away from ALAC and back to FLAC. That way there is no ultimate platform lock-in for Mac OS X because of my music collection. It would only be a point to consider in terms of momentum required to get off the platform. If I were making a switch for other reasons, i
Lossless? (Score:2)
Re: (Score:2)
Re: (Score:2)
Some of my friends haven't understood that. I go to their homes, listen to a mixed tracks CD and the volume is all out of the place. Also what I hate about normalization is a single event near clipping (like a badly ripped CD - see my previous rant in parent) will make ONE track sound very thin and everything else horribly loud.
Anyways, like I said, one can have fun with volume AFTER they have safely secured their original version on a DVD.
Re: (Score:2)
Audiograbber [com-us.net] has a nice feature that allows you to normalise a batch of files as though they were a single file. I like this feature very much and use it frequently. Windows-only, though.
For curiosity, I'd like to know what sample rate is "good enough" for digitising vinyl. How does 96 KHz do? the higher the better, obviously, but at some point there's going to have to be a trade-off between file size and sample rate.
Re:Lossless? - vinyl rip (Score:2)
Look at the specifications of your cartridge, if it's 20Hz-50KHz for example, double that to be conservative. Remember analog is discrete, 50KHz means you can get a pure 49.999KHz sound. Digital is quantized, 50KHz would mean you can get a pure 50KHz, then a pure 25KHz, pure 16.67KHz, and so on. To get a proper recording, and get proper quantization, 2x or 4x the maximum is adequate, I would go to 96KHz. Or if you have the equipment, try
Re: (Score:2)
Disk Image (Score:2)
The disk image files themselves are made with Roxio Toast in Digidesign's Sound Designer II format. Yes, it's proprietary software and a proprietary format, but the likelihood that this format will be readabl
Re: (Score:2)
Re: (Score:2)
1. Insert CD to be imaged into the CD drive.
2. Type in dd if=/dev/hda of=/home/yourname/name_of_cd.iso
3. Move the
To read the image on a UNIX machine:
mkdir
mount -t iso9660
and then the contents of th
Re: (Score:2)
"mount -o loop -t iso9660
Thanks for the correction. I'd briefly used other kinds of UNIX machines and almost all of the commands I'd used were identical to Linux, so I though this would be to.I guess not.
Anal Retentive MP3 (Score:2)
Why Compress? (Score:2)
As other replies on this thread have observed, lossless compressi
The answer is: It doesn't matter! It's Lossless! (Score:2)
Transcoding convenience has been the issue for me (Score:2)
My ideal setup would be:
Have a bunch of lossless files on a portable harddrive
Have a portable media player that can play high-quality lossy files
Have an "update" process, that when I move music to the portable media player, automatically encodes it to the lossy system, so I can fit more onto it, but keep the lo
Use Flac & don't worry about re-ripping (Score:2)
The great thing about any lossless format (as long as you have the uncompression tool) is that you may at any time choose to convert to any other format without loss. I regularly convert Monkey audio downloads to Flac without loss.
That's the real problem with MP3 library's you can nev
Re: (Score:3, Informative)
Re: (Score:2, Informative)
I believe it is only possible to have the same rip twice under windows using EAC.
Please read the documentation for EAC. You may be surprised to find that EAC was written because the author wanted a Windows port for the Linux program 'cdparanoia'. EAC's entire goal is to mimic the pre-existing Linux program 'cdparanoia' in its bit-for-bit redundant checking and matching.
There are decent frontends for cdparanoia if you don't want to run it via command line. GRip happens to be excellent. But please check wha