Follow Slashdot stories on Twitter

 


Forgot your password?
Close
typodupeerror
×
Music Media

Dial-Up Audio Public Listening Test Opened 124

Posted by simoniker from the cha-cha-crackle dept.
CaptainCheese writes "Hydrogenaudio.org's Roberto Amorim just announced the opening of their 32kps multi-format listening test, intended to test the current 'dial-up' quality codecs. From the Announcement: "The formats featured are Nero Digital Audio (HE-AAC+PS), Ogg Vorbis, WMA9 Std., MP3pro, Real Audio and QDesign Music Codec. Lame MP3 is being used as low anchor, and a lowpass at 7kHz is being used as high anchor." These codec tests are unusual in that they adhere to ITU-R BS.1116-1. The test is open until July 11th and all are invited to participate. There's more info in the original test discussion, which indicates the originator is interested in 'testing formats working on dial-up streaming bitrates' - the test page notes: 'The real arena where codecs are competing, and most development is going, is at low bitrates.'"
This discussion has been archived. No new comments can be posted.

Dial-Up Audio Public Listening Test Opened More

Dial-Up Audio Public Listening Test Opened

Comments Filter:

  • Results may be flawed (Score:2, Interesting)

    by Atario ( 673917 )
    ...since I'm liable to vote for whichever one sounds most like the Centurions from Battlestar Galactica, or the voice communications from THX-1138. Not best quality, not most understandable, just coolest.

    • Re:Results may be flawed (Score:5, Insightful)

      by FlipmodePlaya ( 719010 ) on Friday July 02, 2004 @05:40PM (#9595971) Journal
      Good point. Can a study that will probably have relatively small survey size of an opinionated tech crown likely to exhibit bias be trusted? I don't know too much, admittedly, but wouldn't an automated test that just compared the output of a compressed audio track to the original be more accurate? Or is there more truth than I think to certain frequencies being worthless and inaudible by human ears?
      • Yeah, and probably some people have some crappy speakers. Some people would probably even try it with thier on-board speaker...

        --
        Only 5 Gmail invitations left! [retailretreat.com]
      • Err... if a codec *sounds* closer to the original, who cares whether the actual waveform is closer? The whole point of psychoacoustic compression is that it exploits the weaknesses of the human ear to its advantage.

      • Re:Results may be flawed (Score:4, Informative)

        by izx ( 460892 ) on Friday July 02, 2004 @07:23PM (#9596626)
        The problem is that "automated comparisons" don't mimic human system responses (the ear, or the eye for video). Take video: the eye would finds grainy VHS tape more pleasing than a digital video that displayed some blocking. The blocked digital video, mathematically, is much closer to the original than the the VHS with its added noise...

        These types of psychovisual (or psychoacoustic) responses are what make automated tools almost useless for judging the perceived quality of any lossy encoder. Perceived, that's the key word....it may not be mathematically up to scratch with the original, but if you PERCEIVE it to be as good as the original, thats what matters (this is of course for CD-quality high bitrate tests).

      • ...wouldn't an automated test that just compared the output of a compressed audio track to the original be more accurate? Or is there more truth than I think to certain frequencies being worthless and inaudible by human ears?

        The whole idea behind lossy audio codecs is that the human brain and ear aren't that good at what they do :) As was pointed out [vorbis.com] on the Ogg Vorbis mailing list a looong time ago, technical tests like you're proposing would only tell you what computers would find more pleasant to list

      • I don't know about that. In exactly what direction is the tech crowd biased in favor of? Remember we are talking 32 kpbs here! This isn't going to be high fidelity listening. If it were a slashdot crowd you might see some bias in favor of ogg and against Real and Windows (wma) codecs, but at this bitrate differences are usually extreme. Even just for voice.
    • Any listening test worth its weight in... sound, will not tell the listener which audio codec compressed sound they are listening to,
      but rather, will give the listener, say, 5 different sounds (labeled for example 1,2,3,4 but with no relation to their codec) and ask them to report which one sounded the best.
      • The only problem is that at these low bitrates, signature sounds are quite noticeable. It is pretty easy to recognize one codec or another if you have heard them before. So ideally, at these bitrates, the testers should have never (non-blindly) compared these codecs before. Then it would be truly "blind" testing.

  • Get the news out to portable music player (Score:4, Insightful)

    by ralphart ( 70342 ) on Friday July 02, 2004 @05:36PM (#9595949)
    Now if only the companies who manufacture digital players would take a look and see that there is life beyond MP3. Nice that a few are starting to offer Ogg Vobis, but they are few and far between.

  • Is this a double-blind test? (Score:3, Insightful)

    by Anonymous Coward on Friday July 02, 2004 @05:37PM (#9595957)
    If it isn't, you'll only find out the most popular format, not the best.

    • Re:Is this a double-blind test? (Score:4, Insightful)

      by sploo22 ( 748838 ) <dwahlerNO@SPAMgmail.com> on Friday July 02, 2004 @05:50PM (#9596039)
      AS a matter of fact, it certainly is. Follow the link at the bottom of the page (to here [rjamorim.com]) and you'll see this information:

      One of the most acclaimed methods of comparing codec quality is by performing so-called "Double Blind Listening Tests". In this sort of test, the participant compares various encoded samples against each other and against an uncompressed reference sample. The blind part means that the participant doesn't know which sample was encoded by which encoder. That guarantees there'll be no psychological bias towards his/her favorite codec, or against the codec he/she dislikes.
    • On this site there is a useful little utility [pcabx.com] written by Arny Krugar to let you do your own DBT testing at any bitrate with any codec you want.

      It does take a bit of preperation, but the results are legit. Not really suited for large organized polls, but fine to see your personal tastes and to understand exactly what a double-blind test is and how it works and why it is the only valid way to scientifically test.

    • It only needs to be a blind test, unless you worry that the computer administering the test is biased.
    • Well yes, it says so in the summary:

      These codec tests are unusual in that they adhere to ITU-R BS.1116-1.

      Follow that link and you get:

      Methods for the subjective assessment of small impairments in audio systems including multichannel sound systems which is a fancy way of saying double blind listening tests.

  • What's the point? (Score:4, Interesting)

    by TommydCat ( 791543 ) on Friday July 02, 2004 @05:38PM (#9595963) Homepage
    I personally don't get the goal of a test like this. Listening at that low quality doesn't have as much commercial, and quite frankly, personal appeal as it did back in the 90's.

    I've seen the double-blind tests done at 128kbps and again fail to see the point.

    What I really want to see is a rating of codecs that are able to achieve DBT-proven audible transparency and see them rated in terms of storage space (thus allowing the VBR schemes to finally compete).

    Of course FLAC would come in last (considering WAV is the 'source'), but can my high quality VBR LAME MP3 pass for the original and take less space than MPC?

    • Re:What's the point? (Score:2, Insightful)

      by Sam3.14 ( 792129 )
      I agree that there isn't that much point to testing such low bitrates. People are quickly switching to DSL and Cable, And I think that there won't be many Dial-up users in 2010.

    • Re:What's the point? (Score:4, Insightful)

      by Nakito ( 702386 ) on Friday July 02, 2004 @05:58PM (#9596082)
      Testing at low bitrates emphasizes certain weaknesses of a codec. At high bitrates, it takes a more sensitive and trained ear to detect the artifacts and flaws. But as you reduce the bitrate, the differences become apparent. If you've never tried experimenting with it you might find it interesting, because the various codecs produce very different sounds with the same source file when the bitrate is drastically reduced. But I wonder if this is a proper way to evaluate the best design overall, since some of these codecs are certainly not optimized for low bitrates, and I do not know if there is necessarily a correlation between the flaws of a codec at low bitrates and those at higher bitrates.

    • Re:What's the point? (Score:3, Informative)

      by bcombee ( 5301 )
      Actually, low bitrates will be more important in the near future, as more people use streaming audio over PCS data services. For example, I listen to low-bitrate streams over Shoutcast several times a week on my PalmOne Treo 600, and 32Kbps streams much better than 64Kbps, while higher than that just isn't feasible on Sprint's PCS network. While this isn't as much of an issue for home users, mobile devices on relatively low-speed networks are going to be big.
    • A lot of people are still on dial-up, and I suspect it'll stay that way -- especially if you look outside the developed world. Also, as someone mentioned, we'll probably see people streaming music to their cell phones or other wireless devices with limited bandwidth. Besides, regardless of how much bandwidth you have, you might not always want to use up all of it on background music. Lower bitrate audio also means that a server can serve a higher number of simutaneous listeners (assuming bandwidth is finite

    • Listening at that low quality doesn't have as much commercial, and quite frankly, personal appeal as it did back in the 90's.

      Not every location is set up for wireless broadband Internet access. Can you get affordable broadband on your mobile phone? GSM mobile phones receive and transmit voice at 13 kbps using the GSM RPE-LTP codec; one often has to pay extra just to get 32 kbps data. Also think about digital radio; lower bitrate for a given perceptual quality allows for more music choices in the same fr

    • In a word, video.
      The less bandwidth the audio takes up on a video stream, the more space you have to increase the frame rate and the resolution of the picture.

      Also, I know quite few people at work who won't get broadband because it is still more expensive than a dial up and they don't think they would use broadband enough to justify the expense. Then there are countries where broadband is some way off into the future and when it does arrive only the wealthy elite will be able to afford it.
    • Low bitrates are useful too, just not so much for pure music listening. Certain codecs (like realaudio) can do pretty well with voice or a mixture of voice and music at these low bitrates. It can be particularly useful when you want to store large quantities of audio information.

      For instance I have an application where I want to record audio from my TV tuner card 24 hours a day. You can hold a lot more data at 32 kbps than at 128.

      Unfortunately, AFAIK the better low bitrate codecs cannot be natively edited

  • Why bother? (Score:1, Interesting)

    by TexasDex ( 709519 )
    I really have no use for low-bitrate music. I have a nice 1.5Mbit connection. Really, there's no point in listening to a low bitrate music stream because no matter how good the codecs are, 32kb/s music just doesn't cut it.

    BUT...
    Although it wouldn't help for internet music, better low-bitrate codecs could make internet talk radio more feasible. It lets companies save bandwidth on the server side and still maintain quality that at worst is a bit better than the phone connections of people calling in (Vo

    • VoIPo(Dial Up) == cheap phone
      Noone in his sane mind would listen to 32kbps music anyways
    • Actually, with talk radio, the lower the bit rate, the better, if you catch my drift.
    • " I really have no use for low-bitrate music. I have a nice 1.5Mbit connection."

      There's still a large dial-up market out there. How many millions of users does AOL alone have?

    • Re:Why bother? (Score:5, Insightful)

      by evilviper ( 135110 ) on Friday July 02, 2004 @06:03PM (#9596105) Journal
      no matter how good the codecs are, 32kb/s music just doesn't cut it.

      Damn, this is the kind of crap that gets modded-up these days...

      Codecs continue to get better and better. Vorbis is pretty good even at 48K (artifacts are subtle). And even if this was 1997, and 32K sounded like crap with current codecs, you're statement is just like the famed "640K is enough for anybody", and "there is a world market for maybe a dozen computers". It's absolutely guaranteed to be proven wrong with time.

    • Re:Why bother? (Score:4, Funny)

      by geeber ( 520231 ) on Friday July 02, 2004 @06:11PM (#9596161)
      Cancel the Test! TexasDex has no use for low-bitrate music. Since he clearly speaks for everyone, the entire affair is clearly a frivolous waste of time.

      Go about your business, people.

    • Re:Why bother? (Score:1, Insightful)

      by Anonymous Coward
      You might find it useful for:

      - streaming video + sound
      - voice chat while playing a game that is already sucking up a good portion of your bandwidth
      - running a voice-chat server (ex: TeamSpeak), because (a) you're streaming out to multiple people, and (b) your upstream bandwidth is usually the limiting factor since most cable and dsl have shitty upstreams.

      There are probably a lot of other uses but those two just stuck out in my mind.

    • I have a nice 1.5Mbit connection.

      Which can feed 7 listeners at 192 kbps or 46 listeners at 32 kbps, as you seem to recognize with talk radio. Wideband Speex, an audio codec designed for talk radio and telephony, sounds listenable even at 12 kbps (listen [speex.org]). However, more listeners for talk radio does mean a bigger audience for conservative spokesmen, whether you agree with them or not.

      32kb/s music just doesn't cut it.

      Have you actually tried listening to a recent codec at 32 kbps? Sure, it's not tran

  • Slashdot Low Bitrate Ethnocentrism (Score:4, Insightful)

    by neildiamond ( 610251 ) on Friday July 02, 2004 @05:53PM (#9596050)
    Wow! I thought people on this site would have been a little more understanding. Believe it or not there are other places in the world (such as Africa) where high-speed Internet is not the norm or even available. Plus if you stream audio, any attempt to lower bandwidth is a plus as it lowers your bills.

    Get over yourselves please.

    By the way, did you ever notice the lack of multimedia even on this site? Why might that be? Hmmm...

    • By the way, did you ever notice the lack of multimedia even on this site? Why might that be? Hmmm...

      Because high bandwidth multimedia has no place on this type of news site; because it's unbelievably annoying; ...

    • Perhaps you should stop seing everything on the basis of race?

      ethnocentrism
      n : belief in the superiority of one's own ethnic group
    • It's also useful for things like VoIP, wireless, gaming, and so on. Just because you have a large connection, doesn't mean you want all, or even most, of it devoted to audio. Like game communication. You generally recieve one stream for each person talking. Well if each stream were 256k, that would nail your connection right quick. If each stream is 32k, not such a problem. There is, of course, the server end too, which much send one stream for each user speaking to each client listening. I'd rather not nee

    • Re:Slashdot Low Bitrate Ethnocentrism (Score:1, Informative)

      by Anonymous Coward
      It's not even available in some places in Sydney (and other major Australian cities), because of the short-sightendness of our telecommunication companies in the 1980s and 1990s. They decided it would save money to multiplex the phone lines using a system called "pair gain". Now, anybody who has had this done to them can't get DSL.

      A technician from the telco told me that we will get DSL in my area "some time after hell freezes over".
  • I remember back when I had dialup, it totally sucked, it took me absolutely forever to download ANYTHING, I would read a magazine while I used the computer because it would take so long for pages to load... when I got broadband at home it was a very happy day... but lots of people I know still don't have broadband in their area, which is why I think it is nice for people to do things like this, but also I think you could use this for voice over ip... just my two cents
    • Satellite is available anywhere there is uninhibited view of the sky, which is most places that don't have cable or DSL. However, satellite is expensive to set up... $500 installation fee from DirecTV last I checked.

      Secondly, dial-up is not that bad, and it's definitely not as bad as your exaggeration. It's not comparable to broadband, but it's not unbarable. To further speed up dialup browsing, one should use a web cache, which is very helpful.

      • Spot-beam satellite (i.e. DirecTV's offering, the British company whose name escapes me) is not available anywhere there is an uninhibited view of the sky. If you look at the contour maps for those products, you will see they are pretty tightly focused on their target market. I suspect it's even more of a problem on the uplink side -- those systems are running with really tight link budgets, and I don't think you're going to get an acceptable uplink BER if your antenna is 10dB off boresite.

        While there

        • This sounds like a business opportunity. We got any venture capitalists in the audience? Let's put together a team to do this at much cheaper rates. We'll still make money of course, but it looks like there's a lot of wiggle room.
          • This sounds like a business opportunity. We got any venture capitalists in the audience? Let's put together a team to do this at much cheaper rates. We'll still make money of course, but it looks like there's a lot of wiggle room.

            Set up a sattelite system, and charge people for using it to access the internet, yeah sounds real simple...
      • yeah i do agree with that, but dialup can be really extremely slow, sometimes it will work just fine, and be nowhere comparable, but other times it will be excessively and nastily slow, not to bash on dialup or anything, but definitely what this article is talking about (the dialup friendly codecs) is a good thing, because however not bad dialup may be, it still ends up being awful at downloading media such as big images, and definitely audio / video.
    • I too used to read a magazine, when I had had both cable and DSL (cable is generally faster, by about 3x), and went back to dial-up for a year and a half.

      Thankfully I'm back on cable. But I did get to read more. ;-)

  • Still too bandwidth intensive to be useful? (Score:3, Interesting)

    by Wild Bill TX ( 787533 ) on Friday July 02, 2004 @06:12PM (#9596164) Homepage
    No matter how optimized it is, won't it will still use too much bandwidth for dial-up users who actually want to do something else with their connection? All of the streams I ever tried to listen to, including the 8kbps ones, gladly used all of my available bandwidth. I don't know about anybody else, but I'm not interested in only getting a fraction of my 2 KB/sec max for browsing, using chats, or other tasks.
  • Why don't you just USE THE PHONE?

    Yes, I know there are applications for this, like doing some other thing while listening to audio, and the prohibitive internatioanl call rates, but still..

    • Because audio over the phone is like (roughly) 56kbits uncompressed. It's optimised for the frequencies associated with speech, but it's still a lousy way to send audio.

      Compare a 56kbit wav to a 56kbit mp3 and you'll hear a huge difference...

      • Compare a 56kbit wav to a 56kbit mp3 and you'll hear a huge difference...

        To save Slashdot readers the trouble of going in and encoding it yourself, I've done it for you. Hear it here [jk0.org].

    • Maybe, say a MMORPG. Let's then say that I juin a team/guild/association. And let's say that, while we play and collarbate, we all want to chat. Well what if we were to use the phone system as you suggest? First we'd need to get a teleconferencing centre to conference the calls, which isn't free. Then we'd all have to call long distance, and pay per minute. Those in other countries would have to pay a LOT per minute. So for a game that we pay less than $20/month to play, we'd each be paying anywhere from $5

  • Interesting results (Score:4, Informative)

    by sploo22 ( 748838 ) <dwahlerNO@SPAMgmail.com> on Friday July 02, 2004 @06:16PM (#9596189)
    I just took the test with sample 9, one of the speech ones, and it's amazing how much variability there is in the various codecs. One of them was so good I could only reliably hear the difference after a dozen repeated listenings, and another sounded like a cellphone in a tunnel. I'll be interested to see the results in a week or so.
  • When I first read this story's title on this, I immediately thought of They Might Be Giants' infamous Dial-A-Song [wikipedia.org] program, which can incidentally be reached at (718) 387-6962.

    Something, though, tells me that this test isn't going to apply to this sort of dial-up audio.

  • More interested in 32kbps speech (Score:4, Interesting)

    by Dr. Spork ( 142693 ) on Friday July 02, 2004 @07:39PM (#9596715)
    I am in the process of starting a project [syr.edu] which needs accurate speech encoding at 32kbps. For now we're going with LAME at --alt-preset -b 32 -a --resample 22 --lowpass 6 -Z based on informal tests we did (ideas also came from here [hydrogenaudio.org]), but I'd love to see something more formal.

    Notice all the different non-standard switches I had to use, which together help noticably. That's the sort of stuff you need to do to LAME before it produces acceptable results at very low bitrates. It is optimized only for 44.1KHz, so we should keep that in mind when we see the results. Notice now that none of these switches are being used for this test, so I'm almost certain that LAME will come out looking much worse than it is.

    I would love for there to be a LAME-based encoder that is optimized for speech, low bitrates and sample rates. If it is made, I am prepared to re-encode all the readings that are (and are about to be) posted on my site.

    • Speex [speex.org].

      Sound great on 8 and 16 khz material, even up to 22 khz, but sounds terrible in its "super wideband" mode. Another recommendation is Ogg Vorbis, though so far it's ending up near the bottom of the quality scale for my ears in this 32 kbps test.
      • Nooo! On the tradeoff between quality and compatibility, I'll take compatibility any day. Lame is already good enough to be useful. Students load their assignments in their ipods, on CDs they play back in MP3-aware cd-players (or dvd players), and a million other things. For any audio book project, it's mp3 or nuthing. I'd be stupid to use anything else.

  • This sounds terrible!!! (Score:3, Funny)

    by rice_burners_suck ( 243660 ) on Friday July 02, 2004 @10:22PM (#9597398)
    I just listened to this, and I can tell you that it sounds totally like crap. Here at home, I have a sound room with a bitchen 100% analog system... Vacuum tube amplifiers, gold wiring, the works. I play my records on this thing and they have that wonderful warm sound. But this 32 kps sound sounds like garbage.

  • one use: books! (Score:2, Interesting)

    by timothy ( 36799 )
    A lot of people are complaining here that low-bitrate recording is useless / stupid / so 1988 etc. Fine for them :) I'm glad that people are fanatic about sound quality and that storage prices make it reasonable for many people to use nothing but lossless codecs etc., and to care about the difference between 192kbps and 256kbps MP3s. Lossless is certainly a good storage answer for the long term, as the file can be inflated and re-squashed with the latest n' greatest lossy codecs as appropriate.

    However, the
  • In terms of voice I did some testing of my own a while back and actually (as much as I hate to admit it) realaudio came out ahead in most of my voice tests right down to below 24 kpbs testing. I think it really helps when you've specifically designed your codec for these ultra-low bitrates. IIRC, WMA didn't do that bad at these low bitrates either.

    For 16 kbps and lower, it was pretty tough to find anything that sounded ok. This is where speex starts to look a bit better (although it didn't fare well in my
    • I have been a proponent of RealAudio's codecs from day one. Especially in the face of WMA and low-bitrate mp3's.

      I'm very informed when it comes to audio, pre/post production, etc...but I never got into codecs, the types of codecs, and how they work. (aside from a basic understanding of the frouenhoffer algorithm).

      RealAudio, seriously, sounds quite good. Alot smoother, far less top-end aliasing, and when you start cranking up the bitrates it just gets better.

      I was always disapointed that it did

Slashdot Top Deals

"Buy land. They've stopped making it." -- Mark Twain

Close