I believe it was the opposite; S3 was lauded due to its wolfson dac (as well as the original Galaxy S), while S2 was condemned for not using that same DAC.
Hopefully all gets measured so we can have some objectivity involved.
Yeah but talking about a specific DAC chip is audio nerds looking at just a piece of the equation and using it as a potential red herring. The analog amplifiers and output, and overall implementation, matter much more than which specific DAC chip is used as long as it's not garbage, meaning basically any modern DAC chip.
i9100 had yamaha DAC and was dismissed by many as poor choice moving away from wolfson DAC which sounded especially great with voodoo sound. As for S3, US variants with Snapdragon did not come with Wolfson DAC
I own an S3 and I hate that it's output has a bass boost profile similar to the HTC One with its "Beats" profile. It's probably great for earbuds but for any other listening device the bass is overdriven. If it just produced a flat output I would be completely happy with it.
Can't it be turned off somewhere? HTC's Beats glorified EQ can be turned off and ignored entirely, well, except for the persistent notification that lets you know that it is indeed turned off (seriously).
Xda devs is full of discussions about things like boeffla Kernel tweaks and hacks which expose more of the hardware controls of Wolfson DACs and thus allow customization of sound EQ.
It does seem to be the case that you really do have to research a specific phone and all it's variants to be sure of getting the audio chain you want.
I'd like to see iPod touch models added to the mix also, as whilst not actually a smartphone, it is in many ways very similar and music listening on it through the headphone jack is a major selling point of it. The thing is, as you don't need to hear it "ring" in a loud room, it has a much weaker internal speaker than any smartphone, so is the audio-amp driving the earphone socket as good as that of the iPhone, or have they saved money on it to hit the lower price point?
I too would like to see this as well. Especially between the 4th and 5th Gen. I swear the audio quality of my 4th Gen is superior to my 5th Gen. I also wonder the difference between the various models of iPhones.
my buddy's aunt earned 14958 dollar past week. she been working on the laptop and got a 510900 dollar home. All she did was get blessed and put into action the information leaked on this site... http://cpl.pw/OKeIJo
This is perfect, I havn't read it all but I have an ancient Nexus S and it has horrendous headphone output. I've heard the next two Nexii weren't great in that either. Lots of hiss from all the radios, and even if there was no hiss there's just an odd EQ curve or something, it makes the bass muddy. I think the program Voodoo sound helped it but not completely. So audio output will be an important consideration in my next phone.
The Nexus S has some of the best DAC in the industry, actually... Most phones these days use the built-in Snapdragon DAC, but the S has a Wolfson DAC which is way superior.
I think you either had a dud or you have another issue somewhere because the S was lauded for sound quality (just like the original Galaxy S).
The captivate and epic 4G (sprint variant) were substantially different from the galaxy s, although I remember the epic 4G touch (S2) having decent audio quality.
So I've heard, but I'm not the only one with shit audio on it, the running theory is they put in a good DAC but didn't bother properly making drivers or something for it.
Or perhaps improper shielding of the audio circuit from the radios, since the static seemed to be radio related. Even modern phones have some of that, even the best sounding ones like the iPhone 5S, but it's much much reduced.
The DAC really means nothing without knowing everything else that goes into the circuit. Yes, the DAC might have superior SNR and THD+N but if you use an amplifier, or a volume control circuit, that are worse than other phones, you've now mitigated that advantage. It's there in the DAC but by the time it gets to the output stage it's been buried by noise elsewhere in the system.
Using a better DAC is nice. But you can't just drop it in and get better results, everything else needs to be engineered around it as well.
Using a higher end separate DAC hints that the manufacturer at least somewhat cares about sound quality. Using the built-in SoC DAC says that it's an afterthought.
It's not a guarantee, but it's certainly an indicator.
Totally agree with cheinonen, any high end audio stereo setup is as strong as it's weakest link. Take the excellent nuforce DAC-100 at ~1200$CAD, use it with a poor sub 400$ multi channel receiver and you won't get much out of it. The nuforce got plenty of amazing review, but it always depends on what it is paired with.
The galaxy sure got a nice DAC but it's amplifier section is poorly engineered.
Actually (and I know audiophiles everywhere will disagree with me on this, but whatever...), you can get audibly perfect sound with a dac worth only a couple hundred dollars at most, and an amp of similar cost. There's really no point in buying a >$1k dac, and the same goes for an amplifier (though expensive multi channel receivers can be justified, not for the quality of their amplifiers, but for the quality of their DSP, room correction, video upscaling, and variety of inputs and outputs). For an audio system, by far the most important component is the speakers. You should spend the vast majority of your audio budget on the transducers (speakers/headphones), then buy an amp with sufficient power to drive them to whatever level you need without distortion/clipping (which is almost always less power than recommended by audiophiles). The rest of the system can really be quite inexpensive - the built in DAC in most modern audio products is audibly flawless, and it's impossible to improve on 14AWG zip cord for the cables (from an audibility standpoint) unless you're running your wires over an extremely long distance.
Now, that isn't to say that modern smartphones have reached that audibly flawless point yet - many of them haven't (especially when they have such blatant flaws as the single channel clipping shown in the review above), but many of them are a lot closer than people realize, and it really doesn't cost much to make a product audibly flawless. I remember seeing testing done on the latest iPod, and it was good enough to not contain any audible flaws. I would assume that the iPhone is similar, and the GS4 looks to also be audibly perfect in the review above (so long as 500mV p-p is sufficient to drive your headphones - this may not be the case for some less sensitive models).
The S4 powers moderately hard to drive headphones just fine - sure, it won't drive something like an HD650 adequately, but the majority of consumer headphones out there will go plenty loud driven straight out of an S4. That having been said, it would be nice if it had a bit more power for hard to drive headphones.
I can't believe how many people on the internet are singing this song... Does it really matter who makes the DAC if the output sounds like crap? The DAC is just one of many components that affect the sound quality. You can have the best components in the world and still produce absolutely dreadful sound. Ask any wannabe audiophile who's dabbled in assembling home systems out of separates. This is akin to saying that your car must be very powerful BECAUSE it has Bosch coil packs --and Bosch makes the BEST coil packs (just play along for the sake of the analogy, I'm not actually asserting that Bosch makes the best coils --I don't know).
This has been needed badly for some time. Phones displacing media players this matters a whole lot. To me far more than actually anything else about a phone.
You also have to remember with some measurements about what is actually audible and what isn't. It's crazy seeing people moaning about nanoseconds of jitter on DACs yet they're quite happy with milliseconds of the equivalent of jitter on vinyl. Then again some of these companies sell ethernet cable at £1600/m and claim it makes a difference to sound ;)
Great to see something like this on anand! I would like to see some older high-end phones compared to the ones we have now - for example the Nokia N900, or maybe even some of the ooold Sony-Ericsson Walkman phones (W810 and up). I'd find such comparison interesting ^^
I was also wondering if it's possible to add some ancient phone with cult following from audio geeks. You know, Nokia N91 and similar. According to some, sound quality on the phones were never better, so that'd be nice to see what is really happening.
YES! Thank you so much for doing this, its awesome. I didn't realize audio/amp quality was a huge thing until I upgraded my headphones. Being able to hear the difference between the same audio file on two different devices made me realize how little manufacturers care about audio quality. Then using an FM transmitter in my car showed me how some devices can't pump out enough power over the headphone jack. My Galaxy Nexus has pretty good audio quality (and does optical audio out through the dock, also awesome) served me fine for pumping up music through the FM transmitter. The Galaxy S3 I'm temporarily using does not though, and produces noticeable clipping at max volume. It also isn't properly shielded and when charging there is interference on the headphone jack (something you should also test if you haven't thought of it). I'm looking at upgrading if I can't fix my Nexus, and hope the HTC One is as good as it in terms of audio quality (with Beats disabled of course).
The One is very interesting in terms of quality. There is a huge difference between the stock music, google play music, power amp, and astro. Astro is the best quality.
You know, you briefly alluded to the single biggest difference maker and a ton of people would still take it for granted... If you're using crappy headphones much of this is wholly irrelevant. At the same time, expensive headphones aren't always an upgrade... For one thing you've got the Beats and Bose of the world getting by on marketing, and even beyond that not every headphone will be a good match to any given phone depending on how hard to drive it is (or isn't) and how out of whack the phone's output impedance is (something that's often taken for granted even with dedicated audio components).
I always thought the reference for audio testing equipment, especially in telecommunications, was HEAD acoustics. Still a good and interesting article.
Great! This is exactly what I've been waiting for.
I've noticed the audio quality on my Droid 4 is sufficiently poor that I continue to carry around an iPod Classic along with the D4. I'm hoping that the sound quality on my next phone, likely either a Note 3 or the next gen HTC One (One Two?), will be good enough that this isn't necessary.
As a mild audiophile who occasionally searches for electrical measurements of DAPs/smartphones on IF/GE/GSMA/Voldemort's Blog/etc, I am very pleased seeing this article in Anandtech! Had a mild laugh seeing the HTC One 'Beats Enabled' frequency response (is the treble peak still there without Beats?), and I'm very much looking forward to seeing some Nvidia Shield/SIII i9300 measurements.
With Beats disabled, you actually see a bit of a roll-off in the bass and the treble. It's +/- 0.230 dB so not a level you'll likely hear. Overall the response is very flat if you turn off Beats. I'm still testing something else I found with the HTC One before I have full numbers up there.
No more than it would be with anything else I would think. For the Nexus 5 I tested two different samples (one from Brian, one from a friend of mine) and both exhibited this issue. So you might see small variations, as you would with any display or anything else, but nothing major I wouldn't think.
Another factor to consider: certain cell companies can enable enhanced audio quality for cell phone calls, but only on selected cell phone models. The IPhone 5 has that ability, as I recall. Not sure about the specifics, but I believe that both ends of the cell call must be using supported phones, as well as: the cell provider must enable that feature. That would enable cell calls to have better voice quality than land line calls, via increased audio frequency response.
It's not useful to measure noise/dynamic range at the highest volume setting. You have to measure at one of the lowest settings to determine whether the phone truncates bits, whether the noise floor does not decrease with the volume setting, etc.
I have an htc dna and it is completely, totally, utterly useless for playing music with sensitive IEMs; I suspect many android phones with "-90db" thd+n measurements are similarly bad in practice.
We have stepped output level charts as well that measure this, they just aren't included here right now. I can start to pull those out for current and future tests if we want to use them.
I saw that one graph included when you compared the Nexus 5 and G2 at different levels and I think it should be in all the reviews. I always run my IEM's in the lower half of the volume range so I am quite interested in how they perform. As others have said, excellent work!
Same here -- I always run my good IEM's below half volume at least (usually more like a third.) No point in having good headphones & music if the volume is turned up so loud it hurts....
BTW Great job Chris! I have always wanted more in-depth reviews about smartphone audio quality. It's very important to me, but so far there haven't been any reliable/objective tests available. Thanks! Just a thought, but maybe it would be possible to test against a pro-quality amp/DAC? When I use my GNex, the quality is obviously very different from my audio interface that I use with ProTools. It'd be interesting to see just how much of that is measurable...
That would be great. I believe most display tests are run with the same light output to have more reasonable comparisons... I would like to see the same principle here. As a curiosity (or a warning) it would be good to know that above certain levels the sound will deteriorate.
I would love to see the distortion of the apple on a standard headphone (16 ohms). They have notoriously bad sound quality all they way back to the first ipod. Apple is a profit company not a quality company.
I would also like to see a real comparison of battery life on apple products. Like battery life VS screen size in SQ inches. The screen is so small who cares if it can brows the web for 10 hours if you cant see it.
Also frame rate VS pixel count. Once again I can refresh a 1 pixel screen at 1Mhz but it does not do me any good nor does a 1M Pixel screen at 1 Hz but the spec is the same 1M pixels a second.
I am not an Apple evangelist having only owned Android (and WM and Palm) smartphone devices. So with that said, this is a truly ignorant statement. Apple has led the way on several different design and hardware fronts, including display quality and size, camera quality, GPU power, etc., and it's not clear that without their leadership there would be the kind of robust hardware competition there is in the PC and Android space.
"They have notoriously bad sound quality all they way back to the first iPod"
Sure, Bob, whatever you say. Of course, there are authoritative sources that disagree with you. For example: http://www.stereophile.com/content/apple-ipod-port... which summarizes by stating, "Excellent, cost-effective audio engineering from an unexpected source."
I think I'll take Stereophile's word over yours, Bob…
deasys hit the nail on the head Bob---Not sure which iPod/iOS device YOU'VE hear/listened to/had experience with....but as an absolute audio nerd, I can assure you decent audio files sound excellent....my favorite cans right now are the B&W P5s currently, but I've also got 2 pairs of Grados, Sony MDR 7520s, and Sennheiser HD800s for our studio mixing (with B&W Nautilus 802 speakers and Focal SM9 studio monitors). You couldn't be more dead wrong about Apple's sound quality. Perhaps you need to find a new way to 'rip' your music or quit listening to low bitrate MP3s to judge sound quality? As far as screen/battery life----WTF does that have to do with this incredibly extensive, exhaustive sound quality review and comparison/contrast between three Android phones and an iPhone? Bone to pick, eh? LoL----Doesn't matter does it....regardless of the article, review, discussion----always SOMETHING to do with Apple isn't it?
The devil is always in the detail... Unfortunately deasy, the detail you failed to pick up on is that the 'stereophile' tests were performed into the mackintosh powerbook 'line in' impedance and so did not uncover the limitations of the analogue output stage when listening via headphones.
You're dead right Bob (whatever the other numpties say). The problem with the ipods (except for the first gen iPod nano, which was superb) is down to an underpowered analogue output stage. There is no problem when driving a high impedance 'line out', but it results in clipping and poor bass response when driving (low impedance) headphones. For those of us listening via headphones, the most crucial objective test is performed when driving into a low impedance. If anandtech do the low impedance tests, you will be vindicated for identifying the shortcomings of the apple devices. And the numpties who know no better will eat humble pie.
I commend you for taking this initiative. I agree that audio testing by sites like Anandtech could eventually lead to phones with better sound quality, which is something that has been neglected so far.
However, I wish you tested the audio quality of phone calls as well. Cell phones are phones, after all. It doesn't look to me that these tests measure the ability of a phone to selectively capture the spoken voice in a loud environment (without omitting the first syllable of a sentence) and reproduce the voice reasonably accurately and reasonably loudly through its speaker, which is what a cell phone must do to function as a phone in real life.
If no objective and reproducible test currently exists to do this, why not invent one?
As for me, I have an S3, and I find the speaker volume barely sufficient to hear the caller's voice. I consider this a significant and entirely unnecessary weakness of the S3. A phone's ability to carry a conversation is far more important than its competence as a camera or music player.
Those might be your priorities, not necessarily everyone else's though. I can tell you I'd be much more concerned how a phone would fare with music than with calls... I use maybe 200-300 voice minutes a month but I probably spend at least twice as long listening to music/podcasts on my phone (if not thrice as long).
In the same vein, I couldn't care less about the camera as long as it's usable enough for basic stuff like snapping a pic of something as a reminder while I'm at a store... I still use my pocket camera or my micro four thirds camera for any picture of any moment I'd truly like to remember.
However I KNOW that's not a majority view and for many many people a smartphone is now their primary camera, so I can appreciate the efforts Brian puts towards evaluating those. I'd imagine that anyone who speaks a ton on their smartphone probably uses a Bluetooth device and I'd bet that's ultimately a bigger factor in call quality (along with the network).
Not saying it wouldn't be interesting to test mind you, just adding some perspective.
Aside from lacking more precise test methodology, and having way too many variables compared to reality to make the tests reliable in reality - such evaluations are already fairly regularly performed during reviews of the device.
Do keep in mind most carriers will not support frequencies outside the 300-3400mHz range; inadequate for a decent voice conversation.
If you are concerned about voice quality, your first priority should be to get the carriers to support wideband audio. Without it, the phone manufacturers themselves can't do that much.
Apple only introduced it with the iPhone 5, but others like Nokia have supported it in their phones far longer, even prior to the release of Windows Phone (7), including support in virtually every device they've released. (Whether its a low end Asha, Lumia 500/520, or a high end 900/920). Even the popular Galaxy 2, 3, and 4 support it - so there is no shortage of devices with the capability.
I'm curious how well your Grado headphone's are holding up? I bought a pair of SR80's for use at work last winter; but the wire started to develop damage a month or two ago. If I move the wrong way while wearing them I can get brief bursts of static in one ear, and can mute that ear by pinching the cable just above the Y. I suspect the damage was caused by the post in the headband allowing the earcups to spin freely, combined with the unmarked cable making it hard to notice anything less than a half dozen or so revolutions of twist. I'm wondering how much of this is bad luck on my part vs poor design/manufacturing.
I had similiar concens with my Grados (actually mine are Allesandro ms1i). I sent grado an email suggesting that they put a stripe on the "Y" wires so that you could more easily straighten them out. They said that they would pass it along to their design team. Then they sent me a free T-Shirt. Classy company for sure.
Quite a good initiative thanks, it is too hard to get these numbers nowadays.
I would suggest you guys build up a database over time of phones performance (see headphoneinfo awesome job for instance). I also suggest that you add to your test the maximum output delivered (power or voltage swing into load). This is interesting, because if a phone clips at high volumes, but its output power is 10dB above the others in average, then the normal user will simply not see the drawback (altough I admit this is initially poor job from the company in tuning the audio system). It also helps to chase the brands which deliver lower output power, that can turn to a problem on more demanding headphones (high impedance requiring higher voltage swing). Some users will fancy some extra power on their headphone output (even if this might not be safe for their ears). Last point, some high-end IEMs have quite low impedance, that demand fairly high current specially in the high energy low frequency, creating bass roll off. A simple frequency response check on a low impedance IEM would show this.
I like the idea of audio testing, but I am disappointed by the methods used in this article: why would you bother testing a device at maximum volume when you know it is clipping badly? You should reduce the volume to a setting where it does not clip and then continue the review. You can then report the maximum useable volume setting on the device.
The maximum volume on an iPhone is reported to be in excess of 100 dB. Listening at this volume for even a short period (15 minutes) on a consistent basis will permanently damage your hearing. Why not test these devices at reasonable volume levels? http://www.theatlantic.com/health/archive/2011/12/...
(hopefully not too flawed analogy follows...) If you are comparing two overclocked computers for maximum performance, you set them to their highest stable clock rate and then benchmark. You do not set one to a clock rate that causes continual crashing, and then report that it failed several of the benchmarks. I think this is comparable to audio review for the clipping cellphones. You might argue that the device should support any user-accessible volume level, but historically it is very common for audio amplifiers to allow users to adjust the gain until the output clips. Apple is an unusual case that limits the user to only access non-clipping gain settings.
audio systems are tested at max performance (there are many reasons for that, including the fact that when you sell something, all usage range of the system should be good), so analogy with OC is not ideal. I agree with you though that testing at nominal volume could help, as an adder only of max volume testing
Like I said in my previous comment, it is common for audio amplifiers to allow you to adjust the gain past where the amplifier will start to clip. You should never expect a car stereo or home theater amplifier to allow you to run at maximum gain without clipping, so why should you expect a phone's headphone amplifier to behave differently?
The proper way to run this test is to adjust the amplifier to maximum non-clipped gain and then run the test.
The day has finally arrived? Good data with some surprising results. I think I'm mostly surprised at how well all of the devices perform. I think dynamic range is perhaps the most important test here simply because most people won't be listening at max volume on headphones and pushing the noise floor down as low as possible is important for quiet listening.
Were these tests done on the AKG K701? That is well known as a difficult to drive pair of cans without an amp. If a phone can drive those loudly with good measurements then it's certainly good enough for anything I'd use it for. Testing should be done worst case and if there's time more typical cases. When using my phone as a line out I'll typically leave it 3 steps below max because I expected there to be output stage power issues (seen as dramatic clipping on LG's stuff :x) on my phone. Any lower and as you noted the static noise floor lowers the SNR.
I was a little surprised at the weak channel separation in the otherwise amazing iphone. Channel separation is already a p big issue. Even with expensive headphones it's easy to test and ballpark a crosstalk of worse than -60dB by ear just from the jack to the drivers.
I'd like to make a request for some data of testing devices (1 iphone and 1 iconic android per year?) going backwards to see a progression (or maybe lack thereof) of audio quality in smartphones over the past 4 or 5 years.
Curious how the test result gonna be for XPERIA Z, ZU, and Z1 against Lumias :) and...can it be used to test ASUS Xonar Vs Onkyo soundcards or other audiophile soundcards...
It can test anything. I use it to test Blu-ray players, preamps, amps, receivers and more. The report it spit out for a receiver for me today was well over 150 pages.
Testing some other gear might be interesting context wise... i.e. How does a smartphone compare to a Xonar DGX or STX, or to some of the cheaper amps out there (O2? Magni?). Adding stuff like the venerable SanDisk Clip Zip might be even more relevant as far as comparisons go, since that's a great $30 solution for anyone with a phone with disappointing audio.
Even better than a SanDisk Clip Zip is the older SanDisk Sansa Clip+, which can be found quite cheaply. Here's what will give you a fantastic audio experience: 1. Add a 32gb microSDHC card to the Clip Plus with your music encoded as flac (the Clip+ supports flac [lossless] playback. 2. Install the excellent RockBox (http://rockbox.org) replacement firmware. 3. Add the superb FiiO E6 headphone amp. Note: beware of counterfeits on eBay! 4. Use decent or better headphones (at work I use Grado SR60; cycling I use various good quality earbuds). 5. Enjoy your music as you've never heard it on a phone or iPod.
I have a Nexus 5 and when I plug headphones in I usually listen to level 6 or 7 (out of 15) and have found the audio quality to be to my liking (vs. my old Samsung Galaxy S2).
However, typically when I listen to mobile devices via an amplifier I would turn the device up to MAX volume and then modulate volume via the amp - this does not seem to be the best case for the Nexus 5, I guess stop 12 would be the best volume.
Is amplifier clipping a common occurrence for mobile devices? I consider this a design flaw, i.e. max volume available for the device should be prior to any clipping.
To be honest i have heard many smartphones through one of the best earphones and headphones i dint like the sound quality from any of the smartphones that i have heard compared to the ones that my mp3 players deliver. This is where i hate my smartphone and still love my media players.
I hope we get to see how the Xperia Z Ultra performs even though I have not seen a review for the phone itself so I guess not much of a change there :/
This article is fantastic. I love that you are going to be testing audio quality. Obviously testing anything takes a lot of time, but I think it would be nice to see how these phones perform at "reasonable" listening levels. It would be great if you could pick dB level that is pleasant through the apple earpods and test a 2nd time at that, sort of how you use 200 nits for panel tests. That would give a more helpful representation of performance. Another thing that I would really appreciate is an explanation of how bad (or good) performance needs to be for most people to hear the difference (with bad and with good headphones). Thanks for the piece.
Can you add built in speaker testing? I have an HTC one, and while I prefer to turn beats audio off when I use my phone in the car (listening to music on the car's speakers), I much prefer the sound of the built in speakers with beats audio on. As others have pointed out, all of the pieces in the chain are important to sound quality. My guess is that the frequency response of the built in speakers is much less at low frequency, and the beats audio increase in gain shown in your chart (below 100Hz) is designed to compensate for the frequency response characteristics of the built in speakers. In other words, if you tested the frequency response of the built in speakers (using a microphone with a known linear frequency response to gather the data), would the curve be much flatter when using the built in speakers with beats audio turned on versus turned off? Would the HTC one with beats audio on in fact have a much better (flatter) frequency response through the built in speakers than other phones do through their built in speakers?
It would be very useful to be able to compare the sound quality of the whole sound reproduction chain including the built in speakers – for me, the built in speaker sound quality and volume level is a major discriminator between phones – right up there with display color accuracy and camera quality.
Excellent initiative. The HTC One frequency response looks dreadful. I'm looking forward to graphs with beats enhancement off.
As the next step, I would really like to see where different types of audio devices compare to each other... smartphones, ordinary media players (ipods), audiophile media players (HM-801), entry level/top level PC sound cards, realtek onboard chipsets.
I'm not sure if it's necessary to go through the effort of including PC soundcards, motherboard solutions, etc., but including media/MP3 players would definitely be a nice addition. The place that mobile phones have now as media consumption devices in addition to being phones came as an evolution from the original media device.
However, the media device, given its singular purpose, prominently had sound quality as a major review component, and consumers did look at the DACs and opamps used. When they evolved to all-purpose smartphones, suddenly there were a lot of other features that took precedence, including processing power, display quality, etc.
Now that AnandTech is shepherding in a new look at audio quality, comparing the state of mobile phones to the heyday of media device would definitely be useful.
I'll second this and dishayu above, it'd be a great reference.
Hell, if we could get you to test output of a few BT receivers out there that might also be interesting and valuable to readers... I know BT's another can of worms as it introduces more compression etc, but it's actually gotten quite decent and those receivers end up replicating (and thus bypassing) many of the same components in phones.
There's not a lot of in depth reviews out there for those things but seeing as you can use them with any pair of headphones they'd fit right into your testing and they can be a suitable solution/alternative to a phone's line out. Something like the older Sony MW-300, or the newer models, or three equivalent Samsung/LG models.
Great article, though I have a quibble with the headphones to be used for future testing... Nothing against either as I've owned both, and the SR80 are a fine candidate (common entry level open headphone recommendation). The AKG have a reputation for being hard to drive and they aren't exactly very portable anyway, plus they represent another dynamic open headphone. I think a popular IEM (Ety, Shure, even Apple's dual driver IEM) might be a better second alternative, if it's at all possible with your rig... I know IEM are a pain to test due to seal issues and whatnot. Failing that maybe some other dynamic headphone that's more likely to be used with a mobile device (V-moda M-80? Senn HD25-1 II?), or something cheap yet extremely popular and good for the $ (Koss Portapro or KSC75).
Also, if you could test and report output impedance for the phones that would be a HUGE help for people trying to figure out what kinda headphone would work best... Relatively high output Z isn't uncommon and it can wreck havoc with some lower impedance headphones, particularly sensitive IEM (and specially the multi balanced armature models that are so prevalent now).
I do hope you consider providing the audio tests in the main review on the day it's published, as opposed to tacking it on later. I usually read reviews only once, i.e. on the day they are published and don't keep returning to individual reviews looking for updates, so this would be a major data point readers like me would not be able to take advantage of.
I wouldn't hold my breath. Chris H isn't a typical smartphone reviewer. As a result getting these results at initial launch time would require either buying additional sets of test equipment for the reviewers, buying an additional phone for Chris H to do audio testing on, or delaying the article to ship the phone to Chris H after completing the rest of the testing work.
Audio precision won't let you see pricing information without creating an account on their website. That suggests it's painfully expensive and that getting multiple copies of the hardware won't happen. Getting multiple copies of the phone isn't cheap either and is probably not going to happen except perhaps for a few halo devices. With the peanut gallery raging about any reviews that don't make it out on release day, I'm doubtful that anandtech would choose to delay reviews for a few days for a specialized test.
Ehh, I agree, if it's not realistic to have this testing the day the review is out it's no big deal... If it's a deal breaker for you then you'd wait the same amount of time either way, and if it's not (probably the majority of readers) then there's no point in making the rest wait.
"Chris H isn't a typical smartphone reviewer. As a result getting these results at initial launch time would require either buying additional sets of test equipment for the reviewers, buying an additional phone for Chris H to do audio testing on, or delaying the article to ship the phone to Chris H after completing the rest of the testing work."
Having this on the day-of is going to be a challenge for a number of reasons.
- Brian is in Arizona, and I'm in Oregon. If there is only a single review sample, I have to get it from him. - As mentioned, the Audio Precision is ridiculously expensive. I think the APx 582 used starts at $19,000 before adding the HDMI, Bluetooth, and Digital modules I use (I need it for receivers as well). Audio Precision is just a few minutes from my house and they've been nice enough to let me come in, test everything there, and endlessly bother their QA people to get this right. However, as I have to come in I have to schedule that, and it takes time. - That also makes it far easier to do a batch of these at a time than one at a time. If I had at APx at home it would be easier but right now that's not possible.
So we will try to get all the data, as fast as possible, into the system, but day-of is going to be a logistical challenge. I'd rather have it be accurate than be fast.
OK, I shouldn't say ridiculously expensive. However, the instrument we use costs enough that it's not feasible for us to have them for myself, Anand, Brian, and everyone else that needs one for testing. The Audio Precision gear ranges from $6K to $50K+ depending on what you need and the price still means we can't outfit everyone with one. So testing will happen as fast as possible, but likely won't run with the reviews when they are initially posted.
This audio bench is a tool of truly significant value to anyone hoping to arrive at the best, most fully informed purchasing decision possible. Given the respect & high regard that AT has earned throughout the tech sector, this audio initiative raises the bar for more than smartphone manufacturers alone. Along these lines, does AT have any plans to initiate a similar audio bench for the various motherboard lines that have recently implemented enhanced audio capabilities? These are very positive steps toward driving advances in fundamental, yet long neglected platform capabilities. However, as improvements in audio reproduction are realized, there will still remain one critical, & seemingly intractable obstacle to overcome; What's it going to take for the recording industry to give up on compression & adopt a regimen of decent mastering.? It'd be a shame if the only thing audiophile-grade tech revealed was just how badly most studios butcher great music through compression & lousy mastering.
If I didn't say this on one of my previous comments I'll say it now, thanks for doing this! Audio quality is far too often ignored, specially amongst the PC/tech enthusiast crowd... And it's rarely tested very objectively when it's talked about at all.
I wouldn't mind some subjective impressions atop the objective testing though... Or even some more commentary on Chris' part regarding the data itself. Knowing the Nexus 5 is clipping at max volume is one thing, but the reader might not necessarily realize it's not an issue at lower volume levels etc.
Great review, I'm looking forward to further tests.
A few suggestions: - Such graphs are nice for a detailed analysis but useless for an easy comparison between different smartphones. Find a way to break down the important information in those graphs to one or two numbers, which you then list in a bar diagram to allow a comparison across different devices.
- As you pointed out yourself and also other commenters, testing THD at the max. volume might be industry standard, but it's useless for a normal consumer, especially, again, in such a comparison. As in the display tests where Anandtech adjusts to a fixed display brightness across all devices, you should do the same here. Because different headphones require different volume settings you might chose three settings: A really silent one (fixed dB), a normal one (fixed dB), and the loudest possible (max. power the device offers). For example on my HTC Desire I use Sony in-ear ear-buds which I drive with the lowest volume setting possible, else it's simply too loud for me.
- Those three volumes also have the advantage to go in detail in specific areas: The loudest volume setting can be used to determine how much load the smartphone can drive and the consequences (just what you did with the Nexus 5, excellent). The normal volume setting is a measurement for overall audio quality across all devices, because that's the one most people will use. So there a focus should be kept on dynamic range, frequency response, distortion, ... The silent setting is to determine in detail the background noise. On my HTC Desire noise is audible with the Sony in-ear ear-buds (not with lower end normal ones). And as you said, if the device can output a lot of power, naturally the noise to signal ratio becomes smaller. But that's artificial and a useless measurement when compared to other devices. So keep the volume at a uniform low setting and measure the noise to get comparable and meaningful results across all devices. Also make sure to include some noise measurement while in Airplane mode and while transfering some data over mobile. Then you can judge how well the analog part was designed.
Problem is, the dB values Chris would use would often not correspond with the same volume (and power output) level you'd use, because you're using different headphones with different sensitivities. Settling on different power levels might be more correct but it'd probably leave a lot of people scratching their heads... It might be worth taking a looksee at the way Tyll @ InnerFidelity tests amps and headphones. He's been doing objective tests for quite a while and has found a pretty good balance as far as how to present the data in an easy to digest manner.
Having a couple different fixed power output comparison points should make not matching your exact listening volume less of an issue. At least it's a less arbitrary measurement point than the point where the manufacturer decided to put a virtual stopper on the volume knob. The fact that max volume clips with a specific set of headphones shouldn't matter too much. If the headphone amp is clipping due to limited current available then using higher impedance headphones may be able to use that volume level without clipping. And if aren't able to use it, just don't crank it up that high and problem solved.
Finally I waiting for this for years. Simple and probably stupid question, will you include Windows Phone smartphone? Many thanks in advance for these tests.
Is this novel - or just getting up to par? From what I recall, GSMArena and Mobile-Review have been doing detailed measurement of phone audio performance for years.
I had the HTC One and it played much louder than the G2 without distortions on the same headphones. HTC One is in many ways ahead of newer phones like the G2 or Z1.
However, I don't think the "issue" with the Nexus 5 was explained properly. What is happening is that the volume control on the Nexus goes high enough that it's possible for the amplifier to run out of power against the specific load. This is perfectly normal, and the practical implication is that the maximum power of that headphone amplifier is lower than your published measurements - the result should be at <1% THD.
Also, all results should list the load against which they were measured. If you allow massive amounts of distortion and don't specify a load, it would be easy enough to claim that a 100W <1% THD into 8 Ohms rated amplifier is "discovered to be able to produce 1000W" - just as long as you disregard it being driven into a 2 Ohm load with over 50% THD.
A metric that might also be of use, and practical in predicting a headphone amplifier's real-world performance, is output impedance. Smartphones and tablets are usually used with low impedance, sensitive headphones, and if the output impedance is relatively high, it can affect real-world frequency response massively - sometimes similar in scale as the EQ you pointed out in one of the measurements. Of course, providing measurements made against a range of headphones with different impedance characteristics, as you seem to intend to do, will point at the same issue if there is any. But in that case, please provide impedance curves of the reference headphones.
Ehh, impedance curves for the headphones he suggested (Grado SR80, AKG K701) are easy to find (Inner Fidelity and others have decent databases)... Testing and providing output impedance for the phones would be very valuable indeed though, even if everything else in the chain performs alright that alone can affect the FR significantly with one pair of headphones and not at all with the next... And unfortunately there's never been a realistic standard for output impedance, (other than high quality solid state amps now aiming for >1 ohm), and it's often all over the place.
I agree with this. I think of three main usage scenarios regarding the headphone output: 1.) Quality of HP output driving a high-impedance line-level input for a home or car stereo. 2.) Quality of HP output driving low-impedance, sensitive IEMs. 3.) Quality of HP output driving high-impedance, non-sensitive cans.
For #2 and #3, the output impedance should be known, as well as the amplifier power at a given THD level. There reasons why the iPhones are able to perform well with IEMs, namely low noise floor and low output impedance. Also, iPhones perform well with larger cans because of the decent output power. I think the usage cases I listed above are pretty common, so I think a good approach to testing is to think about the most important parameters for each usage case. BTW, I am excited about AT doing these measurements- very good news, indeed!
I use my phone most of the time as a music player. Audio quality and storage capacity for flac files are major factors in my buying decision. I hope we can have more widespread information on smartphones audio performance
If you actually cared about scientific measurements of audio performance you would use compressed audio instead of flac. ;) Given a decent amount of bitrate, compressed audio is indistinguishable from uncompressed in double blind tests. In my experience FLAC is mostly about the listener feeling good about getting the "correct" experience, like expensive speaker cables.
There's a valid usage case for FLAC as far as ripping and archiving IMO, you might as well if you're ripping a large collection (or ripping often)... Since you can quickly re-transcode or edit files w/o a loss of quality... But yeah, I don't see why anyone would put FLAC files on a phone, transcoding is dead simple and super quick if you have a remotely modern PC. Managing FLAC & MP3 playlists or whatever shouldn't be a hurdle if you're putting the effort to maintain a FLAC library to begin with, just use the MP3 library for everything or use stuff like Media Monkey's smart filters/playlists.
The biasing of the amplifier in the Nexus 5 and LG G2 left channel appears incorrect. Note it is only clipping on the negative portion of the waveform.
Could you compare the phones to an iPod maybe? This way we would've an comparisooon to a mediaplayer where there's no 3G/4G/LTE disturbance.
Oh, and for the general audience discussing the DACs etc... The DAC isn't the cruicial part, never has been. The amp is what it's all about and how good or poor it's powered. There's a reason why audiophiles still use tube-amps, or atleast digital amps with high quality toroidal transformers and good shielding to reduce noise distortion etc.
Oh, and btw... A good mediaplayer needs a microSDHC-slot or the possibility to use an USB-stick. A mediaplyer is no good if I can't carry my whole music-library with it (100+ GB).
Hi Chris, very nice to see smartphones getting measured in terms of audio performance, as many of these phones are used as a music device as well. Personally, I think that an inclusion of output impedance measurements would really great be though, as it is one of the factors that would affect the earphone performance rather significantly.
Very good start. Poor quality needs to be exposed and you've done this with the Nexus 5. I would like to see output impedance since low output impedance is a very important quality of a good headphone output.
"Here we see that Beats is adding a +3.5 dB boost from 60 Hz to 90 Hz, but the deviation from 0 dB goes from 30Hz to 300 Hz. Past 6.5 kHz we also see a rise in the treble."
And people pay 300 bucks for a headphone that does exactly, and only, this. Its a good joke, really.
Check out NwAvGuy's blog if you haven't already. He appeared and disappeared a few years ago bringing with him a headphone amp design and (more importantly) a breath of fresh, objective air in testing audiophile headphone equipment. He has some good data there and comparing testing methodologies might be insightful for things to try here on anandtech.
I appreciate this article, Chris. An improvement would be a summary table of all the models compared on one page, and some sort of analysis beyond the "poor performance" comparison. I've also seen data that the specs will change significantly for worse when headphones are attached to the phones.
Finally some in-depth audio anlysis. I've been waiting for this as I actually use my smartphone as a phone (I know shocker in today's age) and media player most of the time.
Awesome! I have been wanting to see some audio analysis done for a while now. Is there any chance of extending this to test audio on motherboards, sound cards, and laptops as well?
1. Include an output impedance measurement. If you're using these as decent-quality audio players with headphones, this is one of the most important things to know for certain headphones, at least. Who cares what the measured frequency response with a resistor is if your source's output impedance is causing +/- 5 dB swings in response for some balanced armature IEMs?
2. Standardize THD+N tests to a given output level for all phones (say 0.1 V or whatever else; the danger is picking something standard like 0 dBu that some phones could possibly not even reach). Don't just use whatever the max volume is, especially since that's into clipping territory for some phones. People don't scale their playback levels by how much power the electronics is capable of handling. I hope. Referencing a fixed level is more fair.
3. Please do keep reporting which phones run into clipping (and at what load) at volume settings at max or less. Also what some nonstandard settings like Beats Mode do.
4. Make careful distinction of THD with headphones as load and as not. If not loading with headphones-level impedance that is mostly testing the performance when hooked up to say a speaker system with a patch cable, which I don't think many people are doing these days.
5. When reporting maximum level, standardize to point of say 1% THD (or max volume, whichever comes first). Also note if headphones are used or not. It'd be meaningless to quote maximum levels far past the point of clipping.
6. If you have time, see if you can coax and measure some bad behavior out of the phone by using Wi-Fi, Bluetooth, LTE, etc. simultaneously, maybe some CPU/GPU load and seeing if that causes audio issues. Honestly, glitchy or cackling playback are far greater issues to audio quality than looking at 0.3 vs. 1 dB dips in frequency response at 19 kHz or something like that. Or output power levels most people don't need.
I allso agree. Maximum volume level is not the same as guality, you have to find measures that are relevant in normal listening volumes. They vary yeas and depends a lot of used headphones. So I would allso recommended of using headphones with different impedanses! One pair of high impedance hifi headphones. One pair of low impedance hifi headphones and one pair of high guality (but commonly used) low impedance headphones. Otherwise it would be like testing 780 triple sli setup using 320p resolution mini monitor... (Well actually not likely because phones have not been very high guality sound sourses, but I think that you got the point.) Good sound qaulity is important factor to me, so I am very eager to hear more (pun inteded ;-) about these test and allso the results!
As others have stated, and I will reiterate, Only Anandtech! Bravo!
I would love to see a similar article comparing HD Voice for the carriers. I know that test may be more difficult to conduct, but ever since Sprint hyped their HD Voice and didn't deliver, I'd like to see some real engineers do the topic justice.
Congratulations AnandTech, yet again you show the rest of the review industry what they should be doing. Keep up the good work and it should be fascinating to see the full set of results. Thanks.
Is there any comment on the practical issue of noise at low volumes? Noise floor testing is uncommon but phones are noisy and are typically listened to with sensitive buds. Who cares it'd the dynamic range is 100dB if you never turn the volume up past 30%?
I disagree with you testing all phones at maximum volume level 1) the max volume differs, some companies may reasonably make the choice to let quality suffer while volume improves. at the least volume needs to be reported alongside these quality discussions. Preferably it would be set to a base volume and tested there.
Amazing article. I have always been impressed with audio quality on iPhones and disappointed with GS2, GS3, Atrix, Lumia 900 and now OG-Pro. Really shows how much attention Apple pays to the products they build.
Nice to hear you're taking sound semi-seriously with devices that have a semi-important audio task. At least for those who use smartphones for aural communication and music playback.
Nice to see the Audio Precision in action. Great unit. However, please, consider putting a little more thought into the scaling of your FR graphs. What appears visually as a huge peak in the htc Beats graph barely has a 3dB rise. That's not a huge peak.
A 1dB rise at 10k is not much of a rise and since it doesn't display enough data, we can't tell if it's a low Q event or not.
The Audio Precision can scale the vertical axis to make plots more informative rather than sensational.
Great article, would it be possible to test audio input (the mic) in this much detail. Noise cancellation as well as how it does in speakerphone mode / video recording. I know earlier reviews / articles have mentioned this are but it would be great to have detailed numbers like this article.
Chris, fantastic article, way to bring me back to Anand. Please include results of the Iphone 4s and any other phones you get your hands on. My girlfriend and I are phone whores, she's Apple, I'm Android. I'm constantly playing with all of our phones and cans trying to find the best audio combo.
I run Sennheiser HD-25s, Sennheiser Momentums, V-Moda Crossfade M-80s, and Koss Porta Pros (Awesome since the 80s!).
Best sounding phone I've ever heard: Iphone 4S sounds way better than the new Iphone 5. It's definitely louder and fuller. Too bad the 4s doesn't have APT-X codec for Bluetooth.
Runner up: Samsung Galaxy S with Voodoo sound Rom (Note, you need to load a non-standard rom to make it sound great)
The rest: SGS2 (International Model): Too quiet, poor quality, disappointing dynamic range.
SGS3 (North American Model): Poor quality, lots of cross talk, disappointing dynamic range.
HTC One (Beats Audio OFF): Sounds great, really no complaints. A step above most Android phones. Still miss the Wolfson DAC though.
IPhone 5: Still sounds great but it doesn't live up to the 4S. I'd say it sounds slightly better than the HTC One but the difference is really marginal.
Nexus 5: Sounds better than the SGS2 or 3. The V-Moda M80s sound good because they aren't hard to drive. The Sennheiser HD25 needs the top volumes and the clipping is obvious. The HTC One sounds a little better.
Wonderfully well thought out and written article. Thank you! FFora future article my one request would be for a llcomparrison of all phones on each test parameter instead of only comparing 2 or 3 on some metrics. Again, thank you!
Thank you so much for doing this testing. It would also be very helpful if your wrap up section did some side by side comparisons, kind of like they do on Consumer Reports. It's very hard from this article to tell which one comes out on top. Also, it would be nice to see an audio/phone quality report on the HTC One since it is also one of the most popular smartphones available.
In my opinion, audio quality testing is a very welcome adition to Anandtech. I am maily a classical music listner. Good dynamic range, wide and flat frequency response and capacity to respect all harmonics present in recordings are very important for classic music. Can you broaden your testing to in order to give an idea of what smartphones are more capable with classical music?
A major use case for me and my phone is as a spoken language playback device through a speaker (not headphones). Be it podcasts or audible books, a good percentage of my "audio" listening on my phone is via powered "stereo" speakers plugged into the headphone port. Often with the volume on the phone at max so the powered speakers have more range especially if in the garage doing noisy stuff or taking a shower. What if any weight should be placed on these results for such a use case? Additionally, I've historically found phones unable to provide enough power for the various headphones I use (currently Klipsch S4) when using them with mowing the grass or such. Thus I have added a small personal audio amplifier for some uses of my phone and listening to things. In that case I usually have the phone at mid or lower levels and control most of the sound at the amp. Any thought to testing some of the more common portable audio amplifiers?
Anyone know of an app that improves the call quality of the Galaxy S4? I know about Adapt Sound but it doesn't give my phone good phone call sound. Thanks.
What a good article. I'd love to see this data for sound cards vs integrated, and MP3 players, using only high quality reference headphones (or speakers).
From the stepped response, it seems that they are all using 16 bit DACs (16 bits would give you close to 96dB of dynamic range if the only source of noise was quantization for a signal at full power).
Can you confirm this? Does this mean, that having 24 bit encoded music is simply wasteful on a mobile device?
24 bit encoded audio for end-user listening purposes has never been shown to have any advantage or difference from 16 bit in double blind tests. The potential advantage of 24 bit DACs lies in being able to use digital attenuation for volume control without losing any dynamic range. However, with proper dithering after attenuation, you have quite a bit of leeway even with a 16 bit one before it gets audible. So, don't worry about that spec!
Thanks Anandtech, Nice for doing these reviews, one of my top reasons for choosing a smartphone is audio quality, and im not talking about the voice of my wife, but the music reproduced through earbuds. Please keep going testing at full volume not only for those who use them with an external integrated, but because that's the right way, if manufactures bost the volume and allow clipping than thats a flaw, also if a phone measure good at full volume than its super good at low volumes, you also can measure them at a specific sound pressure levels, though the difference should be small, but anyway these are not reviews for the casual listener, and Yes unfourtunately iphones sound very good. I also would be interested in measuring some old and new mp3 players, though in that case its different to be objective, because for a n audio player SQ is a major selling point, but I dont see Lumia selling more than Galaxie because of better SQ. If you cant find any old mp3 player, we are crazy enough to send them to you, hoping to have them back once tested. Thanks
I appreciate the sophistication and thoroughness of this review. But I have to say that I have never had a smartphone where I've thought, boy this audio quality is unacceptable. Especially with earbuds or plugged into an external system. Unless you're an audiophile, they all seem pretty good to me.
Speakers on smartphones are obviously another story.
But to me the real issue is call quality, including earpiece volume, sound quality of the mic (for one's caller), and especially noise cancellation. Anandtech, thankfully, covers noise cancellation in its reviews with a meaningful babble track test. But I wish there were still much more focus on call quality. There are real differences when it comes to call quality, even amongst flagship phones, and this is an area in which, on a daily basis, I experience frustration. It's amazing how much the "phone" functionality of something that is after all phone is ignored.
I agree. Excellent article. I hope to see more like it and on the subject.
I wouldn't mind the "sensational" scales so much if they were at least used consistently. It was difficult to compare for example the THD FFT responses by tabbing between them in browser windows because the scales aren't the same.
While the max volume should really be set correctly, it would nevertheless be interesting to see the THD FFT for what SHOULD have been the max volume (i.e., a step or two below the G2). I have to say I'm a lot less excited about getting a Nexus 5 now - the Galaxy 4 looks much more appealing in spite of the significant price increase. These results even had me consider getting an iPhone for about half a second.
Sorry, but shouldn't you be using lower impedance headphones? The grado is rated at 32 ohms and the K 701 is rated at 64 ohms. The grado might be ok for portable electronics use but the K 701 is more difficult to drive than the 64ohms it is rated at. Not only that, but they are both open headphones which leaks tons of sound and not prefer for anybody using a portable electronic as their music source.
I think you have to inform the reader the difference between a closed and open headphones(assume they don't know the difference). I see people purchasing a HD600/650 or an AKG Q702 and complains about the sound leakage when these headphones are design to do so. Most of the time people will purchase headphones like a B&W P5/P7, KEF M500, Bose, or Beats for portable use because they are closed design with low impedance. I would say those are targeted for smartphone users.
I am not saying you shouldn't test smartphone devices with higher impedance headphones, but The Grado and AKGs are not designed for the average joe who don't know much about headphone technologies.
i listen to a lot of music on headphones, i own a S3, and i want to upgrade to S4 and everywhere i see a review the audio quality and power is very bad. should i go with the HTC One? audio and design is very important to me. (i love the S4 but in terms of audio and design, it's crappy) do any of you guys test or compare those 2 on the same headphones?
It's great to see you pushing the boundaries of phone reviewing by testing things that noone else bothers to, and with a scientific precision no one else can match, but i do have a couple of suggestions: firstly, how about also testing the quality of the built in speakers, rather than just the headphone amp? after all, sound qquality in calls is an important consideration when buying a phone. as for the loudspeaker, while i agree that noone except annoying teenagers would listen to music through their phone's loudspeaker, a lot of people use the speakerphone function on their phones regularly, so it's a legitimate area of testing. secondly, i wonder if it's fair to test all phones with a set of ear buds designed for one phone? if i were a manufacturer i would make sure to bundle a set of buds that sound good driven by the headphone amp i built into my phone, and i would expect it not to do as well with a higher impendance pair of headphones,for example. so to test on apple buds is probably not quite fair on other devices. i would simply pick 3 pairs of headphones, one each of high, low, and medium impendance, made by some reputable audio brand rather than by the manufacturrer of one of the tested devices. good job though, looking forward to future review
One suggestion to include one Xperia smartphone in your list to test. I personally use an discontinued model Arc S for close to 3 years now and have always been pleased with the bravia audio quality and output, sunlight visibility and camera. I am not a techie or a heavy user of a smartphone; but amazed with the audio quality of this device. I am sure the newer models like Xperia Z would have a more upgraded and better sound quality.
Can anyone comment on how these deficiencies translate to A2DP streaming over bluetooth? I have an LG G2 and am looking at finding something with Apt-X / aptX support (or cooking it in via a custom ROM, if possible). If the phone can't output audio properly over any channel then I don't want to waste my time.
while quality of music playback is certainly a major issue, one primary quality question hardly gets any mention, not to speak of objective examination: the quality of the phone call audio itself ! it you conduct crucial negotations over the phone - or even if you just flirt with someone on you portable, the quality of voice call transmission can make a huge difference in how you - your message, business proposition, etc - are perceived on the other end of the call, and what impression yout gain from someone calling you. so where in the respective media is a systematic phone-call audio quality assessment ? ( and no, phone call audio quality is not equal to music file playback audio on a smartphone. we are talking much more elements of the phone involved and influencing phone call audio qu. compared to file playback quality.) a lot of modern smartphones sound very poor on voice calls ....
Represent RIP city main! You did an amazing and scientific report I have high hopes for future articles and I'm really behind your trailblazing (another reference) the audio quality topic on our electronics!
I did a wedding gig with a decent firewire audio interface, ran out of music (as the crowd wanted more current hip-hop) Hooked up my friends iphone and I was a little awestruck. My speaker amp was at about 80% and the iphone 5 was at about 90% volume and the distortion and quality did not seem to dip down at any noticeable amount, through the head phones or the floor speakers. WOW.
I personally just bought the nexus 5 and yes, even at low levels being hooked up to my home stereo (which i do often) the quality and total max dBs are defiencent compared to the iphone. Crap-tastic infact, the left and right speakers even seem noticably out of phase or somehting to that effect. (I'll start using the S.O.'s ipad air if i need an emergency audio source in the future)
Just verifiying this man's findings with real world examples. Thanks!
Good point and I think there's some information to be discovered on that topic. However, I wanted to add, carriers are really responsible for increasing the quality of voice. The local recording of a mobile device is much better than the transmitted voice (not withstanding software enhancements eg. noise canceling). Check out, 3GPP and the ITU (International Telecommunications Union). There's some benchmarks that are measuring voice quality calls, but assisted with network simulators, as the network is also an important factor.
Call and sound quality are a huge part of mobile phone tech. I'm currently looking into the HTC One as it appears to have set a precedent on sound quality. Personally I am absolutely sick and tired of having to ask "huh, what?" over phone calls due to the crappy little garbage speakers in most cellphones. They are either a) loud enough to hear but are not clear or b) too quiet to hear and are not clear either. Either way hearing calls on cellphones is painful and difficult on every cell phone ive owned to date. I can live with a bulky cell phone if it means better sound personally. Sadly no one else thinks this way. Apparently till a phone is the size of a thumbnail and useless for calling people cellphones makers will not be satisfied.
I know that this test does not compare that and is more of a DAC question as the test is geared to headphone sound quality (bonus that one of the test subjects is a grado headphone... grados are amazing... i love my sr325's). None the less, the fact that someone is out there testing audio quality of cell phones means that eventually we may be able to actually hear phone calls when we make/answer.
BTW head-fi.org has many user posts on this same subject. If line level quality is important to you definitely search headfi too. This and many other compares can be found there... http://www.head-fi.org/t/685103/best-phone-for-mus...
I read Anandtech alot and have never posted a comment before, but I signed up just to make this post. Please make this a standard part of your phone reviews. I specially missed this part in the review of the HTC One (M8).
Old article. I would just like to point out the difference between driving apple headphones (or any headphones for that matter) vs a line input. Not to say this data is inaccurate, it is just not particularly useful in comparing smartphones audio quality. Firstly because I hope no one actually uses their smartphone with headphones at volumes even up to 80% where it might also start distorting. This is too loud and is damaging your hearing. I would like to see the distortion figures when connecting the headphone jack to an auxiliary input rather than a headphone and see how these phones compare (willing to bet its much more similar)
It is important here to differentiate between the microphone and the rest of the audio system if you will because if the microphone is providing the input then its intrinsic response curve will apply kind of like a coefficient to what the audiio sys output is. Whereas an aux input other than the microphone would offer an output only affected by the frequency response characteristics of the amplification system, primarily the speakers, but not the microphones.
We’ve updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site’s updated Terms of Use and Privacy Policy.
188 Comments
Back to Article
lmcd - Sunday, December 8, 2013 - link
Only Anandtech :-)Curious to see how (badly) the S3 fares.
Old_Fogie_Late_Bloomer - Sunday, December 8, 2013 - link
Even with the occasional stinker of an article (yay, non-removable batteries...?), I have to say......it's stuff like this that keeps me coming back to the site. Only Anandtech, indeed. :) Keep up the quirky, in-depth work.
althaz - Sunday, December 8, 2013 - link
I don't know about the S3, but the S2 was rated very highly for audio quality.quick brown fox - Sunday, December 8, 2013 - link
I believe it was the opposite; S3 was lauded due to its wolfson dac (as well as the original Galaxy S), while S2 was condemned for not using that same DAC.Hopefully all gets measured so we can have some objectivity involved.
MadMan007 - Sunday, December 8, 2013 - link
Yeah but talking about a specific DAC chip is audio nerds looking at just a piece of the equation and using it as a potential red herring. The analog amplifiers and output, and overall implementation, matter much more than which specific DAC chip is used as long as it's not garbage, meaning basically any modern DAC chip.krazyfrog - Monday, December 9, 2013 - link
AnandTech actually tested the audio of the international Galaxy S II in their review and found it to be quite poor.zShowtimez - Monday, December 9, 2013 - link
After just moving from a S2 to a HTC One, let me tell you how awful the S2 was...Samus - Monday, December 9, 2013 - link
The surface(s) all use a wolfson dac and sound fantastic, too. The implementation means nothing if your base source is some realtek crap.Hemlocke - Sunday, December 8, 2013 - link
The i9100 had the Wolfson DAC, but the U.S. Variants didn't.onslaught86 - Sunday, December 8, 2013 - link
The i9100 did not have a Wolfson DAC. The i9300 did, while its US variants did not. And boy was it night & day coming from the i9100.ph00ny - Thursday, December 12, 2013 - link
i9100 had yamaha DAC and was dismissed by many as poor choice moving away from wolfson DAC which sounded especially great with voodoo sound. As for S3, US variants with Snapdragon did not come with Wolfson DACThe Von Matrices - Sunday, December 8, 2013 - link
I own an S3 and I hate that it's output has a bass boost profile similar to the HTC One with its "Beats" profile. It's probably great for earbuds but for any other listening device the bass is overdriven. If it just produced a flat output I would be completely happy with it.Impulses - Monday, December 9, 2013 - link
Can't it be turned off somewhere? HTC's Beats glorified EQ can be turned off and ignored entirely, well, except for the persistent notification that lets you know that it is indeed turned off (seriously).Samus - Monday, December 9, 2013 - link
A lot of custom roms try to circumvent Samsung audio layer with a combination of audio filters. Its pretty depressing...S3 owner here.speculatrix - Tuesday, December 10, 2013 - link
Xda devs is full of discussions about things like boeffla Kernel tweaks and hacks which expose more of the hardware controls of Wolfson DACs and thus allow customization of sound EQ.It does seem to be the case that you really do have to research a specific phone and all it's variants to be sure of getting the audio chain you want.
barry spock - Monday, December 9, 2013 - link
Excellent stuff. I hope you can do a showdown of the most popular phone models out there at the moment, including the iphone 5s.PrinceGaz - Monday, December 9, 2013 - link
I'd like to see iPod touch models added to the mix also, as whilst not actually a smartphone, it is in many ways very similar and music listening on it through the headphone jack is a major selling point of it. The thing is, as you don't need to hear it "ring" in a loud room, it has a much weaker internal speaker than any smartphone, so is the audio-amp driving the earphone socket as good as that of the iPhone, or have they saved money on it to hit the lower price point?Bansaku - Tuesday, December 10, 2013 - link
I too would like to see this as well. Especially between the 4th and 5th Gen. I swear the audio quality of my 4th Gen is superior to my 5th Gen. I also wonder the difference between the various models of iPhones.JoannWDean - Saturday, December 14, 2013 - link
my buddy's aunt earned 14958 dollar past week. she been working on the laptop and got a 510900 dollar home. All she did was get blessed and put into action the information leaked on this site... http://cpl.pw/OKeIJotipoo - Sunday, December 8, 2013 - link
This is perfect, I havn't read it all but I have an ancient Nexus S and it has horrendous headphone output. I've heard the next two Nexii weren't great in that either. Lots of hiss from all the radios, and even if there was no hiss there's just an odd EQ curve or something, it makes the bass muddy. I think the program Voodoo sound helped it but not completely. So audio output will be an important consideration in my next phone.Friendly0Fire - Sunday, December 8, 2013 - link
The Nexus S has some of the best DAC in the industry, actually... Most phones these days use the built-in Snapdragon DAC, but the S has a Wolfson DAC which is way superior.I think you either had a dud or you have another issue somewhere because the S was lauded for sound quality (just like the original Galaxy S).
tedders - Sunday, December 8, 2013 - link
I must have had a dud Galaxy S then too because my Captivate had abysmal audio.Samus - Monday, December 9, 2013 - link
The captivate and epic 4G (sprint variant) were substantially different from the galaxy s, although I remember the epic 4G touch (S2) having decent audio quality.tipoo - Sunday, December 8, 2013 - link
So I've heard, but I'm not the only one with shit audio on it, the running theory is they put in a good DAC but didn't bother properly making drivers or something for it.tipoo - Sunday, December 8, 2013 - link
Or perhaps improper shielding of the audio circuit from the radios, since the static seemed to be radio related. Even modern phones have some of that, even the best sounding ones like the iPhone 5S, but it's much much reduced.cheinonen - Sunday, December 8, 2013 - link
The DAC really means nothing without knowing everything else that goes into the circuit. Yes, the DAC might have superior SNR and THD+N but if you use an amplifier, or a volume control circuit, that are worse than other phones, you've now mitigated that advantage. It's there in the DAC but by the time it gets to the output stage it's been buried by noise elsewhere in the system.Using a better DAC is nice. But you can't just drop it in and get better results, everything else needs to be engineered around it as well.
Friendly0Fire - Sunday, December 8, 2013 - link
Using a higher end separate DAC hints that the manufacturer at least somewhat cares about sound quality. Using the built-in SoC DAC says that it's an afterthought.It's not a guarantee, but it's certainly an indicator.
Galidou - Sunday, December 8, 2013 - link
Totally agree with cheinonen, any high end audio stereo setup is as strong as it's weakest link. Take the excellent nuforce DAC-100 at ~1200$CAD, use it with a poor sub 400$ multi channel receiver and you won't get much out of it. The nuforce got plenty of amazing review, but it always depends on what it is paired with.The galaxy sure got a nice DAC but it's amplifier section is poorly engineered.
cjl - Monday, December 9, 2013 - link
Actually (and I know audiophiles everywhere will disagree with me on this, but whatever...), you can get audibly perfect sound with a dac worth only a couple hundred dollars at most, and an amp of similar cost. There's really no point in buying a >$1k dac, and the same goes for an amplifier (though expensive multi channel receivers can be justified, not for the quality of their amplifiers, but for the quality of their DSP, room correction, video upscaling, and variety of inputs and outputs). For an audio system, by far the most important component is the speakers. You should spend the vast majority of your audio budget on the transducers (speakers/headphones), then buy an amp with sufficient power to drive them to whatever level you need without distortion/clipping (which is almost always less power than recommended by audiophiles). The rest of the system can really be quite inexpensive - the built in DAC in most modern audio products is audibly flawless, and it's impossible to improve on 14AWG zip cord for the cables (from an audibility standpoint) unless you're running your wires over an extremely long distance.Now, that isn't to say that modern smartphones have reached that audibly flawless point yet - many of them haven't (especially when they have such blatant flaws as the single channel clipping shown in the review above), but many of them are a lot closer than people realize, and it really doesn't cost much to make a product audibly flawless. I remember seeing testing done on the latest iPod, and it was good enough to not contain any audible flaws. I would assume that the iPhone is similar, and the GS4 looks to also be audibly perfect in the review above (so long as 500mV p-p is sufficient to drive your headphones - this may not be the case for some less sensitive models).
speculatrix - Tuesday, December 10, 2013 - link
+1A lot of audiophile ideas simply don't stand up to basic engineering principles, and blind A:B tests show it.
dylan522p - Sunday, December 8, 2013 - link
Just because you have a good DAC,doesn't mean you have great audio. S4 has a great DAC but it can't even power high impedence headphones.cjl - Monday, December 9, 2013 - link
The S4 powers moderately hard to drive headphones just fine - sure, it won't drive something like an HD650 adequately, but the majority of consumer headphones out there will go plenty loud driven straight out of an S4. That having been said, it would be nice if it had a bit more power for hard to drive headphones.oktrav - Monday, December 9, 2013 - link
I can't believe how many people on the internet are singing this song... Does it really matter who makes the DAC if the output sounds like crap? The DAC is just one of many components that affect the sound quality. You can have the best components in the world and still produce absolutely dreadful sound. Ask any wannabe audiophile who's dabbled in assembling home systems out of separates. This is akin to saying that your car must be very powerful BECAUSE it has Bosch coil packs --and Bosch makes the BEST coil packs (just play along for the sake of the analogy, I'm not actually asserting that Bosch makes the best coils --I don't know).nomopofomo - Sunday, December 8, 2013 - link
So glad you went into such depth.Best case scenario, the public and manufacturer are both made aware of the flaws.
drwho9437 - Sunday, December 8, 2013 - link
This has been needed badly for some time. Phones displacing media players this matters a whole lot. To me far more than actually anything else about a phone.probedb - Sunday, December 8, 2013 - link
You also have to remember with some measurements about what is actually audible and what isn't. It's crazy seeing people moaning about nanoseconds of jitter on DACs yet they're quite happy with milliseconds of the equivalent of jitter on vinyl. Then again some of these companies sell ethernet cable at £1600/m and claim it makes a difference to sound ;)matagyula - Sunday, December 8, 2013 - link
Great to see something like this on anand!I would like to see some older high-end phones compared to the ones we have now - for example the Nokia N900, or maybe even some of the ooold Sony-Ericsson Walkman phones (W810 and up). I'd find such comparison interesting ^^
chubbypanda - Monday, December 9, 2013 - link
Great stuff Chris!I was also wondering if it's possible to add some ancient phone with cult following from audio geeks. You know, Nokia N91 and similar. According to some, sound quality on the phones were never better, so that'd be nice to see what is really happening.
wrkingclass_hero - Sunday, December 8, 2013 - link
So, how long before Arstechnica "discovers" the audio problems with the Nexus 5?crabperson - Sunday, December 8, 2013 - link
YES! Thank you so much for doing this, its awesome. I didn't realize audio/amp quality was a huge thing until I upgraded my headphones. Being able to hear the difference between the same audio file on two different devices made me realize how little manufacturers care about audio quality.Then using an FM transmitter in my car showed me how some devices can't pump out enough power over the headphone jack. My Galaxy Nexus has pretty good audio quality (and does optical audio out through the dock, also awesome) served me fine for pumping up music through the FM transmitter. The Galaxy S3 I'm temporarily using does not though, and produces noticeable clipping at max volume. It also isn't properly shielded and when charging there is interference on the headphone jack (something you should also test if you haven't thought of it). I'm looking at upgrading if I can't fix my Nexus, and hope the HTC One is as good as it in terms of audio quality (with Beats disabled of course).
dylan522p - Sunday, December 8, 2013 - link
The One is very interesting in terms of quality. There is a huge difference between the stock music, google play music, power amp, and astro. Astro is the best quality.Impulses - Monday, December 9, 2013 - link
You know, you briefly alluded to the single biggest difference maker and a ton of people would still take it for granted... If you're using crappy headphones much of this is wholly irrelevant. At the same time, expensive headphones aren't always an upgrade... For one thing you've got the Beats and Bose of the world getting by on marketing, and even beyond that not every headphone will be a good match to any given phone depending on how hard to drive it is (or isn't) and how out of whack the phone's output impedance is (something that's often taken for granted even with dedicated audio components).Daniel Egger - Sunday, December 8, 2013 - link
I always thought the reference for audio testing equipment, especially in telecommunications, was HEAD acoustics. Still a good and interesting article.Drumsticks - Sunday, December 8, 2013 - link
Jeez Anandtech rocks. Thanks guys!RoninX - Sunday, December 8, 2013 - link
Great! This is exactly what I've been waiting for.I've noticed the audio quality on my Droid 4 is sufficiently poor that I continue to carry around an iPod Classic along with the D4. I'm hoping that the sound quality on my next phone, likely either a Note 3 or the next gen HTC One (One Two?), will be good enough that this isn't necessary.
pukemon1976 - Sunday, December 8, 2013 - link
2 thumbs up for galaxy note 3 audio quality. was quite surprised.dylan522p - Sunday, December 8, 2013 - link
One definely. The internal amp adds tons to quality the Two or whatever it is called will assuradly be even better.quick brown fox - Sunday, December 8, 2013 - link
As a mild audiophile who occasionally searches for electrical measurements of DAPs/smartphones on IF/GE/GSMA/Voldemort's Blog/etc, I am very pleased seeing this article in Anandtech! Had a mild laugh seeing the HTC One 'Beats Enabled' frequency response (is the treble peak still there without Beats?), and I'm very much looking forward to seeing some Nvidia Shield/SIII i9300 measurements.cheinonen - Sunday, December 8, 2013 - link
With Beats disabled, you actually see a bit of a roll-off in the bass and the treble. It's +/- 0.230 dB so not a level you'll likely hear. Overall the response is very flat if you turn off Beats. I'm still testing something else I found with the HTC One before I have full numbers up there.quick brown fox - Sunday, December 8, 2013 - link
Thanks. Again, this article is excellent; looking forward for more measurements!brusselwilson - Sunday, December 8, 2013 - link
Is sample variation an issue of relevance for smartphone audio systems?cheinonen - Sunday, December 8, 2013 - link
No more than it would be with anything else I would think. For the Nexus 5 I tested two different samples (one from Brian, one from a friend of mine) and both exhibited this issue. So you might see small variations, as you would with any display or anything else, but nothing major I wouldn't think.vailr - Sunday, December 8, 2013 - link
Another factor to consider: certain cell companies can enable enhanced audio quality for cell phone calls, but only on selected cell phone models. The IPhone 5 has that ability, as I recall. Not sure about the specifics, but I believe that both ends of the cell call must be using supported phones, as well as: the cell provider must enable that feature. That would enable cell calls to have better voice quality than land line calls, via increased audio frequency response.shenkey - Sunday, December 8, 2013 - link
Could we also get Windows Phone devices included in the test. Lumia 920 and 1520 should fit in the range.cheinonen - Sunday, December 8, 2013 - link
I'll add whatever I can when I get a chance. This first run has taken almost all my time up since the week of Thanksgiving.hopfenspergerj - Sunday, December 8, 2013 - link
It's not useful to measure noise/dynamic range at the highest volume setting. You have to measure at one of the lowest settings to determine whether the phone truncates bits, whether the noise floor does not decrease with the volume setting, etc.I have an htc dna and it is completely, totally, utterly useless for playing music with sensitive IEMs; I suspect many android phones with "-90db" thd+n measurements are similarly bad in practice.
hopfenspergerj - Sunday, December 8, 2013 - link
Not to mention poor shielding on the dna causes the phone to output chirping and static and other loud extraneous noises whenever it transmits data.cheinonen - Sunday, December 8, 2013 - link
We have stepped output level charts as well that measure this, they just aren't included here right now. I can start to pull those out for current and future tests if we want to use them.evonitzer - Monday, December 9, 2013 - link
I saw that one graph included when you compared the Nexus 5 and G2 at different levels and I think it should be in all the reviews. I always run my IEM's in the lower half of the volume range so I am quite interested in how they perform. As others have said, excellent work!DoctorG - Tuesday, December 10, 2013 - link
Same here -- I always run my good IEM's below half volume at least (usually more like a third.) No point in having good headphones & music if the volume is turned up so loud it hurts....BTW Great job Chris! I have always wanted more in-depth reviews about smartphone audio quality. It's very important to me, but so far there haven't been any reliable/objective tests available. Thanks! Just a thought, but maybe it would be possible to test against a pro-quality amp/DAC? When I use my GNex, the quality is obviously very different from my audio interface that I use with ProTools. It'd be interesting to see just how much of that is measurable...
UsernameAlreadyExists - Friday, December 13, 2013 - link
That would be great. I believe most display tests are run with the same light output to have more reasonable comparisons... I would like to see the same principle here. As a curiosity (or a warning) it would be good to know that above certain levels the sound will deteriorate.bob11d50 - Sunday, December 8, 2013 - link
I would love to see the distortion of the apple on a standard headphone (16 ohms). They have notoriously bad sound quality all they way back to the first ipod. Apple is a profit company not a quality company.I would also like to see a real comparison of battery life on apple products. Like battery life VS screen size in SQ inches. The screen is so small who cares if it can brows the web for 10 hours if you cant see it.
Also frame rate VS pixel count. Once again I can refresh a 1 pixel screen at 1Mhz but it does not do me any good nor does a 1M Pixel screen at 1 Hz but the spec is the same 1M pixels a second.
Robert
NeoteriX - Monday, December 9, 2013 - link
I am not an Apple evangelist having only owned Android (and WM and Palm) smartphone devices. So with that said, this is a truly ignorant statement. Apple has led the way on several different design and hardware fronts, including display quality and size, camera quality, GPU power, etc., and it's not clear that without their leadership there would be the kind of robust hardware competition there is in the PC and Android space.deasys - Monday, December 9, 2013 - link
"They have notoriously bad sound quality all they way back to the first iPod"Sure, Bob, whatever you say. Of course, there are authoritative sources that disagree with you. For example: http://www.stereophile.com/content/apple-ipod-port... which summarizes by stating, "Excellent, cost-effective audio engineering from an unexpected source."
I think I'll take Stereophile's word over yours, Bob…
akdj - Monday, December 9, 2013 - link
deasys hit the nail on the head Bob---Not sure which iPod/iOS device YOU'VE hear/listened to/had experience with....but as an absolute audio nerd, I can assure you decent audio files sound excellent....my favorite cans right now are the B&W P5s currently, but I've also got 2 pairs of Grados, Sony MDR 7520s, and Sennheiser HD800s for our studio mixing (with B&W Nautilus 802 speakers and Focal SM9 studio monitors). You couldn't be more dead wrong about Apple's sound quality. Perhaps you need to find a new way to 'rip' your music or quit listening to low bitrate MP3s to judge sound quality?As far as screen/battery life----WTF does that have to do with this incredibly extensive, exhaustive sound quality review and comparison/contrast between three Android phones and an iPhone?
Bone to pick, eh?
LoL----Doesn't matter does it....regardless of the article, review, discussion----always SOMETHING to do with Apple isn't it?
winchuff - Wednesday, December 18, 2013 - link
The devil is always in the detail... Unfortunately deasy, the detail you failed to pick up on is that the 'stereophile' tests were performed into the mackintosh powerbook 'line in' impedance and so did not uncover the limitations of the analogue output stage when listening via headphones.winchuff - Wednesday, December 18, 2013 - link
You're dead right Bob (whatever the other numpties say). The problem with the ipods (except for the first gen iPod nano, which was superb) is down to an underpowered analogue output stage. There is no problem when driving a high impedance 'line out', but it results in clipping and poor bass response when driving (low impedance) headphones. For those of us listening via headphones, the most crucial objective test is performed when driving into a low impedance. If anandtech do the low impedance tests, you will be vindicated for identifying the shortcomings of the apple devices. And the numpties who know no better will eat humble pie.FYoung - Sunday, December 8, 2013 - link
I commend you for taking this initiative. I agree that audio testing by sites like Anandtech could eventually lead to phones with better sound quality, which is something that has been neglected so far.However, I wish you tested the audio quality of phone calls as well. Cell phones are phones, after all. It doesn't look to me that these tests measure the ability of a phone to selectively capture the spoken voice in a loud environment (without omitting the first syllable of a sentence) and reproduce the voice reasonably accurately and reasonably loudly through its speaker, which is what a cell phone must do to function as a phone in real life.
If no objective and reproducible test currently exists to do this, why not invent one?
As for me, I have an S3, and I find the speaker volume barely sufficient to hear the caller's voice. I consider this a significant and entirely unnecessary weakness of the S3. A phone's ability to carry a conversation is far more important than its competence as a camera or music player.
Impulses - Monday, December 9, 2013 - link
Those might be your priorities, not necessarily everyone else's though. I can tell you I'd be much more concerned how a phone would fare with music than with calls... I use maybe 200-300 voice minutes a month but I probably spend at least twice as long listening to music/podcasts on my phone (if not thrice as long).In the same vein, I couldn't care less about the camera as long as it's usable enough for basic stuff like snapping a pic of something as a reminder while I'm at a store... I still use my pocket camera or my micro four thirds camera for any picture of any moment I'd truly like to remember.
However I KNOW that's not a majority view and for many many people a smartphone is now their primary camera, so I can appreciate the efforts Brian puts towards evaluating those. I'd imagine that anyone who speaks a ton on their smartphone probably uses a Bluetooth device and I'd bet that's ultimately a bigger factor in call quality (along with the network).
Not saying it wouldn't be interesting to test mind you, just adding some perspective.
DarkXale - Monday, December 9, 2013 - link
Aside from lacking more precise test methodology, and having way too many variables compared to reality to make the tests reliable in reality - such evaluations are already fairly regularly performed during reviews of the device.Do keep in mind most carriers will not support frequencies outside the 300-3400mHz range; inadequate for a decent voice conversation.
If you are concerned about voice quality, your first priority should be to get the carriers to support wideband audio. Without it, the phone manufacturers themselves can't do that much.
Apple only introduced it with the iPhone 5, but others like Nokia have supported it in their phones far longer, even prior to the release of Windows Phone (7), including support in virtually every device they've released. (Whether its a low end Asha, Lumia 500/520, or a high end 900/920).
Even the popular Galaxy 2, 3, and 4 support it - so there is no shortage of devices with the capability.
DanNeely - Sunday, December 8, 2013 - link
I'm curious how well your Grado headphone's are holding up? I bought a pair of SR80's for use at work last winter; but the wire started to develop damage a month or two ago. If I move the wrong way while wearing them I can get brief bursts of static in one ear, and can mute that ear by pinching the cable just above the Y. I suspect the damage was caused by the post in the headband allowing the earcups to spin freely, combined with the unmarked cable making it hard to notice anything less than a half dozen or so revolutions of twist. I'm wondering how much of this is bad luck on my part vs poor design/manufacturing.Marovincian - Monday, December 9, 2013 - link
I had similiar concens with my Grados (actually mine are Allesandro ms1i). I sent grado an email suggesting that they put a stripe on the "Y" wires so that you could more easily straighten them out. They said that they would pass it along to their design team. Then they sent me a free T-Shirt. Classy company for sure.ManuLM - Sunday, December 8, 2013 - link
Quite a good initiative thanks, it is too hard to get these numbers nowadays.I would suggest you guys build up a database over time of phones performance (see headphoneinfo awesome job for instance).
I also suggest that you add to your test the maximum output delivered (power or voltage swing into load). This is interesting, because if a phone clips at high volumes, but its output power is 10dB above the others in average, then the normal user will simply not see the drawback (altough I admit this is initially poor job from the company in tuning the audio system).
It also helps to chase the brands which deliver lower output power, that can turn to a problem on more demanding headphones (high impedance requiring higher voltage swing). Some users will fancy some extra power on their headphone output (even if this might not be safe for their ears).
Last point, some high-end IEMs have quite low impedance, that demand fairly high current specially in the high energy low frequency, creating bass roll off. A simple frequency response check on a low impedance IEM would show this.
RandomUsername3245 - Sunday, December 8, 2013 - link
I like the idea of audio testing, but I am disappointed by the methods used in this article: why would you bother testing a device at maximum volume when you know it is clipping badly? You should reduce the volume to a setting where it does not clip and then continue the review. You can then report the maximum useable volume setting on the device.The maximum volume on an iPhone is reported to be in excess of 100 dB. Listening at this volume for even a short period (15 minutes) on a consistent basis will permanently damage your hearing. Why not test these devices at reasonable volume levels?
http://www.theatlantic.com/health/archive/2011/12/...
(hopefully not too flawed analogy follows...)
If you are comparing two overclocked computers for maximum performance, you set them to their highest stable clock rate and then benchmark. You do not set one to a clock rate that causes continual crashing, and then report that it failed several of the benchmarks. I think this is comparable to audio review for the clipping cellphones. You might argue that the device should support any user-accessible volume level, but historically it is very common for audio amplifiers to allow users to adjust the gain until the output clips. Apple is an unusual case that limits the user to only access non-clipping gain settings.
ManuLM - Sunday, December 8, 2013 - link
audio systems are tested at max performance (there are many reasons for that, including the fact that when you sell something, all usage range of the system should be good), so analogy with OC is not ideal.I agree with you though that testing at nominal volume could help, as an adder only of max volume testing
eio - Monday, December 9, 2013 - link
yes, power of drive is a good factor in a benchmark. but performances at different loads should not be compared directly.eio - Monday, December 9, 2013 - link
a ideal test may have several series of performance graphs with several steps of incrementing loads...RandomUsername3245 - Wednesday, December 11, 2013 - link
Late reply...Like I said in my previous comment, it is common for audio amplifiers to allow you to adjust the gain past where the amplifier will start to clip. You should never expect a car stereo or home theater amplifier to allow you to run at maximum gain without clipping, so why should you expect a phone's headphone amplifier to behave differently?
The proper way to run this test is to adjust the amplifier to maximum non-clipped gain and then run the test.
willis936 - Sunday, December 8, 2013 - link
The day has finally arrived? Good data with some surprising results. I think I'm mostly surprised at how well all of the devices perform. I think dynamic range is perhaps the most important test here simply because most people won't be listening at max volume on headphones and pushing the noise floor down as low as possible is important for quiet listening.Were these tests done on the AKG K701? That is well known as a difficult to drive pair of cans without an amp. If a phone can drive those loudly with good measurements then it's certainly good enough for anything I'd use it for. Testing should be done worst case and if there's time more typical cases. When using my phone as a line out I'll typically leave it 3 steps below max because I expected there to be output stage power issues (seen as dramatic clipping on LG's stuff :x) on my phone. Any lower and as you noted the static noise floor lowers the SNR.
I was a little surprised at the weak channel separation in the otherwise amazing iphone. Channel separation is already a p big issue. Even with expensive headphones it's easy to test and ballpark a crosstalk of worse than -60dB by ear just from the jack to the drivers.
I'd like to make a request for some data of testing devices (1 iphone and 1 iconic android per year?) going backwards to see a progression (or maybe lack thereof) of audio quality in smartphones over the past 4 or 5 years.
willis936 - Sunday, December 8, 2013 - link
Oh, and thanks for the excellent write up and all of your hard work! I'm looking forward to future data.cheinonen - Monday, December 9, 2013 - link
No, for the initial set I used basic Apple earbuds that everyone has. I do have AKG K701s to test them on as well, and plan to do so going forward.charltonv3x - Sunday, December 8, 2013 - link
Curious how the test result gonna be for XPERIA Z, ZU, and Z1 against Lumias :)and...can it be used to test ASUS Xonar Vs Onkyo soundcards or other audiophile soundcards...
cheinonen - Monday, December 9, 2013 - link
It can test anything. I use it to test Blu-ray players, preamps, amps, receivers and more. The report it spit out for a receiver for me today was well over 150 pages.Impulses - Monday, December 9, 2013 - link
Testing some other gear might be interesting context wise... i.e. How does a smartphone compare to a Xonar DGX or STX, or to some of the cheaper amps out there (O2? Magni?). Adding stuff like the venerable SanDisk Clip Zip might be even more relevant as far as comparisons go, since that's a great $30 solution for anyone with a phone with disappointing audio.lookit77 - Wednesday, December 11, 2013 - link
+1 for using the SanDisk Clip Zip as one of the benchmarks.mrnuxi - Wednesday, December 11, 2013 - link
Even better than a SanDisk Clip Zip is the older SanDisk Sansa Clip+, which can be found quite cheaply. Here's what will give you a fantastic audio experience:1. Add a 32gb microSDHC card to the Clip Plus with your music encoded as flac (the Clip+ supports flac [lossless] playback.
2. Install the excellent RockBox (http://rockbox.org) replacement firmware.
3. Add the superb FiiO E6 headphone amp. Note: beware of counterfeits on eBay!
4. Use decent or better headphones (at work I use Grado SR60; cycling I use various good quality earbuds).
5. Enjoy your music as you've never heard it on a phone or iPod.
Morgifier - Sunday, December 8, 2013 - link
This is great information, thanks!I have a Nexus 5 and when I plug headphones in I usually listen to level 6 or 7 (out of 15) and have found the audio quality to be to my liking (vs. my old Samsung Galaxy S2).
However, typically when I listen to mobile devices via an amplifier I would turn the device up to MAX volume and then modulate volume via the amp - this does not seem to be the best case for the Nexus 5, I guess stop 12 would be the best volume.
Is amplifier clipping a common occurrence for mobile devices? I consider this a design flaw, i.e. max volume available for the device should be prior to any clipping.
ruzveh - Monday, December 9, 2013 - link
To be honest i have heard many smartphones through one of the best earphones and headphones i dint like the sound quality from any of the smartphones that i have heard compared to the ones that my mp3 players deliver. This is where i hate my smartphone and still love my media players.Can mobile companies take a note on this?
ruzveh - Monday, December 9, 2013 - link
And the fun is we dont get quality DAC for the premium we pay for these phonesshaolin95 - Monday, December 9, 2013 - link
I hope we get to see how the Xperia Z Ultra performs even though I have not seen a review for the phone itself so I guess not much of a change there :/porphyr - Monday, December 9, 2013 - link
This article is fantastic. I love that you are going to be testing audio quality. Obviously testing anything takes a lot of time, but I think it would be nice to see how these phones perform at "reasonable" listening levels. It would be great if you could pick dB level that is pleasant through the apple earpods and test a 2nd time at that, sort of how you use 200 nits for panel tests. That would give a more helpful representation of performance. Another thing that I would really appreciate is an explanation of how bad (or good) performance needs to be for most people to hear the difference (with bad and with good headphones). Thanks for the piece.99sport - Monday, December 9, 2013 - link
Thanks for the great article.Can you add built in speaker testing? I have an HTC one, and while I prefer to turn beats audio off when I use my phone in the car (listening to music on the car's speakers), I much prefer the sound of the built in speakers with beats audio on. As others have pointed out, all of the pieces in the chain are important to sound quality. My guess is that the frequency response of the built in speakers is much less at low frequency, and the beats audio increase in gain shown in your chart (below 100Hz) is designed to compensate for the frequency response characteristics of the built in speakers. In other words, if you tested the frequency response of the built in speakers (using a microphone with a known linear frequency response to gather the data), would the curve be much flatter when using the built in speakers with beats audio turned on versus turned off? Would the HTC one with beats audio on in fact have a much better (flatter) frequency response through the built in speakers than other phones do through their built in speakers?
It would be very useful to be able to compare the sound quality of the whole sound reproduction chain including the built in speakers – for me, the built in speaker sound quality and volume level is a major discriminator between phones – right up there with display color accuracy and camera quality.
dishayu - Monday, December 9, 2013 - link
Excellent initiative. The HTC One frequency response looks dreadful. I'm looking forward to graphs with beats enhancement off.As the next step, I would really like to see where different types of audio devices compare to each other... smartphones, ordinary media players (ipods), audiophile media players (HM-801), entry level/top level PC sound cards, realtek onboard chipsets.
NeoteriX - Monday, December 9, 2013 - link
I'm not sure if it's necessary to go through the effort of including PC soundcards, motherboard solutions, etc., but including media/MP3 players would definitely be a nice addition. The place that mobile phones have now as media consumption devices in addition to being phones came as an evolution from the original media device.However, the media device, given its singular purpose, prominently had sound quality as a major review component, and consumers did look at the DACs and opamps used. When they evolved to all-purpose smartphones, suddenly there were a lot of other features that took precedence, including processing power, display quality, etc.
Now that AnandTech is shepherding in a new look at audio quality, comparing the state of mobile phones to the heyday of media device would definitely be useful.
stunta - Monday, December 9, 2013 - link
Excellent article! It would be fantastic to see results from commonly used mp3 players. iPod, Sandisk Sansa (Clip, Fuze etc.) etc.How do you feed the test tone into the phone? Does the Audio Precision device come with test files you simply drop on the phone and play?
Also, do we know if these phones use the same amplifier for the built-in speakers and headphone outputs?
Thanks again!
Impulses - Monday, December 9, 2013 - link
I'll second this and dishayu above, it'd be a great reference.Hell, if we could get you to test output of a few BT receivers out there that might also be interesting and valuable to readers... I know BT's another can of worms as it introduces more compression etc, but it's actually gotten quite decent and those receivers end up replicating (and thus bypassing) many of the same components in phones.
There's not a lot of in depth reviews out there for those things but seeing as you can use them with any pair of headphones they'd fit right into your testing and they can be a suitable solution/alternative to a phone's line out. Something like the older Sony MW-300, or the newer models, or three equivalent Samsung/LG models.
Panzerknacker - Monday, December 9, 2013 - link
Please test the Oppo Find 5! I use that phone and it should have good audio capability. Wonder how it does vs iPhone 5.juhatus - Monday, December 9, 2013 - link
Now if only apple would enable their audio tech (noise-cancel atleast) on bluetooth.I downgraded from nokia 800 to iphone5 and atleast using the in-car-bluetooth the audio on calls went really bad.
On other comment, the graphs are huge in this article, maybe little scaling or something? some bare flatlines there and very big size graphs.
Nice article :)
Impulses - Monday, December 9, 2013 - link
Great article, though I have a quibble with the headphones to be used for future testing... Nothing against either as I've owned both, and the SR80 are a fine candidate (common entry level open headphone recommendation). The AKG have a reputation for being hard to drive and they aren't exactly very portable anyway, plus they represent another dynamic open headphone. I think a popular IEM (Ety, Shure, even Apple's dual driver IEM) might be a better second alternative, if it's at all possible with your rig... I know IEM are a pain to test due to seal issues and whatnot. Failing that maybe some other dynamic headphone that's more likely to be used with a mobile device (V-moda M-80? Senn HD25-1 II?), or something cheap yet extremely popular and good for the $ (Koss Portapro or KSC75).Impulses - Monday, December 9, 2013 - link
Also, if you could test and report output impedance for the phones that would be a HUGE help for people trying to figure out what kinda headphone would work best... Relatively high output Z isn't uncommon and it can wreck havoc with some lower impedance headphones, particularly sensitive IEM (and specially the multi balanced armature models that are so prevalent now).psuedonymous - Monday, December 9, 2013 - link
Any chance of a test/roundup of bluetooth receivers?Impulses - Monday, December 9, 2013 - link
I'd love to see that.deathdemon89 - Monday, December 9, 2013 - link
I do hope you consider providing the audio tests in the main review on the day it's published, as opposed to tacking it on later. I usually read reviews only once, i.e. on the day they are published and don't keep returning to individual reviews looking for updates, so this would be a major data point readers like me would not be able to take advantage of.DanNeely - Monday, December 9, 2013 - link
I wouldn't hold my breath. Chris H isn't a typical smartphone reviewer. As a result getting these results at initial launch time would require either buying additional sets of test equipment for the reviewers, buying an additional phone for Chris H to do audio testing on, or delaying the article to ship the phone to Chris H after completing the rest of the testing work.Audio precision won't let you see pricing information without creating an account on their website. That suggests it's painfully expensive and that getting multiple copies of the hardware won't happen. Getting multiple copies of the phone isn't cheap either and is probably not going to happen except perhaps for a few halo devices. With the peanut gallery raging about any reviews that don't make it out on release day, I'm doubtful that anandtech would choose to delay reviews for a few days for a specialized test.
Impulses - Monday, December 9, 2013 - link
Ehh, I agree, if it's not realistic to have this testing the day the review is out it's no big deal... If it's a deal breaker for you then you'd wait the same amount of time either way, and if it's not (probably the majority of readers) then there's no point in making the rest wait.xaml - Saturday, December 14, 2013 - link
"Chris H isn't a typical smartphone reviewer. As a result getting these results at initial launch time would require either buying additional sets of test equipment for the reviewers, buying an additional phone for Chris H to do audio testing on, or delaying the article to ship the phone to Chris H after completing the rest of the testing work."Or buying an additional Chris H...
cheinonen - Monday, December 9, 2013 - link
Having this on the day-of is going to be a challenge for a number of reasons.- Brian is in Arizona, and I'm in Oregon. If there is only a single review sample, I have to get it from him.
- As mentioned, the Audio Precision is ridiculously expensive. I think the APx 582 used starts at $19,000 before adding the HDMI, Bluetooth, and Digital modules I use (I need it for receivers as well). Audio Precision is just a few minutes from my house and they've been nice enough to let me come in, test everything there, and endlessly bother their QA people to get this right. However, as I have to come in I have to schedule that, and it takes time.
- That also makes it far easier to do a batch of these at a time than one at a time. If I had at APx at home it would be easier but right now that's not possible.
So we will try to get all the data, as fast as possible, into the system, but day-of is going to be a logistical challenge. I'd rather have it be accurate than be fast.
cheinonen - Thursday, December 12, 2013 - link
OK, I shouldn't say ridiculously expensive. However, the instrument we use costs enough that it's not feasible for us to have them for myself, Anand, Brian, and everyone else that needs one for testing. The Audio Precision gear ranges from $6K to $50K+ depending on what you need and the price still means we can't outfit everyone with one. So testing will happen as fast as possible, but likely won't run with the reviews when they are initially posted.Bobs_Your_Uncle - Monday, December 9, 2013 - link
This audio bench is a tool of truly significant value to anyone hoping to arrive at the best, most fully informed purchasing decision possible. Given the respect & high regard that AT has earned throughout the tech sector, this audio initiative raises the bar for more than smartphone manufacturers alone.Along these lines, does AT have any plans to initiate a similar audio bench for the various motherboard lines that have recently implemented enhanced audio capabilities?
These are very positive steps toward driving advances in fundamental, yet long neglected platform capabilities. However, as improvements in audio reproduction are realized, there will still remain one critical, & seemingly intractable obstacle to overcome; What's it going to take for the recording industry to give up on compression & adopt a regimen of decent mastering.?
It'd be a shame if the only thing audiophile-grade tech revealed was just how badly most studios butcher great music through compression & lousy mastering.
Impulses - Monday, December 9, 2013 - link
I think that battle's beyond Anandtech'srealm, though the more cognizant the average reader/listener is the better.Impulses - Monday, December 9, 2013 - link
If I didn't say this on one of my previous comments I'll say it now, thanks for doing this! Audio quality is far too often ignored, specially amongst the PC/tech enthusiast crowd... And it's rarely tested very objectively when it's talked about at all.I wouldn't mind some subjective impressions atop the objective testing though... Or even some more commentary on Chris' part regarding the data itself. Knowing the Nexus 5 is clipping at max volume is one thing, but the reader might not necessarily realize it's not an issue at lower volume levels etc.
UpSpin - Monday, December 9, 2013 - link
Great review, I'm looking forward to further tests.A few suggestions:
- Such graphs are nice for a detailed analysis but useless for an easy comparison between different smartphones. Find a way to break down the important information in those graphs to one or two numbers, which you then list in a bar diagram to allow a comparison across different devices.
- As you pointed out yourself and also other commenters, testing THD at the max. volume might be industry standard, but it's useless for a normal consumer, especially, again, in such a comparison. As in the display tests where Anandtech adjusts to a fixed display brightness across all devices, you should do the same here. Because different headphones require different volume settings you might chose three settings:
A really silent one (fixed dB), a normal one (fixed dB), and the loudest possible (max. power the device offers).
For example on my HTC Desire I use Sony in-ear ear-buds which I drive with the lowest volume setting possible, else it's simply too loud for me.
- Those three volumes also have the advantage to go in detail in specific areas:
The loudest volume setting can be used to determine how much load the smartphone can drive and the consequences (just what you did with the Nexus 5, excellent).
The normal volume setting is a measurement for overall audio quality across all devices, because that's the one most people will use. So there a focus should be kept on dynamic range, frequency response, distortion, ...
The silent setting is to determine in detail the background noise. On my HTC Desire noise is audible with the Sony in-ear ear-buds (not with lower end normal ones). And as you said, if the device can output a lot of power, naturally the noise to signal ratio becomes smaller. But that's artificial and a useless measurement when compared to other devices. So keep the volume at a uniform low setting and measure the noise to get comparable and meaningful results across all devices. Also make sure to include some noise measurement while in Airplane mode and while transfering some data over mobile. Then you can judge how well the analog part was designed.
Impulses - Monday, December 9, 2013 - link
Problem is, the dB values Chris would use would often not correspond with the same volume (and power output) level you'd use, because you're using different headphones with different sensitivities. Settling on different power levels might be more correct but it'd probably leave a lot of people scratching their heads... It might be worth taking a looksee at the way Tyll @ InnerFidelity tests amps and headphones. He's been doing objective tests for quite a while and has found a pretty good balance as far as how to present the data in an easy to digest manner.stepz - Monday, December 9, 2013 - link
Having a couple different fixed power output comparison points should make not matching your exact listening volume less of an issue. At least it's a less arbitrary measurement point than the point where the manufacturer decided to put a virtual stopper on the volume knob. The fact that max volume clips with a specific set of headphones shouldn't matter too much. If the headphone amp is clipping due to limited current available then using higher impedance headphones may be able to use that volume level without clipping. And if aren't able to use it, just don't crank it up that high and problem solved.supergex - Monday, December 9, 2013 - link
Finally I waiting for this for years.Simple and probably stupid question, will you include Windows Phone smartphone?
Many thanks in advance for these tests.
Osamede - Monday, December 9, 2013 - link
Is this novel - or just getting up to par? From what I recall, GSMArena and Mobile-Review have been doing detailed measurement of phone audio performance for years.tom5 - Monday, December 9, 2013 - link
I had the HTC One and it played much louder than the G2 without distortions on the same headphones. HTC One is in many ways ahead of newer phones like the G2 or Z1.eio - Monday, December 9, 2013 - link
Bravo! you guys always set the standard of a proper product review.eio - Monday, December 9, 2013 - link
It would be even more great if interference can be benchmarked, like the noise of I/O, communication while playing a quiet music.panda-fu - Monday, December 9, 2013 - link
This is a step in a good direction!However, I don't think the "issue" with the Nexus 5 was explained properly. What is happening is that the volume control on the Nexus goes high enough that it's possible for the amplifier to run out of power against the specific load. This is perfectly normal, and the practical implication is that the maximum power of that headphone amplifier is lower than your published measurements - the result should be at <1% THD.
Also, all results should list the load against which they were measured. If you allow massive amounts of distortion and don't specify a load, it would be easy enough to claim that a 100W <1% THD into 8 Ohms rated amplifier is "discovered to be able to produce 1000W" - just as long as you disregard it being driven into a 2 Ohm load with over 50% THD.
A metric that might also be of use, and practical in predicting a headphone amplifier's real-world performance, is output impedance. Smartphones and tablets are usually used with low impedance, sensitive headphones, and if the output impedance is relatively high, it can affect real-world frequency response massively - sometimes similar in scale as the EQ you pointed out in one of the measurements. Of course, providing measurements made against a range of headphones with different impedance characteristics, as you seem to intend to do, will point at the same issue if there is any. But in that case, please provide impedance curves of the reference headphones.
Impulses - Monday, December 9, 2013 - link
Ehh, impedance curves for the headphones he suggested (Grado SR80, AKG K701) are easy to find (Inner Fidelity and others have decent databases)... Testing and providing output impedance for the phones would be very valuable indeed though, even if everything else in the chain performs alright that alone can affect the FR significantly with one pair of headphones and not at all with the next... And unfortunately there's never been a realistic standard for output impedance, (other than high quality solid state amps now aiming for >1 ohm), and it's often all over the place.ssddaydream - Monday, December 9, 2013 - link
I agree with this.I think of three main usage scenarios regarding the headphone output:
1.) Quality of HP output driving a high-impedance line-level input for a home or car stereo.
2.) Quality of HP output driving low-impedance, sensitive IEMs.
3.) Quality of HP output driving high-impedance, non-sensitive cans.
For #2 and #3, the output impedance should be known, as well as the amplifier power at a given THD level.
There reasons why the iPhones are able to perform well with IEMs, namely low noise floor and low output impedance. Also, iPhones perform well with larger cans because of the decent output power.
I think the usage cases I listed above are pretty common, so I think a good approach to testing is to think about the most important parameters for each usage case.
BTW, I am excited about AT doing these measurements- very good news, indeed!
Anand_user123 - Monday, December 9, 2013 - link
I use my phone most of the time as a music player. Audio quality and storage capacity for flac files are major factors in my buying decision. I hope we can have more widespread information on smartphones audio performancestepz - Monday, December 9, 2013 - link
If you actually cared about scientific measurements of audio performance you would use compressed audio instead of flac. ;) Given a decent amount of bitrate, compressed audio is indistinguishable from uncompressed in double blind tests. In my experience FLAC is mostly about the listener feeling good about getting the "correct" experience, like expensive speaker cables.Impulses - Monday, December 9, 2013 - link
There's a valid usage case for FLAC as far as ripping and archiving IMO, you might as well if you're ripping a large collection (or ripping often)... Since you can quickly re-transcode or edit files w/o a loss of quality... But yeah, I don't see why anyone would put FLAC files on a phone, transcoding is dead simple and super quick if you have a remotely modern PC. Managing FLAC & MP3 playlists or whatever shouldn't be a hurdle if you're putting the effort to maintain a FLAC library to begin with, just use the MP3 library for everything or use stuff like Media Monkey's smart filters/playlists.NaterGator - Monday, December 9, 2013 - link
The biasing of the amplifier in the Nexus 5 and LG G2 left channel appears incorrect. Note it is only clipping on the negative portion of the waveform.vshah - Monday, December 9, 2013 - link
Can't wait to see htc one results!jrs77 - Monday, December 9, 2013 - link
Could you compare the phones to an iPod maybe? This way we would've an comparisooon to a mediaplayer where there's no 3G/4G/LTE disturbance.Oh, and for the general audience discussing the DACs etc... The DAC isn't the cruicial part, never has been. The amp is what it's all about and how good or poor it's powered.
There's a reason why audiophiles still use tube-amps, or atleast digital amps with high quality toroidal transformers and good shielding to reduce noise distortion etc.
Oh, and btw... A good mediaplayer needs a microSDHC-slot or the possibility to use an USB-stick. A mediaplyer is no good if I can't carry my whole music-library with it (100+ GB).
Leezhunjin - Monday, December 9, 2013 - link
Hi Chris, very nice to see smartphones getting measured in terms of audio performance, as many of these phones are used as a music device as well. Personally, I think that an inclusion of output impedance measurements would really great be though, as it is one of the factors that would affect the earphone performance rather significantly.CSMR - Monday, December 9, 2013 - link
Very good start. Poor quality needs to be exposed and you've done this with the Nexus 5.I would like to see output impedance since low output impedance is a very important quality of a good headphone output.
Origin64 - Monday, December 9, 2013 - link
"Here we see that Beats is adding a +3.5 dB boost from 60 Hz to 90 Hz, but the deviation from 0 dB goes from 30Hz to 300 Hz. Past 6.5 kHz we also see a rise in the treble."And people pay 300 bucks for a headphone that does exactly, and only, this. Its a good joke, really.
willis936 - Monday, December 9, 2013 - link
To Chris:Check out NwAvGuy's blog if you haven't already. He appeared and disappeared a few years ago bringing with him a headphone amp design and (more importantly) a breath of fresh, objective air in testing audiophile headphone equipment. He has some good data there and comparing testing methodologies might be insightful for things to try here on anandtech.
cheinonen - Monday, December 9, 2013 - link
I've read his full blog and wish it was still updated.Impulses - Monday, December 9, 2013 - link
Tyll's Inner Fidelity blog is another great resource.adityanag - Monday, December 9, 2013 - link
This is excellent. Also the reason why I've been reading Anandtech since 1998. Keep up the great work, guys.Gadgety - Monday, December 9, 2013 - link
I appreciate this article, Chris. An improvement would be a summary table of all the models compared on one page, and some sort of analysis beyond the "poor performance" comparison. I've also seen data that the specs will change significantly for worse when headphones are attached to the phones.xodusgenesis - Monday, December 9, 2013 - link
Finally some in-depth audio anlysis. I've been waiting for this as I actually use my smartphone as a phone (I know shocker in today's age) and media player most of the time.ZoSo - Monday, December 9, 2013 - link
Would like to see some results of a few current WP8 phones, Nokia in particular.asgallant - Monday, December 9, 2013 - link
Awesome! I have been wanting to see some audio analysis done for a while now. Is there any chance of extending this to test audio on motherboards, sound cards, and laptops as well?lever_age - Monday, December 9, 2013 - link
Glad to see this. My suggestions are as follows:1. Include an output impedance measurement. If you're using these as decent-quality audio players with headphones, this is one of the most important things to know for certain headphones, at least. Who cares what the measured frequency response with a resistor is if your source's output impedance is causing +/- 5 dB swings in response for some balanced armature IEMs?
2. Standardize THD+N tests to a given output level for all phones (say 0.1 V or whatever else; the danger is picking something standard like 0 dBu that some phones could possibly not even reach). Don't just use whatever the max volume is, especially since that's into clipping territory for some phones. People don't scale their playback levels by how much power the electronics is capable of handling. I hope. Referencing a fixed level is more fair.
3. Please do keep reporting which phones run into clipping (and at what load) at volume settings at max or less. Also what some nonstandard settings like Beats Mode do.
4. Make careful distinction of THD with headphones as load and as not. If not loading with headphones-level impedance that is mostly testing the performance when hooked up to say a speaker system with a patch cable, which I don't think many people are doing these days.
5. When reporting maximum level, standardize to point of say 1% THD (or max volume, whichever comes first). Also note if headphones are used or not. It'd be meaningless to quote maximum levels far past the point of clipping.
6. If you have time, see if you can coax and measure some bad behavior out of the phone by using Wi-Fi, Bluetooth, LTE, etc. simultaneously, maybe some CPU/GPU load and seeing if that causes audio issues. Honestly, glitchy or cackling playback are far greater issues to audio quality than looking at 0.3 vs. 1 dB dips in frequency response at 19 kHz or something like that. Or output power levels most people don't need.
Impulses - Monday, December 9, 2013 - link
All great suggestions IMO.haukionkannel - Tuesday, December 10, 2013 - link
I allso agree. Maximum volume level is not the same as guality, you have to find measures that are relevant in normal listening volumes. They vary yeas and depends a lot of used headphones. So I would allso recommended of using headphones with different impedanses! One pair of high impedance hifi headphones. One pair of low impedance hifi headphones and one pair of high guality (but commonly used) low impedance headphones.Otherwise it would be like testing 780 triple sli setup using 320p resolution mini monitor...
(Well actually not likely because phones have not been very high guality sound sourses, but I think that you got the point.) Good sound qaulity is important factor to me, so I am very eager to hear more (pun inteded ;-) about these test and allso the results!
DaveSimmons - Monday, December 9, 2013 - link
Thanks for doing this. When my 120 GB iPod dies I'll probably replace it with a smartphone or tablet, so this will help me choose which one to use.estarkey7 - Monday, December 9, 2013 - link
As others have stated, and I will reiterate, Only Anandtech! Bravo!I would love to see a similar article comparing HD Voice for the carriers. I know that test may be more difficult to conduct, but ever since Sprint hyped their HD Voice and didn't deliver, I'd like to see some real engineers do the topic justice.
hlovatt - Monday, December 9, 2013 - link
Congratulations AnandTech, yet again you show the rest of the review industry what they should be doing. Keep up the good work and it should be fascinating to see the full set of results. Thanks.willis936 - Monday, December 9, 2013 - link
Is there any comment on the practical issue of noise at low volumes? Noise floor testing is uncommon but phones are noisy and are typically listened to with sensitive buds. Who cares it'd the dynamic range is 100dB if you never turn the volume up past 30%?pr1mal0ne - Monday, December 9, 2013 - link
I disagree with you testing all phones at maximum volume level1) the max volume differs, some companies may reasonably make the choice to let quality suffer while volume improves. at the least volume needs to be reported alongside these quality discussions. Preferably it would be set to a base volume and tested there.
Tuxedo - Monday, December 9, 2013 - link
Amazing article. I have always been impressed with audio quality on iPhones and disappointed with GS2, GS3, Atrix, Lumia 900 and now OG-Pro. Really shows how much attention Apple pays to the products they build.edwh - Monday, December 9, 2013 - link
what about output impedance?fishjunk - Monday, December 9, 2013 - link
I don't really use the audio jack on my Nexus 5. Are there any tests done on the phone mic and speakers.JasonQG - Monday, December 9, 2013 - link
I'll be curious to see how the iPhone 5s compares to the iPhone 5. I noticed a definite improvement when I upgraded, at least to my ears' perception.ClockHound - Tuesday, December 10, 2013 - link
Nice to hear you're taking sound semi-seriously with devices that have a semi-important audio task. At least for those who use smartphones for aural communication and music playback.Nice to see the Audio Precision in action. Great unit. However, please, consider putting a little more thought into the scaling of your FR graphs. What appears visually as a huge peak in the htc Beats graph barely has a 3dB rise. That's not a huge peak.
A 1dB rise at 10k is not much of a rise and since it doesn't display enough data, we can't tell if it's a low Q event or not.
The Audio Precision can scale the vertical axis to make plots more informative rather than sensational.
pandemonium - Tuesday, December 10, 2013 - link
Awesome stuff. Keep it up AT!kreacher - Tuesday, December 10, 2013 - link
Great article, would it be possible to test audio input (the mic) in this much detail. Noise cancellation as well as how it does in speakerphone mode / video recording. I know earlier reviews / articles have mentioned this are but it would be great to have detailed numbers like this article.Cyleo - Tuesday, December 10, 2013 - link
This is awesome. I truly love this, please continue this work ;)Any change one of the more recent Sony models makes the test (Xperia z comes to mind)
Pastuch - Tuesday, December 10, 2013 - link
Chris, fantastic article, way to bring me back to Anand. Please include results of the Iphone 4s and any other phones you get your hands on. My girlfriend and I are phone whores, she's Apple, I'm Android. I'm constantly playing with all of our phones and cans trying to find the best audio combo.I run Sennheiser HD-25s, Sennheiser Momentums, V-Moda Crossfade M-80s, and Koss Porta Pros (Awesome since the 80s!).
Best sounding phone I've ever heard:
Iphone 4S sounds way better than the new Iphone 5. It's definitely louder and fuller. Too bad the 4s doesn't have APT-X codec for Bluetooth.
Runner up:
Samsung Galaxy S with Voodoo sound Rom (Note, you need to load a non-standard rom to make it sound great)
The rest:
SGS2 (International Model): Too quiet, poor quality, disappointing dynamic range.
SGS3 (North American Model): Poor quality, lots of cross talk, disappointing dynamic range.
HTC One (Beats Audio OFF): Sounds great, really no complaints. A step above most Android phones. Still miss the Wolfson DAC though.
IPhone 5: Still sounds great but it doesn't live up to the 4S. I'd say it sounds slightly better than the HTC One but the difference is really marginal.
Nexus 5: Sounds better than the SGS2 or 3. The V-Moda M80s sound good because they aren't hard to drive. The Sennheiser HD25 needs the top volumes and the clipping is obvious. The HTC One sounds a little better.
vision33r - Tuesday, December 10, 2013 - link
HTC One audio is a joke. Which features a software sound enhancer AKA Beats Audio which any custom rom can cook into their roms.I've had the S3 with the Beats Audio software and took it off flashed a better DAC amp app.
Traum - Tuesday, December 10, 2013 - link
As a head-fi audiophile, I want to THANK YOU for doing this!MWisBest - Tuesday, December 10, 2013 - link
Nice article, definitely curious as to how my Galaxy Nexus would fare in this, as I'm a bit of an audiophile.hrrmph - Wednesday, December 11, 2013 - link
Awesome topic and write-up :)Keep these coming, please.
Scootiep7 - Wednesday, December 11, 2013 - link
Wonderfully well thought out and written article. Thank you! FFora future article my one request would be for a llcomparrison of all phones on each test parameter instead of only comparing 2 or 3 on some metrics. Again, thank you!qualitycounts - Wednesday, December 11, 2013 - link
Thank you so much for doing this testing. It would also be very helpful if your wrap up section did some side by side comparisons, kind of like they do on Consumer Reports. It's very hard from this article to tell which one comes out on top. Also, it would be nice to see an audio/phone quality report on the HTC One since it is also one of the most popular smartphones available.sergoliv - Wednesday, December 11, 2013 - link
In my opinion, audio quality testing is a very welcome adition to Anandtech. I am maily a classical music listner. Good dynamic range, wide and flat frequency response and capacity to respect all harmonics present in recordings are very important for classic music. Can you broaden your testing to in order to give an idea of what smartphones are more capable with classical music?mike8675309 - Wednesday, December 11, 2013 - link
A major use case for me and my phone is as a spoken language playback device through a speaker (not headphones). Be it podcasts or audible books, a good percentage of my "audio" listening on my phone is via powered "stereo" speakers plugged into the headphone port. Often with the volume on the phone at max so the powered speakers have more range especially if in the garage doing noisy stuff or taking a shower. What if any weight should be placed on these results for such a use case?Additionally, I've historically found phones unable to provide enough power for the various headphones I use (currently Klipsch S4) when using them with mowing the grass or such. Thus I have added a small personal audio amplifier for some uses of my phone and listening to things. In that case I usually have the phone at mid or lower levels and control most of the sound at the amp. Any thought to testing some of the more common portable audio amplifiers?
skynet11 - Wednesday, December 11, 2013 - link
Could you please test on-board speakers in like manner?BobN - Wednesday, December 11, 2013 - link
Anyone know of an app that improves the call quality of the Galaxy S4? I know about Adapt Sound but it doesn't give my phone good phone call sound. Thanks.AnnonymousCoward - Wednesday, December 11, 2013 - link
What a good article. I'd love to see this data for sound cards vs integrated, and MP3 players, using only high quality reference headphones (or speakers).hmaarrfk - Friday, December 13, 2013 - link
Interesting article.From the stepped response, it seems that they are all using 16 bit DACs (16 bits would give you close to 96dB of dynamic range if the only source of noise was quantization for a signal at full power).
Can you confirm this? Does this mean, that having 24 bit encoded music is simply wasteful on a mobile device?
hmaarrfk - Friday, December 13, 2013 - link
Or did you simply use 16 bit audio? Have you tried your tests with 24 bit audio?panda-fu - Thursday, December 19, 2013 - link
24 bit encoded audio for end-user listening purposes has never been shown to have any advantage or difference from 16 bit in double blind tests. The potential advantage of 24 bit DACs lies in being able to use digital attenuation for volume control without losing any dynamic range. However, with proper dithering after attenuation, you have quite a bit of leeway even with a 16 bit one before it gets audible. So, don't worry about that spec!sonci - Friday, December 13, 2013 - link
Thanks Anandtech,Nice for doing these reviews, one of my top reasons for choosing a smartphone is audio quality, and im not talking about the voice of my wife, but the music reproduced through earbuds.
Please keep going testing at full volume not only for those who use them with an external integrated, but because that's the right way, if manufactures bost the volume and allow clipping than thats a flaw, also if a phone measure good at full volume than its super good at low volumes, you also can measure them at a specific sound pressure levels, though the difference should be small, but anyway these are not reviews for the casual listener,
and Yes unfourtunately iphones sound very good.
I also would be interested in measuring some old and new mp3 players, though in that case its different to be objective, because for a n audio player SQ is a major selling point, but I dont see Lumia selling more than Galaxie because of better SQ.
If you cant find any old mp3 player, we are crazy enough to send them to you, hoping to have them back once tested.
Thanks
Mokona512 - Saturday, December 14, 2013 - link
Can you do a comparison of smartphones and dedicated mp3 players?for example my sandisk sansa running rockbox firmare gives much better audio quality than my smartphone, especially when driving my akg k240.
cb474 - Saturday, December 14, 2013 - link
I appreciate the sophistication and thoroughness of this review. But I have to say that I have never had a smartphone where I've thought, boy this audio quality is unacceptable. Especially with earbuds or plugged into an external system. Unless you're an audiophile, they all seem pretty good to me.Speakers on smartphones are obviously another story.
But to me the real issue is call quality, including earpiece volume, sound quality of the mic (for one's caller), and especially noise cancellation. Anandtech, thankfully, covers noise cancellation in its reviews with a meaningful babble track test. But I wish there were still much more focus on call quality. There are real differences when it comes to call quality, even amongst flagship phones, and this is an area in which, on a daily basis, I experience frustration. It's amazing how much the "phone" functionality of something that is after all phone is ignored.
kevmitch - Sunday, December 15, 2013 - link
I agree. Excellent article. I hope to see more like it and on the subject.I wouldn't mind the "sensational" scales so much if they were at least used consistently. It was difficult to compare for example the THD FFT responses by tabbing between them in browser windows because the scales aren't the same.
While the max volume should really be set correctly, it would nevertheless be interesting to see the THD FFT for what SHOULD have been the max volume (i.e., a step or two below the G2). I have to say I'm a lot less excited about getting a Nexus 5 now - the Galaxy 4 looks much more appealing in spite of the significant price increase. These results even had me consider getting an iPhone for about half a second.
vang024 - Monday, December 16, 2013 - link
Sorry, but shouldn't you be using lower impedance headphones? The grado is rated at 32 ohms and the K 701 is rated at 64 ohms. The grado might be ok for portable electronics use but the K 701 is more difficult to drive than the 64ohms it is rated at. Not only that, but they are both open headphones which leaks tons of sound and not prefer for anybody using a portable electronic as their music source.I think you have to inform the reader the difference between a closed and open headphones(assume they don't know the difference). I see people purchasing a HD600/650 or an AKG Q702 and complains about the sound leakage when these headphones are design to do so. Most of the time people will purchase headphones like a B&W P5/P7, KEF M500, Bose, or Beats for portable use because they are closed design with low impedance. I would say those are targeted for smartphone users.
I am not saying you shouldn't test smartphone devices with higher impedance headphones, but The Grado and AKGs are not designed for the average joe who don't know much about headphone technologies.
bogdan.anghel - Thursday, December 19, 2013 - link
i listen to a lot of music on headphones, i own a S3, and i want to upgrade to S4 and everywhere i see a review the audio quality and power is very bad. should i go with the HTC One? audio and design is very important to me. (i love the S4 but in terms of audio and design, it's crappy) do any of you guys test or compare those 2 on the same headphones?manveruppd - Friday, December 20, 2013 - link
It's great to see you pushing the boundaries of phone reviewing by testing things that noone else bothers to, and with a scientific precision no one else can match, but i do have a couple of suggestions:firstly, how about also testing the quality of the built in speakers, rather than just the headphone amp? after all, sound qquality in calls is an important consideration when buying a phone. as for the loudspeaker, while i agree that noone except annoying teenagers would listen to music through their phone's loudspeaker, a lot of people use the speakerphone function on their phones regularly, so it's a legitimate area of testing.
secondly, i wonder if it's fair to test all phones with a set of ear buds designed for one phone? if i were a manufacturer i would make sure to bundle a set of buds that sound good driven by the headphone amp i built into my phone, and i would expect it not to do as well with a higher impendance pair of headphones,for example. so to test on apple buds is probably not quite fair on other devices. i would simply pick 3 pairs of headphones, one each of high, low, and medium impendance, made by some reputable audio brand rather than by the manufacturrer of one of the tested devices.
good job though, looking forward to future review
synaesthetic - Thursday, December 26, 2013 - link
output impedance please!seshraj - Wednesday, January 1, 2014 - link
Thanks for another wonderful post.One suggestion to include one Xperia smartphone in your list to test. I personally use an discontinued model Arc S for close to 3 years now and have always been pleased with the bravia audio quality and output, sunlight visibility and camera. I am not a techie or a heavy user of a smartphone; but amazed with the audio quality of this device. I am sure the newer models like Xperia Z would have a more upgraded and better sound quality.
jcazes - Tuesday, January 7, 2014 - link
Can anyone comment on how these deficiencies translate to A2DP streaming over bluetooth? I have an LG G2 and am looking at finding something with Apt-X / aptX support (or cooking it in via a custom ROM, if possible). If the phone can't output audio properly over any channel then I don't want to waste my time.Impulses - Wednesday, August 13, 2014 - link
It wouldn't really be a factor, the Bluetooth device's amp would be what's driving your headphones, not the phone's circuitry.blade 7 - Monday, February 17, 2014 - link
while quality of music playback is certainly a major issue, one primary quality question hardly gets any mention, not to speak of objective examination: the quality of the phone call audio itself !it you conduct crucial negotations over the phone - or even if you just flirt with someone on you portable, the quality of voice call transmission can make a huge difference in how you - your message, business proposition, etc - are perceived on the other end of the call, and what impression yout gain from someone calling you. so where in the respective media is a systematic phone-call audio quality assessment ? ( and no, phone call audio quality is not equal to music file playback audio on a smartphone. we are talking much more elements of the phone involved and influencing phone call audio qu. compared to file playback quality.) a lot of modern smartphones sound very poor on voice calls ....
Excerpt - Wednesday, February 26, 2014 - link
Represent RIP city main! You did an amazing and scientific report I have high hopes for future articles and I'm really behind your trailblazing (another reference) the audio quality topic on our electronics!I did a wedding gig with a decent firewire audio interface, ran out of music (as the crowd wanted more current hip-hop) Hooked up my friends iphone and I was a little awestruck. My speaker amp was at about 80% and the iphone 5 was at about 90% volume and the distortion and quality did not seem to dip down at any noticeable amount, through the head phones or the floor speakers. WOW.
I personally just bought the nexus 5 and yes, even at low levels being hooked up to my home stereo (which i do often) the quality and total max dBs are defiencent compared to the iphone. Crap-tastic infact, the left and right speakers even seem noticably out of phase or somehting to that effect. (I'll start using the S.O.'s ipad air if i need an emergency audio source in the future)
Just verifiying this man's findings with real world examples. Thanks!
Excerpt - Wednesday, February 26, 2014 - link
Good point and I think there's some information to be discovered on that topic. However, I wanted to add, carriers are really responsible for increasing the quality of voice. The local recording of a mobile device is much better than the transmitted voice (not withstanding software enhancements eg. noise canceling). Check out, 3GPP and the ITU (International Telecommunications Union). There's some benchmarks that are measuring voice quality calls, but assisted with network simulators, as the network is also an important factor.@paulkind3d - Thursday, March 6, 2014 - link
Call and sound quality are a huge part of mobile phone tech. I'm currently looking into the HTC One as it appears to have set a precedent on sound quality. Personally I am absolutely sick and tired of having to ask "huh, what?" over phone calls due to the crappy little garbage speakers in most cellphones. They are either a) loud enough to hear but are not clear or b) too quiet to hear and are not clear either. Either way hearing calls on cellphones is painful and difficult on every cell phone ive owned to date. I can live with a bulky cell phone if it means better sound personally. Sadly no one else thinks this way. Apparently till a phone is the size of a thumbnail and useless for calling people cellphones makers will not be satisfied.I know that this test does not compare that and is more of a DAC question as the test is geared to headphone sound quality (bonus that one of the test subjects is a grado headphone... grados are amazing... i love my sr325's). None the less, the fact that someone is out there testing audio quality of cell phones means that eventually we may be able to actually hear phone calls when we make/answer.
BTW head-fi.org has many user posts on this same subject. If line level quality is important to you definitely search headfi too. This and many other compares can be found there... http://www.head-fi.org/t/685103/best-phone-for-mus...
H0rtOn - Thursday, May 8, 2014 - link
I read Anandtech alot and have never posted a comment before, but I signed up just to make this post. Please make this a standard part of your phone reviews. I specially missed this part in the review of the HTC One (M8).notsure123 - Friday, April 29, 2016 - link
Old article. I would just like to point out the difference between driving apple headphones (or any headphones for that matter) vs a line input. Not to say this data is inaccurate, it is just not particularly useful in comparing smartphones audio quality. Firstly because I hope no one actually uses their smartphone with headphones at volumes even up to 80% where it might also start distorting. This is too loud and is damaging your hearing. I would like to see the distortion figures when connecting the headphone jack to an auxiliary input rather than a headphone and see how these phones compare (willing to bet its much more similar)Photosynthecis Media - Friday, March 6, 2020 - link
It is important here to differentiate between the microphone and the rest of the audio system if you will because if the microphone is providing the input then its intrinsic response curve will apply kind of like a coefficient to what the audiio sys output is. Whereas an aux input other than the microphone would offer an output only affected by the frequency response characteristics of the amplification system, primarily the speakers, but not the microphones.