More Than Throughput: Next Generation Wi-Fi Testing with Ixia's WaveDevice
by Joshua Ho on March 25, 2016 8:00 AM ESTWi-Fi Basics: L2/MAC Layer
The next layer up in OSI model is the data link layer, which helps to encapsulate raw bits into something more manageable. The key to understanding this subject (and many other technology concepts) is seeing everything in layers of abstraction, because otherwise it would be incredibly difficult to talk about and analyze aspects of electronics without getting lost in an excess of mostly irrelevant information.
In the case of Wi-Fi, the MAC layer, which is within the OSI data link layer, is where a ton of the intelligence resides on the device (this in particular setting it apart from LTE). At its most basic form, the MAC layer hides the reality of the physical link layer with an abstracted network. To the parts of the networking stack operating above the MAC layer, the network appears to be solely composed of endpoints. Furthermore this abstract network appears to be full duplex, which means that data can be received and transmitted simultaneously.
Obviously, infrastructure-mode Wi-Fi isn’t solely composed of endpoints, and neither is Wi-Fi full duplex. Rather, due to the nature of how radios work, Wi-Fi is half duplex (radios can only talk or receive), as trying to transmit in full-duplex mode results in self-interference due to the receiver picking up the signal from the transmitter. There’s actually a significant amount of research going into solving the duplex problem to improve data rates and spectral efficiency, but from a commercial perspective most consumer devices don’t have this kind of technology yet.
In order to enable this kind of abstraction, there are a lot of systems in place in the Wi-Fi standard. While it might be relatively simple to abstract away the fact that the network actually has an access point that routes communications from one endpoint to another, but emulating full-duplex communication in a half-duplex medium is surprisingly complex. In general, only one device on the access point/spectrum can transmit at any given time, and if this rule is broken you end up with a lot of interference effects and whatever data was being transmitted is pretty much as good as lost. However it should be noted that there is an exception here with the latest generation devices supporting MU-MIMO, where multiple antennas are used to create areas of constructive and destructive interference so that “beams” are created to allow for multiple simultaneous transmissions.
As a result of these limitations, the MAC layer has to require that both devices and access points cooperate in sharing the spectrum using a time division scheme known as Carrier Sense Multiple Access with Collision Avoidance, or CSMA/CA. Devices connected to the AP have to make sure to listen to the channel with a receiver to first verify that the channel is clear, then can transmit data by first sending a request to send packet and waiting for a clear to send packet. Once the clear to send is received, the device can transmit its data. Because there’s no guarantee here that some other device didn’t also simultaneously transmit, the device has to listen for an acknowledgement by the access point that the data was received. If an ack is not received by a certain period of time, the device has to assume that someone else was also attempting to transmit and respond by backing off on transmissions for a specified period of time before transmitting again.
WaveDevice in turn is actually able to specifically test this part of the MAC layer using its ecosystem performance test, in which clients are simulated by WaveDevice and the device of interest is tested to see whether its CSMA/CA algorithms are designed so that appropriate throughput levels are maintained in the face of competing traffic. It turns out that some devices can be too aggressive and collide with other traffic, or too passive and spend too much time backing off. Getting too far from ideal in either direction will seriously affect throughput, so from a validation standpoint this is a test that is of interest as soon as you’re in environments like a convention center or press conference where there may be hundreds, if not thousands of other devices in the vicinity all on the same few channels.
Another part of the MAC layer that is important to understand for the purposes of Wi-Fi testing is rate adaptation. While WaveDevice allows for manual control of the Modulation and Coding Scheme (MCS) used by the device in addition to the number of spatial streams for MIMO and other settings like guard interval and channel bandwidth, it’s important that a device selects all of these things automatically and correctly. This is necessary in order to ensure that packet loss and retransmission isn’t happening at excessive rates in higher parts of the networking stack and that throughput at higher layers is maximized. Importantly, unlike the cellular world, Wi-Fi lacks channel quality indicators that allow for the device and access point to directly determine what the ideal modulation and encoding scheme is. This means that rate adaptation has to happen based upon factors like packet/frame loss rates.
Meanwhile it’s also important for the device to avoid transmitting signals at excessive power levels, as power consumed by the power amplifier directly affects battery life. Given that power amplifiers generally have a power-added efficiency of somewhere around 40% in modern mobile devices, it’s not entirely surprising to have a power amplifier consume somewhere around 1W of power alone, even before considering other parts of the RF chain or the rest of the device. Using a real world example here, our web browsing battery life test is long enough that even an average difference of 200 mW can cause a runtime difference measured in hours, so proper control of transmit power is definitely important. It's also important for the Wi-Fi chain to go to appropriate sleep states in order to save power. When implemented improperly, there can be some pretty serious knock-on effects in terms of idle battery life because unnecessary wake-ups can lead to waking the main CPU, which is relatively enormous in terms of power consumption on a mobile device.
From a testing perspective, these aspects can also be tested on WaveDevice by looking at how a device performs by testing for throughput while steadily decreasing transmit power on the access point. This rate vs range test can also be a test of the RF front end/physical layer, though it requires that the test chamber is set up properly to ensure that the device receives a constant transmit power and multipath propagation regardless of angle to avoid issues with anisotropy (in the real world, devices vary in their transmit and receive capabilities based on their angle and orientation). This test also allows for direct measurement of the ability of a device’s Wi-Fi chipset to demodulate and decode the signal in the face of decreasing SNR and received power, in addition to its ability to select the ideal MCS rate to maximum throughput and reduce packet loss.
44 Comments
View All Comments
JoshHo - Saturday, March 26, 2016 - link
I was aware before the testing that the Pixel C had something wrong with WiFi, but the goal was to try and separate software issues from hardware ones.It was subjectively obvious but difficult to understand what was causing the problems in a reproducible manner.
at80eighty - Saturday, March 26, 2016 - link
Have to concur with everyone. Happy to see anandtech still keeping that edge to their content. Great work Joshua.mannyvel - Sunday, March 27, 2016 - link
We have one of these systems at work, and you have to be a wifi genius to use it.That said, what our guy said was this: test your throughout with multiple clients at multiple distances. There also some subtleties about wifi that are odd. One is that the anount of airtime that a given adapter uses is more important sometimes than its actual speed. There is only so much airtime, and a badly behaving adapter that's further away will prevent faster devices with better signal from using the AP because of how PHY rates are negotiated (not sure what the term is). Basically an adapter further away will scream louder and more slowly to see if the AP can hear it - but the slow screaming takes airtime away from the faster devices that have better signal.
Also, since it's RF more clients means more interference. I suspect you will find, as we did, that all your wifi testing has been completely wrong for pretty much forever. With the ixia stuff you can tell how bad your AP really performs in, say, and apartment-like configuration.
One more confounding factor is not all chips and drivers behave consistently. An AP that works great with Windows 10 and an atheros chipset may work crappily with the same chip in win7...as you've seen in the iPad Pro tests.
bman212121 - Monday, March 28, 2016 - link
I agree 100% with everything mannyvel said. One of the biggest hurdles people face is range vs density. Everyone thinks that throwing a bigger antenna will give you better wifi and that having an AP with increased range is a good thing. In reality, most APs these days are turned down in power so they match closer to the clients capabilities, and in the case of roaming a minimum RSSI is enforced. Only having "1 bar" hurts overall performance more than having a second AP that can provide higher signal strength. We used to have one high powered Aironet that could cover a huge area. In order to increase performance you might take down that one AP and put up 3 in it's place to get the proper coverage while getting the performance you need.I'd really be interested in some minimum RSSI tests. A great performing antenna might not have the highest power, but you can easily make up for power by having better sensitivity. Even APs of the same power output will have different receiving abilities, and higher end APs can pickup and work with much weaker signals and still maintain solid connections.
skavi - Sunday, March 27, 2016 - link
Holy shit, I can't wait for these WiFi performance and roaming tests for more devices. This is so important, but there is rarely any information specific to devices. My S6 is never able too roam correctly and forces me to reconnect each time I move somewhere, maybe these kinds of tests will pressure OEMs into paying more attention to these issues. It seems Apple at least has gotten it down.Powervano - Monday, March 28, 2016 - link
Very nice and informative article!Could you please test Surface Book and Surface Pro 4 using the WaveDevice? Would be nice to shed some light around the Wi-Fi issues on these devices.
Conficio - Monday, March 28, 2016 - link
Thanks Josh,you are on your way to become a hero for mobile device users. I appreciate the hard work flowing into this article.
I wonder which parts of this stack are software updatable? As there is talk of the Pixel-C having a broken Wifi, is there hope it gets fixed by a software update? In the same interest did I miss the specification of the exact relevant software versions on the devices?
P.S.: The Baseband package on my device influences WiFi or is it only for the WAN portion of my Android device?
cuyapo - Monday, March 28, 2016 - link
Great article, thanks! One thing only:CSMA/CA = Carrier Sense Multiple Access with Collission Avoidance
James5mith - Tuesday, March 29, 2016 - link
Still reading through the article, but it's worth noting that SmallNetBuilder has also switched to this test setup for wireless testing going forward. Interestingly the one thing this doesn't take into account is antenna design. While it's great to test throughput vs. attenuation/power/etc. None of that takes into account if the antenna is poorly designed, oriented incorrectly, etc. It does give a good baseline to work from though. And it should help people track down problems for wifi if they are using a router that tests solidly without taking factors like antenna design into account.James5mith - Tuesday, March 29, 2016 - link
http://www.smallnetbuilder.com/wireless/wireless-f...