Test on Real Mobile Devices without Breaking the Bank

Mobile Web Devices

Mobile is the future of the web, so it’s time to start investing in some mobile devices. Testing on actual devices is now an absolutely essential part of web design. Stephanie Rieger explains that there are crucial aspects of a web experience you simply can’t realize by resizing your browser or using emulators: true site performance, device capabilities, form factor, pixel density and the impact of the network.

What devices should you get? How much is it going to set you back? Obviously you can’t test on every device under the sun, so here’s my recommendation:

iOS

iOS traffic accounts for more than half of mobile traffic, so if you don’t have your iOS bases covered, get on it.

iPhone (iOS5)

Cost: $199 w/ plan
This device is likely already in your pocket, but if you don’t own one you need to make the investment. Aso make sure to download and test on Opera Mini.

iPod touch (iOS4)

Cost: $65-100 (used 2nd Gen)
It’s incredibly important to test on older versions of iOS and on older hardware, as Stephanie recently stressed, especially considering the fact that the iPhone 3GS is still around and free. The iPod setup works really well for me as it’s not terribly expensive. It isn’t used as my main device so I don’t feel like I’m missing out leaving it un-updated.

Android phone(s) (2.1, 2.2, 2.3, 4.0)

Cost: $45-?
Get at least one Android device, but I would recommend getting a large smattering of Android devices. Look to get a variety of OS versions, sizes, processing power, resolutions, and manufacturers. I’ve had lots of issues with browsers on HTC devices (zoom issues, touch events, layout), and it’s definitely good to have at least one running the vanilla OS. In addition to the stock browsers, you gain access to these wonderful browsers:

Android’s growth and diversity has been tremendous, and the device capabilities range from highly capable smartphones to bargain devices. Even if you can’t afford multiple devices right now, be aware that Android isn’t just a singular device, so while your experience might test well on one Android device, you might have trouble elsewhere.

Windows Phone 7

Cost: ~$125 and up
I noticed a lot of people on Twitter post-holiday mentioning how they’ve acquired a Windows Phone to test/play with. I think the Metro UI is a solid interface and the device is a worthy investment. This year the Microsoft partnership with Nokia goes into full-swing, and could see a boost as manufacturers start experimenting with non-Android offerings.

The Mango update brought a version of Internet Explorer 9 instead of the original IE7, but the browser still don’t support javascript touch events amongst other things. It’s also worth noting that after spending some time with a real Windows Phone, web experiences that resemble iOS and Android seem oddly out of place.

WebOS device (Pre or Pixi)

Cost: $25-100
Even though WebOS’s long-term future is up in the air, the devices are dirt-cheap. I got a Palm Pixi on a daily deals site for $25 so the price is most certainly right. The smaller form factor also makes for good testing.

Blackberry <6 device (Bold or Curve)

Cost: $17-200
I think these devices are very important to own. It’s a humbling experience testing on a non-Webkit Blackberry running 4.5, 4.6, 5.0. It keeps me honest as it butchers a lot JS and CSS. There’s lots of false positives and might be my biggest reminder that “support” is in no way a binary thing. It’s also a great reminder to accommodate for non-touch interfaces in your designs.

I just picked up a Blackberry Curve for $17, so the price is definitely right. And while BB 6.0+ now sports a capable Webkit browser, the majority of Blackberry subscribers are still sub 6.0 (and yes, they do use the web on these phones).

Nokia devices

Cost: $100-?
Obviously the rest of the world is different than the US so I don’t have specific recommendations as far as Nokia devices go, but don’t forget about the mobile giant and its massive reach. I use an N-95 to test on and it serves as a great “lowest common denominator” device. It supports some Javascript, has a d-pad input method, has a screen size less than 320 pixels and is a great little device despite being 5+ years old.

Nokia isn’t all old feature phones though, their N8 and N9 devices are definitely extremely capable smartphones. There’s a massive array of Nokia devices, so I’ll defer to the Riegers and their excellent and thorough breakdown of Nokia browsers.

Kindle Fire

Cost: $199
Getting a Kindle Fire for testing is absolutely a worthwhile purchase. It’s increasingly popular and is extremely affordable. The Silk browser is similar to Android 2.3 browser and the form factor is unique as it’s a larger than a mobile phone, but smaller than an iPad.

iPad

Cost: $499
Obviously not the cheapest thing in the world, but insanely popular (and super fun to own!). This device is extremely influential and we’re already seeing 7% of holiday online sales coming from iPads, just 18 months after the original iPad launch (via lukew). Tablet computing is most certainly on the rise, and the iPad2’s browser is currently one of the most capable browsers out there.

Miscellaneous Android Tablets

Cost: $250-499
The smattering of Android tablets out there are nice to have, but I wouldn’t say required to start with. If budget allows, definitely snag Motorola Xoom and/or Galaxy Tab running Honeycomb.

Considerations

Getting a bunch of used wi-fi enabled devices is a great way to go. Not having to deal with a ton of carrier plans is of course a good thing, and once you have the wi-fi set up you’re good to go.
Obviously having a device or three with carrier plans help for field testing. You should always tast your experiences in the real world over real connections, including 3G, EDGE and crappy free Wi-fi spots. It can be a sobering endeavor, but this is the world we live in and we should be considerate of the on-the-go aspect of mobile context.
Check out what devices other mobile web people are testing. Here’s what devices we have at R/GA.
The good people at Cloudfour are in the process of putting together a very thorough device testing lab in Portland. I’m excited to see the results.
And just because emulators aren’t the real thing, that doesn’t mean they aren’t useful. Check out Max Firtman’s very thorough rundown of mobile emulators.
Paul Irish has a nice thread collecting some relevant links
View some of the comments on the mobile web mailing list (which is where this post originated from).

Testing on real mobile devices is now an essential part of the web creation process. You can get a pretty representational cross-section of the mobile landcape for a couple hundred bucks. Make an investment in the future of the web. People are accessing the web through the lens of their mobile phones and we need to do our best to give them great user experiences.