Mobile Zone is brought to you in partnership with:

Matt has been paid to develop software for the past 12 years. He specializes in mobile and web development and has recently been doing a lot with Windows Phone 7. He runs DevEvening (http://devevening.co.uk/) a .net focused user group in Surrey and the Windows Phone User Group (http://wpug.net/) in London. He blogs at http://blog.mrlacey.co.uk/ and tweets at @mrlacey & @wpug. Matt is a DZone MVB and is not an employee of DZone and has posted 102 posts at DZone. You can read more from them at their website. View Full User Profile

Is There a Need for Emulators If You Have to Test on Actual Devices?

11.29.2012
| 4380 views |
  • submit to reddit

This is in response to a comment on my post about the undocumented way I found of creating a xap file you can't deploy to a #wp8 device

In that post I claimed that it was important to test on physical devices as well as in the emulator. The comment stated 

"this seems to break the whole "emulator" thing, since it seems that it doesn't properly emulate a physical device!"

Firstly, I'm not aware of anyone ever claiming that an emulator completely does away with the need for testing on real devices. 

I've only heard people claim you MUST test on physical hardware in addition to an emulator/simulator. This goes back many, many years and has always been essential for phones as emulators have never given a complete representation of actual devices. They've gotten a lot better but by their very nature they're not perfect.

This is important:
An emulator (or simulator) emulates (or simulates) a real device. It doesn't claim to give a complete representation of a real device.

What is an emulator actually good for then?

  • Testing quickly
  • Testing multiple devices or configurations and switching between them quickly
  • Automated testing
  • Testing before devices are available.
  • Testing on devices you can't buy (due to financial or location restrictions) - but beware that this means you won't know about the experience of the app on those devices. This may or may not be an issue.
  • Avoiding issues with the number of side-loaded apps installed on an actual device. (Or maybe I just need to keep track of what I side-load better)
  • Testing use in different locations (without needing to create your own mocking layer)
  • Testing use of the accelerometer (without needing to create your own mocking layer)
  • Testing different network conditions (network speed & signal strength) - again without having to create your own mocking layer.
  • Testing app behaviour when reminders are triggered.

What must you test on an actual device?

  • What it's really like to use the app in the hand (your thumb and a mouse pointer are not the same!)
  • What the actual performance is like on actual devices
  • Any functionality that isn't available in the emulator (interruption from incoming phone calls, etc.)
  • That the behaviour of the app on an actual device is the same as the emulator.

This list assumes use of VS2012 for testing Windows Phone (8) apps. For other platforms and emulators/simulators the above lists may vary slightly.

Published at DZone with permission of Matt Lacey, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)