Epitiro announced its first ISP broadband service performance results for 2009 today, and it was Slingshot and Orcon who headed the performance rankings.
“As broadband services have evolved, Epitiro has also evolved its methodology. We now look at ISP performance from two perspectives:
1. “In The Home” - User experience analysis: the experience customers have of an ISP’s service and;
2. “In The Lab” - Service performance analysis: the performance of the actual service in controlled conditions.
“Broadband is tricky to measure, because the ISP doesn’t always have control over all the factors that determine the quality of their service. This was one of the main drivers behind LLU. So when ranking ISPs, we use different methodologies to assess service performance on the one hand, and user experience of that performance on the other.”
“IN THE HOME” - USER EXPERIENCE ANALYSIS
Because a range of factors can affect a user’s experience of broadband, it’s important to understand what actual user experience in the home is.
“This approach includes the effects of some factors which are beyond the ISPs’ control, such as house wiring, phone line and modem quality.
If we have enough measurement points out there in the real world, these factors will average out for each ISP, and we’ll get a sense of what their services are like to the user, despite these other factors. We do this by providing incentives for people to download our software onto their PC. This software then runs regular tests on their ISP’s service.”
“Over the last year, we have measured ISPs in well over 12,000 homes all over the country and across different service plans, so we believe we have good coverage.
We measure performance for common things users do like downloading, surfing and gaming. We do this on a daily basis, which means we have millions of data points to construct a performance picture from.”
However, we have removed ISP results from plans which are rate-limited, as some ISPs offer these services and some don’t, and to include these would unfairly disadvantage those ISPs providing these services. We have only included data for the Big 5: Telecom, Vodafone, TCL, Orcon and Slingshot.
“Orcon’s performance is particularly strong in Auckland, where its unbundled services perform well, but it also tends to be a good performer in most other regions of New Zealand too. Slingshot also performs well. ”
“However, everybody should run tests from their own home and compare the results to other ISPs locally before making a purchase decision. Broadband service can vary from street to street. We encourage all users to download our freeware if you want your service experience to be included in our data.”
A RURAL / URBAN DIVIDE
There is a significant difference between ISP performance in those regions that contain one of New Zealand’s five large cities, and those that don’t. “To use line speed as an example; download speeds outside the big five city regions are a third slower than those within them. Upload speeds are half as fast.
“2008 was a big year for broadband investment. Cabinetisation, content caching, ADSL2+ and unbundling have all had positive impacts on consumer experience of broadband services. But the impacts of these thus far have been felt more in the larger urban areas.”
Auckland and Wellington Regions; Consistently The Best Unsurprisingly, Greater Auckland and the Wellington region top the regional performance charts across all measurements, including downloading and browsing.
Anyone can download Epitiro’s tester for free from www.isposure.co.nz. This tool is unique in that it allows users to compare their ISP’s price and performance to a range of other ISPs in their local area. Users can also vote on their ISP’s performance.
“IN THE LAB” - SERVICE PERFORMANCE ANALYSIS
“Service performance analysis allows us to isolate the components of a broadband plan that an individual ISP can control and measure their performance based just on those factors. We do this in a lab-type environment in 11 city sites around New Zealand.
We create an environment in which we control as many of the external factors as possible, leaving the main variable factor to be the quality of service each ISP provides. We focus on ISPs’ premium plans to do this.”
This tool is particularly good for monitoring network performance, and its data forms the basis for the Commerce Commission reports. Slingshot and Orcon took the top two spots, with Telecom also performing strongly.
METHODOLOGICAL NOTE
Epitiro removed the results from slower services such as those with limited upload and download speeds. “This is because some ISPs offer these services and others don’t, so it would artificially pull down the average of those offering slower and cheaper services.”
ABOUT EPITIRO
Epitiro offers a range of services to Internet Service Providers (ISPs) enabling them to benchmark customer performance across dial up, cable, broadband and wireless connections.
The ISP-I and Isposure platforms and technology from Epitiro emulate an Internet user’s activity and gathers a range of detailed statistics including connection speed, reliability, performance and email delivery for independent analysis.
Media release date: 11 August 2009
