This page is a work in progress. The testing engine is still under development, so final measurement parameters have not been finalized. This page will detail the reviewing process for both the subjective and objective reviewing methods.

Why Test?

One of the major differences between this site’s reviews and reviews you find elsewhere is in the structured and objective testing that is preformed. Reading subjective and direct experience reviews is important but they generally do lack objective information and a way of directly comparing one hosting provider to another. The ideal way to compare web hosts is with “apples to apples” comparison.

It has always been important to objectively compare any service of this type for a variety of reasons.

• First and most importantly moving from one host to another can involve some degree of complication, data loss, downtime, loss of site visitors and loss of positioning in search engines. Picking a reliable host that works for you lets you concern yourself with your web site and not the technical details.
• Many hosting providers are moving to yearly (or longer) pre-payment plans, increasing your commitment. In these cases it is important to know what you are getting into before you commit.
• Getting the most for your money. “You get what you pay for”, has never been less true than in the hosting market. Pricing and marketing practices lead to a situation where price is not always an indicator of the best deal.

Testing Process

Initial Survey. The purpose of the initial survey is to exclude web hosting providers that do not provide a certain minimum feature set and / or already have an excessive number of negative reviews. The result of this survey determines what type of reviewing will be done.

This evaluation is a subjective process that may accidentally exclude respectable, quality hosts. This site does not intent to include every web hosting provider but rather a selection of those that seem to provide the best services.

After the initial survey, providers are placed into one of three groups. The results of this step are kept confidential.

1. Distinguished. These are hosts who have a multitude of positive unsolicited reviews and meet minimum feature requirements. This group will go on to the subjective and objective review phases.
   
2. Acceptable. Hosts in the group may not have many positive reviews or a few mixed reviews. At the reviewer’s discretion both a subjective and object review will be done or only an objective review.
3. Rejected. If a host doesn’t meet the minimum feature requirements and / or has excessive negative reviews, they will not be included for either subjective or objective reviewing. They may however be re-evaluated in the future.

Subjective Review Process

The purpose of the subjective review is to give you an example of what working with the provider is like.

1. Pre-Sales Question. The host is sent a pre-sales question(s) via email. Questions center around typical information a new customer may want to know. e.g. What is the account upgrade process? Is it possible to have a unique IP address? What spam filtering option does the plan come with?
2. Account Sign Up. Any notable details are reported in this step.
3. Control Panel. This step explains what type of control panel the plan includes, details on the speed and responsiveness of the control panel and notes about its ease of use. In cases where the control panel is unique it will be briefly described.
4. CMS Install. A CMS based website is created. Typically Wordpress, Drupal or Joomla are selected. If an automated script installer (like Fantastico) is available, the CMS is installed with it.
5. Site Creation. The CMS system is configured and content is added on an ongoing basis.
6. Technical Support Contact. A request for technical support is sent. The request is not disclosed here, but involves a typical technical support request. E.g. How do I access web statistics? How can I get my site re-directed to www.domain.com from a request to domain.com (without the www)?

Objective Review Process

The purpose of the objective review is to generate a summary of actual web site performance data. A set of testing pages are placed on a web site hosted by the reviewed provider. These pages include static html, dynamic PHP/MySQL generated and large static files.

Results are generated by periodically performing the measurements described below. Multiple hosting accounts are used for each host where possible. Measurements are taken from a variety of testing nodes. Results are normalized for each node so ranking can be calculated with aggregate data from all testing nodes.

The individual measurements that comprise the objective review are described below.

1. TCP Connection Time. This test measures how fast a connection is established.
   (Measured in Seconds)
2. Static Page Retrieval. A standardized static page with images and external style sheet is downloaded. Caching is disabled.
   (Measured in Bits per Second)
3. Dynamic Page Retrieval. A standardized dynamic page with included images and external style sheet is downloaded. The server makes calls to a SQL database in order to generate this page. The design of this test is based on a typical page from a CMS. Caching is disabled.
   (Measured in Bits per Second)
4. Large File Download. A standardized file of greater than 10MB is downloaded. Caching is disabled.
   (Measured in Bits per Second)
5. Test Failures. During the normal course of testing, failure of any test is recorded. Regular failures indicate potentially serious performance problems with the service. Occasional failures are considered normal due to network outages that are not controllable by the host. It is not always possible to determine what caused a test to fail but over the average of many tests “gremlin” effects should affect all hosts to a similar degree. Failures are manually investigated to verify the testing engine is functioning properly. The numeric results should not be seen as an indication of absolute uptime since the testing interval is too large to precisely establish uptime.
   (Measure in Percent)

Minimum Feature Standards

• Shared, VPS or dedicated web hosting.
• A common scripting language. E.g. ASP, PHP, Ruby on Rails etc
• SQL database. E.g. MySQL, PostgreSQL, Microsoft SQL etc.
• Web statistics. E.g. Webalizer, AWStats, Analog etc