What should be in an IT survey?

Developing an IT survey can be a daunting task. IT departments that have never run a survey before will tend to ask every combination of questions, making the survey long and difficult to answer.  Others will cover only part of the IT services, getting only a partial view of user satisfaction. We will see the major components of an IT survey and a few guidelines to build your own survey.

1. Demographic information

Ideally, the It department would already have demographic information about its users (department, location, language, etc). But unfortunately, this information is often difficult to access, stored in different databases with various owners or in writing somewhere. If the information is not available, we recommend to ask at least a few questions that will facilitate analysis later on such as the user's department, physical location, language, years with the company and their comfort level with technology.

2. IT Dimensions

These are the static questions we like to use from one survey to another. It is important to keep a set of questions static so that you can compare results from one survey to another. This gives a way for the IT department to benchmark itself over time. Using standardized questions also allows you to benchmark yourself to other IT organizations. Survey companies typically use a set of standard questions that are pre-formulated and valid.

The eight dimensions we use are:

  • Projects: Satisfaction with project management.
  • Communication: Frequency and quality of IT communications with end users.
  • Application: Effectiveness of current application portfolio.
  • Information: Availability and reliability of information.
  • Equipment: Availability, performance and reliability of equipment.
  • Process: Effectiveness of IT processes.
  • Support: Effectiveness of support.
  • Staff: Skills level and friendliness of support staff.

You can of course decide to exclude some dimensions if they are not appropriate for your own objectives.

We also recommend to use a Likert scale with 7 choices:

  • Very dissatisfied
  • Dissatisfied
  • Somewhat dissatisfied
  • neutral
  • Somewhat satisfied
  • Satisfied
  • Very satisfied

Research has shown that users don't feel they can accurately represent their opinion with less than 7 choices while more than 7 simply becomes useless. We've seen surveys with as little as 4 choices which always brought back amazing satisfaction results. When we switched to seven we saw that the reality was entirely different.

3. Survey length

One of the major issue with IT surveys is managing response rate. When a survey is too long, response rate falls dramatically (users stop responding to the survey in the middle of it).

A user should be able to complete a survey in less than 10 minutes and with a minimum of typing. This means most questions should be answerable with a mouse click.

Although it is tempting to ask hundreds of questions, research shows that reliability of the answers drops severely after a while. It is thus better to keep the survey short and follow-up with a second survey  or interviews later on if there are questions left open. The survey should be a quick health-check tool, it shouldn't try to diagnostic all possible issues.

4. Survey invitations

Survey invitations will have a dramatic impact on response rate. IT departments often believe that users will respond to a survey simply because they send it out. The reality is that users are solicited by all of the departments and outside companies every day.

Thus, the invitation should make clear the following:

  • The purpose of the survey
  • Number of question and / or how long the survey will take
  • If the survey is confidential
  • The deadline for responding to the survey
  • Contact info for any questions.

Most survey tool also allows to track respondents. This makes it possible to send reminders to the ones that didn't fill out the survey already. Two reminders in a 10-14 days timeframe is typical.

Or get someone else to do it for you

Web tools have made it easy for IT departments to build and administer surveys. But just like having Photoshop doesn't make me an artist, having access to tools doesn't guarantee a valid survey. IT departments can save effort and mistakes by using survey services built on best practices.

Take-aways

  • IT surveys should be kept short
  • IT surveys are a health-check tool, not a in-depth diagnostic tool
  • Managing response rate is critical
Posted on October 18, 2013 .