Information Gathering

C H A P T E R

4

Information Gathering: Interactive Methods
LEARNING OBJECTIVES
Once you have mastered the material in this chapter you will be able to: 1. Recognize the value of interactive methods for information gathering. 2. Construct interview questions to elicit human information requirements. 3. Structure interviews in a way that is meaningful to users. 4. Understand the concept of JAD and when to use it. 5. Write effective questions to survey users about their work. 6. Design and administer effective questionnaires.

PA RT I I Information Requirements Analysis

There are three key interactive methods that you can use to elicit human information requirements from organizational members.These three methods are interviewing, joint application design (JAD), and surveying people through questionnaires. Although different in their implementation, these methods have a great deal in common, too. The basis of their shared properties is talking with and listening to people in the organization to understand their interactions with technology through a series of carefully composed questions. Each of the three interactive methods for information gathering possesses its own established process for you to follow in interacting with users. If followed, these systematic approaches will help ensure proper design and implementation of interviews, JAD workshops, and questionnaires, as well as support insightful analysis of the resulting data. Unobtrusive methods (sampling, investigation, and observing a decision maker’s behavior and physical environment) that do not require the same degree of interactivity between analysts and users will be covered in an upcoming chapter. By using interactive methods with unobtrusive methods you will achieve a more complete portrait of the organization’s information requirements.

INTERVIEWING
Before you interview someone else, you must in effect interview yourself. You need to know your biases and how they will affect your perceptions. Your education, intellect, upbringing, emotions, and ethical framework all serve as powerful filters for what you will be hearing in your interviews. You need to think through the interview thoroughly before you go. Visualize why you are going, what you will ask, and what will make it a successful interview in your eyes. You must anticipate how to make the interview fulfilling for the individual you interview, as well. An information-gathering interview is a directed conversation with a specific purpose that uses a question-and-answer format. In the interview you want to get the opinions of the interviewee and his or her feelings about the current state of the system, organizational and personal goals, and informal procedures for interacting with information technologies.

103

104

PART II • INFORMATION REQUIREMENTS ANALYSIS

Above all, seek the opinions of the person you are interviewing. Opinions may be more important and more revealing than facts. For example, imagine asking the owner of a traditional store who has recently added an online store how many customer refunds she typically gives for Web transactions each week. She replies, “About 20 to 25 a week.” When you monitor the transactions and discover that the average is only 10.5 per week, you might conclude that the owner is overstating the facts and the problem. Imagine instead that you ask the owner what her major concerns are and that she replies, “In my opinion, customer returns of goods purchased over the Web are way too high.” By seeking opinions rather than facts, you discover a key problem that the owner wants addressed. In addition to opinions, you should try to capture the feelings of the interviewee. Remember that the interviewee knows the organization better than you do. You can understand the organization’s culture more fully by listening to the feelings of the respondent. Goals are important information that can be gleaned from interviewing. Facts that you obtain from hard data may explain past performance, but goals project the organization’s future. Try to find out as many of the organization’s goals as possible from interviewing. You may not be able to determine goals through any other data-gathering methods. The interview is also a valuable time to explore key HCI (human–computer interaction) concerns, including the ergonomic aspects, the system usability, how pleasing and enjoyable the system is, and how useful it is in supporting individual tasks. In the interview you are setting up a relationship with someone who is probably a stranger to you. You need to build trust and understanding quickly, but at the same time you must maintain control of the interview. You also need to sell the system by providing needed information to your interviewee. Do so by planning for the interview before you go so that conducting it is second nature to you. Fortunately, effective interviewing can be learned. As you practice, you will see yourself improving. Later in the chapter we discuss joint application design (JAD) (pronounced as one word, jad, rhymes with add), which can serve as an alternative to one-on-one inˇ terviewing in certain situations.

Five Steps in Interview Preparation
The five major steps in interview preparation are shown in Figure 4.1. These steps include a range of activities from gathering basic background material to deciding who to interview.
READ BACKGROUND MATERIAL. Read and understand as much background information about

the interviewees and their organization as possible. This material can often be obtained on the corporate Web site, from a current annual report, a corporate newsletter, or any publications sent out to explain the organization to the public. Check the Internet for any corporate information such as that in Standard and Poor’s. As you read through this material, be particularly sensitive to the language the organizational members use in describing themselves and their organization. What you are trying to do is build up a common vocabulary that will eventually enable you to phrase interview questions in a way that is understandable to your interviewee. Another benefit of researching your organization is to maximize the time you spend in interviews; without such preparation you may waste time asking general background questions.
ESTABLISH INTERVIEWING OBJECTIVES. Use the background information you gathered as well as

your own experience to establish interview objectives. There should be four to six key areas concerning HCI, information processing, and decision-making behavior about which you will want to ask questions. These areas include HCI concerns (the usefulness and usability of the

FIGURE 4.1
Steps the systems analyst follows in planning the interview. 1. 2. 3. 4. 5.

Steps in Planning the Interview
Read background material. Establish interviewing objectives. Decide whom to interview. Prepare the interviewee. Decide on question types and structure.

CHAPTER 4 • INFORMATION GATHERING: INTERACTIVE METHODS

105

system; how it fits physical aspects; how it suits a user’s cognitive capabilities, whether it is engaging or aesthetically pleasing; and whether using the system is rewarded with desired consequences), information sources, information formats, decision-making frequency, qualities of information, and decision-making style.
DECIDE WHOM TO INTERVIEW. When deciding whom to interview, include key people at all

levels who will be affected by the system in some manner. Strive for balance so that as many users’ needs are addressed as possible. Your organizational contact will also have some ideas about whom should be interviewed.
PREPARE THE INTERVIEWEE. Prepare the person to be interviewed by calling ahead or sending an

email message and allowing the interviewee time to think about the interview. If you are doing an in-depth interview, it is permissible to email your questions ahead of time to allow your interviewee time to think over his or her responses. Because there are many objectives to fulfill in the interview (including building trust and observing the workplace), however, interviews should typically be conducted in person and not via email. Interviews should be kept to 45 minutes or an hour at the most. No matter how much your interviewees seem to want to extend the interview beyond this limit, remember that when they spend time with you, they are not doing their work. If interviews go over an hour, it is likely that the interviewees will resent the intrusion, whether or not they articulate their resentment.
DECIDE ON QUESTION TYPES AND STRUCTURE. Write questions to cover the key areas of HCI and

decision making that you discovered when you ascertained interview objectives. Proper questioning techniques are the heart of interviewing. Questions have some basic forms you need to know. The two basic question types are open-ended and closed. Each question type can accomplish something a little different from the other, and each has benefits and drawbacks. You need to think about the effect each question type will have. It is possible to structure your interview in three different patterns: a pyramid structure, a funnel structure, or a diamond structure. Each is appropriate under different conditions and serves a different function, and each one is discussed later in this chapter.

Question Types
OPEN-ENDED QUESTIONS. Open-ended questions include those such as “What do you think

about putting all the managers on an intranet?” “Please explain how you make a scheduling decision.” “In what ways does the system extend your capability to do tasks that would not be possible otherwise?” Consider the term open-ended. “Open” actually describes the interviewee’s options for responding. They are open. The response can be two words or two paragraphs. Some examples of open-ended questions are found in Figure 4.2. The benefits of using open-ended questions are numerous and include the following: 1. Putting the interviewee at ease. 2. Allowing the interviewer to pick up on the interviewee’s vocabulary, which reflects his or her education, values, attitudes, and beliefs. 3. Providing richness of detail. 4. Revealing avenues of further questioning that may have gone untapped. 5. Making it more interesting for the interviewee. 6. Allowing more spontaneity. FIGURE 4.2
Open-ended interview questions allow the respondent open options for responding. The examples were selected from different interviews and are not shown in any particular order.

Open-Ended Interview Questions
• What’s your opinion of the current state of business-to-business ecommerce in your firm? • What are the critical objectives of your department? • Once the data are submitted via the Web site, how are they processed? • Describe the monitoring process that is available online. • What are some of the common data entry errors made in this department? • What are the biggest frustrations you’ve experienced during the transition to ecommerce?

106

PART II • INFORMATION REQUIREMENTS ANALYSIS

FIGURE 4.3
Closed interview questions limit the respondent’s options. The examples were selected from different interviews and are not shown in any particular order.

Closed Interview Questions
• How many times a week is the project repository updated? • On average, how many calls does the call center receive monthly? • Which of the following sources of information is most valuable to you? ° Completed customer complaint forms ° Email complaints from consumers who visit the Web site ° Face-to-face interaction with customers ° Returned merchandise • List your top two priorities for improving the technology infrastructure. • Who receives this input?

7. Making phrasing easier for the interviewer. 8. Using them in a pinch if the interviewer is caught unprepared. As you can see, there are several advantages to using open-ended questions. There are, however, also many drawbacks: 1. Asking questions that may result in too much irrelevant detail. 2. Possibly losing control of the interview. 3. Allowing responses that may take too much time for the amount of useful information gained. 4. Potentially seeming that the interviewer is unprepared. 5. Possibly giving the impression that the interviewer is on a “fishing expedition” with no real objective for the interview. You must carefully consider the implications of using open-ended questions for interviewing.
CLOSED QUESTIONS. The alternative to open-ended questions is found in the other basic question

type: closed questions. Such questions are of the basic form “Is it easy to use the current system?” and, “How many subordinates do you have?” The possible responses are closed to the interviewee, because he or she can only reply with a finite number such as “None,” “One,” or “Fifteen.” Some examples of closed questions can be found in Figure 4.3. A closed question limits the response available to the interviewee. You may be familiar with closed questions through multiple-choice exams in college. You are given a question and five responses, but you are not allowed to write down your own response and still be counted as having correctly answered the question. A special kind of closed question is the bipolar question. This type of question limits the interviewee even further by only allowing a choice on either pole, such as yes or no, true or false, agree or disagree. Examples of bipolar questions can be found in Figure 4.4. The benefits of using closed questions of either type include the following: 1. 2. 3. 4. 5. 6. Saving time. Easily comparing interviews. Getting to the point. Keeping control over the interview. Covering lots of ground quickly. Getting to relevant data.

FIGURE 4.4
Bipolar interview questions are a special kind of closed question. The examples were selected from different interviews and are not shown in any particular order. • • • •

Bipolar Interview Questions
Do you use the Web to provide information to vendors? Do you agree or disagree that ecommerce on the Web lacks security? Do you want to receive a printout of your account status every month? Does your Web site maintain a FAQ page for employees with payroll questions? • Is this form complete?

CHAPTER 4 • INFORMATION GATHERING: INTERACTIVE METHODS

107

Open-Ended
Low Reliability of Data

Closed
High

FIGURE 4.5
Attributes of open-ended and closed questions.

Low

Efficient Use of Time

High

Low

Precision of Data

High

Much

Breadth and Depth

Little

Much

Interviewer Skill Required

Little

Difficult

Ease of Analysis

Easy

The drawbacks of using closed questions are substantial, however. They include the following: 1. Being boring for the interviewee. 2. Failing to obtain rich detail (because the interviewer supplies the frame of reference for the interviewee). 3. Missing main ideas for the preceding reason. 4. Failing to build rapport between interviewer and interviewee. Thus, as the interviewer, you must think carefully about the question types you will use. Both open-ended and closed questions have advantages and drawbacks, as shown in Figure 4.5. Notice that choosing one question type over the other actually involves a trade-off; although an open-ended question affords breadth and depth of reply, responses to open-ended questions are difficult to analyze.
PROBES. A third type of question is the probe or follow-up. The strongest probe is the simplest:

the question, “Why?” Other probes are “Can you give me an example of a time you did not find the system trustworthy?” and “Will you elaborate on that for me?” Some examples of probing questions can be found in Figure 4.6. The purpose of the probe is to go beyond the initial answer to get more meaning, to clarify, and to draw out and expand on the interviewee’s point. Probes may be either open-ended or closed questions. It is essential to probe. Most beginning interviewers are reticent about probing and consequently accept superficial answers. They are usually grateful that employees have granted interviews and feel somewhat obligated to accept unqualified statements politely.

Arranging Questions in a Logical Sequence
Just as there are two generally recognized ways of reasoning—inductive and deductive—there are two similar ways of organizing your interviews. A third way combines both inductive and deductive patterns. FIGURE 4.6
Probes allow the systems analyst to follow up on questions to get more detailed responses. The examples were selected from different interviews and are not shown in any particular order.

Probes
• Why? • Give an example of how ecommerce has been integrated into your business processes. • Please give an illustration of the security problems you are experiencing with your online bill payment system. • You mentioned both an intranet and an extranet solution. Please give an example of how you think each differs. • What makes you feel that way? • Tell me step by step what happens after a customer clicks the “Submit” button on the Web registration form.

108

PART II • INFORMATION REQUIREMENTS ANALYSIS

C O N S U LT I N G O P P O R T U N I T Y 4 . 1

Strengthening Your Question Types trongbodies, a large, local chain of sports clubs, has experienced phenomenal growth in the past five years. Management would like to refine its decision-making process for purchasing new bodybuilding equipment. Currently, managers listen to customers, attend trade shows, look at advertisements, and put in requests for new equipment purchases based on their subjective perceptions. These are then approved or denied by Harry Mussels. Harry is the first person you will interview. He is a 37-year-old division manager who runs five area clubs. He travels all over the city to their widespread locations. He keeps an office at the East location, although he is there less than a quarter of the time. In addition, when Harry is present at a club, he is busy answering business-related phone calls, solving on-the-spot prob-

S

lems presented by managers, and interacting with club members. His time is short, and to compensate for that he has become an extremely well-organized, efficient divisional manager. He cannot grant you a lot of interview time. However, his input is important, and he feels he would be the main beneficiary of the proposed system. What type of interview question might be most suitable for your interview with Harry? Why is this type most appropriate? How will your choice of question type affect the amount of time you spend in preparation for interviewing Harry? Write 5 to 10 questions of this type. What other techniques might you use to supplement information unavailable through that type of question? Write a paragraph to explain.

USING A PYRAMID STRUCTURE. Inductive organization of interview questions can be visualized

as having a pyramid shape. Using this form, the interviewer begins with very detailed, often closed, questions. The interviewer then expands the topics by allowing open-ended questions and more generalized responses, as shown in Figure 4.7. A pyramid structure should be used if you believe your interviewee needs to warm up to the topic. Using a pyramid structure for question sequencing is also useful when you want an ending

FIGURE 4.7
Pyramid structure for interviewing goes from specific to general questions. sp Pyramid structures start with a … ecific question

What specifically is the problem you are experiencing with your firewall?

Have you considered other methods to improve the security of corporate data?

What do you think would make security more effective here?

In general, how do you feel about the security of data versus the importance of Internet access?

… and end with a general one.

CHAPTER 4 • INFORMATION GATHERING: INTERACTIVE METHODS

109

determination about the topic. Such is the case in the final question, “In general, how do you feel about the security of data versus the importance of Internet access?”
USING A FUNNEL STRUCTURE. In the second kind of structure, the interviewer takes a deductive

approach by beginning with generalized, open-ended questions and then narrowing the possible responses by using closed questions. This interview structure can be thought of as funnel shaped, as that depicted in Figure 4.8. Using the funnel structure method provides an easy, nonthreatening way to begin an interview. A funnel-shaped question sequence is also useful when the interviewee feels emotional about the topic and needs freedom to express those emotions.
USING A DIAMOND-SHAPED STRUCTURE. Often a combination of the two structures, resulting in

a diamond-shaped interview structure, is best. This structure entails beginning in a very specific way, then examining general issues, and finally coming to a very specific conclusion, as shown in Figure 4.9. The interviewer begins with easy, closed questions that provide a warm-up to the interview process. In the middle of the interview, the interviewee is asked for opinions on broad topics that obviously have no “right” answer. The interviewer then narrows the questions again to get specific questions answered, thus providing closure for both the interviewee and the interviewer. The diamond structure combines the strengths of the other two approaches but has the disadvantage of taking longer than either other structure. The end of the interview is a natural place to ask one key question: “Is there anything we haven’t touched on that you feel is important for me to know?” Considered a formula question by the interviewee most of the time, the response will often be “No.” You are interested in the other times, when this question opens the proverbial floodgates and much new data are presented, though. As you conclude the interview, summarize and provide feedback on your overall impressions. Inform the interviewee about the subsequent steps to take and what you and other team members will do next. Ask the interviewee with whom you should talk next. Set up future appointment times for follow-up interviews, thank the interviewee for his or her time, and shake hands.

Writing the Interview Report
Although the interview itself is complete, your work on the interview data is just beginning. You need to capture the essence of the interview through a written report. It is imperative that you write the interview report as soon as possible after the interview. This step is another way you can ensure quality of interview data. The longer you wait to write up your interview, the more suspect the quality of your data becomes.

FIGURE 4.8
What are your reactions to the new Web-based procurement system?

What departme nts

Funnel structure for interviewing begins with broad questions, then funnels to specific questions.

? are involved in implementing it

ures Funnel struct begin with a tion … general ques

What items will be available for purchase on the Web site? Is there any essential item that has been excluded from the site?

th … and end wi specific one. a

110

PART II • INFORMATION REQUIREMENTS ANALYSIS

C O N S U LT I N G O P P O R T U N I T Y 4 . 2

Skimming the Surface ou are about to leave SureCheck Dairy after a preliminary tour when another member of your systems analysis team calls you at the dairy to say he cannot make his interview appointment with the plant manager because of illness. The plant manager is extremely busy, and you want to keep his enthusiasm for the project going by doing things as scheduled. You also realize that without the initial interview data, the rest of your data gathering will be slowed. Although you have no interview questions prepared, you make the decision to go ahead and interview the plant manager on the spot. You have learned that SureCheck is interested in processing its own data on quantities and kinds of dairy products sold so that its

Y

people can use that information to better control production of the company’s large product line (it includes whole, skim, 2 percent, and 1 percent milk, half-and-half, cottage cheese, yogurt, and frozen novelties). Sales managers are currently sending their sales figures to corporate headquarters, 600 miles away, and processing turnaround seems slow. You will base your ad-libbed questions on what you have just found out on the tour. In the few minutes before your interview begins, decide on a structure for it: funnel, pyramid, or diamond. In a paragraph, justify why you would proceed with the interview structure you have chosen based on the unusual context of this interview. Write a series of questions and organize them in the structure you have chosen.

After this initial summary, go into more detail, noting main points of the interview and your own opinions. Review the interview report with the respondent at a follow-up meeting. This step helps clarify the meaning the interviewee had in mind and lets the interviewee know that you are interested enough to take the time to understand his or her point of view and perceptions. FIGURE 4.9
Diamond-shaped structure for interviewing combines the pyramid and funnel structures.
Diamond gin structures be a specific with question …
What five kinds of information are tracked by the free Web site usage service you use?

What are the promotional activities you feature on your Web site in exchange for this service?

… move toward general questions …

What is the value of the usage information to you as a Webmaster?

What are two surprising items concerning end user behavior on your site that you have discovered by using this service?

Are “cookies” a better way to measure end user site usage?

th a … and end wi . on ecific questi sp

CHAPTER 4 • INFORMATION GATHERING: INTERACTIVE METHODS

111

JOINT APPLICATION DESIGN
No matter how adept you become as an interviewer, you will inevitably experience situations in which one-on-one interviews do not seem to be as useful as you would like. Personal interviews are time consuming and subject to error, and their data are prone to misinterpretation. An alternative approach to interviewing users one by one, called joint application design (JAD), was developed by IBM. The motivation for using JAD is to cut the time (and hence the cost) required by personal interviews, to improve the quality of the results of information requirements assessment, and to create more user identification with new information systems as a result of the participative processes. Although JAD can be substituted for personal interviews at any appropriate juncture during the SDLC, it has usually been employed as a technique that allows you, as a systems analyst, to accomplish requirements analysis and to design the user interface jointly with users in a group setting. The many intricacies of this approach can only be learned in a paid seminar demonstrating proprietary methods. We can, however, convey enough information about JAD here to make you aware of some of its benefits and drawbacks in comparison with one-on-one interviews.

Conditions That Support the Use of JAD
The following list of conditions will help you decide when the use of JAD may be fruitful. Consider using joint application design when: 1. User groups are restless and want something new, not a standard solution to a typical problem. 2. The organizational culture supports joint problem-solving behaviors among multiple levels of employees. 3. Analysts forecast that the number of ideas generated via one-on-one interviews will not be as plentiful as the number of ideas possible from an extended group exercise. 4. Organizational workflow permits the absence of key personnel during a two-to-four-day block of time.

Who Is Involved?
Joint application design sessions include a variety of participants—analysts, users, executives, and so on—who will contribute differing backgrounds and skills to the sessions. Your primary concern here is that all project team members are committed to the JAD approach and become involved. Choose an executive sponsor, a senior person who will introduce and conclude the JAD session. Preferably, select an executive from the user group who has some sort of authority over the IS people working on the project. This person will be an important, visible symbol of organizational commitment to the systems project. At least one IS analyst should be present, but the analyst usually takes a passive role, unlike traditional interviewing in which the analyst controls the interaction. As the project analyst, you should be present during JAD to listen to what users say and what they require. In addition, you will want to give an expert opinion about any disproportionate costs of solutions proposed during the JAD session itself. Without this kind of immediate feedback, unrealistic solutions with excessive costs may creep into the proposal and prove costly to discourage later on. From eight to a dozen users can be chosen from any rank to participate in JAD sessions. Try to select users who can articulate what information they need to perform their jobs as well as what they desire in a new or improved computer system. The session leader should not be an expert in systems analysis and design but rather someone who has excellent communication skills to facilitate appropriate interactions. Note that you do not want to use a session leader who reports to another person in the group. To avoid this possibility, an organization may want to retain an outside management consultant to serve as session leader. The point is to get a person who can bring the group’s attention to bear on important systems issues, satisfactorily negotiate and resolve conflicts, and help group members reach a consensus. Your JAD session should also include one or two observers who are analysts or technical experts from other functional areas to offer technical explanations and advice to the group during the sessions. In addition, one scribe from the IS department should attend the JAD sessions to formally write down everything that is done.

112

PART II • INFORMATION REQUIREMENTS ANALYSIS

HYPERCASE® EXPERIENCE 4.1

ell, I did warn you that things weren’t always smooth here at MRE. By now you’ve met many of our key employees and are starting to understand the ‘lay of the land.’Who would have thought that some innocent decisions about systems, like whether to purchase a COMTEX or Shiroma, would cause such hostility? Well, live and learn, I always say. At least now you’ll know what you’re up against when you have to start recommending software and hardware! “It’s funny that not all questions are created equal. I myself favor asking open-ended questions, but when I have to answer them, it is not always easy.

“W

HYPERCASE Questions
1. Using the interview questions posed in HyperCase, give five examples of open-ended questions and five examples of closed questions. Explain why your examples are correctly classified as either open-ended or closed question types. 2. List three probing questions that are part of the Daniel Hill interview. In particular, what did you learn by following up on the questions you asked Daniel? 3. List three probing questions that are part of the Snowden Evans interview. In particular, what did you learn by following up on the questions you asked Snowden?

FIGURE 4.HC1
Pointing to a question in HyperCase will reveal an answer.

Where to Hold JAD Meetings
If at all possible, we recommend holding the two-to-four-day sessions off-site, away from the organization, in comfortable surroundings. Some groups use executive centers or even group decision support facilities that are available at major universities. The idea is to minimize the daily distractions and responsibilities of the participants’ regular work. The room itself should comfortably hold the number of people invited. Minimal presentation support equipment includes two overhead projectors, a whiteboard, a flip chart, and easy access to a copier. Group decision support rooms will also provide networked PCs, a projection system, and software written to facilitate group interaction while minimizing unproductive group behaviors. Schedule your JAD session when all participants can commit to attending. Do not hold the sessions unless everyone who has been invited can actually attend. This rule is critical to the success of the sessions. Ensure that all participants receive an agenda before the meeting, and con-

CHAPTER 4 • INFORMATION GATHERING: INTERACTIVE METHODS

113

C O N S U LT I N G O P P O R T U N I T Y 4 . 3

A Systems Analyst, I Presume? now what I think of the work the last systems analyst team did? The printouts created are a jungle. To figure out the cost of raw materials to us, I have to cut my way through the overgrowth of data, hacking my path with a pen. I cross out everything that’s irrelevant. Sometimes I physically rip out the excess vegetation until I reach the numbers I need,” says Henry Stanley, accounting supervisor for Zenith Glass Company. As you interview him, he points unhappily to an untidy stack of mutilated printouts sprouting beside his desk.

“K

Identify the overriding metaphor Henry is using to describe the printouts he is receiving and the accessibility of information in them. In a paragraph, describe how this step helps you understand Henry’s attitude toward any work proposed by your systems analysis team. In a paragraph, adopt Henry’s metaphor and extend it in a more positive sense during your interview with him.

sider holding an orientation meeting for a half day one week or so before the workshop so that those involved know what is expected of them. Such a premeeting allows you to move rapidly and act confidently once the actual meeting is convened.

Accomplishing a Structured Analysis of Project Activities
IBM recommends that the JAD sessions examine these points in the proposed systems project: planning, receiving, receipt processing/tracking, monitoring and assigning, processing, recording, sending, and evaluating. For each topic, the questions who, what, how, where, and why should also be asked and answered. Clearly, ad hoc interactive systems such as decision support systems and other types of systems dependent on decision-maker style (including prototype systems) are not as easily analyzed with the structured approach of JAD. As the analyst involved with the JAD sessions, you should receive the notes of the scribe and prepare a specifications document based on what happened at the meeting. Systematically present the management objectives as well as the scope and boundaries of the project. Specifics of the system, including details on screen and report layouts, should also be included.

Potential Benefits of Using JAD in Place of Traditional Interviewing
There are four major potential benefits that you, the users, and your systems analysis team should consider when you weigh the possibilities of using joint application design. The first potential benefit is time savings over traditional one-on-one interviews. Some organizations have estimated that JAD sessions have provided a 15 percent time savings over the traditional approach. Hand-in-hand with time savings is the rapid development possible via JAD. Because user interviews are not accomplished serially over a period of weeks or months, the development can proceed much more quickly. A third benefit to weigh is the possibility of improved ownership of the information system. As analysts, we are always striving to involve users in meaningful ways and to encourage users to take early ownership of the systems we are designing. Due to its interactive nature and high visibility, JAD helps users become involved early in systems projects and treats their feedback seriously. Working through a JAD session eventually helps reflect user ideas in the final design. A final benefit of participating in JAD sessions is the creative development of designs. The interactive character of JAD has a great deal in common with brainstorming techniques that generate new ideas and new combinations of ideas because of the dynamic and stimulating environment. Designs can evolve through facilitated interactions, rather than in relative isolation.

Potential Drawbacks of Using JAD
There are three drawbacks or pitfalls that you should also weigh when making a decision on whether to do traditional one-on-one interviews or to use JAD. The first drawback is that JAD requires the commitment of a large block of time from all participants. Because JAD requires a

114

PART II • INFORMATION REQUIREMENTS ANALYSIS

two-to-four-day commitment, it is not possible to do any other activities concurrently or to timeshift any activities, as is typically done in one-on-one interviewing. A second pitfall occurs if preparation for the JAD sessions is inadequate in any regard or if the follow-up report and documentation of specifications is incomplete. In these instances resulting designs could be less than satisfactory. Many variables need to come together correctly for JAD to be successful. Conversely, many things can go wrong. The success of designs resulting from JAD sessions is less predictable than that achieved through standard interviews. Finally, the necessary organizational skills and organizational culture may not be sufficiently developed to enable the concerted effort required to be productive in a JAD setting. In the end you will have to judge whether the organization is truly committed to, and prepared for, this approach.

USING QUESTIONNAIRES
The use of questionnaires is an information-gathering technique that allows systems analysts to study attitudes, beliefs, behavior, and characteristics of several key people in the organization who may be affected by the current and proposed systems. Attitudes are what people in the organization say they want (in a new system, for instance); beliefs are what people think is actually true; behavior is what organizational members do; and characteristics are properties of people or things. Responses gained through questionnaires (also called surveys) using closed questions can be quantified. If you are surveying people via email or the Web, you can use software to turn electronic responses directly into data tables for analysis using a spreadsheet application or statistical software packages. Responses to questionnaires using open-ended questions are analyzed and interpreted in other ways. Answers to questions on attitudes and beliefs are sensitive to the wording chosen by the systems analyst. Through the use of questionnaires, the analyst may be seeking to quantify what was found in interviews. In addition, questionnaires may be used to determine how widespread or limited a sentiment expressed in an interview really is. Conversely, questionnaires can be used to survey a large sample of system users to sense problems or raise important issues before interviews are scheduled. Throughout this chapter, we compare and contrast questionnaires with interviews. There are many similarities between the two techniques, and perhaps the ideal would be to use them in conjunction with each other, either following up unclear questionnaire responses with an interview or designing the questionnaire based on what is discovered in the interview. Each technique, however, has its own specific functions, and it is not always necessary or desirable to use both.

Planning for the Use of Questionnaires
At first glance questionnaires may seem to be a quick way to gather massive amounts of data about how users assess the current system, about what problems they are experiencing with their work, and about what people expect from a new or modified system. Although it is true that you can gather a lot of information through questionnaires without spending time in face-to-face interviews, developing a useful questionnaire takes extensive planning time in its own right. When you decide to survey users via email or the Web, you face additional planning considerations concerning confidentiality, authentication of identity, and problems of multiple responses. You must first decide what you are attempting to gain through using a survey. For instance, if you want to know what percentage of users prefers a FAQ page as a means of learning about new software packages, a questionnaire might be the right technique. If you want an in-depth analysis of a manager’s decision-making process, an interview is a better choice. Here are some guidelines to help you decide whether the use of questionnaires is appropriate. Consider using questionnaires if: 1. The people you need to question are widely dispersed (different branches of the same corporation). 2. A large number of people are involved in the systems project, and it is meaningful to know what proportion of a given group (for example, management) approves or disapproves of a particular feature of the proposed system. 3. You are doing an exploratory study and want to gauge overall opinion before the systems project is given any specific direction.

CHAPTER 4 • INFORMATION GATHERING: INTERACTIVE METHODS

115

4. You wish to be certain that any problems with the current system are identified and addressed in follow-up interviews. Once you have determined that you have good cause to use a questionnaire and have pinpointed the objectives to be fulfilled through its use, you can begin formulating questions.

Writing Questions
The biggest difference between the questions used for most interviews and those used on questionnaires is that interviewing permits interaction between the questions and their meanings. In an interview the analyst has an opportunity to refine a question, define a muddy term, change the course of questioning, respond to a puzzled look, and generally control the context. Few of these opportunities are possible on a questionnaire. Thus, for the analyst, questions must be transparently clear, the flow of the questionnaire cogent, the respondent’s questions anticipated, and the administration of the questionnaire planned in detail. (A respondent is the person who responds to or answers the questionnaire.) The basic question types used on the questionnaire are open-ended and closed, as discussed for interviewing. Due to the constraints placed on questionnaires, some additional discussion of question types is warranted.
OPEN-ENDED QUESTIONS. Recall that open-ended questions (or statements) are those that leave

all possible response options open to the respondent. For example, open-ended questions on a questionnaire might read, “Describe any problems you are currently experiencing with output reports” or “In your opinion, how helpful are the user manuals for the current system’s accounting application?” When you write open-ended questions for a questionnaire, anticipate what kind of response you will get. For instance, if you ask a question such as, “How do you feel about the system?” the responses are apt to be too broad for accurate interpretation or comparison. Therefore, even when you write an open-ended question, it must be narrow enough to guide respondents to answer in a specific way. (Examples of open-ended questions can be found in Figure 4.10.) Open-ended questions are particularly well suited to situations in which you want to get at organizational members’ opinions about some aspect of the system, whether product or process. In such cases you will want to use open-ended questions when it is impossible to list effectively all the possible responses to the question.
CLOSED QUESTIONS. Recall that closed questions (or statements) are those that limit or close the

response options available to the respondent. For example, in Figure 4.11 the statement in question 23 (“Below are the six software packages currently available. Please check the software package(s) you personally use most frequently”) is closed. Notice that respondents are not asked why the package is preferred, nor are they asked to select more than one, even if that is a more representative response. Closed questions should be used when the systems analyst is able to list effectively all the possible responses to the question and when all the listed responses are mutually exclusive, so that choosing one precludes choosing any of the others. Use closed questions when you want to survey a large sample of people. The reason becomes obvious when you start imagining how the data you are collecting will look. If you use only openended questions for hundreds of people, correct analysis and interpretation of their responses becomes impossible without the aid of a computerized content analysis program. There are trade-offs involved in choosing either open-ended or closed questions for use on questionnaires. Figure 4.12 summarizes these trade-offs. Notice that responses to open-ended questions can help analysts gain rich, exploratory insights as well as breadth and depth on a topic. Although open-ended questions can be written easily, responses to them are difficult and time consuming to analyze. When we refer to the writing of closed questions with either ordered or unordered answers, we often refer to the process as scaling. The use of scales in surveys is discussed in detail in a later section.
WORD CHOICE. Just as with interviews, the language of questionnaires is an extremely important

aspect of their effectiveness. Even if the systems analyst has a standard set of questions

116

PART II • INFORMATION REQUIREMENTS ANALYSIS

FIGURE 4.10
Open-ended questions used for questionnaires.
53.

What are th e most freq uent proble experience ms you with comp uter outpu t? A.

B.

C.

54.

Of the prob lems you lis ted above, most troub what is the lesome? single

Open-ended n questions ca k the as respondent for lists …

55.

Why?

… or detailed responses …

… or short answers.

Below are questions about you blanks to rself. Pleas the best o e fill in the f your abili ty. 67. How long have you w orked for this comp any? Years and Months 68. How long have you w orked in th e same ind Years and ustry? Months 69. In what oth er industri es have yo u worked?

concerning systems development, it is wise to write them to reflect the business’s own terminology. Respondents appreciate the efforts of someone who bothers to write a questionnaire reflecting their own language usage. For instance, if the business uses the term supervisors instead of managers, or units rather than departments, incorporating the preferred terms in the questionnaire helps respondents relate to the meaning of the questions. Responses will be easier to interpret accurately, and respondents will be more enthusiastic overall. To check whether language used on the questionnaire is that of the respondents, try some sample questions on a pilot (test) group. Ask them to pay particular attention to the appropriateness of the wording and to change any words that do not ring true. Here are some guidelines to use when choosing language for your questionnaire: 1. Use the language of respondents whenever possible. Keep wording simple. 2. Work at being specific rather than vague in wording. Avoid overly specific questions as well.

CHAPTER 4 • INFORMATION GATHERING: INTERACTIVE METHODS

117

FIGURE 4.11
Answer ques 23.

king the appr opriate box. Below are th e six softwar e packages available. P currently lease check the you person ally use mos software package(s) t frequently . [ ] Microso ft Excel [ ] Microso [ ] Microso ft PowerPoi ft Access nt [ ] Mic [ ] Oracle tions rosoft Windo SCM Closed ques ws Live Mai [ ] Visible l require the Analyst may 24. to "The sales fig respondent ures are usua lly late." eck a box … ch [ ] Agree [ ] Disagre e Answer ques tions 25 an d 26 by circ ling the appr 25. opriate num "When the sa ber. les figures ar e prepared services they by compute are late." r data Never Rarely Sometimes 1 Often 2 Always 3 4 5 … or circle a number …

tions 23 an

d 24 by chec

Closed questions on questionnaires help ensure responses.

Answer qu 45.

estions 45–4

8 by circlin

g the approp d

The division

riate respon

se.

I am curren

Investments Operations Marketing

tly in is calle

the … or circle lf. answer itse

46.

nd can best be describe High Schoo d as l Some Colle ge Bachelor’s Degree Master’s D egree or Hig her My gender is Male Female

My educatio

nal backgrou

Open-Ended
Slow Speed of Completion

Closed
Fast

FIGURE 4.12
Trade-offs between the use of open-ended and closed questions on questionnaires.

High

Exploratory Nature

Low

High

Breadth and Depth

Low

Easy

Ease of Preparation

Difficult

Difficult

Ease of Analysis

Easy

118

PART II • INFORMATION REQUIREMENTS ANALYSIS

3. 4. 5. 6.

Keep questions short. Do not patronize respondents by talking down to them through low-level language choices. Avoid bias in wording. Avoiding bias also means avoiding objectionable questions. Target questions to the correct respondents (that is, those who are capable of responding). Don’t assume too much knowledge. 7. Ensure that questions are technically accurate before including them. 8. Use software to check whether the reading level is appropriate for the respondents.

Using Scales in Questionnaires
Scaling is the process of assigning numbers or other symbols to an attribute or characteristic for the purpose of measuring that attribute or characteristic. Scales are often arbitrary and may not be unique. For example, temperature is measured in a number of ways; the two most common are the Fahrenheit scale (where water freezes at 32 degrees and boils at 212 degrees) and the Celsius scale (where freezing occurs at 0 degrees and boiling at 100 degrees).
MEASUREMENT. There are two different forms of measurement scales commonly used by

systems analysts: 1. nominal scales and 2. interval scales. Nominal scales are used to classify things. A question such as: What type of software do you use the most? 1 2 3 4 A Word Processor A Spreadsheet A Database An Email Program

uses a nominal scale. Obviously, nominal scales are the weakest forms of measurement. Generally, all the analyst can do with them is obtain totals for each classification. Interval scales possess the characteristic that the intervals between each of the numbers are equal. Due to this characteristic, mathematical operations can be performed on the questionnaire data, resulting in a more complete analysis. Examples of interval scales are the Fahrenheit and Celsius scales, which measure temperature. The foregoing example of the Information Center is definitely not that of an interval scale, but by anchoring the scale on either end, the analyst may want to assume the respondent perceives the intervals to be equal: How useful is the support given by the Technical Support Group?
Not Useful at All 1 2 3 4 Extremely Useful 5

If the systems analyst makes this assumption, more quantitative analysis is possible.
VALIDITY AND RELIABILITY. There are two measures of performance in constructing scales:

validity and reliability. The systems analyst should be aware of these concerns. Validity is the degree to which the question measures what the analyst intends to measure. For example, if the purpose of the questionnaire is to determine whether the organization is ready for a major change in computer operations, do the questions measure that? Reliability measures consistency. If the questionnaire was administered once and then again under the same conditions and if the same results were obtained both times, the instrument is said to have external consistency. If the questionnaire contains subparts and these parts have equivalent results, the instrument is said to have internal consistency. Both external and internal consistency are important.

CHAPTER 4 • INFORMATION GATHERING: INTERACTIVE METHODS

119

CONSTRUCTING SCALES. The actual construction of scales is a serious task. Careless construction

of scales can result in one of the following problems: 1. Leniency. 2. Central tendency. 3. Halo effect. Leniency is a problem caused by respondents who are easy raters. A systems analyst can avoid the problem of leniency by moving the “average” category to the left (or right) of center. Central tendency is a problem that occurs when respondents rate everything as average. The analyst can improve the scale (1) by making the differences smaller at the two ends, (2) by adjusting the strength of the descriptors, or (3) by creating a scale with more points. The halo effect is a problem that arises when the impression formed in one question carries into the next question. For example, if you are rating an employee about whom you have a very favorable impression, you may give a high rating in every category or trait, regardless of whether or not it is a strong point of the employee’s. The solution is to place one trait and several employees on each page, rather than one employee and several traits on a page.

Designing the Questionnaires
Many of the same principles that are relevant to the design of forms for data input (as covered in Chapter 12) are important here as well. Although the intent of the questionnaire is to gather information on attitudes, beliefs, behavior, and characteristics whose impact may substantially alter users’ work, respondents are not always motivated to respond. Remember that organizational members as a whole tend to receive too many surveys, many of which are often ill-conceived and trivial. A well-designed, relevant questionnaire can help overcome some of this resistance to respond. Here are some rules for designing a good questionnaire: 1. 2. 3. 4. Allow ample white space. Allow ample space to write or type in responses. Make it easy for respondents to clearly mark their answers. Be consistent in style.

When you design questionnaires for the Web, apply the same rules you use when designing paper questionnaires. Most software packages allow you to insert one of the commonly used data entry formats shown in Figure 4.13. Following the four guidelines should help you gain a better response rate to the questionnaire.

Name
One-line text box

Appearance

Purpose
Used to obtain a small amount of text and limit the answer to a few words Used to obtain one or more paragraphs of text Used to obtain a yes-no answer (e.g., Do you wish to be included on the mailing list?) Used to obtain a yes-no or true-false answer Used to obtain more consistent results (Respondent is able to choose the appropriate answer from a predetermined list [e.g., a list of state abbreviations]) Most often used for an action (e.g., a respondent pushes a button marked “Submit” or “Clear”)

FIGURE 4.13
When designing a Web survey, keep in mind that there are different ways to capture responses.

Scrolling text box

Check box Radio button

Drop-down menu

Push button

120

PART II • INFORMATION REQUIREMENTS ANALYSIS

C O N S U LT I N G O P P O R T U N I T Y 4 . 4

The Unbearable Questionnaire
'm going to go into a depression or at least a slump if someone doesn’t figure this out soon,” say Penny Stox, office manager for Carbon, Carbon, & Rippy, a large brokerage firm. Penny is sitting across a conference table from you and two of her most productive account executives, By Lowe and Sal Hy. You are all mulling over the responses to a questionnaire that has been distributed among the firm’s account executives, which is shown in Figure 4.C1. “We need a crystal ball to understand these,” By and Sal call out together. “Maybe it reflects some sort of optimistic cycle, or something,” Penny says as she reads more of the responses. “Who designed this gem, anyway?”

“I

“Rich Kleintz,” By and Sal call out in unison. “Well, as you can see, it’s not telling us anything!” Penny exclaims. Penny and her staff are dissatisfied with the responses they have received on the unbearable questionnaire, and they feel that the responses are unrealistic reflections of the amount of information account executives want. In a paragraph, state why these problems are occurring. On a separate sheet, change the scaling of the questions to avoid these problems.

Circle the appr 1. Industry Re Less 1 2. Trend Analy Less 1 3. Computer-G Less 1 4. Investment Less 1

opriate numbe

ports

r for each sour ce of informati About the Same 3 About the Same 3

on described.

2 sis

4

More 5

2 enerated Grap 2 Advisory Serv 2 ices hs

4

More 5

About the Same 3 About the Same 3

We need to change this ire. questionna -Penny

4

More 5

4

More 5

5. Point an

d Figure Ch

arts About the Same 3 About the Same 3 About the Same 3

Less 1 6. Compute Less 1 7. Hot Tips Less 1 rized Portfo

2 lio Analysis

4

More 5

2

4

More 5

2

4

More 5

FIGURE 4.C1
Questionnaire developed for the brokerage firm of Carbon, Carbon, & Rippy by Rich Kleintz.

CHAPTER 4 • INFORMATION GATHERING: INTERACTIVE METHODS

121

C O N S U LT I N G O P P O R T U N I T Y 4 . 5

Order in the Courts love my work,” Tennys says, beginning the interview with a volley. “It’s a lot like a game. I keep my eye on the ball and never look back,” he continues. Tennyson “Tennys” Courts is a manager for Global Health Spas, Inc., which has popular health and recreation spas worldwide. “Now that I’ve finished my MBA, I feel like I’m on top of the world with Global,” Tennys says. “I think I can really help this outfit shape up with its computers and health spas.” Tennys is attempting to help your systems group, which is developing a system to be used by all 80 outlets (where currently each group

“I

handles its paperwork in its own way). “Can I bounce this off you?” he asks Terri Towell, a member of your team of systems analysts. “It’s a questionnaire I designed for distribution to all spa managers.” Ever the good sport, Terri tells Tennys that she’d love to take a look at the form. But back in the office, Terri puts the ball in your court. Systematically critique Tennys’s technique as depicted in Figure 4.C2, and explain to him point by point what it needs to be a matchless questionnaire with a winning form. Building on your critique, tell Tennys what he should do to rewrite the form as a Web survey instead.

QUESTIONNAIRE FOR ALL MANAGERS OF HEALTH SPAS ***URGENT***FILL OUT IMMEDIATELY AND RETURN PERSONALLY TO YOUR DIVISION MANAGER. YOUR NEXT PAYCHECK WILL BE WITHHELD UNTIL IT IS CONFIRMED THAT YOU HAVE TURNED THIS IN. In 10 words or fewer, what complaints have you lodged about the current computer system in the last six months to a year? Are there others who feel the same way in your outlet as you do? Who? List their names and positions. 1. 3. 5. 7. 2. 4.
Terri Please help this me improve rm. fo Tennys

What is the biggest problem you have when communicating your information requirements to headquarters? Describe it briefly. How much computer downtime did you experience last year? 1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 - 10 Is there any computer equipment you never use? Description Serial Number Neutral Disagree

Do you want it removed? Agree

In your opinion, what’s next as far as computers and Global Health Spas are concerned?

Thanks for filling this out. • • • • • • • • • • • • • • • •

FIGURE 4.C2
Questionnaire developed for managers of Global Health Spas by Tennys Courts.

QUESTION ORDER. There is no best way to order questions on the questionnaire. Once again, as you

order questions, you must think about your objectives in using the questionnaire and then determine the function of each question in helping you to achieve your objectives. It is also important to see the questionnaire through the respondent’s eyes. Some guidelines for ordering questions are: 1. Place questions that are important to respondents first. 2. Cluster items of similar content together. 3. Introduce less controversial questions first. You want respondents to feel as unthreatened by and interested in the questions being asked as possible, without getting overwrought about a particular issue.

122

PART II • INFORMATION REQUIREMENTS ANALYSIS

Administering Questionnaires
RESPONDENTS. Deciding who will receive the questionnaire is handled in conjunction with the

task of setting up objectives for its results. Sampling, which is covered in Chapter 5, helps the systems analyst to determine what sort of representation is necessary and hence what kind of respondents should receive the questionnaire. Recipients are often chosen as representative because of their rank, length of service with the company, job duties, or special interest in the current or proposed system. Be sure to include enough respondents to allow for a reasonable sample in the event that some questionnaires are not returned or some response sheets are incorrectly completed and thus must be discarded.
METHODS OF ADMINISTERING THE QUESTIONNAIRE. The systems analyst has several options for

administering the questionnaire, and the choice of administration method is often determined by the existing business situation. Options for administering the questionnaire include the following: 1. Convening all concerned respondents together at one time. 2. Personally handing out blank questionnaires and taking back completed ones. 3. Allowing respondents to self-administer the questionnaire at work and drop it in a centrally located box. 4. Mailing questionnaires to employees at branch sites and supplying a deadline, instructions, and return postage. 5. Administering the questionnaire electronically either via email or on the Web. Each of these five methods has advantages and disadvantages. Most commonly, respondents are allowed to self-administer the questionnaire. Response rates with this method are a little lower than with the other methods, because people may forget about the form, lose it, or purposely ignore it. Self-administration, however, allows people to feel that their anonymity is ensured and may result in less guarded answers from some respondents. Both email and Web surveys fall into the category of self-administered questionnaires. Administering the questionnaire electronically, either via email or posted on the Web, is one way to quickly reach current system users. Costs of duplication are minimized. In addition, responses can be made at the convenience of the respondent and then can be automatically collected and stored electronically. Some software permits respondents to begin answering a survey, save their answers, and return to it for completion if they are interrupted. Reminders to respondents can be easily and inexpensively sent via email, as can notifications to the analyst about when the respondent has opened the email. Some software now turns email data into data tables for use in spreadsheet or statistical analysis software. One of the popular services for creating and administering online surveys used by consultants can be found at SurveyMonkey.com, www.surveymonkey.com, a Portland, Oregon, company started in 1999, which recently added email marketing services called MailChimp to their product offerings. Research shows that respondents are willing to answer questions about highly sensitive matters via the Internet. Thus, questions that may be difficult to pose in person regarding systems problems may be acceptable to ask on a Web survey.

SUMMARY
This chapter covers three of the key interactive methods for information gathering that the systems analyst can use, including interviewing, JAD, and construction of questionnaires. During the process of interviewing analysts, listen for HCI concerns relating to ergonomics, aesthetics, usability, and usefulness, as well as goals, feelings, opinions, and informal procedures in interviews with organizational decision makers. Interviews are planned question-and-answer dialogues between two people. Analysts use the interview to develop their relationship with a client, to observe the workplace, and to collect data. Interviews should preferably be conducted in person. The five steps to take in planning the interview are to read background material, establish interviewing objectives, decide whom to interview, prepare the interviewee, and decide on question types and structure. Questions are of two basic types: open-ended or closed. Open-ended questions leave open all response options for the interviewee. Closed questions limit the possible options for response. Probes or follow-up questions can be either open-ended or closed, but they ask the respondent for a more detailed reply. Interviews can be structured in three basic ways: pyramid, funnel, or diamond. Pyramid structures begin with detailed, closed questions and broaden to more generalized questions. Funnel structures begin with