Emerical Research

RESEARCH PROPOSAL

Empirical Research

Presented by:

Santosh

Table of Contents

Abstract....................................................................................................................................1

Objective...................................................................................................................................2

Benefits.....................................................................................................................................3

Empirical Cycle.......................................................................................................................4

Quantitative Methods..............................................................................................................5

Qualitative Methods.................................................................................................................6

Data Analysis............................................................................................................................7

Literature Review.....................................................................................................................8

Empirical Research methods...................................................................................................9

Database Documentation......................................................................................................10

System Service and Utilisation..............................................................................................11

Intend method of Empirical research..................................................................................12

Conclusion..............................................................................................................................13

References..............................................................................................................................14

Abstract

Empirical Research is a research that based on experimentation or observation, i.e. Evidence. Such research is often conducted to answer a specific question or test a hypothesis.

1. Theory.

2. Design.

3. Evaluation.

4. Implementation.

Objective:

• Capture contextual data and complexity.

• Learn from the collective experience of the field.

• Identify, explore, confirm and advance theoretical concepts.

• Enhance educational design.

• Combine rigorous research thorough case study.

• Foster environments to enhanced understanding.

• Relevance of theory is proved by ability to work in a real world.

Benefits:

• Establish relationship between intervention and behavioural response.

• Understand and respond to dynamics of situation.

• Respect of contextual differences.

• Build up what is already known to work.

• Meet accepted professionals of research.

Empirical cycle:

[pic]

Observation: The collecting and organisation of empirical facts.

Induction: Formulating hypothesis.

Deduction: deducting consequences of the hypothesis as testable predictions.

Testing: testing the hypothesis with new empirical material.

Evaluation: evaluating the outcome of testing.

What is Empirical research?

Empirical research methods are the class of research methods in which empirical observations or data are collected on order to answer a particular research questions. While primarily used in academic research, they can also be used in answering particular questions.

The popularity of empirical methods in software engineering researches is in the rise. Surveys, Experiments, Metrics, Case studies, and field studies are examples of empirical methods used to investigate both in software engineering processes and products.

• Research questions.

• Theoretical models.

• Hypothesis.

Resource questions:

Initially a research question is formulated. For examples:

• Is programming language A more effective than programming language B?

• What are the critical success factors in implementing an ERP system?

Theoretical Models:

The theoretical model forms the basis both for collecting and analysis data, and may be modified as a result of research. The theoretical Model is generally developed based on analysis of the literature.

Hypothesis:

A hypothesis defines an expected relationship between variables based on the casual relationships in the theoretical model which can be empirically tested.

Empirical research can be divided into two categories:

• Quantitative research method: such methods collect numerical data (data in the form of numbers) and analyse is using statistical methods.

• Qualitative research methods: such methods collect qualitative data (data in the form of text, images, sounds) drawn from the observation, interviews and documentary evidence, and analyse it using qualitative data analysis methods.

Qualitative methods tend to be more appropriate in the early stages of research and for theory building. Quantitative methods tend to be more appropriate when theory is well developed and for the purpose of refining and testing. A survey may collect qualitative data using open ended questions as well as quantitative data using closed questions; an experiment may include observation of participant’s behaviour as well as measures of response time and accuracy.

Quantitative methods:

The most common quantitative methods are:

• Experiment: apply a treatment, measure results (before or after): this is the only method that can demonstrate casual relationships between variables. Experiment research is associated with the traditional “scientific method” (the “Newtonian” model of science).

• Surveys: ask questions (face to face interviews, telephone, mail, and internet).

• Historical data: look for pattern in historical data (e.g.: IT investment patterns)

While quantitative methods tend to result in more convincing scientific evidence , they are generally more difficult to apply in the real world.

Qualitative method:

The most common qualitative methods are:

• Case study: observation carried out in real world setting (e.g. a software developer project, an opening theatre in a hospital).

• Action Research: apply a research idea in practice, evaluate results, modify idea( cross between an experiment and case study).

Data analysis:

Used to define whether your theory is supported or not supported.

• Quantitative data analysis: use of statistical methods to identify patterns and relationship in the data.

• Qualitative data analysis: analysis is more subjective, and relies heavily on the researcher’s knowledge and experience to identify patterns, extract themes and make generalisation

The qualitative research in research world is about the Information system and products. And those with useful information on the conduct, evaluation and qualitative research. Qualitative research involves data such as interviews, documents and participation observation of data, to understand and explain social phenomenon. Information System, there has been a general shift in Information System research away from technological to managerial and organisational issues, hence an increasing interest in the qualitative research methods.

Overview of Quantitative and Qualitative research methods:

Research methods can be classified in many ways.

|[pic] |[pic] |
|[pic] |[pic] |
|[pic] |[pic] |
|[pic] |[pic] |
|[pic] |[pic] |
|[pic] |[pic] |
| | |
| | |
| | |

• To develop a table of analysis from the data gathered and analysed by means of survey

• To come up with a research paper which lays lights on these evolving issues?

Literature Review:

Selecting a research method for software engineering research is so problematic because the benefits and the challenges to using each method are not yet well catalogued. Many researchers select an appropriate method, that they discuss key question to consider in selecting a method from philosophy consideration about the nature of knowledge.

Software Engineering is a multi-disciplinary field, crossing many social and technological boundaries. To understand how software engineering construct and maintain complex, evolving software systems.

Because of the importance of human activities in software development, many of the research methods that are appropriate to software engineering are drawn from disciplines that study human behaviour both at the individual level and at the team and organisational levels.

Describing in detail the wide variety of possible empirical methods. Identifying and comparing six classes of research method that believe are most relevant to Software Engineering.

• Controlled experiments.

• Case studies.

• Survey research.

• Ethnographies.

• Action research.

• Database research.

For Guiding Example:

Jane is a new PhD student interested in the effectiveness of a novel fisheye-view file navigator. Research is motivated by the fact that navigation is primary activity of software developers requiring a lot of scrolling and many clicks to find files ‘Fisheye-views’ use a distortion technique that, if applied correctly, display information in a compact format that could potentially reduce the amount of scrolling required. Jane’s intuition is that the fisheye-view file navigator is more efficient for file navigator, but critics argue that the more compact information is difficult to read and that developers will not adopt the traditional file navigator. Her research goal therefore is to find evidence that supports or refuses her intuition that the fisheye-view file navigators are more efficient than traditional file navigators for navigation.

Quantitative Research Methods:

[pic]

1. Positivist Research:
Positivist generally assumes that reality is objective given and can be described by measurable properties which are independent of the observer and, in an attempt to increase the predictive understanding of phenomena. Information Research as positivist, if there was evidence of formal propositions, quantifiable measures of variable, hypothesis testing, and the drawing of inferences about a phenomenon from the sample of stated systems. 2. Interpretive Research:
Interpretive research start out with the assumptions that access reality given is only through social constructions such as language, consciousness and shared meanings. Generally it attempts to understand phenomena through the meaning that people assign to them and interpretive methods of research in IS are “aimed at producing an understanding of the context of the information system, and the process whereby the information system influences and is influenced by the context”. Interpretive research does not predefine dependent and independent variables, but focused on the complexity of human sense making a situation. 3. Critical Research:
Critical researchers assume that social reality is historically constituted and that it is produced and reproduced by people. Although people can consciously act to change their social and economical circumstances, critical researchers recognize that their ability to do so is constrained by various forms of social, cultural, and political domination. The main task of critical research is seen as being one of social critique; where by restrictive and alienating conditions of the status quo are brought to light.

Qualitative Research Methods:

Action Research:

There are numerous definitions of action research, however one of the most widely cited. Action research aims to contribute both to the practical concerns of people in an immediate problematic situation and to the goals of social science by joint collaboration within a mutually acceptable ethical framework. Action research has been accepted as a valid research method in applied fields such as organisation development and education. Information systems, however, action research was for a long time largely ignored, part from one or two notable exceptions.

Case Study Research:

The term “case study” has multiple meanings. It can be used to describe a unit of analysis(e.g. a case study of a particular organisation) or to describe a research method. The discussion here concerns the use of the case study a research method.

Case study is an empirical inquiry that: • Investigate a contemporary phenomenon within its real-life context, especially when • The boundaries between phenomenon and the context are not clearly evident.
Clearly, the case study research method is particularly well-suited to Information system research, since the object of our discipline is the study of information systems in organisations, and “interest has shifted to organisation rather than technical issues”.

1. Ethnography:
Ethnographic research comes from the discipline of social and cultural anthropology where an ethnographer is required to spend a significant amount of time in the field. Ethnographers immerse themselves in the lives of the people they study (Lewis 1985,p.380)and seek to place the phenomena studied in their social and cultural context.
In the area of the design and evaluation of information systems, some very interesting work is taking place in a collaborative fashion between ethnographers on the one hand, and designers, IS professionals ,computer scientists and engineers on the other. This collaborative work is especially strong in the UK and Europe and is growing in the US.

Grounded Theory:

Grounded theory is a research method that to develop theory that is grounded in data systematically gathered and analyzed. According to Martin and Timer(1986,) grounded theory is “an inductive, theory discovery methodology that allows the researcher to develop a theoretical account of the general features of a topic while simultaneously grounding the account in empirical observations or data”. The major difference between grounded theory and other methods is its specific approach to theory development – grounded theory suggests that there should be a continuous interplay between data collection and analysis.

Grounded theory approaches are becoming increasingly common in the IS research literature because the method is extremely useful in developing context- based, process-oriented descriptions and explanations of the phenomenon.

Qualitative Techniques of data Store:

Each of the research methods discussed above uses one or more techniques for collecting empirical data (many qualitative researchers prefer the term “empirical materials” to the word “data” since most qualitative data is non-numeric). These techniques range from interviews, observational techniques such as participant observation and fieldwork, through to archival research. Written data sources can include published and unpublished documents, company reports, memos, letters, reports, email messages, faxes, newspaper articles and so forth.

In anthropology and sociology it is a common practice to distinguish between primary and secondary sources of data. Generally speaking, primary sources are those data which are unpublished and which the research has gathered from the people or organization directly
Typically, a case study researcher uses interviews and documentary materials first and foremost, without using participant observation. The distinguishing feature of ethnography, however, is that the researcher spends a significant amount of time in the field. The fieldwork notes and the experience of living there become an important addition to any other data gathering techniques that may be used.

Modes of Analysis:

Although a clear distinction between data gathering and data analysis is commonly made in Quantitative research, such a distinction is problematic for many qualitative researches. For example, from a hermeneutic perspective it is assumed that the researcher’s presuppositions affect the gathering of the data – the questions posed to informants largely determine what you are going to find out. The analysis affects the data and the data affect the analysis in significant ways. Therefore it is perhaps more accurate to speak of “modes of analysis” rather than “data analysis” in qualitative research. These modes of analysis are different approaches to gathering, analyzing and interpreting qualitative data. The common thread is that all qualitative modes of analysis are concerned primarily with textual analysis.

Although there are many different modes of analysis in qualitative research, just three approaches or modes of analysis will be discussed here: hermeneutics, semiotics, and approaches which focus on narrative and metaphor. It could be argued that grounded theory is also a mode of analysis, but since grounded theory has been discussed earlier, that discussion will not be repeated here.

Hermeneutics:

“Interpretation, in the sense relevant to hermeneutics, is an attempt to make clear, to make sense of an object of study. This object must, therefore, be a text, or a text-analogue, which in some way is confused, incomplete, cloudy, seemingly contradictory- in one way or another, unclear. The interpretation aims to bring to light an underlying coherence or sense”

There are different forms of hermeneutic analysis, from”pure” hermeneutics through to “critical” hermeneutics. If hermeneutic analysis is used in an information systems study, the object of the interpretative effort becomes one of attempting to make sense of the organization as a text-analogue. In an organization people (e.g. different stakeholders) can have confused, incomplete, cloudy and contra dictionary views on many issues. The aim of the hermeneutic analysis becomes one of trying to make sense of the whole, and the relationship between people, the organisation and information technology.

2. Semiotics:

Like hermeneutics, semiotics can be treated as both an underlying philosophy and a specific mode of analysis. The following discussion concerns using semiotics as a mode of analysis. Semiotics is primarily concerned with the meaning of signs and symbols in language. The essential idea is that words/signs can be assigned to primary conceptual categories, and these categories represent important aspects of the theory to be tested. The importance of an idea is revealed in the frequency with which it appears in the text.

One form of semiotics is “content analysis.” Krippendorf (1980) defines content analysis as “a research technique for making replicable and valid references from data their contexts.”The researcher searches for structures and patterned regularities in the text and makes inferences on the basis of these regularities.

Another form of semiotics is “discourse analysis.”In conversation analysis, it is assumed that the meanings are shaped in the context of the exchange.

A third form of semiotics is “discourse analysis.”Discourse analysis builds on both content analysis and conversation analysis but focuses on “language games.”A language game refer to a well-defined unit of interaction consisting of a sequence of verbal moves in which turns of phrases, the use of metaphor and allegory all play important part.

3. Narrative and Metaphor:

Narrative is defined by the concise Oxford English dictionary as a “tale, story, recital of facts, especially story told in the first person.”There are many kinds of narrative, from oral narrative through to historical narrative. Metaphor is the application of a name or descriptive term or phrase to an object or action to which it is not literally applicable (e.g. a window in Windows 95).

Narrative and metaphor have long been key terms in literary discussion and analysis. In recent years there has been increase recognition of the role they play in all types of thinking and social practice. Scholars in many disciplines have looked at areas such as metaphor and symbolism in indigenous cultures, oral narrative, narrative and metaphor in organizations, metaphor and medicine, metaphor and psychiatry etc.

In IS the focus has mostly been on understanding language, communication and meaning among system developers and organizational members. In recent year’s narrative, metaphor and symbolic analysis has become a regular theme in the world.

Database documentation for empirical research:

• Browsing.

• Searching.

Text Search fields

List of Values

Yes/No Fields

• Registering.

Browsing:

Browsing is a good way to access documents if you don’t have a specific idea of what you’re looking for. There are number of ways to browse the database, by year, theme/sub-theme, author, HESA subject, publisher, (progress).

To browse the database select Browse from the front page. Then choose which property you wish to browse by e.g. “theme/sub-theme”.

You will be presented with a list of possible values, select one of these, and you will be given a list of references to summaries in the database which match this value. To access a summary, simply click on its reference in the display.

On the abstract page, you should be able to see all the attributes of the summary. Clicking on one of these will take you back to the relevant browse.

Searching:

The database offers two levels of searching, simple and advanced. They are similar, but the advanced form lets you perform a finer-grained search using more fields. Access the simple search using the search link on the front page. To perform an advanced search, use the advanced search link at the top of the simple search page.

Text Search Fields:

These are used to search fields like title, abstract or author. These are the fields where there is a text entry area, and a popup menu just to the right of it. Type your search terms into the box. You can decide how the system will use your search terms by selecting one of the options from the popup menu just to the right of the input box.

Match all, in any order:

In the example shown, the system will search for records in which any of the title, abstract or keywords fields contain both the word “student” and ”feedback”.

Match any:

In the example shown, the system will search for any record with either the term ”student” or “feedback” in any of the title, abstract or keywords fields.

List of Values:

With these you can select one or more values from a list of values for the system to search for. If no value in the list is selected, the system will ignore this field (i.e. it will retrieve records with any value of this field.)
In case where each individual record may have one more than one value attached to the list, you can also change search behaviour by selecting “Any of these” or “All of these” from the popup menu on the right of the list.

Registering:

Certain services require you to register with the system so that it can identify you and allow you access to the information stored in the database. When you register with the system, you can register a username, email address and password which must be confirmed. The system will email you a confirmation URL to visit to enable the password.
In the absence of empirical research on systems of service delivery for this population, we summarized the empirical literature on:
1) Empirical data on prevention and intervention programs that have implications for system and that have data on their effect on children’s social/emotional development and/or challenging behaviour

And

By components of system of service delivery:

2) Comprehensiveness and individualization, family support, collaboration and coordination, eligibility and access, finance, and workforce.

Systems of Service: • Systems must facilitate and support a comprehensive array of services from prevention to intensive intervention. • Systems must support the delivery of high quality services. The quality of early education and care environments is strongly related to the development of social competence. • Research efforts should seek to identify the specific program and system features needed to optimally intervene with young children who are having problem behaviour such as resources, staff ratios, background and training, collaboration with experts, time, family involvement and interagency collaboration. • Systems should be family-centred. • The early care and education, mental health, health and child welfare workforce must be skilled in evidence-based promotion, prevention and intervention strategies. • The science of promotion, prevention, and intervention efforts not expanded far beyond the development of model demonstration programs. Research efforts must examine the transportability of evidence-based practice to usual care settings.

Service Utilization:

• Research is needed to establish system for identifying and replicating evidence-based programs.

• There is a need for longitudinal research, retrospective and prospective, that carefully chart the development of challenging behaviours in the preschool years and research to evaluate screening system that are practical for use at a community level.

The intend method for this research:

In addition, empirical guidelines are often specialised to consider particular types of study e.g. randomised trials, surveys, exploratory studies. Clearly, the particular requirements for a set of guidelines influence their content and format. In the long term, if the software community accepts the need for experimental guidelines, we would expect to find specialised guidelines for different purposes.

We consider guidelines for what to do and what not to do under six basic topic areas: • Experimental context

• Experimental design

• Conduct of the experiment and Data collection

• Analysis

• Presentation of results

• Interpretation of results.

Experimental context:

Experimental context is extremely important for software engineering research.

Experimental context has three elements:

1) Background information about the industrial circumstances in which an empirical study take place or in which a new software engineering technique is developed.
2) Discussion of the research hypotheses and how they were derived.
3) Information about related research.

The main goals of context view are:

1) To ensure that the objectives of the research are properly defined
2) To ensure that the description of the research provides enough detail for other researchers and for practitioners.

Industrial context information is important in two types of empirical software engineering studies:

1) Observational studies (i.e. Studies of industrial practice) and
2) Formal experiments evaluating techniques developed in industry (e.g. inspections, or design patterns).

There is an immense variety to be found in development procedures, organizational culture, and products. This breadth implies that empirical studies based on observing or measuring some aspect of software development in a particular company must report a great deal of contextual information if any results and implications are to be properly understood. Research need to identify the particular factors that might affect the generality and utility of the conclusions.
For example they may need to identify factors such as:

1) The industry in which products are used (e.g. banking, consumer goods, telecommunications, and travel)
2) The nature of the software development organization (e.g. in-house information system department, independent software supplier)
3) The skills and experience of software staff (e.g. with a language, a method, a tool, an application domain)
4) The type of software products used (e.g. a design tool, a compiler)
5) The software processes being used (e.g. a company-standard process, the quality assurance procedures, and the configuration management process).

Such information is essential if the same or other researchers want to replicate a study or include it in a meta-analysis.

Software engineering researchers are becoming more familiar with the concept of stating their scientific hypotheses. However, all too often the so-called hypothesis is simply a statement of the tests to be performed. For example, if researchers want to know whether there is a relationship between the cyclometer number and the number of faults found in a module, they may state the null hypothesis as “there is no correlation between cyclometer number and faults found.” We call this statement a shallow hypothesis, because it does not reflect an underlying, explanatory theory. That is, whatever the result of the experiment, we will not be increasing our software engineering knowledge in any significant way . Furthermore, this approach is ultimately sterile, since it leads to experimental results that are not of great interest.

Experimental Design:

• The population being studied; • The rational and technique for sampling from the population; • The process for allocating and administering the treatments(the term “intervention” is often used as an alternative to treatment); • The methods used to reduce bias and determine sample size. The overall goal of experimental design is to ensure that the design is appropriate for the objectives of the study.

Conducting the Experiment and data collection:

The process of conducting an experiment involves collecting the experimental outcome measures. This is particular problem for software experiments because our measures are not standardized. Thus, one goal of the data collection is to ensure that we have defined our data collection process well enough for an experiment to be replicated.

In order to provide evidence that data collection has been undertaken in an appropriate manner, it is useful to define and report quality control procedures to support the data collection process.

Analysis:

There are two main approaches to analyse experimental results:
1. Classical analysis (often referred as the “frequents” approach). This approach is adopted by most statistical packages.

2. Bayesian analysis. This approach provides a systematic means of making use of “prior information”. Prior information may be obtained from previous studies of the phenomenon of interest or from expert opinion.
Analysis aim to ensure that the experimental results are analysed correctly. Basically, the data should be analysed in accordance with the study design. Thus, the advantage of doing a careful, well-defined study design is that the subsequent analysis is usually straight forward and clear. However, in many software engineering examples the selected design is complex, and the analysis method is inappropriate to cope with it.

Presentation of results:

1. Report information about within person difference when using paired data.
2. Report the magnitude of an effect size.
3. Report confidence limits for inferential statistics including means, means differences, correlation coefficients and regression coefficients.
4. Report the alpha level used for statistical testing.
5. Report whether the tests were two-tailed or one-tailed.
6 Report the value of the software statistics.

We will not repeat all the proceedings context, design, and analysis and data collection restarting them as presentation.

Interpretation of results:

Researchers have a responsibility to discuss any limitations of their study. In our view this is not a major issue as long as you are interested in the evaluating the use of techniques by novice or non-expert software engineers. The techniques of restricted problems with known solutions, it is impossible to be sure that techniques evaluated under such circumstances will scale up to industrial size systems or very novel programming problems.

Conclusion:

Even though the value-based approach to outsourcing has been discussed by the few aforementioned research, there is no value-based mode or empirical research, forward on
Benefits of outsourcing in particular functional area of the organisation, provides the methodology to measure outcome of outsourcing initiative after companies are well into outsourcing initiatives but there is no research or a tool available for companies to decide on the outsourcing before company got onto it. The overall affect of outsourcing initiatives, which will help companies to calculate the impact of outsourcing on overall organisation value before they get into outsourcing.

References

1. Dill man, D.A (2000): Mail and Internet Surveys: The Tailored Design Method (2nd edition), John Wiley & sons, New York.

2. Mehdi Khosrow Pour (2006), Emerging Trends and Challenges in Information Technology Management, By Idea Group Inc (IGI).

3. Peter Flooper (1991), International Economic Transactions: Issues in measurement and empirical research, by University of Chicago Press.

4. Jack Szmatka (2002), the growth of Social Knowledge: theory, simulation, and empirical research in group processes, by Greenwood Publishing Group.

5. Jahrestagung (2003), Exploratory data analysis in empirical research, by Springer.

References: 1. Dill man, D.A (2000): Mail and Internet Surveys: The Tailored Design Method (2nd edition), John Wiley & sons, New York. 2. Mehdi Khosrow Pour (2006), Emerging Trends and Challenges in Information Technology Management, By Idea Group Inc (IGI). 3. Peter Flooper (1991), International Economic Transactions: Issues in measurement and empirical research, by University of Chicago Press. 4. Jack Szmatka (2002), the growth of Social Knowledge: theory, simulation, and empirical research in group processes, by Greenwood Publishing Group. 5. Jahrestagung (2003), Exploratory data analysis in empirical research, by Springer.