Field experiments

Business examples where field experiments would be appropriate are:

(i) a marketing manager wants to test the effects of different types of store displays on sales (i.e. would certain types of displays increase sales more than others);

(ii) a stock broker wants to study the effects of consumer confidence/diffidence on the investment patterns in short-term world market AAA funds before and after a war or major terrorist attack;

(iii) a Finance manager wants to study the effects of changes in current debt to long-term debt ratios, on stock values;

(iv) a manager wants to examine whether dyadic conflict is induced as a function of gender and race differences between the supervisor and the subordinate. In other words, is conflict experienced more when a woman or a minority subordinate reports to a European male supervisor, but not when a European male subordinate reports to a European male supervisor?

Validity and Reliability

1. Why is it important to establish the “goodness” of measures and how is this done?

Answer

The goodness of measures indicates whether the scale we use are reliable and valid. In other words, they indicate to what extent we are accurately and consistently measuring the concept (reliability), and whether we are indeed measuring whatever we are supposed to measure (validity). If we are not measuring the intended concept accurately, then, our study will not yield the correct results.

Reliability is established by assessing the stability of the measure through test-retest reliability and parallel form reliability, and internal consistency of the measure through Cronbach’s alpha.

Validity is established through face validity, content validity, criterion-related validity (concurrent and predictive), and  construct  validity (convergent and discriminant).

2. “Whenever possible, it is advisable to use instruments that have already been developed and repeatedly used in published studies, rather than develop our own instruments for our studies". Do you agree? Discuss the reasons for your answer.

Answer 

It is always advisable to use well-validated instruments that have been repeatedly used in published studies, rather than going through the cumbersome process of developing one's own instruments and laboriously establishing their validity and reliability. However, simply because an instrument is used in some published studies does not necessarily make it a good instrument. It is always wise to examine the psychometric properties established for the instrument before using it. In case this is not easily available, a pilot study can be conducted to see if the measures have at least acceptable reliability.

3. "A valid instrument is always reliable, but a reliable instrument may not always be valid". Comment on this statement.

Answer 

If an instrument is valid, especially in terms of content validity, we would expect to have a reliable instrument. However, an instrument can have both consistency and stability, but may not be measuring what it is intended to measure -- i.e.  have low validity. However, if an instrument does measure the concept that it is supposed to measure, it will be reliable. Thus, a valid instrument will be reliable, but a reliable instrument need not necessarily be valid.

Sampling Design

Question 1

Describe the different between Probability and Non-probability Sampling Designs

Answer

Each of the probability and non-probability sampling designs has been discussed with examples in the book. Overheads provided can be used while discussing the sampling designs. An aspect that needs to be stressed is that the term "random" used in sampling, such as simple random sampling, does not mean haphazardness, but is used to denote a systematic pattern. When we talk of simple random sampling, we mean that every single element in the population has a known and equal chance of being selected. Students seem to confuse the term used in a statistical sense with the layperson's usage of the term to mean ill-arranged, or haphazard. Each of the sampling designs can be discussed with reference to the student population in a university setting. The simple random sampling, and the systematic sampling designs can be used with the university register of students as the sampling frame. Stratified random sampling can be used stratifying the student population along the lines of professional/nonprofessional schools, traditional/nontraditional students, or academic status (undergraduate/graduate), etc. Cluster sampling could use several classes in session, or several narrow specializations, etc. Area-sampling could be explained in terms of location of the campus -- north wing, south wing, etc. Double sampling would be resorted to when a subset of the students sampled are again approached for more information.

In the non-probability sampling designs, Convenience Sampling can be illustrated as the instructor talking to students who are readily available to ascertain how they are experiencing the course. Purposive  Judgment Sampling would constitute sampling the students who can supply the information needed. For instance, if research is conducted on the types of sexual harassment that students experience, it is only those who have undergone sexual harassment in some form or the other, who would be able to provide the needed information. Thus this group would constitute the purposive judgment sample. Such members can perhaps be reached at a workshop conducted on sexual harassment. Snowball sampling can be used to determine a sample of international students who may be experiencing difficulties understanding the language used in lectures.  An initial group may be identified from their assignments, and these students can be asked to indicate students who may be experiencing language difficulties.  Quota sampling can be used to obtain information regarding experiences of say, NZ Europeans, NZ Maoris, and international students, specifying a quota to be sampled from each group, so that minority groups are adequately represented. Another example would be to stipulate quotas from the various disciplines and then sampling the students from each discipline on a convenience basis.

Question2

What kinds of sampling designs would be used for the following:

a. A study to get a quick idea of the medical acceptability of a new aspirin substitute which cannot be dispensed over the counter without prescription.

b. A study involving a sample of 325 students in a university where 2,000 students are enrolled.

c. An investigation of the career salience of professionals in the fields of medicine, engineering, business, and law.

d. The generalisability of the attitudes of blue collar workers from a sample of 184, to the total population of 350 blue collar workers in the entire factory of a particular company.

Answers

a. Purposive Judgment Sampling.

b. A Systematic sampling design (using university listing of students).

c. A Stratified Random sampling with stratification along profession, gender, age, etc.

d. Simple random sampling because of the importance attached to high generalisability.

Type of scale and Types of measurement scales

1. Construct a semantic differential scale to assess the properties of a particular brand of coffee or tea.

Semantic Differential Scale for Good Brew Tea Bags

------------------------------------------------------------------------------------------------------------

                                                                       

Fine  Flavour        8        7        6        5        4         3         2         1    No Flavour

Smooth Brew     8        7        6        5        4         3         2         1      Thick Brew

Good Taste        8        7        6        5        4         3         2         1      Distasteful

Lifts the spirit      8        7        6        5        4         3         2         1     Depresses

Gives Energy      8        7        6        5        4         3         2         1     Debilitates

  
2. Describe the four types of measurement scales.

The four scales are nominal, ordinal, interval, and ratio.

1. The nominal scale separates individuals or items into categories -- e.g. seniors and juniors. 
2. The ordinal scale helps to rank order the separated items. 
3. The interval scale helps to determine the extent to which the items are different from each other. In other words, is the difference between item 1 and 2 the same as the difference between 4 and 5?   
4. The ratio scale helps to determine the proportion of differences among items since it has an absolute zero origin, and is the most powerful of the four scales.

Research Design

9. Below are two scenarios. For each, indicate how the researcher should proceed with the following, giving reasons:

a. The purpose of the study

b. The type of investigation

c. The extent of researcher interference

d. The study setting

e. The time horizon for the study

f. The unit of analysis.

Scenario A

Ms. Aminah, the owner of a small business (a woman's dress boutique), has invited a consultant to tell her how she is different from similar small business within a 60-mile radius, in regard to her usage of the most modern computer technology, sales volume, profit margin, and staff training.

ANSWER

This will be a descriptive study (describing how she compares with the others). Data will be collected on the use of computers, sales volume, profit margin, and training programs and comparisons made. Some of these descriptions might be qualitative, as for example, in describing training methods. This will be a field study (using many similar small businesses), and researcher interference will be minimal. It will be a one-shot study, and the unit of analysis will be small business systems.

Scenario B

Mr. Ali, the owner of several restaurants is concerned about the wide differences in the profit margins of the various restaurants. He would like to try some incentive plans for increasing the efficiency levels of those restaurants that are lagging behind. But, before he introduces this, he would like to be sure that the idea will work. He asks a researcher to help him with this issue.

ANSWER

This would be a causal, hypothesis-testing study. It will be a field experiment, using one or two restaurants to manipulate incentive plans and see if this causes an increase in profit margins in those restaurants. Because of the manipulation, there will be some researcher interference with the natural flow of events. The time horizon for the study would be longitudinal since data will be obtained on sales, profits, etc., both before and after the manipulation. The unit of analysis would be the restaurant.