EXPERIMENTAL DESIGN

Continued from: Research Design

In the previous hub, research design was discussed. One of the constituents of research design was method of collecting data which could be: interviews, surveys or experiments. If the researcher decides to conduct experiment, there are certain requirements like formation of groups, number of tests and their timing and safeguards to ensure that the results are as natural as possible.

In Observation, a researcher only collects and analyzes data without any change. But in an experiment, a researcher would do something other than just observing what is occurring. She would introduce a change or variation in treatment and note down the outcome. The aim is to establish a cause and effect relationship.

As stated by Valerie J. Easton and John H. McColls, “In experiments, a treatment is something that researchers administer to experimental units. For example, a corn field is divided into four, each part is 'treated' with a different fertilizer to see which produces the most corn; a teacher practices different teaching methods on different groups in her class to see which yields the best results; a doctor treats a patient with a skin condition with different creams to see which is most effective. Treatments are administered to experimental units by 'level', where level implies amount or magnitude. For example, if the experimental units were given 5mg, 10mg, 15mg of a medication, those amounts would be three levels of the treatment".

The purpose of research is to draw inferences from the results to establish rules and regulations. For this, it is essential that the information should be without any ambiguity. In order to ensure that results are trust worthy, verifiable and replicable, an appropriate experimental design is the basic requirement.

In experiment design, the nature of study is causal, it can be performed either in lab or in field, the settings are contrived and the researcher would have full command to change the independent variables. The researcher would work out the research design in such a manner that either variables are controlled or their impact can be gauged by some method. The research design should ensure both internal validity and external validity.

Suppose a marketing manager feels that advertisement on private FM radios would be more helpful in sale of his products. In order to prove his point, the manager would have to made some advertisements on FM radio and analyse the result. Let us presume that a period of three months is considered sufficient to settle the issue.

In a rational approach, the manager would pick up two identical products differing only in name i.e. Red Label and Yellow Label. He would divert advertisement of Red Label to FM radio and observe any change in sales. Think for a moment that sale had gone up considerably. Does it prove his contention? No, it is possible that sale of the other product, Yellow Label, have equally increased. It is also possible that a statement by a renowned medical association that ‘tea is good for heart’ may have boosted up the sales. It is also possible that the marketing manager had been more aggressive in pushing up the sale of Red Label. These factors may affect validity of the experiment.

What is validity? Simply put, it is reliability, dependability or being error-free. The internal validity covers the time from start of an experiment to its conclusion and external validity covers conclusion to generalization or applicability. This been stated in the side tables.

The design should be such as to take care of some of the problem. The sketch given in the sideline shows a popular design, known as "True Experimental Design". It has two groups (i) experimental and (ii) control. A pre-test would be taken of both the group to ascertain whether or not they are equal just before start of the experiment.

Experimental group would be treated as per experiment. Later, both groups would take the post-test and the result would be analyzed. Suppose, there were 40 students of the same class. Of these two groups were formed of 20 each. Also suppose that in the pretest, experimental group achieved 65 marks while the control group achieved 63 marks. Now the experiments starts. A teacher wants to prove that use of visual aids in the class room would increase understanding of the subject. One group, the experimental group, would be taught through visuals while the other continued as per traditional lecture. After a certain period of time say one month, post-test would be taken. Suppose, the results in post test were 90 marks for experimental group and 70 marks for control group. Thus we can say that improvement of experimental group was 25 (90-65) while the other group improved by 7 (70-63) or there was a net improvement of 18(25-7) in experimental group. This is attributable to change in teaching method. If there was any history effect, test-retest effect or instrumental effect, that has been taken care of as both the groups were exposed to the same timeframe, same number of tests and same instruments (question paper and grader).

Continuing our example of Red Label and Yellow Label, the same results can be observed whereby increase in sales of Red Label were significantly more than the sales of Yellow Label brand which was due to switching over to FM Radio for advertisement.

In our example, advertisements on FM Radio was independent variable while sales were dependent variable. The marketing manager, acting as researcher, believed that more advertisement in FM would lead to more sales of the product. He manipulated the situation by increasing or decreasing or completely suspending the advertisements and watched the reaction on sales. Thus, he wanted to establish a cause and effect relationship firstly between advertisement and sales and secondly between FM Radio advertisements and Sales. In both cases, he found that advertisements in general increase the sales and also that the advertisement on FM Radio is more effective for his product.

Take another example that a machine tool operator thinks that fumes emitted in the workshop are instrumental in the low efficiency of the operators. She would like to prove this to her supervisor through a research study. She would suggest increasing and decreasing fumes in a certain manner and study its impact on efficiency. If with no-fume, efficiency improves and with high-fume efficiency goes down, the point has been well taken.

External validity is another threat to the experiment. The results of experiment could be valid but when applied in general, the outcome may not be as expected.

The experiment of advertising on FM Radio proved beneficial but FM Radio Station has a short range communication. The experiment was conducted at Karachi, there may be a most popular FM station or a popular announcer or jockey which resulted in increase sale of Red-Label. Same results may not be achieved when FM Stations in other cities are used for advertisement.

Other threats to external validity come from bias, un-natural settings and novelty effect as shown in the side-sketch.

Cross Sectional and Longitudinal

Some experiment are called "one shot" which means data would be collected only once even if the experiment is for 3 months. It is like population survey of a country which takes considerable time. In other experiments, data is collected many times during the experiment to find some pattern. While, first instance is called cross-sectional and second is termed as longitudinal.

Lab and Field Experiment

Experiments could be conducted in laboratories or in the field. In both case, there is interference or manipulation by the researcher. In field, the independent variables cannot be manipulated as easily as in laboratory. Also, the experiment in the field can not be isolated as in laboratory. The field experiments have more external validity as their settings are natural.

MATCHING & RANDOMIZATION

in matching, all batches have the same characters or features or composition like age, gender and experience.

In randomization, the groups are made randomly with no pre-determination. Each member has an equal chance of being assigned to any group.

Difference between matching and randomization:

In matching, individuals are deliberately and consciously matched for various controlled characteristics. In randomization, it is expect that the process will distributes the in-equalities between or among the groups.

CONCLUSION

In this hub, experimental designs were covered in particular reference to lab and field experiment. It was also stated how external forces can exert their influences and what measure be taken to safeguard the experiment as far as possible. It was also stated that result obtained in the experiment may not be valid in genera because of various factor like bias, unrepresentative setting, time and place or sample. The hub suggested certain measures so that the results of an experiment are reliable and replicable.