By Issa Bass
 

The standard error  based t-test can be used to determine the difference between two population means. But what happens if we want to make an inference about more than two population means, let's say three or five means?

Suppose we have three methods of soldering chips on a circuit board and we want to know which one will perform better with the CPUs that we are using. We determine that the difference between the three methods depends on the amount of solder they leave on the board. Twenty one circuit boards are used for the study, at the end of which, we will determine if the methods of soldering have an impact on the heat generated by the CPU.

In this experiment, we are concerned with only one treatment (or factor) which is the amount of solder left on the circuit boards with three levels (the small quantity of solder, medium quantity and heavy quantity of solder) and the response variable which is the heat generated by the CPU.

The intensity of the factor (which values are under control and are varied by the experimenter) determines the levels.

One way to determine the best method would be to use t-tests, comparing two methods at a time: method I will be compared to method II, then to method III, then method II is compared to method III.

Not only is this procedure too long, but it is also prone to multiply the Type I errors. We have seen that if , for a hypothesis testing, there is a 5% chance that the null hypothesis is rejected when it's true. If multiple tests are conducted, chances are that the Type I error will be made several times.

 Another way of doing it would be the Analysis Of Variance (ANOVA). It is used to pinpoint the sources of variation from one or more possible factors. It helps determine whether the variations are due to variability between or within methods. The within method variations are variations due to individual variation within treatment groups, while the between method variations are due to differences between the methods.
In other words, it helps assess the sources of variation that can be linked to the independent variables and determine how those variables interact and affect the predicted variable.

The Analysis Of Variance and the Regression Analysis have this in common: Both seek to analyze the impact of independent variables on response variables. But while ANOVA seeks to define the scope of the variables that will be included in an experiment, the regression analysis determines the coefficients for each variable.

ANOVA is a basic step in the Design Of Experiment (DOE), which is a powerful statistical tool aimed at statistically quantifying interactions between independent variables through their methodical modifications to determine their impact on the predicted variables.

The ANOVA pre requires the following assumptions:

• the treatment data must be normally distributed,
• the variance must be the same for all treatments,
• all samples are randomly selected
• and all the samples are independent

But a violation of these pre requisites does not necessarily lead to false conclusions, the probability for a Type I error will still be lower than if the different methods were compared to one another.

Analysis of variance tests the null hypothesis that all the population means are equal at a significance level : The null hypothesis will be  where  is the mean for method I.

Completely Randomized Experimental Design (One-Way ANOVA)

When performing the One-way ANOVA, a single input factor is varied at different levels with the objective of comparing the means of replications of the experiments. This will enable us to determine the proportion of the variations of the data that are due to the factor level and the variability due to random error (within group variation). The within group variations are variations due to individual variation within treatment groups.

The null hypothesis is rejected when the variation in the response variable is not due to random errors but to variation between treatment levels.

The variability of a set of data depends on the sum of square of the deviations

In the analysis of Variance, the Total variance is subdivided into two independent variances: the variance due to the treatment and the variance due to random error.

Total SS =

Total SS = SSk + SSE

i = a given part of a treatment level
j = a treatment level
k = number of treatment levels
 = number of observations in a treatment level
 = grand mean
 = mean of a treatment group level
 = particular value

So the computation of the ANOVA is done through the Sums of squares of the treatments, the error and their total.

SSk measures the variations between factors; it represents the sum of square of the columns that generate the sum of square between treatments.

The SSE is the sum of Square for errors measures the within- treatment variations.

These two variations (the variation between mean and the variation within sample) determine the difference between and . A greater SSk, compared to SSE indicates an evidence of a difference  and .

The rejection or non-rejection of the null hypothesis depends on the F statistic which is based on the F probability distribution. If the calculated F is greater than the critical F value then the null hypothesis is rejected.

So the test statistic for the null hypothesis (  ) will be based on  Where MSk represents the mean square for the treatment and MSE represents the mean square for error. F  is equal to 1 when MSk and MSE have the same value since both of them are estimates of the same quantity. This would imply that both the means and the variances are equal therefore the null hypothesis cannot be rejected.

Theses two mean squares are ratios of the sum of square of the treatment and the sum of square of the error to their respective degrees of freedom.

The one-way ANOVA table is shown below.

SS k = sum of squares between treatments

SSE = sum of squares due to error
SSE = TSS - SST
TSS = total sum of squares

MS k = mean square for treatments
MSE = mean square for error
t = number of treatment levels
n = number of runs at a particular level
N = total number of runs
F = the calculated F statistic with t -1 and N -t degrees of freedom

If the calculated F is significantly greater than the critical F value then the null hypothesis is rejected.

The critical value of F  for  can be obtained from the F table  which is based on the degree of freedom of the between treatments and the error.

Let's consider the example of the solder on the circuit boards. The following table summarizes the temperatures generated by the CPUs after a half an hours of usage.

The sum of all the observations is1607, the sum of the square of the 21 observations is 123031, so the Total sum of Squares TSS will be:

The sums of the 7 observations of the three different Methods are respectively:  526, 536 and 545.

The sum of square between treatments will be:

Since SSE is nothing but the difference between TSS and SSk,

SSE = 57.24 – 25.8 = 31.44

We have three treatments so the degree of freedom between treatments will be 2 (3-1) and the total number of observations is 21 therefore the degree of freedom for the errors will be 18 (21-3).

From the F table, the critical value of F  for with the degrees of freedom  and  is 3.55.

We can now plot the statistics obtained in an ANOVA chart.

We could have used excel to obtain the following table:

Minitab would have given:

Since the computed value of F, 7.39 exceeds the critical value , we reject the null hypothesis and conclude that there is a difference in the means and that one of the methods is likely to cause the CPU to generate less heat than the other two.

Multiple comparison tests

The reason we used the Analysis Of Variance instead of conducting multiple pair testing was to avoid wasting too much time and above all to avoid multiplying the type I errors. But after we conducted the ANOVA and determined that there is a difference in the means, it becomes necessary to figure out where the difference lies. In order to make that determination without having recourse to the multiple pairwise analyses, we can use a technique known as Multiple Comparison.

The multiple comparisons are made after the ANOVA has determined that there is a difference in the samples' means. means.

Tukey's Honestly Significant Difference (HSD) Test

The T method is a pairwise a posteriori (made after the experiment) test that requires an equality of the sample sizes. The purpose of the test is to determine the critical difference necessary between any two treatment levels means to be significantly different

The T method considers following:

• The number of treatment levels
• The mean square error
• The sample must be independent and of the same size

The HSD is determined by the formula:

Where

 is the protection level covering all possible comparison
n is the number of observation in each treatment
v is the degree of freedom of MSE
t is the number of treatments

The values are computed from the table and two means are said to be significantly different if they differ by  or more.

In the previous example, the degree of freedom was 18; the number of treatments was 3 and  was equal to 0.05, which yields 3.61 from the table.

 Using the formula,

The treatment means were

The absolute values of the differences will be as follows:

Only the absolute value of the difference between the means of Method I and Method III is greater than 1.803, so only the means between these two methods are significantly different.

The Randomized Block Design

In the previous example, we only considered the three methods of soldering and concluded their difference had an impact on the heat generated by the CPU, but other factors that were not included in the analysis (such as the power supplier, the heat sink, the fan ) could well have influenced the results.

In the Randomized Block Design, these variables, referred to as blocking variables are included in the experiment. Since the experimental units are not all homogeneous, homogeneous materials can be found and grouped into blocks so that the means in each block related to the treatment being considered may be compared. Because the comparisons are made within blocks, the error variation does not contain the effects of the blocks or the block to block variations.

If the Randomized block design is to be used for the three methods of soldering, we can subdivide the 21 units in three blocks (A, B and C) and each block will use all three methods. Each cell in the following table displays the average temperature generated by the associated method for every block.

The variables that need to be considered in this experiment are two: The blocks and the treatments (the methods in this case). So the Total Sum of Square the deviations of the predicted variable is divided into three:

• the sum of square of the treatment SST
• the Sum of Square of the Blocks SSB
• and the Sum of Square of the Errors SSE

TSS = SST + SSB + SSE

With:

Where:
i  is the block group
j  is the treatment level
T  is the number of treatment levels
n  is the number of observations in each treatment level
 is the individual observation
 is the treatment mean
 is the grand mean
N  is the total number of observations

As in the case of the Completely Randomized Experimental Design, the mean squares for the blocks, the treatments and the errors are obtained by dividing their sums of square by the respective degrees of freedom.

The degrees of freedom for the treatments and the block are fairly straight forward, it will be total number of treatment minus one and the total number of blocks minus one respectively;  the error it will be the product of these two degrees of freedom:

EQ     

Where  is the F value for the treatments and  is the F value for the Blocks.

We can summarize this information in an ANOVA table:

The null hypothesis for the randomized block design is .

The F value for the treatment is compared to the critical F value from the table. If it greater than the value on the table, the null hypothesis is rejected for the set value.

Let's use the above table as an example.

T = 3; n =3; N = 9

We can verify that TSS =  SSB + SSE + SST = 4.6667 + 4.6667 + 20.6667 = 30

Since the number of treatments and the number of blocks are equal, the degrees of freedom for the blocks and the treatments will be the same: 3 – 1 = 2

The degree of freedom for the SSE will be 9 – 3 – 3 + 1 = 4

The critical value of F  obtained from the table is 9.28, that value is greater than the observed value of F  for treatment, therefore the null hypothesis should not be rejected. In other words, there is not a significant difference between the means that would justify rejecting the null hypothesis.

We could have saved ourselves the pain of having to crunch all these numbers by using Excel.

About the author
Issa Bass is the managing editor of SixSigmaFirst. He can be reached at issa@sixsigmafirst.com

www.manorhouseassociates.com

Tell us what you think about this article. Send a note to the Editor.

www.manorhouseassociates.com