20

Normal Distributions, Standard Deviations, Modality, Skewness and Kurtosis: Understanding concepts

::2014/03/21::

43

Chapter 2: Bi-Modal Hazard rate (BMH) A unique epidemiological feature of breast cancer

::2009/11/24::

From Wikipedia, the free encyclopedia

"Bimodal" redirects here. For the musical concept, see Bimodality.

In statistics, a **bimodal distribution** is a continuous probability distribution with two different modes. These appear as distinct peaks (local maxima) in the probability density function, as shown in Figure 1.

More generally, a **multimodal distribution** is a continuous probability distribution with two or more modes, as illustrated in Figure 3.

When the two modes are unequal the larger mode is known as the major mode and the other as the minor mode. The least frequent value between the modes is known as the antimode. The difference between the major and minor modes is known as the amplitude. In time series the major mode is called the acrophase and the antimode the batiphase.

Galtung introduced a classification system (AJUS) for distributions:^{[2]}

- A: unimodal distribution - peak in the middle
- J: unimodal - peak at either end
- U: bimodal - peaks at both ends
- S: bimodal or multimodal - multiple peaks

This classification has since been modified slightly:

- J (modified) - peak on right
- L: unimodal - peak on left
- F: no peak (flat)

Under this classification bimodal distributions are classified as type S or U.

Bimodal distributions occur both in mathematics and in the natural sciences.

Important bimodal distributions include the arcsine distribution and the beta distribution. Others include the U-quadratic distribution.

The ratio of two normal distributions is also bimodally distributed. Let

where *a* and *b* are constant and *x* and *y* are distributed as normal variables with a mean of 0 and a standard deviation of 1. *R* has a known density that can be expressed as a confluent hypergeometric function.^{[3]}

The distribution of the reciprocal of a *t* distributed random variable is bimodal when the degrees of freedom are more than one. Similarly the reciprocal of a normally distributed variable is also bimodally distributed.

Examples of variables with bimodal distributions include the time between eruptions of certain geysers, the color of galaxies, the size of worker weaver ants, the age of incidence of Hodgkin's lymphoma, the speed of inactivation of the drug isoniazid in US adults, the absolute magnitude of novae, and the circadian activity patterns of those crepuscular animals that are active both in morning and evening twilight. In fishery science multimodal length distributions reflect the different year classes and can thus be used for age distribution- and growth estimates of the fish population.^{[4]} Sediments are usually distributed in a bimodal fashion.

Main article: Mixture distribution

A bimodal distribution most commonly arises as a mixture of two different unimodal distributions (i.e. distributions having only one mode). In other words, the bimodally distributed random variable X is defined as with probability or with probability where *Y* and *Z* are unimodal random variables and is a mixture coefficient.

Mixtures with two distinct components need not be bimodal and two component mixtures of unimodal component densities can have more than two modes. There is no immediate connection between the number of components in a mixture and the number of modes of the resulting density.

A bimodal exponential distribution has been studied by Hassan and Hijazi^{[5]}

Another distribution - the Alpha-skew-normal distribution - has also been studied.^{[6]}

In biology five factors are known to contribute to bimodal distributions of population sizes:

- the initial distribution of individual sizes

- the distribution of growth rates among the individuals

- the size and time dependence of the growth rate of each individual

- mortality rates that may affect each size class differently

- the DNA methylation in human and mouse genome.

The bimodal distribution of sizes of weaver ant workers shown in Figure 2 arises due to existence of two distinct classes of workers, namely major workers and minor workers.^{[1]} In this case, *Y* would be the size of a random major worker, *Z* the size of a random minor worker, and *α* the proportion of worker weaver ants that are major workers.

The distribution of fitness effects of mutations for both whole genomes^{[7]}^{[8]} and individual genes^{[9]} is also frequently found to be bimodal with most mutations being either neutral or lethal with relatively few having intermediate effect.

A mixture of two unimodal distributions with differing means is not necessarily bimodal. The combined distribution of heights of men and women is sometimes used as an example of a bimodal distribution, but in fact the difference in mean heights of men and women is too small relative to their standard deviations to produce bimodality.^{[10]}

Bimodal distributions have the peculiar property that - unlike the unimodal distributions - the mean may be a more robust sample estimator than the median.^{[11]} This is clearly the case when the distribution is U shaped like the arcsine distribution. It may not be true when the distribution has one or more long tails.

Let

where *g*_{i} is a probability distribution and *p* is the mixing parameter.

The moments of *f*(x) are^{[12]}

where

and *S*_{i} and *K*_{i} are the skewness and kurtosis of the *i*^{th} distribution.

It is not uncommon to encounter situations where an investigator believes that the data comes from a mixture of two normal distributions. Because of this, this mixture has been studied in some detail.^{[13]}

A mixture of two normal distributions has five parameters to estimate: the two means, the two variances and the mixing parameter. A mixture of two normal distributions with equal standard deviations is bimodal only if their means differ by at least twice the common standard deviation.^{[10]} Estimates of the parameters is simplified if the variances can be assumed to be equal (the homoscedastic case).

It is obvious that if the means of the two normal distributions are equal that the combined distribution is unimodal. Conditions for unimodality of the combined distribution were derived by Eisenberger.^{[14]} Necessary and sufficient conditions for a mixture of normal distributions to be bimodal have been identified by Ray and Lindsay.^{[15]}

A mixture of two approximately equal mass normal distributions have a negative kurtosis since the two modes on either side of the center of mass effectively flatten out the distribution.

A mixture of two normal distributions with highly unequal mass has a positive kurtosis since the smaller distribution lengthens the tail of the more dominant normal distribution.

Mixtures of other distributions require additional parameters to be estimated.

If the case of equal variance the mixture is unimodal if and only if^{[16]}

or

where *p* is the mixing parameter and *d* is

where *μ*_{1} and *μ*_{2} are the means of the two normal distributions and *σ*_{1} and *σ*_{2} are their standard deviations.

Bimodal distributions are a commonly used example of how summary statistics such as the mean, median, and standard deviation can be deceptive when used on an arbitrary distribution. For example, in the distribution in Figure 1, the mean and median would be about zero, even though zero is not a typical value. The standard deviation is also larger than deviation of each normal distribution.

Although several have been suggested, there is no presently generally agreed summary statistic (or set of statistics) to quantify the parameters of a general bimodal distribution. For a mixture of two normal distributions the mean and standard deviation along with the mixing parameter (a weighing system for the combination) are usually used - a total of five parameters.

A statistic that may be useful is Ashman's D:^{[17]}

where *μ*_{1}, *μ*_{2} are the means and *σ*_{1} *σ*_{2} are the standard deviations.

For a mixture of two normal distributions *D* > 2 is required for a clean separation of the distributions.

The bimodality index assumes that the distribution is a sum of two normal distributions with equal variances but differing means.^{[18]} It is defined as follows:

where *μ*_{1}, *μ*_{2} are the means and *σ* is the common standard deviation.

where *p* is the mixing parameter.

This index assumes that the distribution is a mixture of two normal distributions with means (*μ*_{1} and *μ*_{2}) and standard deviations (*σ*_{1} and *σ*_{2}):^{[19]}

Sarle's bimodality coefficient *b* is^{[20]}

where *γ* is the skewness and *κ* is the kurtosis. The kurtosis is here defined to be the standardised fourth moment around the mean. The value of *b* lies between 0 and 1.^{[21]} The logic behind this coefficient is that a bimodal distribution will have very low kurtosis, an asymmetric character, or both - all of which increase this coefficient.

The formula for a finite sample is^{[22]}

where *n* is the number of items in the sample, *g* is the sample skewness and *k* is the sample excess kurtosis.

The value of *b* for the uniform distribution is 5/9. This is also its value for the exponential distribution. Values greater than 5/9 may indicate a bimodal or multimodal distribution. The maximum value (1.0) is reached only by a Bernoulli distribution with only two distinct values or the sum of two different Dirac delta functions (a bi-delta distribution).

The distribution of this statistic is unknown. It is related to a statistic proposed earlier by Pearson - the difference between the kurtosis and the square of the skewness (*vide infra*).

This is defined as^{[19]}

where *A*_{1} is the amplitude of the smaller peak and *A*_{an} is the amplitude of the antinode.

*A*_{B} is always < 1. Larger values indicate more distinct peaks.

This is the ratio of the left and right peaks.^{[19]} Mathematically

where *A*_{l} and *A*_{r} are the amplitudes of the left and right peaks respectively.

This parameter (*B*) is due to Wilcock.^{[23]}

where *A*_{l} and *A*_{r} are the amplitudes of the left and right peaks respectively and *P*_{i} is the logarithm taken to the base 2 of the proportion of the distribution in the i^{th} interval. The maximal value of *B* is 1.

A bimodality index has been proposed by Sturrock.^{[24]}

This index (*B*) is defined as

When *m* = 2 and *γ* is uniformly distributed, *B* is exponentially distributed.^{[25]}

This statistic is a form of periodogram. It suffers from the usual problems of estimation and spectral leakage common to this form of statistic.

A different bimodality index has been proposed by de Michele and Accatino.^{[26]} Their index (*B*) is

where *μ* is the arithmetic mean of the sample and

where *m*_{i} is number of data points in the *i*^{th} bin,*x*_{i} is the center of the *i*^{th} bin and *L* is the number of bins.

The authors suggested a cut off value of 0.1 for *B* to distinguish between a bimodal (*B* > 0.1)and unimodal (*B* < 0.1) distribution. No statistical justification was offered for this value.

Another index (*B*) has been proposed by Sambrook Smith *et al*^{[27]}

where *p*_{1} and *p*_{2} are the proportion contained in the primary (that with the greater amplitude) and secondary (that with the lesser amplitude) mode and *φ*_{1} and *φ*_{2} are the *φ*-sizes of the primary and secondary mode. The *φ*-size is defined as minus one times the log of the data size taken to the base 2. This transformation is commonly used in the study of sediments.

The authors recommended a cut off value of 1.5 with B being greater than 1.5 for a bimodal distribution and less than 1.5 for a unimodal distribution. No statistical justification for this value was given.

Another bimodality parameter has been proposed by Chaudhuri and Agrawal.^{[28]} This parameter requires knowledge of the variances of the two subpopulations that make up the bimodal distribution. It is defined as

where *n*_{i} is the number of data points in the *i*^{th} subpopulation, *σ*_{i}^{2} is the variance of the *i*^{th} subpopulation, *m* is the total size of the sample and *σ*^{2} is the sample variance.

It is a weighted average of the variance. The authors suggest that this parameter can be used as the optimisation target to divide a sample into two subpopulations. No statistical justification for this suggestion was given.

A number of tests are available to determine if a data set is distributed in a bimodal (or multimodal) fashion.

In the study of sediments particle size is frequently bimodal. Empirically it has been found useful to plot the frequency against the log( size ) of the particles.^{[29]}^{[30]} This usually gives a clear separation of the particles into a bimodal distribution. In geological applications the logarithm is normally taken to the base 2. The log transformed values are referred to as phi (Φ) units. This system is known as the Krumbein (or phi) scale.

An alternative method is to plot the log of the particle size against the cumulative frequency. This graph will usually consist two reasonably straight lines with a connecting line corresponding to the antimode.

- Statistics

Approximate values for several statistics can be derived from the graphic plots.^{[29]}

where *Mean* is the mean, *StdDev* is the standard deviation, *Skew* is the skewness, *Kurt* is the kurtosis and *φ*_{x} is the value of the variate *φ* at the *x*^{th} percentage of the distribution.

A necessary but not sufficient condition for a symmetrical distribution to be bimodal is that the kurtosis be less than three.^{[31]}^{[32]} Here the kurtosis is defined to be the standardised fourth moment around the mean. The reference given prefers to use the *excess kurtosis* - the kurtosis less 3.

Pearson in 1894 was the first to devise a procedure to test whether a distribution could be resolved into two normal distributions.^{[33]} This method required the solution of a ninth order polynomial. In a subsequent paper Pearson reported that for any distribution skewness^{2} + 1 < kurtosis.^{[21]} Later Pearson showed that^{[34]}

where *b*_{2} is the kurtosis and *b*_{1} is the square of the skewness. Equality holds only for the two point Bernoulli distribution or the sum of two different Dirac delta functions. These are the most extreme cases of bimodality possible. The kurtosis in both these cases is 1. Since they are both symmetrical their skewness is 0 and the difference is 1.

Baker proposed a transformation to convert a bimodal to a unimodal distribution.^{[35]}

Several tests of unimodality versus bimodality have been proposed: Haldane suggested one based on second central differences.^{[36]} Larkin later introduced a test based on the F test;^{[37]} Benett created one based on the G test.^{[38]} Tokeshi has proposed a fourth test.^{[39]}^{[40]} A test based on a likelihood ratio has been proposed by Holzmann and Vollmer.^{[16]}

A method based on the score and Wald tests has been proposed.^{[41]} This method can distinguish between unimodal and bimodal distributions when the underlying distributions are known.

Statistical tests for the antimode are known.^{[42]}

- Otsu's method

Otsu's method is commonly employed in computer graphics to determine the optimal separation between two distributions.

To test if a distribution is other than unimodal, several additional tests have been devised: the bandwidth test,^{[43]} the dip test,^{[44]} the excess mass test,^{[45]} the MAP test,^{[46]} the mode existence test,^{[47]} the runt test,^{[48]}^{[49]} the span test,^{[50]} and the saddle test.

The dip test is available for use in R.[1] The values for the dip statistic values range between 0 to 1. Values less than 0.05 indicate significant bimodality and values greater than 0.05 but less than 0.10 suggest bimodality with marginal significance.

Silverman introduced a bootstrap method for the number of modes.^{[43]} The test uses a fixed bandwidth which reduces the power of the test and its interpretability. Under smoothed densities may have an excessive number of modes whose count during bootstrapping is unstable.

Additional tests are available for a number of special cases:

- Mixture of two normal distributions

A study of a mixture density of two normal distributions data found that separation into the two normal distributions was difficult unless the means were separated by 4-6 standard deviations.^{[51]}

In astronomy the Kernel Mean Matching algorithm is used to decide if a data set belongs to a single normal distribution or to a mixture of two normal distributions.

- Beta-normal distribution

This distribution is bimodal for certain values of is parameters. A test for these values has been described.^{[52]}

Assuming that the distribution is known to be bimodal or has been shown to be bimodal by one or more of the tests above, it is frequently desirable to fit a curve to the data. This may be difficult.

Bayesian methods may be useful in difficult cases.

- Two normal distributions

A package for R is available for testing for bimodality.[2] This package assumes that the data are distributed as a sum of two normal distributions. If this assumption is not correct the results may not be reliable. It also includes functions for fitting a sum of two normal distributions to the data.

Assuming that the distribution is a mixture of two normal distributions then the expectation-maximization algorithm may be used to determine the parameters. Several programmes are available for this including Cluster.^{[53]}

- Other distributions

The mixtools package also available for R can test for and estimate the parameters of a number of different distributions.^{[54]}

Another package for a mixture of two right tailed gamma distributions is available.^{[55]}

Several other packages for R are available to fit mixture models; these include flexmix,^{[56]} mcclust,^{[57]} and mixdist.^{[58]}

The programme SWRC ﬁt can fit a number of bimodal distributions.[3]

The statistical programme SAS can also fit a variety of mixed distributions with the command PROCFREQ.

- ^
^{a}^{b}Weber, NA (1946). "Dimorphism in the African*Oecophylla*worker and an anomaly (Hym.: Formicidae)" (PDF).*Annals of the Entomological Society of America***39**: pp. 7–10. **^**Galtung J (1969) Theory and methods of social research. Universitetsforlaget, Oslo ISBN 0043000177**^**Fieller E (1932) The distribution of the index in a noraml bivariate population.*Biometrika*(24):428-440**^**Introduction to tropical fish stock assessment**^**Hassan MY, Hijazi R H (2010) A bimodal exponential power distribution. Pak J Statist 26(2) 379-396**^**Elal-Olivero D (2010) Alpha-skew-normal distribution. Proyecciones J Math 29 (3) 224-240**^**Sanjuán, R (Jun 27, 2010). "Mutational fitness effects in RNA and single-stranded DNA viruses: common patterns revealed by site-directed mutagenesis studies.".*Philosophical transactions of the Royal Society of London. Series B, Biological sciences***365**(1548): 1975–82. doi:10.1098/rstb.2010.0063. PMC 2880115. PMID 20478892.**^**Eyre-Walker, A; Keightley, PD (Aug 2007). "The distribution of fitness effects of new mutations.".*Nature reviews. Genetics***8**(8): 610–8. doi:10.1038/nrg2146. PMID 17637733.**^**Hietpas, RT; Jensen, JD; Bolon, DN (May 10, 2011). "Experimental illumination of a fitness landscape.".*Proceedings of the National Academy of Sciences of the United States of America***108**(19): 7896–901. doi:10.1073/pnas.1016024108. PMC 3093508. PMID 21464309.- ^
^{a}^{b}Schilling, Mark F.; Watkins, Ann E.; Watkins, William (2002). "Is Human Height Bimodal?".*The American Statistician***56**(3): 223–229. doi:10.1198/00031300265. **^**Mosteller F, Tukey JW (1977) Data analysis and regression: a second course in statistics. Reading, Mass, Addison-Wesley Pub Co**^**Kim T-H, White H (2003) On more robust estimation of skewness and kurtosis: Simulation and application to the S & P 500 index**^**Robertson CA, Fryer JG (1969) Some descriptive properties of normal mixtures.*Skandinavisk Aktuarietidskrift*69: 137–146**^**Eisenberger I (1964) Genesis of bimodal distributions.*Technometrics*6 (4) 357-363**^**Ray S, Lindsay BG (2005) The topography of multivariate normal mixtures.*Ann Stat*33 (5) 2042-2065- ^
^{a}^{b}Holzmann H, Vollmer S (2008) A likelihood ratio test for bimodality in two-component mixtures – with application to regional income distribution in the EU.*AStA*2(1)57-69 **^**Ashman KM, Bird CM, Zepf SE (1994)*Astronomical J*108: 2348**^**Wang J, Wen S, Symmans WF, Pusztai L, Coombes KR (2009) The bimodality index: a criterion for discovering and ranking bimodal signatures from cancer gene expression profiling data.*Cancer Inform*7:199-216- ^
^{a}^{b}^{c}Zhang C, Mapes BE, Soden BJ (2003) Bimodality in tropical water vapour.*Q J R. Meteorol Soc*129: 2847–2866 doi:10.1256/qj.02.16 **^**Ellison AM (1987) Effect of seed dimorphism on the density-dependent dynamics of experimental populations of*Atriplex triangularis*(Chenopodiaceae).*Am J Botany*74(8): 1280-1288- ^
^{a}^{b}Pearson K (1916) Mathematical contributions to the theory of evolution, XIX: Second supplement to a memoir on skew variation.*Phil Trans Roy Soc*London. Series A 216 (538–548): 429–457. Bibcode 1916RSPTA.216..429P. doi:10.1098/rsta.1916.0009. JSTOR 91092 **^**SAS Institute Inc. (2012). SAS/STAT 12.1 user’s guide. Cary, NC: Author.**^**Wilcock PR (1993) The critical shear stress of natural sediments.*J Hydraul Engrg ASCE*119:491-505**^**Sturrock P (2008) Analysis of bimodality in histograms formed from GALLEX and GNO solar neutrino data. Solar Physics 249: 1-10**^**Scargle JD (1982) Studies in astronomical time series analysis. II - Statistical aspects of spectral analysis of unevenly spaced data. Astrophys J 263 (1) 835-853**^**de Michele C, Accatino F (2014) Tree cover bimodality in savannas and forests emerging from the switching between two fire dynamics. PLoS One doi:10.1371/journal.pone.0091195**^**Sambrook Smith GH, Nicholas AP, Ferguson RI (1997) Measuring and defining bimodal sediments: Problems and implications. Water Resources Research 33: 1179-1185**^**Chaudhuri D, Agrawal A (2010) Split-and-merge procedure for image segmentation using bimodality detection approach. Defence Sci J 60 (3) 290-301- ^
^{a}^{b}Folk RL, Ward WC (1957) Brazos River bar: a study in the significance of grain size parameters.*J Sedim Petrol*27: 3–26 **^**Dyer KR (1970) Grain-size parameters for sandy gravels.*J Sedim Petrol*40 (2) 616-620**^**Gneddin OY(2010) Quantifying Bimodality.**^**Muratov AL, Gnedin OY (2010) Modeling the metallicity distribution of globular clusters.*Ap J*(submitted) arXiv:1002.1325**^**Pearson K (1894) Contributions to the mathematical theory of evolution: On the dissection of asymmetrical frequency-curves.*Phil Trans Roy Soc*Series A, Part 1, 185: 71-90**^**Pearson K (1929) Editorial note.*Biometrika*21: 370-375**^**Baker GA (1930) Transformations of bimodal distributions.*Ann Math Stat*1 (4) 334-344**^**Haldane JBS (1951) Simple tests for bimodality and bitangentiality.*Ann Eugenics*16 (1) 359–364 doi:10.1111/j.1469-1809.1951.tb02488.x**^**Larkin RP (1979) An algorithm for assessing bimodality vs. unimodality in a univariate distribution.*Behavior Research Methods*11 (4) 467-468 doi:10.3758/BF03205709**^**Bennett SC (1992) Sexual dimorphism of*Pteranodon*and other pterosaurs, with comments on cranial crests.*J Vert Paleont*12 (4) 422-434**^**Tokeshi M (1992) Dynamics and distribution in animal communities; theory and analysis.*Researches in Population Ecology*34:249–273**^**Barreto S, Borges PAV, Guo Q (2003) A typing error in Tokeshi’s test of bimodality.*Global Ecology & Biogeography*12: 173–174**^**Carolan AM, Rayner JCW (2001) One sample tests for the location of modes of nonnormal data.*J Applied Mathematics and Decision Science*5(1) 1–19**^**Hartigan JA (2000) Testing for antimodes.*Studies in Classification, Data Analysis, and Knowledge Organization*169-181- ^
^{a}^{b}Silverman BW (1981) Using kernel density estimates to investigate multimodality.*J Roy Statist Soc*Ser B 43:97-99 **^**Hartigan JA, Hartigan PM (1985) The dip test of unimodality.*Ann Statist*13 (1) 70-84**^**Mueller DW, Sawitzki G (1991) Excess mass estimates and tests for multimodality.*JASA*86, 738 -746**^**Rozál GPM Hartigan JA (1994) The MAP test for multimodality.*J Classification*11 (1) 5-36 doi:10.1007/BF01201021**^**Minnotte MC (1997) Nonparametric testing of the existence of modes.*Ann Statist*25 (4) 1646-1660**^**Hartigan JA, Mohanty S (1992) The RUNT test for multimodality.*J Classification*9: 63-70**^**Andrushkiw RI, Klyushin DD, Petunin YI (2008)*Theory Stoch Processes*14 (1) 1-6**^**Hartigan JA (1988) The span test of multimodality**^**Jackson PR, Tucker GT, Woods HF (1989) Testing for bimodality in frequency distributions of data suggesting polymorphisms of drug metabolism--hypothesis testing.*Br J Clin Pharmacol*28(6) 655–662**^**http://www.amstat.org/sections/srms/Proceedings/y2002/Files/JSM2002-000150.pdf**^**https://engineering.purdue.edu/~bouman/software/cluster/**^**http://cran.r-project.org/web/packages/mixtools/index.html**^**http://cran.r-project.org/web/packages/discrimARTs/discrimARTs.pdf**^**http://cran.r-project.org/web/packages/flexmix/index.html**^**http://cran.r-project.org/web/packages/mclust/index.html**^**http://cran.r-project.org/web/packages/mixdist/index.html

Wikipedia content is licensed under the GFDL License