Normal distribution

/div>

Jump to: navigation, search

This article is about the univariate normal distribution. For normally distributed vectors, see Multivariate normal distribution.

Probability density function The red line is the standard normal distribution
Cumulative distribution function Colors match the image above
Notation	$\mathcal{N}(\mu,\,\sigma^2)$
Parameters	μ ∈ R — mean (location) σ² > 0 — variance (squared scale)
Support	x ∈ R
PDF	$\frac{1}{\sqrt{2\pi\sigma^2}}\,e^{ -\frac{(x-\mu)^2}{2\sigma^2} }$
CDF	$\frac12\left[1 + \operatorname{erf}\left( \frac{x-\mu}{\sqrt{2\sigma^2}}\right)\right]$
Mean	μ
Median	μ
Mode	μ
Variance	σ²
Skewness	0
Ex. kurtosis	0
Entropy	$\frac12 \ln(2 \pi e \, \sigma^2)$
MGF	$\exp\{ \mu t + \frac{1}{2}\sigma^2t^2 \}$
CF	$\exp \{ i\mu t - \frac{1}{2}\sigma^2 t^2 \}$
Fisher information	$\begin{pmatrix}1/\sigma^2&0\\0&1/(2\sigma^4)\end{pmatrix}$

In probability theory, the normal (or Gaussian) distribution is a continuous probability distribution that has a bell-shaped probability density function, known as the Gaussian function or informally the bell curve:^{[nb 1]}

$f(x) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{ -\frac{(x-\mu)^2}{2\sigma^2} }$

where parameter μ is the mean or expectation (location of the peak) and σ² is the variance, the mean of the squared deviation, (a "measure" of the width of the distribution). σ is the standard deviation. The distribution with μ = 0 and σ² = 1 is called the standard normal. A normal distribution is often used as a first approximation to describe real-valued random variables that cluster around a single mean value.

The normal distribution is considered the most prominent probability distribution in statistics. There are several reasons for this:^[1] First, the normal distribution is very tractable analytically, that is, a large number of results involving this distribution can be derived in explicit form. Second, the normal distribution arises as the outcome of the central limit theorem, which states that under mild conditions the sum of a large number of random variables is distributed approximately normally. Finally, the "bell" shape of the normal distribution makes it a convenient choice for modelling a large variety of random variables encountered in practice.

For this reason, the normal distribution is commonly encountered in practice, and is used throughout statistics, natural sciences, and social sciences^[2] as a simple model for complex phenomena. For example, the observational error in an experiment is usually assumed to follow a normal distribution, and the propagation of uncertainty is computed using this assumption. Note that a normally-distributed variable has a symmetric distribution about its mean. Quantities that grow exponentially, such as prices, incomes or populations, are often skewed to the right, and hence may be better described by other distributions, such as the log-normal distribution or Pareto distribution. In addition, the probability of seeing a normally-distributed value that is far (i.e. more than a few standard deviations) from the mean drops off extremely rapidly. As a result, statistical inference using a normal distribution is not robust to the presence of outliers (data that is unexpectedly far from the mean, due to exceptional circumstances, observational error, etc.). When outliers are expected, data may be better described using a heavy-tailed distribution such as the Student's t-distribution.

From a technical perspective, alternative characterizations are possible, for example:

The normal distribution is the only absolutely continuous distribution all of whose cumulants beyond the first two (i.e. other than the mean and variance) are zero.
For a given mean and variance, the corresponding normal distribution is the continuous distribution with the maximum entropy.^[3]^[4]

The normal distributions are a sub-class of the elliptical distributions.

1 Definition
- 1.1 Alternative formulations
2 Characterization
3 Properties
4 Related distributions
- 4.1 Operations on a single random variable
- 4.2 Extensions
5 Normality tests
6 Estimation of parameters
7 Occurrence
8 Generating values from normal distribution
9 Numerical approximations for the normal CDF
10 History
- 10.1 Development
- 10.2 Naming
11 See also
12 Notes
13 Citations
14 References
15 External links

[ Definition

The simplest case of a normal distribution is known as the standard normal distribution, described by the probability density function

$\phi(x) = \frac{1}{\sqrt{2\pi}}\, e^{- \frac{\scriptscriptstyle 1}{\scriptscriptstyle 2} x^2}.$

The factor $\scriptstyle\ 1/\sqrt{2\pi}$ in this expression ensures that the total area under the curve ϕ(x) is equal to one,^[proof] and 1/2 in the exponent makes the "width" of the curve (measured as half the distance between the inflection points) also equal to one. It is traditional in statistics to denote this function with the Greek letter ϕ (phi), whereas density functions for all other distributions are usually denoted with letters f or p.^[5] The alternative glyph φ is also used quite often, however within this article "φ" is reserved to denote characteristic functions.

Every normal distribution is the result of exponentiating a quadratic function (just as an exponential distribution results from exponentiating a linear function):

$f(x) = e^{a x^2 + b x + c}. \,$

This yields the classic "bell curve" shape, provided that a < 0 so that the quadratic function is concave. f(x) > 0 everywhere. One can adjust a to control the "width" of the bell, then adjust b to move the central peak of the bell along the x-axis, and finally one must choose c such that $\scriptstyle\int_{-\infty}^\infty f(x)\,dx\ =\ 1$ (which is only possible when a < 0).

Rather than using a, b, and c, it is far more common to describe a normal distribution by its mean μ = − b/2a and variance σ² = − 1/2a. Changing to these new parameters allows one to rewrite the probability density function in a convenient standard form,

$f(x) = \frac{1}{\sqrt{2\pi\sigma^2}}\, e^{\frac{-(x-\mu)^2}{2\sigma^2}} = \frac{1}{\sigma}\, \phi\!\left(\frac{x-\mu}{\sigma}\right).$

For a standard normal distribution, μ = 0 and σ² = 1. The last part of the equation above shows that any other normal distribution can be regarded as a version of the standard normal distribution that has been stretched horizontally by a factor σ and then translated rightward by a distance μ. Thus, μ specifies the position of the bell curve's central peak, and σ specifies the "width" of the bell curve.

The parameter μ is at the same time the mean, the median and the mode of the normal distribution. The parameter σ² is called the variance; as for any random variable, it describes how concentrated the distribution is around its mean. The square root of σ² is called the standard deviation and is the width of the density function.

The normal distribution is usually denoted by N(μ, σ²).^[6] Commonly the letter N is written in calligraphic font (typed as \mathcal{N} in LaTeX). Thus when a random variable X is distributed normally with mean μ and variance σ², we write

$X\ \sim\ \mathcal{N}(\mu,\,\sigma^2). \,$

[ Alternative formulations

Some authors advocate using the precision instead of the variance, and variously define it as τ = σ⁻² or τ = σ⁻¹.^[7] This parametrization has an advantage in numerical applications where σ² is very close to zero and is more convenient to work with in analysis as τ is a natural parameter of the normal distribution. Another advantage of using this parametrization is in the study of conditional distributions in multivariate normal case.

The question of which normal distribution should be called the "standard" one is also answered differently by various authors. Starting from the works of Gauss the standard normal was considered to be the one with variance σ² = 1/2 :

$f(x) = \frac{1}{\sqrt\pi}\,e^{-x^2}$

Stigler (1982) goes even further and insists the standard normal to be with the variance σ² = 1/2π :

$f(x) = e^{-\pi x^2}$

According to the author, this formulation is advantageous because of a much simpler and easier-to-remember formula, the fact that the pdf has unit height at zero, and simple approximate formulas for the quantiles of the distribution. In Stigler's formulation the density of a normal N(μ, τ) with mean μ and precision τ will be equal to

$f(x;\,\mu,\tau) = \tau\,e^{-\pi\tau^2(x-\mu)^2}$

[ Characterization

In the previous section the normal distribution was defined by specifying its probability density function. However there are other ways to characterize a probability distribution. They include: the cumulative distribution function, the moments, the cumulants, the characteristic function, the moment-generating function, etc.

[ Probability density function

The probability density function (pdf) of a random variable describes the relative frequencies of different values for that random variable. The pdf of the normal distribution is given by the formula explained in detail in the previous section:

$f(x;\,\mu,\sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}} \, e^{-(x-\mu)^2\!/(2\sigma^2)} = \frac{1}{\sigma} \,\phi\!\left(\frac{x-\mu}{\sigma}\right), \qquad x\in\mathbb{R}.$

This is a proper function only when the variance σ² is not equal to zero. In that case this is a continuous smooth function, defined on the entire real line, and which is called the "Gaussian function".

Properties:

Function f(x) is unimodal and symmetric around the point x = μ, which is at the same time the mode, the median and the mean of the distribution.^[8]
The inflection points of the curve occur one standard deviation away from the mean (i.e., at x = μ − σ and x = μ + σ).^[8]
Function f(x) is log-concave.^[8]
The standard normal density ϕ(x) is an eigenfunction of the Fourier transform.
The function is supersmooth of order 2, implying that it is infinitely differentiable.^[9]
The first derivative of ϕ(x) is ϕ′(x) = −x·ϕ(x); the second derivative is ϕ′′(x) = (x² − 1)ϕ(x). More generally, the n-th derivative is given by ϕ⁽ⁿ⁾(x) = (−1)ⁿH_n(x)ϕ(x), where H_n is the Hermite polynomial of order n.^[10]

When σ² = 0, the density function doesn't exist. However a generalized function that defines a measure on the real line, and it can be used to calculate, for example, expected value is

$f(x;\,\mu,0) = \delta(x-\mu).$

where δ(x) is the Dirac delta function which is equal to infinity at x = μ and is zero elsewhere.

[ Cumulative distribution function

The cumulative distribution function (CDF) describes probability of a random variable falling in the interval (−∞, x].

The CDF of the standard normal distribution is denoted with the capital Greek letter Φ (phi), and can be computed as an integral of the probability density function:

$\Phi(x) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^x e^{-t^2/2} \, dt = \frac12\left[\, 1 + \operatorname{erf}\left(\frac{x}{\sqrt{2}}\right)\,\right],\quad x\in\mathbb{R}.$

This integral cannot be expressed in terms of elementary functions, so is simply called a transformation of the error function, or erf, a special function. Numerical methods for calculation of the standard normal CDF are discussed below. For a generic normal random variable with mean μ and variance σ² > 0 the CDF will be equal to

$F(x;\,\mu,\sigma^2) = \Phi\left(\frac{x-\mu}{\sigma}\right) = \frac12\left[\, 1 + \operatorname{erf}\left(\frac{x-\mu}{\sigma\sqrt{2}}\right)\,\right],\quad x\in\mathbb{R}.$

The complement of the standard normal CDF, Q(x) = 1 − Φ(x), is referred to as the Q-function, especially in engineering texts.^[11]^[12] This represents the upper tail probability of the Gaussian distribution: that is, the probability that a standard normal random variable X is greater than the number x. Other definitions of the Q-function, all of which are simple transformations of Φ, are also used occasionally.^[13]

Properties:

The standard normal CDF is 2-fold rotationally symmetric around point (0, ½): Φ(−x) = 1 − Φ(x).
The derivative of Φ(x) is equal to the standard normal pdf ϕ(x): Φ′(x) = ϕ(x).
The antiderivative of Φ(x) is: ∫ Φ(x) dx = x Φ(x) + ϕ(x).

For a normal distribution with zero variance, the CDF is the Heaviside step function (with H(0) = 1 convention):

$F(x;\,\mu,0) = \mathbf{1}\{x\geq\mu\}\,.$

[ Quantile function

The inverse of the standard normal CDF, called the quantile function or probit function, is expressed in terms of the inverse error function:

$\Phi^{-1}(p) \equiv z_p = \sqrt2\;\operatorname{erf}^{-1}(2p - 1), \quad p\in(0,1).$

Quantiles of the standard normal distribution are commonly denoted as z_p. The quantile z_p represents such a value that a standard normal random variable X has the probability of exactly p to fall inside the (−∞, z_p] interval. The quantiles are used in hypothesis testing, construction of confidence intervals and Q-Q plots. The most "famous" normal quantile is 1.96 = z_0.975. A standard normal random variable is greater than 1.96 in absolute value in 5% of cases.

For a normal random variable with mean μ and variance σ², the quantile function is

$F^{-1}(p;\,\mu,\sigma^2) = \mu + \sigma\Phi^{-1}(p) = \mu + \sigma\sqrt2\,\operatorname{erf}^{-1}(2p - 1), \quad p\in(0,1).$

[ Characteristic function and moment generating function

The characteristic function φ_X(t) of a random variable X is defined as the expected value of e^itX, where i is the imaginary unit, and t ∈ R is the argument of the characteristic function. Thus the characteristic function is the Fourier transform of the density ϕ(x). For a normally distributed X with mean μ and variance σ², the characteristic function is ^[14]

$\varphi(t;\,\mu,\sigma^2) = \int_{-\infty}^\infty\! e^{itx}\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac12 (x-\mu)^2/\sigma^2} dx = e^{i\mu t - \frac12 \sigma^2t^2}.$

The characteristic function can be analytically extended to the entire complex plane: one defines φ(z) = e^{iμz − 1/2σ²z²} for all z ∈ C.^[15]

The moment generating function is defined as the expected value of e^tX. For a normal distribution, the moment generating function exists and is equal to

$M(t;\, \mu,\sigma^2) = \operatorname{E}[e^{tX}] = \varphi(-it;\, \mu,\sigma^2) = e^{ \mu t + \frac12 \sigma^2 t^2 }.$

The cumulant generating function is the logarithm of the moment generating function:

$g(t;\,\mu,\sigma^2) = \ln M(t;\,\mu,\sigma^2) = \mu t + \frac{1}{2} \sigma^2 t^2.$

Since this is a quadratic polynomial in t, only the first two cumulants are nonzero.

[ Moments

The normal distribution has moments of all orders. That is, for a normally distributed X with mean μ and variance σ², the expectation E[|X|^p] exists and is finite for all p such that Re[p] > −1. Usually we are interested only in moments of integer orders: p = 1, 2, 3, ….

Central moments are the moments of X around its mean μ. Thus, a central moment of order p is the expected value of (X − μ)^p. Using standardization of normal random variables, this expectation will be equal to σ^p · E[Z^p], where Z is standard normal.

$\mathrm{E}\left[(X-\mu)^p\right] = \begin{cases} 0 & \text{if }p\text{ is odd,} \\ \sigma^p\,(p-1)!! & \text{if }p\text{ is even.} \end{cases}$

Here n!! denotes the double factorial, that is the product of every odd number from n to 1.
Central absolute moments are the moments of |X − μ|. They coincide with regular moments for all even orders, but are nonzero for all odd p's.

$\operatorname{E}\left[|X-\mu|^p\right] = \sigma^p(p-1)!! \cdot \left.\begin{cases} \sqrt{2/\pi} & \text{if }p\text{ is odd}, \\ 1 & \text{if }p\text{ is even}, \end{cases}\right\} = \sigma^p \cdot \frac{2^{\frac{p}{2}}\Gamma\left(\frac{p+1}{2}\right)}{\sqrt{\pi}}$

The last formula is true for any non-integer p > −1.
Raw moments and raw absolute moments are the moments of X and |X| respectively. The formulas for these moments are much more complicated, and are given in terms of confluent hypergeometric functions ₁F₁ and U.^{[citation needed]}

$\begin{align} & \operatorname{E} \left[ X^p \right] = \sigma^p \cdot (-i\sqrt{2}\sgn\mu)^p \; U\left( {-\frac{1}{2}p},\, \frac{1}{2},\, -\frac{1}{2}(\mu/\sigma)^2 \right), \\ & \operatorname{E} \left[ |X|^p \right] = \sigma^p \cdot 2^{\frac p 2} \frac {\Gamma\left(\frac{1+p}{2}\right)}{\sqrt\pi}\; _1F_1\left( {-\frac{1}{2}p},\, \frac{1}{2},\, -\frac{1}{2}(\mu/\sigma)^2 \right). \\ \end{align}$

These expressions remain valid even if p is not integer. See also generalized Hermite polynomials.
First two cumulants are equal to μ and σ² respectively, whereas all higher-order cumulants are equal to zero.

Order	Raw moment	Central moment	Cumulant
1	μ	0	μ
2	μ² + σ²	σ²	σ²
3	μ³ + 3μσ²	0	0
4	μ⁴ + 6μ²σ² + 3σ⁴	3σ⁴	0
5	μ⁵ + 10μ³σ² + 15μσ⁴	0	0
6	μ⁶ + 15μ⁴σ² + 45μ²σ⁴ + 15σ⁶	15σ⁶	0
7	μ⁷ + 21μ⁵σ² + 105μ³σ⁴ + 105μσ⁶	0	0
8	μ⁸ + 28μ⁶σ² + 210μ⁴σ⁴ + 420μ²σ⁶ + 105σ⁸	105σ⁸	0

[ Properties

[ Standardizing normal random variables

As a consequence of property 1, it is possible to relate all normal random variables to the standard normal. For example if X is normal with mean μ and variance σ², then

$Z = \frac{X - \mu}{\sigma}$

has mean zero and unit variance, that is Z has the standard normal distribution. Conversely, having a standard normal random variable Z we can always construct another normal random variable with specific mean μ and variance σ²:

$X = \sigma Z + \mu. \,$

This "standardizing" transformation is convenient as it allows one to compute the PDF and especially the CDF of a normal distribution having the table of PDF and CDF values for the standard normal. They will be related via

$F_X(x) = \Phi\left(\frac{x-\mu}{\sigma}\right), \quad f_X(x) = \frac{1}{\sigma}\,\phi\left(\frac{x-\mu}{\sigma}\right).$

[ Standard deviation and confidence intervals

Dark blue is less than one standard deviation from the mean. For the normal distribution, this accounts for about 68% of the set, while two standard deviations from the mean (medium and dark blue) account for about 95%, and three standard deviations (light, medium, and dark blue) account for about 99.7%.

For more details on this topic, see 68-95-99.7 rule (Empirical Rule).

About 68% of values drawn from a normal distribution are within one standard deviation σ away from the mean; about 95% of the values lie within two standard deviations; and about 99.7% are within three standard deviations. This fact is known as the 68-95-99.7 rule, or the empirical rule, or the 3-sigma rule. To be more precise, the area under the bell curve between μ − nσ and μ + nσ is given by

$F(\mu+n\sigma;\,\mu,\sigma^2) - F(\mu-n\sigma;\,\mu,\sigma^2) = \Phi(n)-\Phi(-n) = \mathrm{erf}\left(\frac{n}{\sqrt{2}}\right),$

where erf is the error function. To 12 decimal places, the values for the 1-, 2-, up to 6-sigma points are:^[16]

$\scriptstyle\; n\;$	$\;\scriptstyle\mathrm{erf}\left(\frac{n}{\sqrt{2}}\right)\;$	i.e. 1 minus ...	or 1 in ...
1	0.682689492137	0.317310507863	3.15148718753
2	0.954499736104	0.045500263896	21.9778945080
3	0.997300203937	0.002699796063	370.398347345
4	0.999936657516	0.000063342484	15,787.1927673
5	0.999999426697	0.000000573303	1,744,277.89362
6	0.999999998027	0.000000001973	506,797,345.897

The next table gives the reverse relation of sigma multiples corresponding to a few often used values for the area under the bell curve. These values are useful to determine (asymptotic) confidence intervals of the specified levels based on normally distributed (or asymptotically normal) estimators:^[17]

$\scriptstyle\;\mathrm{erf}\left(\frac{n}{\sqrt{2}}\right)\;$	n	$\scriptstyle\;\mathrm{erf}\left(\frac{n}{\sqrt{2}}\right)\;$	n
0.80	1.281551565545	0.999	3.290526731492
0.90	1.644853626951	0.9999	3.890591886413
0.95	1.959963984540	0.99999	4.417173413469
0.98	2.326347874041	0.999999	4.891638475699
0.99	2.575829303549	0.9999999	5.326723886384
0.995	2.807033768344	0.99999999	5.730728868236
0.998	3.090232306168	0.999999999	6.109410204869

where the value on the left of the table is the proportion of values that will fall within a given interval and n is a multiple of the standard deviation that specifies the width of the interval.

[ Central limit theorem

Main article: Central limit theorem

The theorem states that under certain (fairly common) conditions, the sum of a large number of random variables will have an approximately normal distribution. For example if (x₁, …, x_n) is a sequence of iid random variables, each having mean μ and variance σ², then the central limit theorem states that

$\sqrt{n}\left( \frac{1}{n}\sum_{i=1}^n x_i - \mu \right)\ \xrightarrow{d}\ \mathcal{N}(0,\,\sigma^2).$

The theorem will hold even if the summands x_i are not iid, although some constraints on the degree of dependence and the growth rate of moments still have to be imposed.

The importance of the central limit theorem cannot be overemphasized. A great number of test statistics, scores, and estimators encountered in practice contain sums of certain random variables in them, even more estimators can be represented as sums of random variables through the use of influence functions — all of these quantities are governed by the central limit theorem and will have asymptotically normal distribution as a result.

As the number of discrete events increases, the function begins to resemble a normal distribution

Another practical consequence of the central limit theorem is that certain other distributions can be approximated by the normal distribution, for example:

The binomial distribution B(n, p) is approximately normal N(np, np(1 − p)) for large n and for p not too close to zero or one.
The Poisson(λ) distribution is approximately normal N(λ, λ) for large values of λ.
The chi-squared distribution χ²(k) is approximately normal N(k, 2k) for large ks.
The Student's t-distribution t(ν) is approximately normal N(0, 1) when ν is large.

Whether these approximations are sufficiently accurate depends on the purpose for which they are needed, and the rate of convergence to the normal distribution. It is typically the case that such approximations are less accurate in the tails of the distribution.

A general upper bound for the approximation error in the central limit theorem is given by the Berry–Esseen theorem, improvements of the approximation are given by thee Edgeworth expansions.

[ Miscellaneous

The family of normal distributions is closed under linear transformations. That is, if X is normally distributed with mean μ and variance σ², then a linear transform aX + b (for some real numbers a and b) is also normally distributed:

$aX + b\ \sim\ \mathcal{N}(a\mu+b,\,<HT>Source: this <A HREF=http://en.wikipedia.org/wiki/Normal+distribution>wiki<I></I>pedia article</A>, under <A HREF=$ CC-BY-SA.

Normal distribution

Normal distribution

Contents

[ Definition

[ Alternative formulations

[ Characterization

[ Probability density function

[ Cumulative distribution function

[ Quantile function

[ Characteristic function and moment generating function

[ Moments

[ Properties

[ Standardizing normal random variables

[ Standard deviation and confidence intervals

[ Central limit theorem

[ Miscellaneous

Tutors Answer Your Questions about Probability-and-statistics (FREE)