{"id":23123,"date":"2018-07-29T04:00:36","date_gmt":"2018-07-29T04:00:36","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=23123"},"modified":"2026-04-29T17:16:08","modified_gmt":"2026-04-29T11:46:08","slug":"python-statistics","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/python-statistics\/","title":{"rendered":"Python Statistics – Python p-Value, Correlation, T-test, KS Test"},"content":{"rendered":"

In this Python<\/strong> <\/a>Statistics tutorial, we will learn how to calculate the p-value and Correlation in Python. Moreover, we will discuss the T-test and the KS Test with examples and code in Python Statistics.<\/span><\/p>\n

So, let’s start the Python Statistics Tutorial.<\/p>\n

p-value in Python Statistics<\/strong><\/h3>\n

When talking about statistics, a p-value for a statistical model is the probability that, when the null hypothesis is true, the statistical summary is equal to or greater than the actual observed results. This is also termed \u2018<\/span>probability value<\/span><\/i>\u2019 or \u2018<\/span>asymptotic significance<\/span><\/i>\u2019.<\/span><\/p>\n

Do you know about Python Decorators<\/a>?<\/strong><\/p>\n

The null hypothesis states that two measured phenomena experience no relationship to each other. We denote this as H or H<\/span>0<\/span>. One such null hypothesis can be that the number of hours spent in the office affects the amount of salary paid. For a significance level of 5%, if the p-value falls lower than 5%, the null hypothesis is invalidated. <\/span><\/p>\n

Then it is discovered that the number of hours you spend in your office will not affect the amount of salary you will take home. Note that p-values can range from 0% to 100%, and we write them in decimals. A p-value for 5% will be 0.05.<\/span><\/p>\n

A smaller p-value bears more significance as it can tell you that the hypothesis may not explain the observation fairly. If one or more of these probabilities turn out to be less than or equal to <\/span>\u03b1, the level of significance, we reject the null hypothesis. For a true null hypothesis, p can take on any value between 0 and 1 with equal likelihood. For a true alternative hypothesis, p-values likely fall closer to 0.<\/span><\/p>\n

\"Python<\/a>

Python p-Value<\/p><\/div>\n

T-test in Python Statistics<\/strong><\/h3>\n

Let\u2019s talk about T-tests. Such a test tells us whether a sample of numeric data strays or differs significantly from the population. It also talks about two samples- whether they\u2019re different. In other words, it gives us the probability of the difference between populations. The test involves a t-statistic. For small samples, we can use a T-test with two samples.<\/span><\/p>\n

\"Python<\/a>

Python Statistics – Python\u00a0T-test<\/p><\/div>\n

Let’s discuss\u00a0Python Generators<\/a><\/strong><\/p>\n

a. One-sample T-test with Python<\/strong><\/h4>\n

Let\u2019s try this on a single sample. The test will tell us whether the means of the sample and the population are different. Consider the voting populace in India and in Gujarat. Does the average age of Gujarati voters differ from that of the population? Let\u2019s find out.<\/span><\/p>\n

>>> import numpy as np\r\n>>> import pandas as pd\r\n>>> import scipy.stats as stats\r\n>>> import matplotlib.pyplot as plt\r\n>>> import math\r\n>>> np.random.seed(6)\r\n>>> population_ages1=stats.poisson.rvs(loc=18,mu=35,size=150000)\r\n>>> population_ages2=stats.poisson.rvs(loc=18,mu=10,size=100000)\r\n>>> population_ages=np.concatenate((population_ages1,population_ages2))\r\n>>> gujarat_ages1=stats.poisson.rvs(loc=18,mu=30,size=30)\r\n>>> gujarat_ages2=stats.poisson.rvs(loc=18,mu=10,size=20)\r\n>>> gujarat_ages=np.concatenate((gujarat_ages1,gujarat_ages2))\r\n>>> population_ages.mean()<\/pre>\n
43.000112<\/span><\/strong><\/p>\n
>>> gujarat_ages.mean()<\/pre>\n
39.26<\/strong><\/p>\n
>>> stats.ttest_1samp(a=gujarat_ages,popmean=population_ages.mean())<\/pre>\n
Ttest_1sampResult(statistic=-2.5742714883655027, pvalue=0.013118685425061678)<\/strong>
\nNow this value of -2.574 tells us how aberrant the sample mean is from the null hypothesis.<\/span><\/p>\n
b. Two-sample T-test With Python<\/strong><\/h4>\n
Such a test tells us whether two data samples have different means. Here, we take the null hypothesis that both groups have equal means. We don\u2019t need a known population parameter for this.<\/span>
\nLet’s revise Recursion in Python<\/a><\/strong><\/p>\n
>>> np.random.seed(12)\r\n>>> maharashtra_ages1=stats.poisson.rvs(loc=18,mu=33,size=30)\r\n>>> maharashtra_ages2=stats.poisson.rvs(loc=18,mu=13,size=20)\r\n>>> maharashtra_ages=np.concatenate((maharashtra_ages1,maharashtra_ages2))\r\n>>> maharashtra_ages.mean()<\/pre>\n
42.26<\/span><\/strong><\/p>\n
>>> stats.ttest_ind(a=gujarat_ages,b=maharashtra_ages,equal_var=False)<\/pre>\n
Ttest_indResult(statistic=-1.4415218453964938, pvalue=0.1526272389714945)<\/span><\/strong>
\nThe value of 0.152 tells us there\u2019s a 15.2% chance that the sample data is so far apart for two identical groups. This is greater than the 5% confidence level.<\/span><\/p>\n
c. Paired T-test With Python<\/strong><\/h4>\n
When you want to check how different samples from the same group are, you can go for a paired T-test. Let\u2019s take an example.<\/span><\/p>\n
>>> np.random.seed(11)\r\n>>> before=stats.norm.rvs(scale=30,loc=250,size=100)\r\n>>> after=before+stats.norm.rvs(scale=5,loc=-1.25,size=100)\r\n>>> weight_df=pd.DataFrame({\"weight_before\":before,\r\n                         \"weight_after\":after,\r\n                         \"weight_change\":after-before})\r\n>>> weight_df.describe()<\/pre>\n
\"Python<\/a>
Paired Sample T-test<\/p><\/div>\n
>>> stats.ttest_rel(a=before,b=after)<\/pre>\n
Ttest_relResult(statistic=2.5720175998568284, pvalue=0.011596444318439857)<\/strong>
\nSo, we see we have just 1% chances to find such huge differences between samples.<\/span>
\nDo you know about Python Errors<\/a>?<\/strong><\/p>\n
KS Test in Python Statistics<\/strong><\/h3>\n
This is the Kolmogorov-Smirnov<\/span> test. It lets us test the hypothesis that the sample is a part of the standard t-distribution. Let\u2019s take an example.<\/span><\/p>\n
>>> stats.kstest(x,'t',(10,))<\/pre>\n
KstestResult(statistic=0.023682909426459897, pvalue=0.6289865281325614)<\/strong><\/span><\/p>\n
>>> stats.kstest(x,'norm')<\/pre>\n
KstestResult(statistic=0.019334747291889, pvalue=0.8488119233062457)<\/strong>
\nPay attention to the p-values in both cases.<\/span><\/p>\n
a. Two samples<\/strong><\/h4>\n
What we saw above was the KS test for one sample. Let\u2019s try two.<\/span><\/p>\n
>>> stats.ks_2samp(gujarat_ages,maharashtra_ages)<\/pre>\n
Ks_2sampResult(statistic=0.26, pvalue=0.056045859714424606)<\/strong><\/p>\n
Correlation in Python Statistics<\/strong><\/h3>\n
Correlation measures the strength and direction of a relationship between two variables. The most common type is Pearson correlation, which ranges from -1 to +1. A value close to +1 indicates a strong positive relationship, while -1 indicates a strong negative relationship.<\/p>\n
Limitations of correlation:<\/strong><\/p>\n