Statistics Fundamentals:
Learn Data & Probability
From the Ground Up
Master statistics fundamentals for data science, business, and research — from descriptive statistics and probability to hypothesis testing, regression, and Bayesian inference. Free, structured, and always online.
Precise, plain-language definitions of the ideas that underpin all statistics fundamentals.
descriptive vs inferential statistics
Descriptive vs. Inferential Statistics
Descriptive statistics summarize and describe the data you have collected — using mean, median, and standard deviation. Inferential statistics use that sample data to draw conclusions about a larger population, relying on probability theory and hypothesis testing to quantify uncertainty.
A p-value is the probability of observing results at least as extreme as yours, assuming the null hypothesis is true. A p-value below 0.05 suggests the result is statistically significant — unlikely to be due to chance alone. It does not measure effect size or practical importance.
The Central Limit Theorem states that when you take sufficiently large random samples from any population — regardless of its shape — the sampling distribution of the mean will approximate a normal distribution. This holds for n ≥ 30, making it the foundation of most inferential methods.
Statistics fundamentals are the foundational concepts used to collect, organize, analyze, and interpret data. They form the backbone of every data-driven field — from business intelligence and clinical research to machine learning and public policy. Once you understand these fundamentals, you can read data critically, identify patterns, and draw evidence-based conclusions.
1. Defining Statistics
Statistics can be defined as the science of learning from data. Its scope encompasses: data collection (surveys, experiments, observational studies); data organization (structuring raw information for analysis); data analysis (applying mathematical techniques to extract meaning); and interpretation and communication (translating findings into actionable insights — see our guide to statistical interpretation).
2. Data Types
All statistical analysis begins with understanding the data at hand. Correct classification determines which methods are appropriate. Learn more in our guide to types of data.
Statistical Data Types — Classification and Examples
Regression analysis — linear regression, R², residuals, correlation, predictive modeling
4. Statistical Inference
Point Estimation vs. Interval Estimation
Point estimation provides a single best-guess value for a population parameter (e.g., the sample mean as an estimate of the population mean). Interval estimation provides a range — a confidence interval — within which the true parameter is expected to fall with a specified probability. Larger samples yield narrower, more precise intervals — see margin of error for details.
5. Bayesian Inference
Bayesian vs. Frequentist Statistics
Frequentist statistics treats probability as the long-run frequency of events. Bayesian statistics treats probability as a degree of belief — prior knowledge is formally incorporated and updated as new evidence arrives, producing a posterior probability distribution. Read more on Bayesian vs. frequentist statistics and the Bayes Factor.
6. Regression Analysis and Predictive Modelling
Regression Analysis — Key Concepts
Concept
Purpose
Key Metric
Linear Regression
Predict a continuous outcome from one or more predictors
EDA is the structured first step in any analytical workflow. Its purpose is to summarise distributions with descriptive statistics, visualise relationships using scatter plots, heatmaps, and Q-Q plots, identify missing values, anomalies, and outliers, and generate hypotheses for inferential analysis. See our data visualization guide and normality tests for the diagnostic toolkit.
8. Practical Applications
Field
Application
Healthcare & Medicine
Clinical trial design and analysis; epidemiological modelling; diagnostic test evaluation
From descriptive statistics and probability to inferential statistics, Bayesian methods, and regression — structured for real understanding.
Core Statistics
Descriptive Statistics
Descriptive statistics summarize and describe the main features of a dataset using measures of central tendency (mean, median, mode) and measures of dispersion (variance, standard deviation, IQR, skewness, kurtosis).
Probability is the mathematical measure of how likely an event is to occur, expressed as a value between 0 and 1. Key concepts include conditional probability, the multiplication and addition rules, Bayes' theorem, and expected value.
The Central Limit Theorem guarantees that the sampling distribution of the mean approaches normality for n ≥ 30, enabling confidence intervals and hypothesis tests on real-world data.
A confidence interval is a range of plausible values for an unknown population parameter, estimated from sample data. A 95% CI means 95% of such intervals would contain the true value.
Hypothesis testing is the formal method used to determine whether sample data provides sufficient evidence to reject a null hypothesis. It relies on significance levels, test statistics, and p-values.
t-tests are the most commonly used inferential statistical tests for comparing means. Paired t-tests compare before-and-after measurements; two-sample t-tests compare independent groups.
Simple linear regression models the relationship between a dependent variable and a single independent variable using least-squares. Model fit is assessed using R², residual plots, and diagnostic tests.
Data visualization is the graphical representation of information to make patterns, trends, and relationships easier to understand. Core tools include histograms, scatter plots, box plots, and probability trees.
Bayesian inference updates the probability assigned to a hypothesis as new evidence accumulates, combining prior beliefs with observed data to produce a posterior probability distribution.
How to Perform Hypothesis Testing: A 6-Step Framework
The complete process for any statistical test — from stating your hypothesis to stating your conclusion.
1
State the null and alternative hypotheses
Define H₀ (null hypothesis) and H₁ (alternative hypothesis). Be precise — vague hypotheses produce ambiguous conclusions. See null vs. alternative hypothesis.
2
Choose a significance level (α)
Select your significance level, typically α = 0.05. This represents a 5% risk of a Type I error — falsely rejecting a true null hypothesis.
3
Select the appropriate statistical test
Choose based on your data type: Z-test (large sample, known σ), t-test (small sample), chi-square (categorical data), or ANOVA (3+ group means).
4
Calculate the test statistic
Compute the test statistic from your sample data. This converts raw data into a standardised value (z, t, χ², or F) for comparison against a reference distribution — see degrees of freedom.
If p < α, reject the null hypothesis. If p ≥ α, fail to reject it. State your conclusion in plain language relative to the original research question — see our decision rule guide.
Understanding distribution assumptions in algorithms
Research & Science
Statistics ensures research findings are reliable and reproducible. It helps researchers design experiments, test hypotheses, and draw valid conclusions.
Explore U.S. household debt, savings, spending, credit card balances, and personal finance data using verified statistics from the Federal Reserve, BLS, BEA, CFPB, Experian, and Vanguard.
Explore historical stock market returns, volatility, long-term compounding, and key investment statistics using data from S&P Dow Jones Indices, NYU Stern, the Federal Reserve, and the SEC.
Explore cryptocurrency adoption, market capitalization, stablecoin growth, user demographics, ownership patterns, and crypto risk data using current statistics from leading research sources.
Explore AI adoption, market growth, ROI data, business use cases, and emerging trends with verified statistics from McKinsey, Gartner, PwC, IBM, WEF, Stanford HAI, and Grand View Research.
Explore small business failure rates, survival statistics, major causes of business failure, industry benchmarks, geographic differences, and the financial factors that influence long-term business survival.
Find Lilliefors critical D values by sample size and significance level. Compare your observed D with the critical value to determine whether the normality assumption is supported.
View table
FAQ
Frequently Asked Questions About Statistics
Common questions from beginners and professionals — answered clearly and precisely.
What is the difference between descriptive and inferential statistics?+
Descriptive statistics summarize and describe the data you already have — for example, calculating the average exam score for a class. Inferential statistics use a sample to make predictions or draw conclusions about a larger population. Both branches are covered in depth in the learning library above.
What is a p-value and why does it matter?+
A p-value is the probability of observing your results — or more extreme results — assuming the null hypothesis is true. A small p-value (below 0.05) suggests the observed effect is unlikely to have occurred by chance. However, a p-value alone does not measure effect size — always interpret it alongside confidence intervals and effect size measures like Cohen's d.
What is the Central Limit Theorem and why is it important?+
The Central Limit Theorem (CLT) states that the sampling distribution of the sample mean approaches a normal distribution as the sample size increases — regardless of the shape of the original population. This is why so many statistical methods work reliably on real-world data. A sample size of 30 or more is typically sufficient.
What is the difference between a z-test and a t-test?+
Use a z-test when the population standard deviation (σ) is known and the sample size is large (typically n ≥ 30). Use a t-test when the population standard deviation is unknown — which is almost always the case in practice. The t-distribution has heavier tails to account for this additional uncertainty.
How do I know which statistical test to use?+
The right test depends on: (1) your data type — continuous or categorical; (2) the number of groups — one, two, or more; (3) whether groups are independent or paired; and (4) your sample size. Our statistical test selector includes a full decision tree to guide you.
What is a confidence interval in simple terms?+
A confidence interval is a range of plausible values for an unknown population parameter estimated from sample data. A 95% CI means that if you repeated the study many times, 95% of intervals constructed would contain the true population value. A narrower interval means a more precise estimate, which typically requires a larger sample size. Use our CI calculator to compute one instantly.
What is the difference between correlation and causation?+
Correlation measures statistical association between two variables. Causation means one variable directly causes changes in another. Correlation does not imply causation — two variables may be correlated because a third variable (a confounder) influences both, or the association is coincidental. Establishing causation requires well-designed experiments — see our study design guide.
What is standard deviation and how is it different from variance?+
Both measure the spread or dispersion in a dataset. Variance is the average of the squared differences from the mean. Standard deviation is the square root of the variance and is more interpretable because it is expressed in the same units as your original data. See our standard deviation guide for worked examples.
Our Team
Built by Experts in Statistics, Data Science & Analytics
Our team combines academic training and applied experience to make statistics accurate, accessible, and practical.
Holds a background in Statistics with strong academic training in probability theory, regression analysis, and experimental design. Focused on simplifying complex statistical concepts into clear, structured lessons.
Descriptive StatisticsRegressionProbability Theory
KA
Kinza A
Data Science & ML Writer
Background in Applied Mathematics and Data Science with expertise in statistical modeling, hypothesis testing, and machine learning foundations. Connects statistics fundamentals to real-world applications.
Experienced in business analytics and data interpretation with practical work in dashboards, data visualization, and statistical reporting. Contributes real-world insights and case-based examples.
Data VisualizationBusiness AnalyticsStatistical Reporting
Browse Everything
Complete Statistics Fundamentals Directory
Every guide, calculator, table, and tool on the site — organized by topic so you can jump straight to what you need.
Use the pre-formatted citations below for academic papers, research reports, and course assignments. All content is reviewed by qualified contributors with backgrounds in statistics and data science.
APA (7th Edition)
StatisticsFundamentals.com. (2026, June 20). Statistics fundamentals: The complete beginner's guide. https://statisticsfundamentals.com/
MLA (9th Edition)
"Statistics Fundamentals: The Complete Beginner's Guide." StatisticsFundamentals.com, 20 June 2026, statisticsfundamentals.com/.
Chicago (Author-Date)
StatisticsFundamentals.com. 2026. "Statistics Fundamentals: The Complete Beginner's Guide." Accessed June 20, 2026. https://statisticsfundamentals.com/.
Last reviewed and updated: June 20, 2026
Content Contributors & Reviewers
MA
Minsa A
Senior Statistics Editor
Descriptive Statistics · Probability Theory · Regression