View source for Cross Validation

= Cross-Validation =

'''Cross-Validation''' is a statistical method used to estimate the performance of machine learning models on unseen data. It helps ensure that the model generalizes well and reduces the risk of overfitting.

== Why Cross-Validation? ==

When training a model, it is important to test how well it performs on data it has never seen before. Simply evaluating a model on the same data it was trained on can lead to overly optimistic results. Cross-validation provides a more reliable estimate of model performance.

== How Cross-Validation Works ==

The most common method is '''k-fold cross-validation''', which involves the following steps:

# Split the dataset into '''k''' equal parts called '''folds'''.
# For each fold:
# 
# * Use the fold as the '''validation set'''.
# * Use the remaining k-1 folds as the '''training set'''.
# Train the model on the training set and evaluate on the validation set.
# Calculate the performance metric (e.g., accuracy, F1 score).
# Average the results from all k folds to get an overall performance estimate.

== Example of 5-Fold Cross-Validation ==

If you have 100 data points and choose k=5:

* The data is split into 5 parts of 20 points each.
* The model is trained 5 times, each time leaving out one part for validation and training on the other 80 points.
* The average accuracy over the 5 runs is reported.

== Types of Cross-Validation ==

* '''k-Fold Cross-Validation:''' Standard method described above.
* '''Stratified k-Fold:''' Ensures each fold has roughly the same class distribution, important for imbalanced datasets.
* '''Leave-One-Out (LOO):''' Special case where k equals the number of data points. Each example is used once as validation.
* '''Repeated Cross-Validation:''' Repeat k-fold multiple times with different splits to get a more stable estimate.

== Advantages of Cross-Validation ==

* Provides a better measure of model performance on unseen data.
* Helps detect overfitting and underfitting.
* Efficient use of data since all data points are used for training and validation.

== Limitations ==

* More computationally expensive than a simple train-test split.
* Choice of k affects bias and variance of the estimate:
  * Smaller k (e.g., 5) reduces computation but may increase bias.
  * Larger k (e.g., 10 or LOO) gives less bias but higher variance and more computation.

== Related Pages ==

* [[Model Selection]]
* [[Overfitting]]
* [[Underfitting]]
* [[Evaluation Metrics]]
* [[Train-Test Split]]

== SEO Keywords ==

cross validation, k fold cross validation, stratified cross validation, model validation techniques, overfitting prevention, estimating model performance, machine learning model evaluation
[[Category:Artificial Intelligence]]