A hyperparameter is a parameter that controls the training process of a neural network, but is set before training begins, as opposed to model parameters like weights, which are learned during training. Examples of hyperparameters include the learning rate, batch size, and the number of layers or neurons in the network.
Hyperparameters can significantly affect a model's performance, so selecting the right values is crucial. For example, a learning rate that is too high may lead to poor convergence, while one that is too low could slow down training.
Hyperparameter tuning is typically done using methods like grid search, random search, or Bayesian optimization. Fine-tuning these parameters helps the network learn efficiently and avoid overfitting.