Embeddings and one-hot encoding are both methods for representing categorical data, but they differ significantly in how they represent information.
One-hot encoding creates a vector with the same length as the number of possible categories, where each category is represented by a unique position set to 1, and all other positions are set to 0. For example, in a three-category system ("cat," "dog," "bird"), the word "dog" might be represented as [0, 1, 0]. The drawback of one-hot encoding is that it leads to sparse vectors, and it doesn’t capture any semantic relationships between the categories.
Embeddings, on the other hand, represent categories as dense, low-dimensional vectors. For example, in word embeddings, words with similar meanings are mapped to nearby points in the vector space, so relationships like similarity or analogy can be captured. Embeddings are learned through models that optimize the vector space to preserve these relationships, making them more powerful for tasks like clustering, search, and classification.