How is the k-nearest neighbor algorithm different from k-means clustering?
Most of the answers suggest that KNN is a classification technique and K-means is a clustering technique. I will add a graphical representation for you to understand what is going on there.In a KNN algorithm, a test sample is given as the class of majority of its nearest neighbours. In plain words, if you are similar to your neighbours, then you are one of them. Or if apple looks more similar to banana, orange, and melon (fruits) than monkey, cat and rat (animals), then most likely apple is a fruit. Below is an example, we have three classes and the goal is to find a class label for the unknown example [math]x_j [/math]. In this case we use the Euclidean distance and a value of k=5 neighbors. Of the 5 closest neighbors, 4 belong to [math] \omega_1[/math] and 1 belongs to [math] \omega_3[/math], so [math]x_j [/math] is assigned to [math] \omega_1[/math], the predominant class.(Source - Non-Parametric Techniques)The situation with K-means is that given some data you cluster them in K-groups or clusters[1]. K-means belongs to the family of moving centroid algorithms, i.e. at every iteration the center (or centroid) of the cluster moves slightly to minimize the objective function. In the figure below, you start with an initial guess for means for two clusters [math] m_1[/math] and [math] m_2[/math], use these means to group the objects, then update these means then regroup and so on until either the means [math] m_1[/math] and [math] m_2[/math] stop to move (or change) or some threshold is reached (e.g. number of iterations).(Source - Clustering - K-means)To understand the difference between supervised and unsupervised algorithms, you may like to read here - Shehroz Khan's answer to What is the difference between supervised and unsupervised learning algorithms?Footnotes[1] Shehroz Khan's answer to What is the k-Means algorithm and how does it work?