Who invented K-means?

The term "k-means" was first used by James MacQueen in 1967, though the idea goes back to Hugo Steinhaus in 1956. The standard algorithm was first proposed by Stuart Lloyd of Bell Labs in 1957 as a technique for pulse-code modulation, although it was not published as a journal article until 1982.

en.wikipedia.org - K-means clustering - Wikipedia

How to boost Instagram followers?

How can I increase my followers on Instagram? Use the right Instagram hashtags. Steal your competitors' followers. Use geotags on Instagram posts....

Last updated Jan 5, 2018

Does PR get rejected?

There are many reasons why permanent residence applications can get refused such as criminality, health, misrepresentation, not meeting eligibility...

Last updated Jun 28, 2021

Promotion

Vector quantization algorithm minimizing the sum of squared deviations

Not to be confused with k-nearest neighbors algorithm

k-means clustering is a method of vector quantization, originally from signal processing, that aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean (cluster centers or cluster centroid), serving as a prototype of the cluster. This results in a partitioning of the data space into Voronoi cells. k-means clustering minimizes within-cluster variances (squared Euclidean distances), but not regular Euclidean distances, which would be the more difficult Weber problem: the mean optimizes squared errors, whereas only the geometric median minimizes Euclidean distances. For instance, better Euclidean solutions can be found using k-medians and k-medoids. The problem is computationally difficult (NP-hard); however, efficient heuristic algorithms converge quickly to a local optimum. These are usually similar to the expectation-maximization algorithm for mixtures of Gaussian distributions via an iterative refinement approach employed by both k-means and Gaussian mixture modeling. They both use cluster centers to model the data; however, k-means clustering tends to find clusters of comparable spatial extent, while the Gaussian mixture model allows clusters to have different shapes. The unsupervised k-means algorithm has a loose relationship to the k-nearest neighbor classifier, a popular supervised machine learning technique for classification that is often confused with k-means due to the name. Applying the 1-nearest neighbor classifier to the cluster centers obtained by k-means classifies new data into the existing clusters. This is known as nearest centroid classifier or Rocchio algorithm.

Description [ edit ]

Given a set of observations (x 1 , x 2 , ..., x n ), where each observation is a d-dimensional real vector, k-means clustering aims to partition the n observations into k (≤ n) sets S = {S 1 , S 2 , ..., S k } so as to minimize the within-cluster sum of squares (WCSS) (i.e. variance). Formally, the objective is to find: a r g m i n S ∑ i = 1 k ∑ x ∈ S i ‖ x − μ i ‖ 2 = a r g m i n S ∑ i = 1 k | S i | Var ⁡ S i {displaystyle {underset {mathbf {S} }{operatorname {arg,min} }}sum _{i=1}^{k}sum _{mathbf {x} in S_{i}}left|mathbf {x} -{oldsymbol {mu }}_{i} ight|^{2}={underset {mathbf {S} }{operatorname {arg,min} }}sum _{i=1}^{k}|S_{i}|operatorname {Var} S_{i}} where μ i is the mean of points in S i . This is equivalent to minimizing the pairwise squared deviations of points in the same cluster: a r g m i n S ∑ i = 1 k 1 | S i | ∑ x , y ∈ S i ‖ x − y ‖ 2 {displaystyle {underset {mathbf {S} }{operatorname {arg,min} }}sum _{i=1}^{k},{frac {1}{|S_{i}|}},sum _{mathbf {x} ,mathbf {y} in S_{i}}left|mathbf {x} -mathbf {y} ight|^{2}} The equivalence can be deduced from identity | S i | ∑ x ∈ S i ‖ x − μ i ‖ 2 = ∑ x ≠ y ∈ S i ‖ x − y ‖ 2 {displaystyle |S_{i}|sum _{mathbf {x} in S_{i}}left|mathbf {x} -{oldsymbol {mu }}_{i} ight|^{2}=sum _{mathbf {x} eq mathbf {y} in S_{i}}left|mathbf {x} -mathbf {y} ight|^{2}} . Since the total variance is constant, this is equivalent to maximizing the sum of squared deviations between points in different clusters (between-cluster sum of squares, BCSS),.[1] This deterministic relationship is also related to the law of total variance in probability theory.

History [ edit ]

The term "k-means" was first used by James MacQueen in 1967,[2] though the idea goes back to Hugo Steinhaus in 1956.[3] The standard algorithm was first proposed by Stuart Lloyd of Bell Labs in 1957 as a technique for pulse-code modulation, although it was not published as a journal article until 1982.[4] In 1965, Edward W. Forgy published essentially the same method, which is why it is sometimes referred to as the Lloyd–Forgy algorithm.[5]

Algorithms [ edit ]

Standard algorithm (naive k-means) [ edit ]

k-means Convergence of-means

The most common algorithm uses an iterative refinement technique. Due to its ubiquity, it is often called "the k-means algorithm"; it is also referred to as Lloyd's algorithm, particularly in the computer science community. It is sometimes also referred to as "naïve k-means", because there exist much faster alternatives.[6] Given an initial set of k means m 1 (1),...,m k (1) (see below), the algorithm proceeds by alternating between two steps:[7] Assignment step: Assign each observation to the cluster with the nearest mean: that with the least squared Euclidean distance.[8] (Mathematically, this means partitioning the observations according to the Voronoi diagram generated by the means.) S i ( t ) = { x p : ‖ x p − m i ( t ) ‖ 2 ≤ ‖ x p − m j ( t ) ‖ 2 ∀ j , 1 ≤ j ≤ k } , {displaystyle S_{i}^{(t)}=left{x_{p}:left|x_{p}-m_{i}^{(t)} ight|^{2}leq left|x_{p}-m_{j}^{(t)} ight|^{2} forall j,1leq jleq k ight},} where each x p {displaystyle x_{p}} S ( t ) {displaystyle S^{(t)}} Update step: Recalculate means (centroids) for observations assigned to each cluster. m i ( t + 1 ) = 1 | S i ( t ) | ∑ x j ∈ S i ( t ) x j {displaystyle m_{i}^{(t+1)}={frac {1}{left|S_{i}^{(t)} ight|}}sum _{x_{j}in S_{i}^{(t)}}x_{j}} The algorithm has converged when the assignments no longer change. The algorithm is not guaranteed to find the optimum.[9] The algorithm is often presented as assigning objects to the nearest cluster by distance. Using a different distance function other than (squared) Euclidean distance may prevent the algorithm from converging. Various modifications of k-means such as spherical k-means and k-medoids have been proposed to allow using other distance measures.

Initialization methods [ edit ]

Commonly used initialization methods are Forgy and Random Partition.[10] The Forgy method randomly chooses k observations from the dataset and uses these as the initial means. The Random Partition method first randomly assigns a cluster to each observation and then proceeds to the update step, thus computing the initial mean to be the centroid of the cluster's randomly assigned points. The Forgy method tends to spread the initial means out, while Random Partition places all of them close to the center of the data set. According to Hamerly et al.,[10] the Random Partition method is generally preferable for algorithms such as the k-harmonic means and fuzzy k-means. For expectation maximization and standard k-means algorithms, the Forgy method of initialization is preferable. A comprehensive study by Celebi et al.,[11] however, found that popular initialization methods such as Forgy, Random Partition, and Maximin often perform poorly, whereas Bradley and Fayyad's approach[12] performs "consistently" in "the best group" and k-means++ performs "generally well".

Demonstration of the standard algorithm

1. k initial "means" (in this case k =3) are randomly generated within the data domain (shown in color). 2. k clusters are created by associating every observation with the nearest mean. The partitions here represent the Voronoi diagram generated by the means. 3. The centroid of each of the k clusters becomes the new mean.

4. Steps 2 and 3 are repeated until convergence has been reached.

The algorithm does not guarantee convergence to the global optimum. The result may depend on the initial clusters. As the algorithm is usually fast, it is common to run it multiple times with different starting conditions. However, worst-case performance can be slow: in particular certain point sets, even in two dimensions, converge in exponential time, that is 2Ω( n ).[13] These point sets do not seem to arise in practice: this is corroborated by the fact that the smoothed running time of k-means is polynomial.[14] The "assignment" step is referred to as the "expectation step", while the "update step" is a maximization step, making this algorithm a variant of the generalized expectation-maximization algorithm.

Complexity [ edit ]

Finding the optimal solution to the k-means clustering problem for observations in d dimensions is: NP-hard in general Euclidean space (of d dimensions) even for two clusters, [15] [16] dimensions) even for two clusters, NP-hard for a general number of clusters k even in the plane, [17] even in the plane, if k and d (the dimension) are fixed, the problem can be exactly solved in time O ( n d k + 1 ) {displaystyle O(n^{dk+1})} n is the number of entities to be clustered.[18] Thus, a variety of heuristic algorithms such as Lloyd's algorithm given above are generally used. The running time of Lloyd's algorithm (and most variants) is O ( n k d i ) {displaystyle O(nkdi)} ,[9][19] where:

n is the number of d -dimensional vectors (to be clustered)

is the number of -dimensional vectors (to be clustered) k the number of clusters

the number of clusters i the number of iterations needed until convergence.

On data that does have a clustering structure, the number of iterations until convergence is often small, and results only improve slightly after the first dozen iterations. Lloyd's algorithm is therefore often considered to be of "linear" complexity in practice, although it is in the worst case superpolynomial when performed until convergence.[20] In the worst-case, Lloyd's algorithm needs i = 2 Ω ( n ) {displaystyle i=2^{Omega ({sqrt {n}})}} superpolynomial. [20] superpolynomial. Lloyd's k -means algorithm has polynomial smoothed running time. It is shown that [14] for arbitrary set of n points in [ 0 , 1 ] d {displaystyle [0,1]^{d}} 0 and variance σ 2 {displaystyle sigma ^{2}} k -means algorithm is bounded by O ( n 34 k 34 d 8 log 4 ⁡ ( n ) / σ 6 ) {displaystyle O(n^{34}k^{34}d^{8}log ^{4}(n)/sigma ^{6})} n , k , d and 1 / σ {displaystyle 1/sigma } -means algorithm has polynomial smoothed running time. It is shown that for arbitrary set of points in and variance -means algorithm is bounded by , , and Better bounds are proven for simple cases. For example, it is shown that the running time of k-means algorithm is bounded by O ( d n 4 M 2 ) {displaystyle O(dn^{4}M^{2})} n points in an integer lattice { 1 , … , M } d {displaystyle {1,dots ,M}^{d}} [21]

How much do you earn with 10k views?

The actual rates an advertiser pays varies, usually between $0.10 to $0.30 per view, but averages out at $0.18 per view. On Average, the YouTube...

Last updated Apr 18, 2021

What job would be good for someone with anxiety?

Counselor or Psychologist This may seem like an odd job choice for a person with anxiety issues, but counselors and psychologists who successfully...

Last updated Aug 16, 2019

Promotion

Lloyd's algorithm is the standard approach for this problem. However, it spends a lot of processing time computing the distances between each of the k cluster centers and the n data points. Since points usually stay in the same clusters after a few iterations, much of this work is unnecessary, making the naïve implementation very inefficient. Some implementations use caching and the triangle inequality in order to create bounds and accelerate Lloyd's algorithm.[9][22][23][24][25]

Variations [ edit ]

Hartigan–Wong method [ edit ]

Hartigan and Wong's method[9] provides a variation of k-means algorithm which progresses towards a local minimum of the minimum sum-of-squares problem with different solution updates. The method is a local search that iteratively attempts to relocate a sample into a different cluster as long as this process improves the objective function. When no sample can be relocated into a different cluster with an improvement of the objective, the method stops (in a local minimum). In a similar way as the classical k-means, the approach remains a heuristic since it does not necessarily guarantee that the final solution is globally optimum. Let φ ( S j ) {displaystyle varphi (S_{j})} be the individual cost of S j {displaystyle S_{j}} defined by ∑ x ∈ S j ( x − μ j ) 2 {displaystyle sum _{xin S_{j}}(x-mu _{j})^{2}} , with μ j {displaystyle mu _{j}} the center of the cluster. Assignment step: Hartigan and Wong's method starts by partitioning the points into random clusters { S j } j ∈ { 1 , ⋯ k } {displaystyle {S_{j}}_{jin {1,cdots k}}} . Δ ( m , n , x ) = φ ( S n ) + φ ( S m ) − φ ( S n ∖ { x } ) − φ ( S m ∪ { x } ) . {displaystyle Delta (m,n,x)=varphi (S_{n})+varphi (S_{m})-varphi (S_{n}smallsetminus {x})-varphi (S_{m}cup {x}).} For the x , n , m {displaystyle x,n,m} that reach this maximum, x {displaystyle x} moves from the cluster S n {displaystyle S_{n}} to the cluster S m {displaystyle S_{m}} . Termination: The algorithm terminates once Δ ( m , n , x ) {displaystyle Delta (m,n,x)} is less than zero for all x , n , m {displaystyle x,n,m} . Different move acceptance strategies can be used. In a first-improvement strategy, any improving relocation can be applied, whereas in a best-improvement strategy, all possible relocations are iteratively tested and only the best is applied at each iteration. The former approach favors speed, whether the latter approach generally favors solution quality at the expense of additional computational time. The function Δ {displaystyle Delta } used to calculate the result of a relocation can also be efficiently evaluated by using equality[35] Δ ( x , n , m ) = ∣ S n ∣ ∣ S n ∣ − 1 ⋅ ‖ μ n − x ‖ 2 − ∣ S m ∣ ∣ S m ∣ + 1 ⋅ ‖ μ m − x ‖ 2 . {displaystyle Delta (x,n,m)={frac {mid S_{n}mid }{mid S_{n}mid -1}}cdot lVert mu _{n}-x Vert ^{2}-{frac {mid S_{m}mid }{mid S_{m}mid +1}}cdot lVert mu _{m}-x Vert ^{2}.}

Global optimization and metaheuristics [ edit ]

The classical k-means algorithm and its variations are known to only converge to local minima of the minimum-sum-of-squares clustering problem defined as a r g m i n S ∑ i = 1 k ∑ x ∈ S i ‖ x − μ i ‖ 2 . {displaystyle {underset {mathbf {S} }{operatorname {arg,min} }}sum _{i=1}^{k}sum _{mathbf {x} in S_{i}}left|mathbf {x} -{oldsymbol {mu }}_{i} ight|^{2}.} Many studies have attempted to improve the convergence behavior of the algorithm and maximize the chances of attaining the global optimum (or at least, local minima of better quality). Initialization and restart techniques discussed in the previous sections are one alternative to find better solutions. More recently, global optimization algorithms based on branch-and-bound and semidefinite programming have produced ‘’provenly optimal’’ solutions for datasets with up to 4,177 entities and 20,531 features.[36] As expected, due to the NP-hardness of the subjacent optimization problem, the computational time of optimal algorithms for K-means quickly increases beyond this size. Optimal solutions for small- and medium-scale still remain valuable as a benchmark tool, to evaluate the quality of other heuristics. To find high-quality local minima within a controlled computational time but without optimality guarantees, other works have explored metaheuristics and other global optimization techniques, e.g., based on incremental approaches and convex optimization,[37] random swaps[38] (i.e., iterated local search), variable neighborhood search[39] and genetic algorithms.[40][41] It is indeed known that finding better local minima of the minimum sum-of-squares clustering problem can make the difference between failure and success to recover cluster structures in feature spaces of high dimension.[41]

Discussion [ edit ]

k-means convergence to a local minimum. In this example, the result of k-means clustering (the right figure) contradicts the obvious cluster structure of the data set. The small circles are the data points, the four ray stars are the centroids (means). The initial configuration is on the left figure. The algorithm converges after five iterations presented on the figures, from the left to the right. The illustration was prepared with the Mirkes Java applet.[42] A typical example of the-means convergence to a local minimum. In this example, the result of-means clustering (the right figure) contradicts the obvious cluster structure of the data set. The small circles are the data points, the four ray stars are the centroids (means). The initial configuration is on the left figure. The algorithm converges after five iterations presented on the figures, from the left to the right. The illustration was prepared with the Mirkes Java applet. k-means clustering vs. k-means to produce equal-sized clusters leads to bad results here, while EM benefits from the Gaussian distributions with different radius present in the data set. -means clustering vs. EM clustering on an artificial dataset ("mouse"). The tendency of-means to produce equal-sized clusters leads to bad results here, while EM benefits from the Gaussian distributions with different radius present in the data set. Three key features of k-means that make it efficient are often regarded as its biggest drawbacks: Euclidean distance is used as a metric and variance is used as a measure of cluster scatter. The number of clusters k is an input parameter: an inappropriate choice of k may yield poor results. That is why, when performing k -means, it is important to run diagnostic checks for determining the number of clusters in the data set. is an input parameter: an inappropriate choice of may yield poor results. That is why, when performing -means, it is important to run diagnostic checks for determining the number of clusters in the data set. Convergence to a local minimum may produce counterintuitive ("wrong") results (see example in Fig.). A key limitation of k-means is its cluster model. The concept is based on spherical clusters that are separable so that the mean converges towards the cluster center. The clusters are expected to be of similar size, so that the assignment to the nearest cluster center is the correct assignment. When for example applying k-means with a value of k = 3 {displaystyle k=3} onto the well-known Iris flower data set, the result often fails to separate the three Iris species contained in the data set. With k = 2 {displaystyle k=2} , the two visible clusters (one containing two species) will be discovered, whereas with k = 3 {displaystyle k=3} one of the two clusters will be split into two even parts. In fact, k = 2 {displaystyle k=2} is more appropriate for this data set, despite the data set's containing 3 classes. As with any other clustering algorithm, the k-means result makes assumptions that the data satisfy certain criteria. It works well on some data sets, and fails on others. The result of k-means can be seen as the Voronoi cells of the cluster means. Since data is split halfway between cluster means, this can lead to suboptimal splits as can be seen in the "mouse" example. The Gaussian models used by the expectation-maximization algorithm (arguably a generalization of k-means) are more flexible by having both variances and covariances. The EM result is thus able to accommodate clusters of variable size much better than k-means as well as correlated clusters (not in this example). In counterpart, EM requires the optimization of a larger number of free parameters and poses some methodological issues due to vanishing clusters or badly-conditioned covariance matrices. K-means is closely related to nonparametric Bayesian modeling.[43]

Applications [ edit ]

k-means clustering is rather easy to apply to even large data sets, particularly when using heuristics such as Lloyd's algorithm. It has been successfully used in market segmentation, computer vision, and astronomy among many other domains. It often is used as a preprocessing step for other algorithms, for example to find a starting configuration.

Vector quantization [ edit ]

Two-channel (for illustration purposes – red and green channels only) color image

k-means Vector quantization of colors present in the image above into Voronoi cells using-means

Is there an app that helps with scheduling?

Calendly makes it incredibly easy to schedule work events like interviews and meetings. The platform streamlines the process of finding when people...

Last updated Oct 24, 2018

What will be big in marketing in 2022?

A few digital marketing trends of 2022 can create a competitive advantage when used efficiently. Influencer marketing, video marketing, omnichannel...

Last updated Nov 18, 2019

Promotion

k-means originates from signal processing, and still finds use in this domain. For example, in computer graphics, color quantization is the task of reducing the color palette of an image to a fixed number of colors k. The k-means algorithm can easily be used for this task and produces competitive results. A use case for this approach is image segmentation. Other uses of vector quantization include non-random sampling, as k-means can easily be used to choose k different but prototypical objects from a large data set for further analysis.

Cluster analysis [ edit ]

In cluster analysis, the k-means algorithm can be used to partition the input data set into k partitions (clusters). However, the pure k-means algorithm is not very flexible, and as such is of limited use (except for when vector quantization as above is actually the desired use case). In particular, the parameter k is known to be hard to choose (as discussed above) when not given by external constraints. Another limitation is that it cannot be used with arbitrary distance functions or on non-numerical data. For these use cases, many other algorithms are superior.

Feature learning [ edit ]

k-means clustering has been used as a feature learning (or dictionary learning) step, in either (semi-)supervised learning or unsupervised learning.[44] The basic approach is first to train a k-means clustering representation, using the input training data (which need not be labelled). Then, to project any input datum into the new feature space, an "encoding" function, such as the thresholded matrix-product of the datum with the centroid locations, computes the distance from the datum to each centroid, or simply an indicator function for the nearest centroid,[44][45] or some smooth transformation of the distance.[46] Alternatively, transforming the sample-cluster distance through a Gaussian RBF, obtains the hidden layer of a radial basis function network.[47] This use of k-means has been successfully combined with simple, linear classifiers for semi-supervised learning in NLP (specifically for named entity recognition)[48] and in computer vision. On an object recognition task, it was found to exhibit comparable performance with more sophisticated feature learning approaches such as autoencoders and restricted Boltzmann machines.[46] However, it generally requires more data, for equivalent performance, because each data point only contributes to one "feature".[44]

Relation to other algorithms [ edit ]

Gaussian mixture model [ edit ]

The slow "standard algorithm" for k-means clustering, and its associated expectation-maximization algorithm, is a special case of a Gaussian mixture model, specifically, the limiting case when fixing all covariances to be diagonal, equal and have infinitesimal small variance.[49]: 850 Instead of small variances, a hard cluster assignment can also be used to show another equivalence of k-means clustering to a special case of "hard" Gaussian mixture modelling.[50]: 354, 11.4.2.5 This does not mean that it is efficient to use Gaussian mixture modelling to compute k-means, but just that there is a theoretical relationship, and that Gaussian mixture modelling can be interpreted as a generalization of k-means; on the contrary, it has been suggested to use k-means clustering to find starting points for Gaussian mixture modelling on difficult data.[49]: 849 Another generalization of the k-means algorithm is the k-SVD algorithm, which estimates data points as a sparse linear combination of "codebook vectors". k-means corresponds to the special case of using a single codebook vector, with a weight of 1.[51]

Principal component analysis [ edit ]

The relaxed solution of k -means clustering, specified by the cluster indicators, is given by principal component analysis (PCA).[52][53] The intuition is that k-means describe spherically shaped (ball-like) clusters. If the data has 2 clusters, the line connecting the two centroids is the best 1-dimensional projection direction, which is also the first PCA direction. Cutting the line at the center of mass separates the clusters (this is the continuous relaxation of the discrete cluster indicator). If the data have three clusters, the 2-dimensional plane spanned by three cluster centroids is the best 2-D projection. This plane is also defined by the first two PCA dimensions. Well-separated clusters are effectively modelled by ball-shaped clusters and thus discovered by k-means. Non-ball-shaped clusters are hard to separate when they are close. For example, two half-moon shaped clusters intertwined in space do not separate well when projected onto PCA subspace. k-means should not be expected to do well on this data.[54] It is straightforward to produce counterexamples to the statement that the cluster centroid subspace is spanned by the principal directions.[55]

Mean shift clustering [ edit ]

Basic mean shift clustering algorithms maintain a set of data points the same size as the input data set. Initially, this set is copied from the input set. Then this set is iteratively replaced by the mean of those points in the set that are within a given distance of that point. By contrast, k-means restricts this updated set to k points usually much less than the number of points in the input data set, and replaces each point in this set by the mean of all points in the input set that are closer to that point than any other (e.g. within the Voronoi partition of each updating point). A mean shift algorithm that is similar then to k-means, called likelihood mean shift, replaces the set of points undergoing replacement by the mean of all points in the input set that are within a given distance of the changing set.[56] One of the advantages of mean shift over k-means is that the number of clusters is not pre-specified, because mean shift is likely to find only a few clusters if only a small number exist. However, mean shift can be much slower than k-means, and still requires selection of a bandwidth parameter. Mean shift has soft variants.

Independent component analysis [ edit ]

Under sparsity assumptions and when input data is pre-processed with the whitening transformation, k-means produces the solution to the linear independent component analysis (ICA) task. This aids in explaining the successful application of k-means to feature learning.[57]

Bilateral filtering [ edit ]

k-means implicitly assumes that the ordering of the input data set does not matter. The bilateral filter is similar to k-means and mean shift in that it maintains a set of data points that are iteratively replaced by means. However, the bilateral filter restricts the calculation of the (kernel weighted) mean to include only points that are close in the ordering of the input data.[56] This makes it applicable to problems such as image denoising, where the spatial arrangement of pixels in an image is of critical importance.

Software implementations [ edit ]

Different implementations of the algorithm exhibit performance differences, with the fastest on a test data set finishing in 10 seconds, the slowest taking 25,988 seconds (~7 hours).[1] The differences can be attributed to implementation quality, language and compiler differences, different termination criteria and precision levels, and the use of indexes for acceleration.

Accord.NET contains C# implementations for k -means, k -means++ and k -modes.

-means, -means++ and -modes. ALGLIB contains parallelized C++ and C# implementations for k -means and k -means++.

-means and -means++. AOSP contains a Java implementation for k -means.

-means. CrimeStat implements two spatial k -means algorithms, one of which allows the user to define the starting locations. -means algorithms, one of which allows the user to define the starting locations. ELKI contains k -means (with Lloyd and MacQueen iteration, along with different initializations such as k -means++ initialization) and various more advanced clustering algorithms. -means (with Lloyd and MacQueen iteration, along with different initializations such as -means++ initialization) and various more advanced clustering algorithms. Smile contains k -means and various more other algorithms and results visualization (for java, kotlin and scala). -means and various more other algorithms and results visualization (for java, kotlin and scala). Julia contains a k -means implementation in the JuliaStats Clustering package. -means implementation in the JuliaStats Clustering package. KNIME contains nodes for k -means and k -medoids.

-means and -medoids. Mahout contains a MapReduce based k -means.

-means. mlpack contains a C++ implementation of k -means.

-means. Octave contains k -means.

-means. OpenCV contains a k -means implementation.

-means implementation. Orange includes a component for k -means clustering with automatic selection of k and cluster silhouette scoring. -means clustering with automatic selection of and cluster silhouette scoring. PSPP contains k -means, The QUICK CLUSTER command performs k -means clustering on the dataset. -means, The QUICK CLUSTER command performs -means clustering on the dataset. R contains three k -means variations.

-means variations. SciPy and scikit-learn contain multiple k -means implementations.

-means implementations. Spark MLlib implements a distributed k -means algorithm.

-means algorithm. Torch contains an unsup package that provides k -means clustering.

package that provides -means clustering. Weka contains k-means and x-means.

Proprietary [ edit ]

The following implementations are available under proprietary license terms, and may not have publicly available source code.

Who invented K-means?

How to boost Instagram followers?

Does PR get rejected?

Vector quantization algorithm minimizing the sum of squared deviations

Not to be confused with k-nearest neighbors algorithm

Description [ edit ]

History [ edit ]

Algorithms [ edit ]

Standard algorithm (naive k-means) [ edit ]

k-means Convergence of-means

Initialization methods [ edit ]

Demonstration of the standard algorithm

4. Steps 2 and 3 are repeated until convergence has been reached.

Complexity [ edit ]

n is the number of d -dimensional vectors (to be clustered)

the number of clusters i the number of iterations needed until convergence.

How much do you earn with 10k views?

What job would be good for someone with anxiety?

Variations [ edit ]

Hartigan–Wong method [ edit ]

Global optimization and metaheuristics [ edit ]

Discussion [ edit ]

Applications [ edit ]

Vector quantization [ edit ]

Two-channel (for illustration purposes – red and green channels only) color image

Is there an app that helps with scheduling?

What will be big in marketing in 2022?

Cluster analysis [ edit ]

Feature learning [ edit ]

Relation to other algorithms [ edit ]

Gaussian mixture model [ edit ]

Principal component analysis [ edit ]

Mean shift clustering [ edit ]

Independent component analysis [ edit ]

Bilateral filtering [ edit ]

Similar problems [ edit ]

Software implementations [ edit ]

Accord.NET contains C# implementations for k -means, k -means++ and k -modes.

-means and -means++. AOSP contains a Java implementation for k -means.

-means and -medoids. Mahout contains a MapReduce based k -means.

-means. mlpack contains a C++ implementation of k -means.

-means. Octave contains k -means.

-means. OpenCV contains a k -means implementation.

-means variations. SciPy and scikit-learn contain multiple k -means implementations.

-means implementations. Spark MLlib implements a distributed k -means algorithm.

-means algorithm. Torch contains an unsup package that provides k -means clustering.

package that provides -means clustering. Weka contains k-means and x-means.

Proprietary [ edit ]

See also [ edit ]

What is the coolest job on Earth?

What is my referral code?

Is it better to live without social media?

Why do people get depressed because of social media?