Say you want to minimize intra-cluster variance (that is your optimization objective), then you can formulate the problem as a dynamic programming problem: to minimize the objective function, should I cut here? or not? and you traverse the tree recursively looking for the best cuts giving you k clusters with the smallest objective function value (intra-variance or something else).ДЄwasthi P., Blum A., Sheffet O., 2012. The coarse idea of the paper solution is to extract the flat partition by cutting at different levels in the dendrogram. Some academic paper is giving a precise answer to that problem, under some separation assumptions (stability/noise resilience) on the clusters of the flat partition. In this example, you would use three segments to maximize profit. To maximize profit, keep segmenting until the marginal revenue from segmenting equals the marginal cost of segmenting. With three segments, a slightly higher response rate, and your revenues are 9000. If you segmented your customers into 2 segments, and wrote tailored messages to each segment, your response rate will be higher. Say by using one message, your revenue will be 5000. Say paying a writer to write a custom message costs 1000, two cost 2000 and so on.

Three messages tailored to three segments will be somewhere in between.

A generic message tailored to the average will have the lowest cost. This is the revenue side.ДЄ message that is tailored to each individual will have the highest cost. Having say three messages tailored to three segments will be somewhere in between. A generic message tailored to the average will have the lowest response rate. However, you can think about it from a profit perspective.ДЇor example, in marketing one uses segmentation, which is much like clustering.ДЄ message (an advertisement or letter, say) that is tailored for each individual will have the highest response rate. Look at Comparing Clusterings - An Overview from Wagner and Wagner. There are some measure that try to match clusters that have a maximum absolute or relative overlap, as was suggested to you in your preceding question. Now, if your clusters change over time, this is a bit more tricky why choosing the first cluster-solution rather than another? Do you expect that some individuals move from one cluster to another as a result of an underlying process evolving with time? Also worth to give a try is the clValid package ( described in the Journal of Statistical Software).

You will find useful resources in the CRAN Task View Cluster, including pvclust, fpc, clv, among others. The concordance with Ward hierarchical clustering gives an idea of the stability of the cluster solution (You can use matchClasses() in the e1071 package for that). I also use k-means, with several starting values, and the gap statistic ( mirror) to determine the number of clusters that minimize the within-SS. Basically, we want to know how well the original distance matrix is approximated in the cluster space, so a measure of the cophenetic correlation is also useful. silhouette plots, and some kind of numerical criteria, like DunnвЂ™s validity index, Hubert's gamma, G2/G3 coefficient, or the corrected Rand index. Several clues were given in a related question, What stop-criteria for agglomerative hierarchical clustering are used in practice? I generally use visual criteria, e.g. There is no definitive answer since cluster analysis is essentially an exploratory approach the interpretation of the resulting hierarchical structure is context-dependent and often several solutions are equally good from a theoretical point of view.

Scipy dendrogram