Unsupervised Learning: When There Are No Human Labels, Structure Comes from Assumptions
Using clustering, dimensionality reduction, density estimation, and generative modeling to understand the objective design, non-uniqueness, Pseudo-structure, and validation when there is “no standard answer”.
1No labels does not mean no objective functionDefinition
On what basis can an algorithm judge that one representation or grouping is better?
Unsupervised learning can be summarized as: when there are no human per-example answers, use a researcher-chosen objective to find candidate structures. It takes unlabeled samples and outputs groupings, low-dimensional representations, density models, or generative models, turning many records without ready-made answers into results that can be compared, observed, or used further.
These outputs correspond to different problems.Clusteringplaces similar samples together;Dimensionality Reductionuses fewer new coordinates to summarize the original multiple features, making it easier to compress, plot, or pass to downstream models;Density estimationlearns where data appears more often, used to discover rare samples, or to generate and complete samples similar to the training data. Because uses differ, one cannot list only algorithm names without specifying which kind of structure is desired.
When computing a candidate structure, training first chooses assumptions such as distance, reconstruction, or probability, then optimizes the corresponding objective, and finally validates the result using stability and usefulness. K-means keeps samples in the same group as close together as possible.PCA is a dimensionality reduction method, it tries to preserve the directions of greatest variation in the data. Autoencoders reconstruct the compressed data,Reconstruction error is the difference between the reconstructed result and the original data, and the smaller it is, the less information is lost after compression.Likelihood indicates how plausible a set of model parameters makes the already observed data appear., and density models choose parameters that give the entire batch of data higher likelihood.
After optimization is complete, the numerical value must be interpreted back within the objective just chosen: a better objective value only means it fits this set of assumptions better; it does not prove that the algorithm discovered the only true structure. With a different distance, representation, or use, another structure may also be equally reasonable; this is exactly why stability and external validation cannot be omitted.
2Four types of tasks answer different questionsTask Map
Why are grouping, compression, and generation all called unsupervised?
These four types of tasks describe four different results that unlabeled data can produce: clustering looks for groups, dimensionality reduction replaces the data with fewer coordinates, density estimation answers how common a location is, and generative modeling tries to produce new samples. This division is used to answer “what do we actually want to get from unlabeled data,” rather than mixing all methods without human labels into the same task. The common input is a collection of samples, and the outputs are clusters or membership probabilities, low-dimensional coordinates, location density, and new samples that can be generated.
| Task | How the result is obtained | How to read the output | Main boundaries |
|---|---|---|---|
| Clustering | When computing the objective, clustering is first optimized for within-group similarity, local density, or graph cuts. | Clusters or membership probabilities represent candidate groupings under the current similarity assumptions. | The number of clusters and semantics are not unique. |
| Dimensionality Reduction | Preserves selected information according to variance, neighborhood, or reconstruction objectives. | Low-dimensional coordinates are used for compression and visualization, and their meaning depends on the relationships preserved. | Projection may lose or distort part of the structure. |
| Density Estimation | Fits the sample distribution through objectives such as maximum likelihood. | p(x) describes how concentrated a location is under the training distribution | Common does not mean high quality, and rare does not mean wrong. |
| Generative Modeling | Learns how to reproduce the distribution through likelihood, denoising, or adversarial objectives. | New samples represent the model's learned approximation of the training distribution. | It can also reproduce biases in the training data. |
Results should be interpreted according to the original question: clustering should be tied back to the purpose of grouping, dimensionality reduction should verify which relationships are preserved, and density and generative results should be considered in terms of the distribution. A beautiful two-dimensional plot cannot replace the value of grouping, and high density should not be directly understood as high quality.
The same data can have multiple reasonable structures at the same time, for example grouping by consumption scale, category preference, or activity cycle. The objective function determines which structure the algorithm prioritizes, and the final choice still depends on the use case and subsequent validation.
3Scale is an undeclared weightStep-by-step calculation
First clarify who is being compared and why, then look at why spending amount overwhelms purchase count.
Task scenario:Suppose we are going to use distance to group users with similar behavior. The smaller the distance, the more similar two users are temporarily treated as. Here we compare only two features:x₁ is monthly purchase count,x₂ is monthly spending amount (yuan). This section restates the data to be used, so readers do not need to look back at the problem statement.
| User | Monthly purchase count x₁ | Monthly spending amount x₂ (yuan) |
|---|---|---|
| A | 1 | 100 |
| B | 2 | 110 |
| C | 8 | 900 |
| D | 9 | 920 |
First define the calculation rule:Two-dimensional Euclidean distance squares the differences in the two features separately, adds them, and takes the square root. For user u=(u₁,u₂) and v=(v₁,v₂):
Substitute A and B:purchase counts differ by 2−1=1, amount differs by 110−100=10. So the distance is 1²+10²=101≈10.05. In the sum of squares, the count contributes 1²=1, the amount contributes 10²=100; although the two users also differ in purchase count, about 99% of the original distance already comes from the amount term.
Now substitute B and C:counts differ by 8−2=6, amounts differ by 900−110=790, so the distance is 6²+790²≈790.02. This does not prove that amount is more important in business; it is just that the numerical range of 'yuan' is far larger than that of 'times', and the calculation gives amount a higher weight without declaring it.
How can we compare the two dimensions according to their respective typical fluctuations?For each feature, we can apply z-score standardization:
Here i indicates which user,j indicates which feature;μⱼ and σⱼ are respectively the j feature's mean and standard deviation among the four users. After standardization, the value indicates 'how many standard deviations away from this feature's mean', and count and amount are no longer compared directly in 'times' and 'yuan'.
| Distance | Original units | After z-score (approx.) | Interpretation |
|---|---|---|---|
| A to B | 10.05 | 0.284 | Both users are in a low-frequency, low-spending region, with very small differences. |
| B to C | 790.02 | 2.595 | Both relative deviations in count and amount contribute to the distance. |
Standardization gives purchase count a visible influence again, but it also implicitly assumes 'each dimension is roughly equally weighted according to its own variance', which is not inherently correct. If amount indeed represents a higher business cost, equal weighting would instead lose value judgment; in this case, explicit business weights should be added, with reasons and sensitivity analysis recorded.
| Treatment | What mainly determines the distance | Explanation |
|---|---|---|
| Original units | the feature with the largest numerical range | This example implicitly gives amount high weight |
| z-score | deviation of each dimension relative to its own fluctuation | approximately equal weights by variance of each dimension |
| Business weighting | explicitly defined cost or value | requires recording the weight rationale and validating robustness |
This numerical example shows how scale silently changes similarity: the inputs are users' purchase count and amount, and the output is the distance under original units, standardization, or business weighting. The calculation first takes differences per dimension, then squares, sums, and takes the square root; standardization first converts each per-dimension difference into units relative to its own fluctuation. That the distance is mainly determined by amount only shows its numerical range is larger; both the original scale and z-score carry weight assumptions and cannot automatically replace business judgment.
4Original figure: The same data can be carved into different structures by different objectives.Visualization
Is the algorithm "discovering clusters," or is it executing a set of geometric assumptions?
Scroll horizontally to view the full diagram on small screens.
This figure is a representation sensitivity check used to identify whether candidate structure depends on feature scale. It takes the same set of points and two scaling schemes as input and outputs two geometric layouts; first change each dimension's contribution to distance, then observe how groups or neighbors change accordingly. A clear change in structure indicates that the algorithm is executing different geometric assumptions, not that one plot is necessarily wrong; two-dimensional projections and a small number of points provide only clues, and external use and repeated experiments are still needed.
5What dimensionality reduction preserves is determined by the optimization objectiveCompression
First understand "compressing many features into a few coordinates," then compare what PCA and t-SNE/UMAP each promise to preserve.
What is dimensionality reduction?A user may originally be described by many features such as purchase count, monetary amount, active days, and number of categories. Dimensionality reduction means using fewer new coordinates to summarize these features. It necessarily chooses "which differences are worth preserving"; different methods use different criteria, so the resulting two-dimensional plots cannot substitute for one another.
First consider compressing two dimensions into one:Draw each user as a point on a plane. If "people with high purchase counts also tend to spend more," these points will roughly arrange along a diagonal line. PCA chooses this principal direction and uses a single number to represent each point's position along the diagonal; the original two features are then compressed into a single "overall activity level" coordinate.
Using x to denote the feature vector of a particular user,μ to denote the average position of all users,w to denote one direction that we choose. We require that w has length 1, so that the scale of the new coordinate reflects only the data and is not arbitrarily amplified. A user's one-dimensional coordinate along this direction is:
The "transpose" symbol here T only means taking the direction w and the offset x−μ as an inner product, giving how far along that direction one has traveled. If the one-dimensional coordinate is placed back into the original space, an approximate position can be obtained x̂=μ+zw. The difference between the original point and the approximate position is the information lost through compression; PCA chooses the direction that minimizes the total loss across all users.
Generalizing from one direction to multiple directions:When the original data has many features and you want to preserve two or more new coordinates, place several directions side by side into the matrix W. Therefore, W is not some new data that suddenly appears, but rather a collection of "the several compression directions we are looking for." The formal expression below states the same objective:
| Symbol or term | What it means here |
|---|---|
W | The several projection directions to be found; each column is a direction |
WᵀW=I | Each direction has length 1 and the directions are perpendicular to one another, avoiding recording the same direction twice |
Σ(covariance matrix) | Summarizes how features vary together; for example, when the count rises, whether the monetary amount also tends to rise |
WᵀΣW | How much variation each new coordinate still retains after the data are projected onto these directions |
Tr(trace) | Taking the sum of the diagonal entries of the matrix; here, that means adding up the variation retained by each new coordinate. |
max | Among all qualified directions, select the set that preserves the maximum total variation. |
Here, "variance" can first be read as "how spread out the data are." PCA preserves the directions with the greatest spread; equivalently, under a squared-error criterion, it makes the reconstructed data after compression as close as possible to the original data. It is good for summarizing overall trends and can approximately reconstruct the original features, but "largest variation" does not guarantee "most important for the current business": an irrelevant feature with large numerical fluctuations may be preserved first.
t-SNE and UMAP are doing a different thing.They usually first determine which neighbors each point has in the original high-dimensional space, then try to place these neighbors still close together in the two-dimensional plot. The so-called "local neighborhood" is simply the several closest points around a point; the so-called "high-dimensional" merely means the original data has many features—nothing mysterious.
| Method | What it mainly wants to preserve | How to read the two-dimensional plot | What not to readily infer |
|---|---|---|---|
| PCA | Linear directions with large overall variation | Directions, relative positions, and retained variance have clear meanings | Cannot guarantee preserving low-variance but important signals |
| t-SNE | Points that are very close still stay close as much as possible | Suitable for observing local neighbors and candidate small groups | The distance between different "islands" and the size of islands usually cannot be directly compared |
| UMAP | Local adjacency relationships, and it also tries to preserve some larger-scale structure | Suitable for exploring neighborhoods and continuous variation | Two-dimensional distances still do not equal the exact distances in the original space, and results are affected by parameters and randomness |
The appearance of several "islands" in the plot only shows that this method placed the points this way under the current parameters, random seed, and representation; it does not by itself prove that the data contain several natural categories. To judge whether stable groups really exist, one should go back to the original feature space to inspect distances and neighbors, redraw under different samples and parameters, and validate with external knowledge or downstream tasks.
Dimensionality reduction is a representation method that converts multidimensional features into a small number of new coordinates: input the multidimensional features of each sample, output the small number of new coordinates and an optional approximate reconstruction. In computation, PCA first centers the data, then finds orthogonal directions that preserve the maximum total variance and projects onto them; t-SNE and UMAP place more emphasis on keeping original-space neighbors close on the plot. PCA coordinates indicate a sample's position along the projection directions, whereas islands in a nonlinear plot are mainly neighborhood cues; any method loses information, and no method can declare natural categories solely from two-dimensional distances.
6Density Estimation First Learns “Where Things Are Common,” Then Serves Specific TasksProbability
What does density estimation actually output, and why can it help detect anomalies, impute data, or generate new samples?
First, the task:Give the model many unlabeled samples, for example a large number of users’ “purchase count—spending amount” records; density estimation needs to learn a distribution map that shows which regions have concentrated data and which regions rarely appear.Probability density expresses how concentrated the data is near a given location, and can be used to compare which regions are more common; it does not mean that a particular continuous numerical point itself has some amount of probability.
| After learning “where things are common” | How to use it | Still need to note |
|---|---|---|
| Anomaly screening | Send new records that fall in low-density regions for review | Rare does not equal error or fraud |
| Generate samples | Draw new samples similar to the training data from near high-density structures | Will replicate biases in the training distribution |
| Missing value imputation | Choose a more reasonable missing value under known feature conditions | After the distribution changes, the old model may become invalid |
How does the model learn this map?Use xᵢ to denote the ith training sample,n denotes the total number of training samples,pθ(xᵢ) denotes the probability density that the model with parameter θ gives to this sample. Parameter θ controls the shape of the map. Training compares different parameters and looks for a set of θ*.
Log-likelihood is simply taking the logarithm of each sample’s density first, then adding the results together, and use it to compare which set of parameters better explains the whole batch of data. Taking logarithms turns multiplication of many densities into addition, making computation more stable while not changing the ranking of parameter quality:
For example, most users fall into the “low frequency, low spending” or “high frequency, high spending” regions; a sudden record of “extremely low frequency, extremely high spending” may receive a lower density and thus enter the manual review queue. But this only indicates that it is rare: new business customers, holiday orders, or entry errors can all produce the same result.
Density estimation turns “where things are common” into a queryable distribution model, used for anomaly review, generation, or missing value imputation. The input is unlabeled samples, and the output is a model that computes density for new locations or draws samples; training first computes density for the samples, then aggregates log-likelihood and adjusts parameters. A lower density indicates that a record is rare under the training distribution, not that it is an error, fraud, or low value; continuous density also depends on units, model family, and distribution stability, and values across different specifications cannot be directly compared.
7Self-supervised learning and unsupervised learning are related by inclusion yet differ in training form.Disambiguation
No one labels masked tokens—so why do we say there is an 'answer'?
The difference between self-supervised and traditional unsupervised learning lies in how the supervisory signal is generated. The input is the original sample; after masking, cropping, or pairing, it forms an automatic task, and the output is the hidden content or a consistent representation. The answer comes from the sample itself, not from manual item-by-item annotation, so even without human labels a clear training objective can still be obtained.
Taking masked tokens as an example, the system first hides part of a sentence, passes the unmasked context to the model, and then uses the original tokens to check the predictions. That is, training first constructs targets from the data, then computes a loss as in an ordinary prediction task and updates parameters based on the error; for two crops of an image, the representations of the same object can be required to be close to each other. In this way, 'how inputs become tasks and where answers come from' can both be traced through one training process.
A lower prediction loss means the model is better at completing this automatic task, but it does not guarantee that the representation is suitable for all downstream tasks. For example, a representation that is good at recovering local words is not necessarily the best for determining the topic of a long text; it should still be validated separately on practical tasks such as retrieval, classification, or generation.
| Paradigm | Automatic objective | How results are evaluated |
|---|---|---|
| Self-supervised prediction | Yes, recoverable from data | Can directly compute held-out loss, but the representation is still not unique |
| Clustering | Only a global geometric objective | Granularity and semantics are not unique; external validation is required. |
Broadly, self-supervised learning is often classified under unsupervised representation learning; in the narrower training form, it resembles a prediction task with a clear answer. Whether self-supervised learning is classified as unsupervised is a matter of taxonomy; what truly affects method choice is the source of the signal and the evaluation approach: prediction tasks can compute loss on held-out data, while cluster semantics must be validated with external knowledge and use cases.
8Internal metrics first check "does it look like separate groups?", then external evidence determines "is it useful?"Evaluation
When there are no standard labels, how do you compare two clustering results? What exactly does the silhouette coefficient measure?
First, clarify the roles:Density estimation is a learning task that outputs a distribution model;Internal metrics are evaluation scores computed without using human-provided correct answers, using only input data and algorithm results. It helps us make preliminary comparisons among several candidate results, for example comparing 2 groups versus 3 groups, or before versus after standardization, but it does not itself produce new groupings.
The silhouette coefficient specifically evaluates clustering results, used to determine whether a sample is closer to its own group rather than to other groups. For the current sample,a is its average distance to other points in the same group,b is its average distance to the nearest other group,s is this sample's silhouette coefficient;max(a,b) takes the larger of the two:
| Numerical example | Calculation | How to interpret |
|---|---|---|
a=2, b=8 | s=(8−2)/8=0.75 | Close to its own group and far from other groups; the current assignment is relatively clear |
a=5, b=4 | s=(4−5)/5=−0.20 | Instead, closer to another group; it may be misassigned or on the boundary |
For a single sample, s lies between −1 and 1; averaging over all samples allows comparison of several candidate clusterings. Close to 1 usually indicates within-group compactness and between-group separation; close to 0 indicates boundary location; less than 0 indicates it may be more like another group. However, a high score only proves that the geometric separation is relatively clear under the current features and distance, not that these groups have business meaning.
Stability checks answer a different question:After changing the random seed, resampling a subset of samples, changing the time window, or slightly perturbing features, are the groups still roughly the same? It is used to judge whether the current structure is merely a chance result. Finally, external or downstream validation is also needed: ask experts to interpret the groups, check with a small amount of known labels, or check whether the grouping actually improves retrieval and decision-making.
| Evidence layer | Specific purpose | Cannot prove alone |
|---|---|---|
| Internal metrics | Whether the result conforms to the chosen distance and objective | Whether the grouping has business meaning |
| Stability | Whether the result depends on randomness or chance samples | Whether a stable grouping is worth using |
| External/downstream | Whether it matches expert knowledge and improves real tasks | All future data remain valid |
The unlabeled evaluation chain is a combination of evidence composed of internal metrics, stability checks, and external or downstream validation, solving the problem of "how to compare candidate clusterings when there are no standard labels." It takes as input data, distance, groupings, and results from multiple reruns, and outputs three layers of evaluation evidence: first compute the silhouette coefficient to examine the current geometry, then perturb the data and parameters to check reproducibility, and finally verify expert semantics or task benefits. A high silhouette coefficient only indicates that within-group distances are small and between-group distances are large; stability only indicates that the structure does not readily disappear under perturbations—neither alone can prove that the grouping is worth using.
9Pseudo-structure often comes from batches, devices, and missing-data mechanismsFailure boundaries
Could the two groups found by an algorithm be just two instruments?
If two clusters happen to correspond to two instruments, do not rush to give them business names. Pseudo-structure is candidate structure caused by collection sources or processing workflows but misread as target semantics; collection dates, regions, devices, file formats, missing-value imputation, and crawler sources may all be easier for the algorithm to separate than the differences you actually care about.
To investigate, put source clues and candidate groupings together: input metadata such as cluster assignment, device, time, region, and missingness patterns; output the predictive power of confounding variables and stratified comparison results. First use probes to determine whether the source can predict the cluster, then perform resampling or counterfactual replacement; for example, have the two devices contribute similar numbers of samples, or replace background metadata, and then check whether the original grouping still appears.
If the source variable has strong explanatory power, and the clusters disappear after balancing the source, this indicates that the current structure may be reproducing the collection process. But probe correlation does not mean that all causal mechanisms have been found: the device may also be associated with time, region, or population at the same time, and we still need to compare layer by layer and retain uncertainty.
Naming clusters reifies statistical clumps, especially in contexts such as people, healthcare, and credit. Sensitive attributes or proxy features may cause differential treatment; once grouping affects individuals, high-impact uses must also have a legal basis and provide fairness evaluation, human review, and appeal channels.
10Moving from exploration to production requires versioning the entire structure discovery processWorkflow
If the model, features, or data change, can old cluster numbers still be used?
A versioned workflow turns a single exploration into a deployable object that can be reproduced, compared, and rolled back. It takes frozen data, features, objectives, random seeds, and validation evidence as inputs, and outputs a versioned model, cluster mapping, and monitoring rules; in this way, results after retraining have a basis for comparison with previous versions and can be rolled back in case of anomalies.
- Specify the problem to be solved, why manual labels are not used, and what actions the results will support.
- Freeze the data snapshot, feature definitions, missing-data handling, distance metric, and random seeds so that candidate approaches can be reproduced.
- Compare multiple reasonable objectives, and record which structural assumption each approach changes.
- Use stability and external value to select candidate versions, not just high internal scores or nice-looking plots.
- Save cluster matching rules, model versions, and baseline distributions; after retraining, re-establish the correspondence between old and new clusters.
- If the results drive actions, first conduct controlled experiments and risk reviews, then continuously monitor drift after deployment.
In the selection phase, first record definitions and compare candidate approaches, then select based on stability and external value; this record lets the team know why a version was adopted and distinguishes changes in data from changes in algorithm settings.
Cluster numbers may be permuted, split, or merged after retraining, which indicates that the structure version has changed. During migration, combine centroid distance, sample overlap, and semantic rules; do not treat old cluster numbers as permanent identities, nor assume the meaning is unchanged just because the numbers are the same.
When the input distribution drifts, feature processing changes, or the purpose of use changes, the previous validation conclusions no longer automatically hold even if the old model can still run; you should re-examine stability, external value, and action risk, and roll back to the previous version if necessary.
11Common Misconceptions and Learning PathMisconceptions and dependencies
Without labels, it is even more important to clarify your assumptions.
| Misconception | More accurate understanding |
|---|---|
| Unsupervised learning has no loss function | The objective is defined by distance, density, or reconstruction assumptions. |
| Algorithms will discover natural categories | Structure depends on representation, scale, and purpose. |
| A separated two-dimensional plot proves clusters exist in high dimensions. | Projection distorts distance and density. |
| High internal metrics have business value. | Still requires stability and downstream evidence. |
| Cluster numbers can be reused over the long term. | After retraining, they may be permuted, split, or merged. |
| Level | Dependencies and extensions |
|---|---|
| Prerequisites | Probability, distance, feature scaling |
| Core of this page | Objective non-uniqueness, stability, external validation |
| Methods | Clustering, dimensionality reduction, generative models, anomaly detection |
| Extensions | Self-supervised learning, causal confounding, data governance |
12Connect the causal chainSynthesis
How does this concept connect all the way from the problem to verifiable practice?
- Clarify purpose and unlabeled constraints
- Choose representation, scale, and structural assumptions
- Optimize grouping/compression/density objectives
- Compare multiple objectives and random seeds
- Investigate confounding and perform external validation
- Version results and monitor action risks
13Misconceptions and Self-TestSelf-Test
Can you explain its mechanism, boundaries, and validation methods without memorizing terminology?
- Why is unsupervised learning not without a goal?
- What is the approximate original Euclidean distance from A to B?
- What does standardization solve, and what does it introduce?
- What does density estimation learn? Why can't low-density records be directly judged as anomalies or fraud?
- For a sample, the average distance to points in the same cluster is a=2, and the average distance to points in the nearest other cluster is b=8. What is the silhouette coefficient? Does it prove that the grouping has business value?
- Why can't two-dimensional visualization prove clusters?
- Without ground truth, what three layers of evidence are needed at minimum?
- Assume that "Unsupervised Learning: When There Are No Human Labels, Structure Comes from Assumptions" performs normally on offline examples, but core results decline after launch. How would you locate the issue according to input, internal transformation, output feedback, and applicable boundaries?
Reference Answers
- It still optimizes objectives chosen by the researcher, such as distance, density, reconstruction, or probability.
- 101≈10.05.
- It prevents the numerical scale from unintentionally dominating, but it is equivalent to giving roughly equal weight to the variance of each dimension.
- It learns where data is more concentrated, which can be used for anomaly screening, generation, and missing value imputation; low density only indicates rarity, and business changes, holidays, new customers, or entry errors can all cause rarity.
- s=(8−2)/8=0.75, indicating that under the current distance this point is closer to its own cluster; it cannot prove that the cluster has business semantics, and stability and external/downstream validation are still needed.
- Nonlinear projection may distort global distances, areas, and densities.
- Internal metrics, stability, and external knowledge or downstream value.
- First, save the same failed sample and environment, and confirm that inputs, permissions, and preconditions have not drifted; then record key intermediate states and check whether the mechanism completed the transformation as described on this page; next, compare the raw output with independent metrics and manual final verification; finally, retest with boundary examples and controlled experiments. Only by locating the first link that deviates from expectations can you decide whether to modify the data, mechanism, evaluation, or usage boundaries.
- The Elements of Statistical Learning: clustering, density, and unsupervised learning
- Deep Learning — Representation Learning: representation learning objectives
- Visualizing Data using t-SNE: local neighborhood visualization
- How to Use t-SNE Effectively: projection misinterpretation boundaries