Learning to Cluster Faces

This repo provides an official implementation for [1, 2] and a re-implementation of [3].

Paper

1. Learning to Cluster Faces on an Affinity Graph, CVPR 2019 (Oral) [Project Page]
2. Learning to Cluster Faces via Confidence and Connectivity Estimation, CVPR 2020 [Project Page]
3. Linkage-based Face Clustering via Graph Convolution Network, CVPR 2019

Requirements

• Python >= 3.6
• PyTorch >= 0.4.0
• faiss
• mmcv

Setup and get data

Install dependencies

conda install faiss-gpu -c pytorch
pip install -r requirements.txt

Datasets

Please refer to DATASET.md for data preparation.

Model zoo

Pretrained models are available in the model zoo.

Run

0. Fetch code & Create soft link

git clone git@github.com:yl-1993/learn-to-cluster.git
cd learn-to-cluster
ln -s xxx/data data

1. Run algorithms

Follow the instructions in dsgcn, vegcn and lgcn to run algorithms.

Results on part1_test (584K)

Method	Precision	Recall	F-score
Chinese Whispers (k=80, th=0.6, iters=20)	55.49	52.46	53.93
Approx Rank Order (k=80, th=0)	99.77	7.2	13.42
MiniBatchKmeans (ncluster=5000, bs=100)	45.48	80.98	58.25
KNN DBSCAN (k=80, th=0.7, eps=0.25, min=1)	95.25	52.79	67.93
FastHAC (dist=0.72, single)	92.07	57.28	70.63
DaskSpectral (ncluster=8573, affinity='rbf')	78.75	66.59	72.16
CDP (single model, th=0.7)	80.19	70.47	75.02
L-GCN (k_at_hop=[200, 10], active_conn=10, step=0.6, maxsz=300)	74.38	83.51	78.68
GCN-D (2 prpsls)	95.41	67.77	79.25
GCN-D (5 prpsls)	94.62	72.59	82.15
GCN-D (8 prpsls)	94.23	79.69	86.35
GCN-D (20 prplss)	94.54	81.62	87.61
GCN-D + GCN-S (2 prpsls)	99.07	67.22	80.1
GCN-D + GCN-S (5 prpsls)	98.84	72.01	83.31
GCN-D + GCN-S (8 prpsls)	97.93	78.98	87.44
GCN-D + GCN-S (20 prpsls)	97.91	80.86	88.57
GCN-V	92.45	82.42	87.14
GCN-V + GCN-E	92.56	83.74	87.93

Note that the prpsls in above table indicate the number of parameters for generating proposals, rather than the actual number of proposals. For example, 2 prpsls generates 34578 proposals and 20 prpsls generates 283552 proposals.

Benchmarks (5.21M)

1, 3, 5, 7, 9 denotes different scales of clustering. Details can be found in Face Clustering Benchmarks.

Pairwise F-score	1	3	5	7	9
CDP (single model, th=0.7)	75.02	70.75	69.51	68.62	68.06
LGCN	78.68	75.83	74.29	73.7	72.99
GCN-D (2 prpsls)	79.25	75.72	73.90	72.62	71.63
GCN-D (5 prpsls)	82.15	77.71	75.5	73.99	72.89
GCN-D (8 prpsls)	86.35	82.41	80.32	78.98	77.87
GCN-D (20 prpsls)	87.61	83.76	81.62	80.33	79.21
GCN-V	87.14	83.49	81.51	79.97	78.77
GCN-V + GCN-E	87.93	84.04	82.1	80.45	79.3

BCubed F-score	1	3	5	7	9
CDP (single model, th=0.7)	78.7	75.82	74.58	73.62	72.92
LGCN	84.37	81.61	80.11	79.33	78.6
GCN-D (2 prpsls)	78.89	76.05	74.65	73.57	72.77
GCN-D (5 prpsls)	82.56	78.33	76.39	75.02	74.04
GCN-D (8 prpsls)	86.73	83.01	81.1	79.84	78.86
GCN-D (20 prpsls)	87.76	83.99	82	80.72	79.71
GCN-V	85.81	82.63	81.05	79.92	79.08
GCN-V + GCN-E	86.09	82.84	81.24	80.09	79.25

NMI	1	3	5	7	9
CDP (single model, th=0.7)	94.69	94.62	94.63	94.62	94.61
LGCN	96.12	95.78	95.63	95.57	95.49
GCN-D (2 prpsls)	94.68	94.66	94.63	94.59	94.55
GCN-D (5 prpsls)	95.64	95.19	95.03	94.91	94.83
GCN-D (8 prpsls)	96.75	96.29	96.08	95.95	95.85
GCN-D (20 prpsls)	97.04	96.55	96.33	96.18	96.07
GCN-V	96.37	96.01	95.83	95.69	95.6
GCN-V + GCN-E	96.41	96.03	95.85	95.71	95.62

Results on YouTube-Faces

Method	Pairwise F-score	BCubed F-score	NMI
Chinese Whispers (k=160, th=0.75, iters=20)	72.9	70.55	93.25
Approx Rank Order (k=200, th=0)	76.45	75.45	94.34
Kmeans (ncluster=1436)	67.86	75.77	93.99
KNN DBSCAN (k=160, th=0., eps=0.3, min=1)	91.35	89.34	97.52
FastHAC (dist=0.72, single)	93.07	87.98	97.19
GCN-D (4 prpsls)	94.44	91.33	97.97

Results on DeepFashion

Method	Pairwise F-score	BCubed F-score	NMI
Chinese Whispers (k=5, th=0.7, iters=20)	31.22	53.25	89.8
Approx Rank Order (k=10, th=0)	25.04	52.77	88.71
Kmeans (ncluster=3991)	32.02	53.3	88.91
KNN DBSCAN (k=4, th=0., eps=0.1, min=2)	25.07	53.23	90.75
FastHAC (dist=0.4, single)	22.54	48.77	90.44
Meanshift (bandwidth=0.5)	31.61	56.73	89.29
Spectral (ncluster=3991, affinity='rbf')	29.6	47.12	86.95
DaskSpectral (ncluster=3991, affinity='rbf')	24.25	44.11	86.21
CDP (single model, k=2, th=0.5, maxsz=200)	28.28	57.83	90.93
L-GCN (k_at_hop=[5, 5], active_conn=5, step=0.5, maxsz=50)	30.7	60.13	90.67
GCN-D (2 prpsls)	29.14	59.09	89.48
GCN-D (8 prpsls)	32.52	57.52	89.54
GCN-D (20 prpsls)	33.25	56.83	89.36
GCN-V	33.59	59.41	90.88
GCN-V + GCN-E	38.47	60.06	90.5

Face Recognition

For training face recognition and feature extraction, you may use any frameworks below, including but not limited to:

https://github.com/yl-1993/hfsoftmax

https://github.com/XiaohangZhan/face_recognition_framework

Citation

Please cite the following paper if you use this repository in your reseach.

@inproceedings{yang2019learning,
  title={Learning to Cluster Faces on an Affinity Graph},
  author={Yang, Lei and Zhan, Xiaohang and Chen, Dapeng and Yan, Junjie and Loy, Chen Change and Lin, Dahua},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2019}
}
@inproceedings{yang2020learning,
  title={Learning to Cluster Faces via Confidence and Connectivity Estimation},
  author={Yang, Lei and Chen, Dapeng and Zhan, Xiaohang and Zhao, Rui and Loy, Chen Change and Lin, Dahua},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year={2020}
}