现在人脸对齐比较冷门, 说起来尴尬, menpo 竞赛总共才八九只队伍. 有些方法比如OpenFace CNN expert, Unconstrained Face Alignment without Face Detection 等借鉴意义不是很大. CLM 已经过时了, “without Face Detection” 有一种标题党的味道. FA 流行的做法仍然是级联式的, 不管是基于patch 还是传递上个级联的信息. CSR在过去是FA的主要方法, 现在有了和CNN相结合的CSR(RobustFEC-CNN), 我觉得这个方法挺有意思, 或许是CNN做跟踪的思路. 对于静态图的方法可借鉴 “Deep Alignment Network”, 级联间通过传递热度图和特征, 摒弃了截取Patch的方式. 精度最好的文章是采用了沙漏网络, 不过输入也很大256x256, 貌似这样的网络对于FA很有帮助.

论文简述

  1. Stacked Hourglass Network for Robust Facial Landmark Localisation

    步骤:

    1. 预测5点, 校正crop
    2. 预测19点, 校正crop
    3. 256x256 输入 Hourglass 网络
  2. RobustFEC-CNN: A High Accuracy Facial Landmark Detection System

    第一步免不了用粗精度的点, 校正截图范围. 其中FEC-CNN 来自A Fully End-to-End Cascaded CNN for Facial Landmark Detection(右图)

  3. Deep Alignment Network: A convolutional neural network for robust face alignment

    特点:

    1. 每个级联输入全脸图像 + 热度图 + 上个级联特征图, 112
    2. 上一级联输出点位热度图和特征图. 热度图计算如下; 特征图由fc1层产生.
    3. 这个过程实现 End to End, 先训练第一级联, 然后训练全部效果更好
  4. Leveraging Intra and Inter-Dataset Variations for Robust Face Alignment

    从发论文的角度看, 做法很有趣, 看下来对实际应用感觉意义不大.

    要点:

    1. 共享权重融合不同的数据集
    2. 通过镜像图片得到两组预测
    3. 用CD-Net 融合两组预测.判别网络判别两组点哪个更准, 最后选取更准的那一组. CD-Net的训练过程是:
      用原图和镜像图跑出两组点, 然后构造两组点构造标签, 接近真值的为1, 否则为0:

      然后用这样的数据训练判别网络.
  5. Delving Deep into Coarse-to-fine Framework for Facial Landmark Localization

要点:

  1. 角度估计 -> 图像转正
  2. 划分内无关和外轮廓分别回归,
  3. 划分六个部分: left eyebrow, right eyebrow, left eye, right eye, nose, and mouth, 截patch方式六个网络分别回归(竟然不进行复用模型!!) loss 下降 9.75%
  4. 用每个点临近的3, 5, 7点截取三个尺度的patch, 回归单点坐标, loss 下降 1.77%
  1. 3D-assisted Coarse-to-fine Extreme-pose Facial Landmark Detection

    卖点是:

    1. Deep Regression Feature 回归3D模型, 得到2D点的粗估计, 做两次. 第一次扰动大, 第二次用上一次的估计校正输入图
    2. 用局部特征精化预测. 可用 SDM, ESR, LBP, 文章用 LBF

    文章可借鉴的地方不是很大, 像要得到较为准确的初始位置, 远不用去拟合一个3D模型那么麻烦.

  2. Unconstrained Face Alignment without Face Detection

    要点:

    1. 用热度图和关系图做人脸定位和姿态估计
    2. 裁图一般的人脸点对齐
  3. Face Detection, Bounding Box Aggregation and Pose Estimation for Robust Facial Landmark Localisation in the Wild

    聚合多个人脸检测器(dlib, MTCNN, 作者自己训练的Faster R-CNN)结果. 这一步是为了找到Menpo比赛每张图中正确的人脸 –> 进行角度估计 –> 级联形状回归, 类似SDM这种

  4. Convolutional Experts Constrained Local Model for Facial Landmark Detection

    OpenFace 作者搞得, 同样的框架, 只是用CNN做 patch expert, 其它没什么好说的.

参考文献

  1. J. Yang, Q. Liu, and K. Zhang. Stacked Hourglass Net- work for Robust Facial Landmark Localisation. In Proceed- ings of the International Conference on Computer Vision & Pattern Recognition (CVPRW), Faces-in-the-wild Work- shop/Challenge, 2017. 3, 5, 6

  2. Z.He,J.Zhang,M.Kan,S.Shan,andX.Chen.RobustFEC- CNN: A High Accuracy Facial Landmark Detection System. In Proceedings of the International Conference on Computer Vision & Pattern Recognition (CVPRW), Faces-in-the-wild Workshop/Challenge, 2017. 3, 5, 6

  3. M. Kowalski, J. Naruniec, and T. Trzcinski. Deep Align- ment Network: A convolutional neural network for robust face alignment. In Proceedings of the International Confer- ence on Computer Vision & Pattern Recognition (CVPRW), Faces-in-the-wild Workshop/Challenge, 2017. 3, 5, 6

  1. W. Wu and S. Yang. Leveraging Intra and Inter-Dataset Variations for Robust Face Alignment. In Proceed- ings of the International Conference on Computer Vision & Pattern Recognition (CVPRW), Faces-in-the-wild Work- shop/Challenge, 2017. 3, 5, 6

  2. X. Chen, E. Zhou, J. Liu, and Y. Mo. Delving Deep into Coarse-to-fine Framework for Facial Landmark Localiza- tion. In Proceedings of the International Conference on Computer Vision & Pattern Recognition (CVPRW), Faces- in-the-wild Workshop/Challenge, 2017. 3, 5, 6

  3. S. Xiao, J. Li, Y. Chen, Z. Wang, J. Feng, Y. Shuicheng, and A. Kassim. 3D-assisted Coarse-to-fine Extreme-pose Facial Landmark Detection. In Proceedings of the International Conference on Computer Vision & Pattern Recognition (CVPRW), Faces-in-the-wild Workshop/Challenge, 2017. 3, 5, 6

  4. X.-H. Shao, J. Xing, J. Lv, C. Xiao, P. Liu, Y. Feng, C. Cheng, and F. Si. Unconstrained Face Alignment without Face Detection. In Proceedings of the International Confer- ence on Computer Vision & Pattern Recognition (CVPRW), Faces-in-the-wild Workshop/Challenge, 2017. 3, 5, 6

  5. Z.-H. Feng, J. Kittler, M. Awais, P. Huber, and X. Wu. Face Detection, Bounding Box Aggregation and Pose Estimation for Robust Facial Landmark Localisation in the Wild. In Proceedings of the International Conference on Computer Vision & Pattern Recognition (CVPRW), Faces-in-the-wild Workshop/Challenge, 2017. 3, 5, 6

  6. A. Zadeh, T. Baltrusaitis, and L.-P. Morency. Convolutional Experts Network for Facial Landmark Detection. In Proceedings of the International Conference on Computer Vision & Pattern Recognition (CVPRW), Faces-in-the-wild Workshop/Challenge, 2017. 3, 5, 6

此处应有打赏