人脸姿态校正算法,自动红眼移除算法

作者: 编程技术  发布:2019-09-25

在一些特殊情况下,经常需要依据图像中的人脸,对图片进行倾斜矫正。

说起红眼算法,这个话题非常古老了。

例如拍照角度幅度过大之类的情况,而进行人工矫正确实很叫人头大。

百度百科上的描述:

那是不是可以有一种算法,可以根据人脸的信息对图片进行角度的修复呢?

“红眼”一般是指在人物摄影时,当闪光灯照射到人眼的时候,瞳孔放大而产生的视网膜泛红现象。

答案肯定是确认的。

由于红眼现象的程度是根据拍摄对象色素的深浅决定的,如果拍摄对象的眼睛颜色较深,红眼现象便不会特别明显。

再次例如,想要通过人脸的特征对人物的表情和情绪进行精准判断,

“红眼”也指传染性结膜炎。

那么这个时候如果能确保人脸没有发现严重倾斜,无疑对准确率判断有一定的帮助。

近些年好像没有看到摄影会出现这样的情况,毕竟科技发展迅速。

那么假如一张图片只有一个人脸,其实很好判断,通过眼睛的位置的坐标,根据两眼的直线角度,

记得最早看到红眼移除算法是在ACDSee这个看图软件的编辑功能区。

就可以计算出修正的角度。

当然,当时ACDSee也没有能力做到自动去红眼,也需要进行手工操作。

然后旋转图片到对应角度即可。

红眼移除不难,其实就是把眼睛区域的颜色修正一下。

但是如果,一张图片存在多张人脸的时候该怎么办?

但是难就难在修复之后,不要显得太过突兀,或者破坏眼睛周围的颜色 。

有两种方法:

这就有点难办了。

1.找到最大的那个人脸,以它为基准

当然其实最简单的思路,就是转色域空间处理后再转回RGB。

2.找到频次最高的人脸角度,以频次为基准

记得在2015年的时候,

当然在大多数情况,方法1是比较合理的。

曾经一度想要寻找红眼移除过度自然的算法思路,

这两个种情况就留给各位看官去实现了。

当时仅仅是好奇,想要学习之。

本人仅仅考虑一张人脸的情况,演示如何实现该功能。

直到2016年,在一个Delphi图像控件的源码里看到了一个红颜移除算法函数。

倾斜角度计算的代码如下:

把代码转写成C之后验证了一下,效果不错,过度很自然。

    float diffEyeX = right_eye_x - left_eye_x;    float diffEyeY = right_eye_y - left_eye_y;    float fAngle;    float M_PI = 3.1415926535897932384626433832795f;    if (fabs < 0.0000001f)        fAngle = 0.f;    else        fAngle = atanf(diffEyeY / diffEyeX) * 180.0f / M_PI;

貌似好像有点暴露年龄了,

如果看不明白,需要好好补一下高中数学基础。

俺也曾经是Delphi程序员来的,无比怀念Delphi7。

为了节约时间,直接复用《自动红眼移除算法 附c++完整代码》的代码。

贴上红眼算法的Delphi源码:

增加函数如下:

procedure _IERemoveRedEyes(bitmap: TIEBitmap; fSelx1, fSely1, fSelx2, fSely2: integer; fOnProgress: TIEProgressEvent; Sender: TObject);var  row, col: integer;  nrv, bluf, redq, powr, powb, powg: double;  per1: double;  px: PRGB;begin  fSelX2 := imin(fSelX2, bitmap.Width);  dec;  fSelY2 := imin(fSelY2, bitmap.Height); dec;  per1 := 100 / (fSelY2 - fSelY1 + 0.5);  for row := fSelY1 to fSelY2 do  begin    px := bitmap.Scanline[row];    for col := fSelX1 to fSelX2 do    begin      nrv := px^.g + px^.b;      if nrv < 1 then        nrv := 1;      if px^.g > 1 then        bluf := px^.b / px^.g      else        bluf := px^.b;      bluf := dMax(0.5, dMin(1.5, Sqrt;      redq := (px^.r / nrv) * bluf;      if redq > 0.7 then      begin        powr := 1.775 - (redq * 0.75 + 0.25);        if powr < 0 then          powr := 0;        powr := powr * powr;        powb := 1 - (1 - powr) / 2;        powg := 1 - (1 - powr) / 4;        with px^ do        begin          r := Round(powr * r);          b := Round(powb * b);          g := Round(powg * g);        end;      end;      inc;    end;    if assigned(fOnProgress) then      fOnProgress(Sender, trunc(per1 * (row - fSelY1 + 1)));      Application.ProcessMessages;  end;end;
void RotateBilinear(unsigned char *sourceData, int width, int height, int Channels, int RowBytes,                    unsigned char *destinationData, int newWidth, int newHeight, float angle, bool keepSize = true,                    int fillColorR = 255, int fillColorG = 255, int fillColorB = 255) {    if (sourceData == NULL || destinationData == NULL) return;    float oldXradius = (float) (width - 1) / 2;    float oldYradius = (float) (height - 1) / 2;    float newXradius = (float) (newWidth - 1) / 2;    float newYradius = (float) (newHeight - 1) / 2;    double MPI = 3.14159265358979323846;    double angleRad = -angle * MPI / 180.0;    float angleCos = (float) cos;    float angleSin = (float) sin;    int srcStride = RowBytes;    int dstOffset = newWidth * Channels - ((Channels == 1) ? newWidth : newWidth * Channels);    unsigned char fillR = fillColorR;    unsigned char fillG = fillColorG;    unsigned char fillB = fillColorB;    unsigned char *src = (unsigned char *) sourceData;    unsigned char *dst = (unsigned char *) destinationData;    int ymax = height - 1;    int xmax = width - 1;    if (Channels == 1) {        float cy = -newYradius;        for (int y = 0; y < newHeight; y++) {            float tx = angleSin * cy + oldXradius;            float ty = angleCos * cy + oldYradius;            float cx = -newXradius;            for (int x = 0; x < newWidth; x++, dst++) {                float ox = tx + angleCos * cx;                float oy = ty - angleSin * cx;                int ox1 = (int) ox;                int oy1 = (int) oy;                if ((ox1 < 0) || (oy1 < 0) || (ox1 >= width) || (oy1 >= height)) {                    *dst = fillG;                } else {                    int ox2 = (ox1 == xmax) ? ox1 : ox1 + 1;                    int oy2 = (oy1 == ymax) ? oy1 : oy1 + 1;                    float dx1 = 0;                    if ((dx1 = ox - (float) ox1) < 0)                        dx1 = 0;                    float dx2 = 1.0f - dx1;                    float dy1 = 0;                    if ((dy1 = oy - (float) oy1) < 0)                        dy1 = 0;                    float dy2 = 1.0f - dy1;                    unsigned char *p1 = src + oy1 * srcStride;                    unsigned char *p2 = src + oy2 * srcStride;                    *dst = (unsigned char) (dy2 * (dx2 * p1[ox1] + dx1 * p1[ox2]) +                                            dy1 * (dx2 * p2[ox1] + dx1 * p2[ox2]));                }                cx++;            }            cy++;            dst += dstOffset;        }    } else if (Channels == 3) {        float cy = -newYradius;        for (int y = 0; y < newHeight; y++) {            float tx = angleSin * cy + oldXradius;            float ty = angleCos * cy + oldYradius;            float cx = -newXradius;            for (int x = 0; x < newWidth; x++, dst += Channels) {                float ox = tx + angleCos * cx;                float oy = ty - angleSin * cx;                int ox1 = (int) ox;                int oy1 = (int) oy;                if ((ox1 < 0) || (oy1 < 0) || (ox1 >= width) || (oy1 >= height)) {                    dst[0] = fillR;                    dst[1] = fillG;                    dst[2] = fillB;                } else {                    int ox2 = (ox1 == xmax) ? ox1 : ox1 + 1;                    int oy2 = (oy1 == ymax) ? oy1 : oy1 + 1;                    float dx1 = 0;                    if ((dx1 = ox - (float) ox1) < 0)                        dx1 = 0;                    float dx2 = 1.0f - dx1;                    float dy1 = 0;                    if ((dy1 = oy - (float) oy1) < 0)                        dy1 = 0;                    float dy2 = 1.0f - dy1;                    unsigned char *p1 = src + oy1 * srcStride;                    unsigned char *p2 = p1;                    p1 += ox1 * Channels;                    p2 += ox2 * Channels;                    unsigned char *p3 = src + oy2 * srcStride;                    unsigned char *p4 = p3;                    p3 += ox1 * Channels;                    p4 += ox2 * Channels;                    dst[0] = (unsigned char) (                            dy2 * (dx2 * p1[0] + dx1 * p2[0]) +                            dy1 * (dx2 * p3[0] + dx1 * p4[0]));                    dst[1] = (unsigned char) (                            dy2 * (dx2 * p1[1] + dx1 * p2[1]) +                            dy1 * (dx2 * p3[1] + dx1 * p4[1]));                    dst[2] = (unsigned char) (                            dy2 * (dx2 * p1[2] + dx1 * p2[2]) +                            dy1 * (dx2 * p3[2] + dx1 * p4[2]));                }                cx++;            }            cy++;            dst += dstOffset;        }    } else if (Channels == 4) {        float cy = -newYradius;        for (int y = 0; y < newHeight; y++) {            float tx = angleSin * cy + oldXradius;            float ty = angleCos * cy + oldYradius;            float cx = -newXradius;            for (int x = 0; x < newWidth; x++, dst += Channels) {                float ox = tx + angleCos * cx;                float oy = ty - angleSin * cx;                int ox1 = (int) ox;                int oy1 = (int) oy;                if ((ox1 < 0) || (oy1 < 0) || (ox1 >= width) || (oy1 >= height)) {                    dst[0] = fillR;                    dst[1] = fillG;                    dst[2] = fillB;                    dst[3] = 255;                } else {                    int ox2 = (ox1 == xmax) ? ox1 : ox1 + 1;                    int oy2 = (oy1 == ymax) ? oy1 : oy1 + 1;                    float dx1 = 0;                    if ((dx1 = ox - (float) ox1) < 0)                        dx1 = 0;                    float dx2 = 1.0f - dx1;                    float dy1 = 0;                    if ((dy1 = oy - (float) oy1) < 0)                        dy1 = 0;                    float dy2 = 1.0f - dy1;                    unsigned char *p1 = src + oy1 * srcStride;                    unsigned char *p2 = p1;                    p1 += ox1 * Channels;                    p2 += ox2 * Channels;                    unsigned char *p3 = src + oy2 * srcStride;                    unsigned char *p4 = p3;                    p3 += ox1 * Channels;                    p4 += ox2 * Channels;                    dst[0] = (unsigned char) (                            dy2 * (dx2 * p1[0] + dx1 * p2[0]) +                            dy1 * (dx2 * p3[0] + dx1 * p4[0]));                    dst[1] = (unsigned char) (                            dy2 * (dx2 * p1[1] + dx1 * p2[1]) +                            dy1 * (dx2 * p3[1] + dx1 * p4[1]));                    dst[2] = (unsigned char) (                            dy2 * (dx2 * p1[2] + dx1 * p2[2]) +                            dy1 * (dx2 * p3[2] + dx1 * p4[2]));                    dst[3] = 255;                }                cx++;            }            cy++;            dst += dstOffset;        }    }}void facialPoseCorrection(unsigned char *inputImage, int Width, int Height, int Channels, int left_eye_x, int left_eye_y,                    int right_eye_x, int right_eye_y) {    float diffEyeX = right_eye_x - left_eye_x;    float diffEyeY = right_eye_y - left_eye_y;    float fAngle;    float M_PI = 3.1415926535897932384626433832795f;    if (fabs < 0.0000001f)        fAngle = 0.f;    else        fAngle = atanf(diffEyeY / diffEyeX) * 180.0f / M_PI;    size_t numberOfPixels = Width * Height * Channels * sizeof(unsigned char);    unsigned char *outputImage = (unsigned char *) malloc(numberOfPixels);    if (outputImage != nullptr) {        RotateBilinear(inputImage, Width, Height, Channels, Width * Channels, outputImage, Width, Height, fAngle);        memcpy(inputImage, outputImage, numberOfPixels);        free(outputImage);    }}

非常非常简单的代码。

上效果图片。

但是思路很巧妙。

原图:

不多说,各位看官自己品味一下。

图片 1

先上个效果图:

红眼修复+倾斜矫正:

图片 2

图片 3

图片 4

项目地址:

说明下本文背景前提:

人脸识别暂时采用MTCNN,示例不考虑判断是否存在红眼。

命令行参数:

人脸检测部分,详情见博文《MTCNN人脸检测 附完整C++代码》

mtcnn 模型文件路径图片路径

算法步骤:

例如: mtcnn ../models ../sample.jpg

检测人脸,对齐得到人脸五个特征点。

用cmake即可进行编译示例代码,详情见CMakeLists.txt。

算出两眼球之间的距离,

若有其他相关问题或者需求也可以邮件联系俺探讨。

估算眼球的大概大小,

邮箱地址是:
gaozhihan@vip.qq.com

(示例代码采用两眼球之间的距离的九分之一)

计算相应的半径,

按圆形修复眼球颜色即可。

完整示例代码献上:

#include "mtcnn.h"#include "browse.h"#define USE_SHELL_OPEN#ifndef  nullptr#define nullptr 0#endif#if defined#define _CRT_SECURE_NO_WARNINGS#include <windows.h> #else#include <unistd.h>#endif#define STB_IMAGE_STATIC#define STB_IMAGE_IMPLEMENTATION#include "stb_image.h"//ref:https://github.com/nothings/stb/blob/master/stb_image.h#define TJE_IMPLEMENTATION#include "tiny_jpeg.h"//ref:https://github.com/serge-rgb/TinyJPEG/blob/master/tiny_jpeg.h#include <stdint.h>#include "timing.h"char saveFile[1024];unsigned char *loadImage(const char *filename, int *Width, int *Height, int *Channels) {    return stbi_load(filename, Width, Height, Channels, 0);}void saveImage(const char *filename, int Width, int Height, int Channels, unsigned char *Output) {    memcpy(saveFile + strlen, filename, strlen);    *(saveFile + strlen + 1) = 0;    //保存为jpg    if (!tje_encode_to_file(saveFile, Width, Height, Channels, true, Output)) {        fprintf(stderr, "save JPEG fail.n");        return;    }#ifdef USE_SHELL_OPEN    browse;#endif}void splitpath(const char *path, char *drv, char *dir, char *name, char *ext) {    const char *end;    const char *p;    const char *s;    if (path[0] && path[1] == ':') {        if  {            *drv++ = *path++;            *drv++ = *path++;            *drv = '';        }    }    else if         *drv = '';    for (end = path; *end && *end != ':';)        end++;    for (p = end; p > path && *--p != '\' && *p != '/';)        if (*p == '.') {            end = p;            break;        }    if         for (s = end; (*ext = *s++);)            ext++;    for (p = end; p > path;)        if (*--p == '\' || *p == '/') {            p++;            break;        }    if  {        for (s = p; s < end;)            *name++ = *s++;        *name = '';    }    if  {        for (s = path; s < p;)            *dir++ = *s++;        *dir = '';    }}void getCurrentFilePath(const char *filePath, char *saveFile) {    char drive[_MAX_DRIVE];    char dir[_MAX_DIR];    char fname[_MAX_FNAME];    char ext[_MAX_EXT];    splitpath(filePath, drive, dir, fname, ext);    size_t n = strlen;    memcpy(saveFile, filePath, n);    char *cur_saveFile = saveFile + (n - strlen;    cur_saveFile[0] = '_';    cur_saveFile[1] = 0;}void drawPoint(unsigned char *bits, int width, int depth, int x, int y, const uint8_t *color) {    for (int i = 0; i < min(depth, 3); ++i) {        bits[(y * width + x) * depth + i] = color[i];    }}void drawLine(unsigned char *bits, int width, int depth, int startX, int startY, int endX, int endY,    const uint8_t *col) {    if (endX == startX) {        if (startY > endY) {            int a = startY;            startY = endY;            endY = a;        }        for (int y = startY; y <= endY; y++) {            drawPoint(bits, width, depth, startX, y, col);        }    }    else {        float m = 1.0f * (endY - startY) / (endX - startX);        int y = 0;        if (startX > endX) {            int a = startX;            startX = endX;            endX = a;        }        for (int x = startX; x <= endX; x++) {            y = (int)(m * (x - startX) + startY);            drawPoint(bits, width, depth, x, y, col);        }    }}void drawRectangle(unsigned char *bits, int width, int depth, int x1, int y1, int x2, int y2, const uint8_t *col) {    drawLine(bits, width, depth, x1, y1, x2, y1, col);    drawLine(bits, width, depth, x2, y1, x2, y2, col);    drawLine(bits, width, depth, x2, y2, x1, y2, col);    drawLine(bits, width, depth, x1, y2, x1, y1, col);}#ifndef MAX#define MAX  > : #endif#ifndef MIN#define MIN  > : #endifunsigned char ClampToByte(int Value) {    return ((Value | ((signed int) (255 - Value) >> 31)) & ~((signed int) Value >> 31));}int Clamp(int Value, int Min, int Max) {    if (Value < Min)        return Min;    else if (Value > Max)        return Max;    else        return Value;}void RemoveRedEyes(unsigned char *input, unsigned char *output, int width, int height, int depth, int CenterX, int CenterY,              int Radius) {    if (depth < 3) return;    if ((input == nullptr) || (output == nullptr)) return;    if ((width <= 0) || (height <= 0)) return;    int Left = Clamp(CenterX - Radius, 0, width);    int Top = Clamp(CenterY - Radius, 0, height);    int Right = Clamp(CenterX + Radius, 0, width);    int Bottom = Clamp(CenterY + Radius, 0, height);    int PowRadius = Radius * Radius;    for (int Y = Top; Y < Bottom; Y++) {        unsigned char *in_scanline = input + Y * width * depth + Left * depth;        unsigned char *out_scanline = output + Y * width * depth + Left * depth;        int OffsetY = Y - CenterY;        for (int X = Left; X < Right; X++) {            int OffsetX = X - CenterX;            int dis = OffsetX * OffsetX + OffsetY * OffsetY;            if (dis <= PowRadius) {                float bluf = 0;                int Red = in_scanline[0];                int Green = in_scanline[1];                int Blue = in_scanline[2];                int nrv = Blue + Green;                if (nrv < 1) nrv = 1;                if (Green > 1)                    bluf = (float) Blue / Green;                else                    bluf = (float) Blue;                bluf = MAX(0.5f, MIN(1.5f, sqrt;                float redq = (float) Red / nrv * bluf;                if (redq > 0.7f) {                    float powr = 1.775f - (redq * 0.75f +                                           0.25f);                    if (powr < 0) powr = 0;                    powr = powr * powr;                    float powb = 0.5f + powr * 0.5f;                    float powg = 0.75f + powr * 0.25f;                    out_scanline[0] = ClampToByte(powr * Red + 0.5f);                    out_scanline[1] = ClampToByte(powg * Green + 0.5f);                    out_scanline[2] = ClampToByte(powb * Blue + 0.5f);                }            }            in_scanline += depth;            out_scanline += depth;        }    }}int main(int argc, char **argv) {    printf("mtcnn face detectionn");    printf("blog:http://cpuimage.cnblogs.com/n");    if (argc < 2) {        printf("usage: %s  model_path image_file n ", argv[0]);        printf("eg: %s  ../models ../sample.jpg n ", argv[0]);        printf("press any key to exit. n");        getchar();        return 0;    }    const char *model_path = argv[1];    char *szfile = argv[2];    getCurrentFilePath(szfile, saveFile);    int Width = 0;    int Height = 0;    int Channels = 0;    unsigned char *inputImage = loadImage(szfile, &Width, &Height, &Channels);    if (inputImage == nullptr || Channels != 3) return -1;    ncnn::Mat ncnn_img = ncnn::Mat::from_pixels(inputImage, ncnn::Mat::PIXEL_RGB, Width, Height);    std::vector<Bbox> finalBbox;    MTCNN mtcnn(model_path);    double startTime = now();    mtcnn.detect(ncnn_img, finalBbox);    double nDetectTime = calcElapsed(startTime, now;    printf("time: %d ms.n ", (int)(nDetectTime * 1000));    int num_box = finalBbox.size();    printf("face num: %u n", num_box);    bool draw_face_feat = false;    for (int i = 0; i < num_box; i++) {        if (draw_face_feat) {            const uint8_t red[3] = {255, 0, 0};            drawRectangle(inputImage, Width, Channels, finalBbox[i].x1, finalBbox[i].y1,                          finalBbox[i].x2,                          finalBbox[i].y2, red);            const uint8_t blue[3] = {0, 0, 255};            for (int num = 0; num < 5; num++) {                drawPoint(inputImage, Width, Channels, (int) (finalBbox[i].ppoint[num] + 0.5f),                          (int) (finalBbox[i].ppoint[num + 5] + 0.5f), blue);            }        }        int left_eye_x = (int) (finalBbox[i].ppoint[0] + 0.5f);        int left_eye_y = (int) (finalBbox[i].ppoint[5] + 0.5f);        int right_eye_x = (int) (finalBbox[i].ppoint[1] + 0.5f);        int right_eye_y = (int) (finalBbox[i].ppoint[6] + 0.5f);        int dis_eye = (int) sqrtf((right_eye_x - left_eye_x) * (right_eye_x - left_eye_x) +                                  (right_eye_y - left_eye_y) * (right_eye_y - left_eye_y));        int radius = MAX(1, dis_eye / 9);        RemoveRedEyes(inputImage, inputImage, Width, Height, Channels, left_eye_x, left_eye_y, radius);        RemoveRedEyes(inputImage, inputImage, Width, Height, Channels, right_eye_x, right_eye_y, radius);    }    saveImage("_done.jpg", Width, Height, Channels, inputImage);    free(inputImage);    printf("press any key to exit. n");    getchar();    return 0;}

算法见RemoveRedEyes ,这个技巧可以用于类似的图片颜色处理。

要看人脸检测的结果,把draw_face_feat改为 true 即可。

项目地址:

参数也很简单,

mtcnn 模型文件路径图片路径

例如: mtcnn ../models ../sample.jpg

用cmake即可进行编译示例代码,详情见CMakeLists.txt。

若有其他相关问题或者需求也可以邮件联系俺探讨。

邮箱地址是:
gaozhihan@vip.qq.com

本文由贝博体育app发布于编程技术,转载请注明出处:人脸姿态校正算法,自动红眼移除算法

关键词: