How to create a webcam face detector

We are going to create a JAVA program that makes a basic face dection, it will draw a box around the detected faces. To do that we will use the OpemIAMJ library. All this code is available in github.

Project creation

First of all we will create an empty project using maven

mvn -B archetype:generate -DarchetypeGroupId=org.apache.maven.archetypes -DgroupId=com.calabozo.facedetector  -DartifactId=facedetector

The structure directory that we have is this one:

facedetector/
├── pom.xml
└── src
    ├── main
    │   └── java
    │       └── com
    │           └── calabozo
    │               └── facedetecor
    │                   └── App.java
    └── test
        └── java
            └── com
                └── calabozo
                    └── facedetecor
                        └── AppTest.java

We can check that everything is in order compiling and testing the code:

cd facedetector/
mvn compile

We are going to use eclipse as our IDE so we convert this project into an eclipse folder:

mvn eclipse:eclipse

To import this project in eclipse choose: File->Import->Maven->Existing Maven Projects. Then select the folder where your project has been created.

Once we have imported our project into eclipse we can modify our pom.xml. We will need the libraries from OpemIAMJ to perform the video processing.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.calabozo.facedetector</groupId>
  <artifactId>facedetector</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>facedetector</name>
  <url>http://maven.apache.org</url>
  <properties>
    <openimaj.version>1.3.5</openimaj.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>
  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <artifactId>image-processing</artifactId>
      <groupId>org.openimaj</groupId>
      <version>${openimaj.version}</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <artifactId>core-video-capture</artifactId>
      <groupId>org.openimaj</groupId>
      <version>1.3.5</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>org.openimaj</groupId>
      <artifactId>faces</artifactId>
      <version>${openimaj.version}</version>
      <scope>compile</scope>
    </dependency> 
    <dependency>
      <groupId>org.openimaj</groupId>
      <artifactId>video-processing</artifactId>
      <version>${openimaj.version}</version>
      <scope>compile</scope>
    </dependency>
  </dependencies>
</project>

Basic face detection

Now we can start writing the JAVA code. Inside the main in the App.java class write the following:

public static void main(String[] args) throws VideoCaptureException	{
		List<Device> videoDevices = VideoCapture.getVideoDevices();
		System.out.println(videoDevices.size() + " video devices detected.");

		Video<MBFImage> video;
		video = new VideoCapture(640, 480, 24, videoDevices.get(0));

		VideoDisplay<MBFImage> display = VideoDisplay.createVideoDisplay(video);
	}

This will open a video stream from your main webcam and will display it at a resolution of 640×480 at 24fps. But there is no face detector yet in this example, to do that we can use the class HaarCascadeDetector, so we can write at the bottom of our main the following lines:

public static void main(String[] args) throws VideoCaptureException	{
	[...]
	VideoDisplay<MBFImage> display = VideoDisplay.createVideoDisplay(video);

display.addVideoListener(new VideoDisplayListener<MBFImage>() {
	FaceDetector<DetectedFace, FImage> fd = new HaarCascadeDetector(100);
	public void beforeUpdate(MBFImage frame) {
		List<DetectedFace> faces = fd.detectFaces(Transforms.calculateIntensity(frame));
		for (DetectedFace face : faces)
			frame.drawShape(face.getBounds(), RGBColour.RED);

	}
	public void afterUpdate(VideoDisplay<MBFImage> display) {}
});

	}

The HaarCascadeDetector searches for faces in boxes up to 100 pixels (what we defined in the constructor) per every frame we receive in the beforeUpdate method inside the video listener.

The call to frame.drawShape draws a red box around the detected face.

helen1

Face keypoints detection

OpenIAMJ includes the class FKEFaceDetector which not only searchs for faces but it can also give some face keypoints. It can detect eyes, nose and mouth. Obviously this will require more computer power so, depending on the computer, it can be necessary to lower the screen resolution of the frames per second.

display.addVideoListener(new VideoDisplayListener&amp;lt;MBFImage&amp;gt;() {
	FaceDetector<KEDetectedFace, FImage> fd = new FKEFaceDetector();

	public void beforeUpdate(MBFImage frame) {
		Lis<KEDetectedFace> faces = fd.detectFaces(Transforms.calculateIntensity(frame));
		for (KEDetectedFace face : faces) {
			Shape shape = face.getShape();
			frame.drawShape(shape, RGBColour.RED);
			Point2d faceOrigin = new Point2dImpl(shape.minX(), shape.minY());
			FacialKeypoint[] keypoints = face.getKeypoints();
			for (FacialKeypoint point : keypoints) {
				point.position.translate(faceOrigin);
				frame.drawPoint(point.position, RGBColour.WHITE, 10);
			}
		}
	}
	public void afterUpdate(VideoDisplay<MBFImage>display) {}
});

This is an example of a frame:
helen2

Deploying

Add the following snippet to your pom.xml inside the project labels:

  <build>
    <plugins>
      <plugin>
        <artifactId>maven-assembly-plugin</artifactId>
        <configuration>
          <archive>
            <manifest>
              <mainClass>com.calabozo.facedetector.App</mainClass>
            </manifest>
          </archive>
          <descriptorRefs>
            <descriptorRef>jar-with-dependencies</descriptorRef>
          </descriptorRefs>
        </configuration>
      </plugin>
    </plugins>
  </build>

Then you cam compile the jar with all the depencies executing inside the facedector folder:

mvn compile assembly:single

To execute the program just:

java -jar target/facedetector-1.0-SNAPSHOT-jar-with-dependencies.jar

More information can be found in the OpenIMAJ documentation: http://openimaj.org/tutorial/index.html

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s