Facial and Gesture Recognition
Following up from my post on affecting computing, my peers and I have concluded that use of OpenCV for video capture and facial recognition would be the best solution. To achieve what we need in our limited time frame and because this is what the library was designed for. In this post however, I'm only sharing a small experiment on what I originally intended to be my take on basic computer vision without OpenCV. In the end I opted for a simpler route, focusing less on detection of facial features on more on general movement. Although currently limited to a singular entity, detection of multiple entities would be easy enough
to introduce with some slight modifications.
The purpose of this experiment was to see how difficult it would be to isolate the face of a user and eventually their forehead, for observation of colour changes in their skin. I chose Unity since this is the end target SDK that data must be provided for. After some code to initialise and read pixels from the webcam, I had it detect changes in the image with variables for specifying sensitivity and tolerance levels. I chose to isolate the green channel, as it is far less noisy and seemed to produce a brighter image compared to that of red and blue. From here I had Unity draw a green box and red cross, indicating the area where movement occurred and where it was most prominent respectively.
After some tinkering of variables to match the lighting conditions of my room, I was able to detect movement of my head and limbs with reasonable accurately. Supporting multiple objects from here would be a matter of grouping chunks with neighbours found nearby, using another variable for specification of range. I say chunks since not every individual pixel is queried, currently not implemented correctly since large groups of pixels are skipped. If I were to ever continue work on this project for the sake of experimentation, having some form of background removal should be fairly trivial to implement, and would allow for capabilities somewhat similar to an EyeToy.
The purpose of this experiment was to see how difficult it would be to isolate the face of a user and eventually their forehead, for observation of colour changes in their skin. I chose Unity since this is the end target SDK that data must be provided for. After some code to initialise and read pixels from the webcam, I had it detect changes in the image with variables for specifying sensitivity and tolerance levels. I chose to isolate the green channel, as it is far less noisy and seemed to produce a brighter image compared to that of red and blue. From here I had Unity draw a green box and red cross, indicating the area where movement occurred and where it was most prominent respectively.
After some tinkering of variables to match the lighting conditions of my room, I was able to detect movement of my head and limbs with reasonable accurately. Supporting multiple objects from here would be a matter of grouping chunks with neighbours found nearby, using another variable for specification of range. I say chunks since not every individual pixel is queried, currently not implemented correctly since large groups of pixels are skipped. If I were to ever continue work on this project for the sake of experimentation, having some form of background removal should be fairly trivial to implement, and would allow for capabilities somewhat similar to an EyeToy.
Hi Tim, was it hard to connect OpenCV to Unity? How did you make it?
ReplyDelete