<<< Previous Lecture ––––––– Next Lecture >>>
Computer Vision
Arduino is great for sensing many kinds of inputs, but sometimes you need make your installation see the world in a bit more detailed way. That’s when computer vision comes into play.
With the help of different cameras, you can detect different features from the environment. Normal RGB cameras are good when you need color tracking, infra-red cameras are nice for situations where you have projections or dark spaces and Kinect or other depth cameras are great for detecting people. See the article linked below for pros and cons of different cameras.
Articels
- Guide to Camera Types for Interactive Installations
- Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers
- Hands Up! A Visual Essay on the ‘Media Art Pose’
Artists
Tools
How can we do things like these ourselves? Let’s go through some tools that are available for us.
Processing
The built-in video library for Processing has a couple of examples for basic video tracking. Open the examples window from Processing (File–>Examples…) and take a look at the following sketches:
- BackgroundSubtraction – You define a background and the program then compares what is different in the current frame compared to the background
- BrightnessTracking – Tracks the brightest pixel
- FrameDifferencing – Similar to the first example, but instead of a pre-defined background the current frame is compared to the previous one.
You can also try installing the OpenCV library for Processing to be able to do some more sophisticated tracking (face tracking, blob tracking):
- Follow the instruction here: http://ubaa.net/shared/processing/opencv/ to install OpenCV to your computer
- Open Processing
- Select Sketch –> Import Library… –> Add Library…
- Search for OpenCV
- Install the OpenCV library
- Relaunch Processing
- Open face_detection sketch from the OpenCV examples
- You will probably get an error when you try to run it. Follow this instruction to fix the error.
See this tutorial for more details.
Other software
Processing will get you pretty far, but it’s not the fastest for analyzing video. There are many programs developed just for tracking purposes and they will send the tracking data via OSC.
Community Core Vision
CCV is a handy blob tracking software originally developed by/for DIY multitouch table developers, but it can be used for other purposes also. The software sends tracking information via a protocol called TUIO. Works nicely with infrared cameras for tracking objects with IR LEDs.
- Download CCV
- Install the Processing library (http://vialab.science.uoit.ca/smt/)
- You can also receive the tracking data in other programming environments.
reacTIVision
reacTIVision is a software that is able to track specific amoeba-like shapes called fiducials from the camera image. You can just print out the symbols and attach them to the objects you wish to track (the camera of course needs to see the symbols)
- Download reacTIVision
- You need a TUIO implementation for your programming environment to receive the data
FaceOSC
FaceOSC is a very accurate face tracking software that is able to detect one face at a time. It sends the tracking information via OSC.
openFrameworks
openFrameworks is a creative coding toolkit similar to Processing, but it runs much faster since it’s based on C++ instead of Java. It comes with some simple tracking examples, but you can do much more advanced things with additional libraries. FaceOSC and CCV are both made with openFrameworks.