Hardware configuration

1. Projector + usb web camera
2. Projector + usb web camera + motion sensor
3. DLP projector synchronized with FireWire camera + motion sensor
4. DLP projector synchronized and colocated with FireWire camera

1st version

Devices:

In the very first systems, we used the simple pro-cam setup. Attaching a camera rigidly to a projector allows a compact mobile device whose geometric transformation between projector coordinates and camera coordinates can be considered as roughly constant on a planar surface. This setup is the simplest one, suits for a beginner but is difficult to do geometric registration precisely without referencing features on a projection surface.

2nd version

Devices:

Attaching a small motion sensor to the pro-cam device, pro-cam geometric calibration can be performed in real time without any feature points on a surface. Offline calibration is required to find relationship between sensor values and pro-cam geometries. However, this calibration needs to be performed only once and re-calibration is not required unless there are changes in relative positions and orientations among the three devices. This setup is capable with planar or slant surfaces only.

3rd version

Devices:

For robust interactive techniques, we moved from a small mobile projector (0.6 kg, 400 ANSI lumen) to a bigger but brighter one (1.1 kg, 1400 ANSI lumen). Usb web camera was replaced with a FireWire high-speed camera that supporting an external trigger. Projector is synchronized with the camera using a VGA splitter to tap the v-sync signal (60 Hz) coming from the computer.

With the synchronized pro-cam device, projection and visual sensing (for real-time scene recognition) can be performed simultaneously without interfering each other. The motion sensor is still used for real-time geometric pro-cam calibration in this configuration. Therefore, no fiducial is required on a surface.

4th version

Devices:

Because ipProjector implies use in an unknown environment, a planar projection surface cannot be guaranteed and the 3D motion sensor (as shown in the 3rd version) may fail to geometrically calibrate the projector and camera. To solve the problem of an unknown surface, we coaxialized the projector and camera using a plate beam splitter while the two devices remain synchronized as in the previous configuration.

With the colocating design, the camera sees exactly what the projector is projecting and there is no parallax between the two field-of-views. Furthermore, shapes of the projected image (as seen by the colocated camera) are barely affected by geometries of the projection surface. Consequently, a precise pro-cam geometric calibration can be achieved and no real-time calibration is required. In other words, this colocating design allows pro-cam geometric calibration that is independent to the projection surface.

(1): Simple pro-cam colocation. (2): Snapshot of the projection surface. (3): Snapshot of the surrounding environment. (4): Image captued by the colocated camera.

Our prototype is still an open system. Therefore, the colocated camera sees reflection of light from the surrounding environment as well. This effect is not preferable but can be solved easily by adding a black curtain (or other light-absorbing material) to the prototype. As a result, no ghost reflection appears in the captured images any more.

Captured images of the colocated camera without and with the curtain.