GENTORO

ABSTRACT: A system called GENTORO that uses a robot and a handheld projector for supporting children's storytelling activities is described. GENTORO differs from many existing systems in that children can make a robot play their own story in a physical space augmented by mixed-reality technologies. Pilot studies have been conducted to clarify the design requirements of GENTORO from both technological and practical viewpoints. A user study of GENTORO has been conducted in collaboration with elementary school children and teachers. This study indicates that GENTORO's features can enhance children's embodied participation in, and their level of engagement with, their storytelling activities, and can support children in designing and expressing creative and original stories.

Children create their stories at various times in their daily life. For example, they will often improvise a story by using puppets while playing. Making children create and tell stories is one of the teaching methods used in primary education. There has been much research into supporting such story creation and story expression activity via the use of computational media. In accordance with this research, the term "storytelling" in this paper is used to refer both to children's "story creation" and to "story expression". Storytelling support systems for children are divided into the following two categories.

Desktop-based storytelling support systems: A typical feature of systems in this category is that children create a story and make story characters play in the virtual world. Some systems allow children to interact with characters shown on a fixed monitor using conventional input devices such as mice and keyboards or novel input devices.
Physical-space-based storytelling support systems: A typical feature of systems in this category is that children play the roles of characters in their story in a physical space. They can interact with other children or artifacts in an immersive environment enhanced by mixed-reality technologies.

The GENTORO system (Figure 1) belongs to the latter category, but differs from existing systems. In GENTORO, children can make robots play characters of their story in a physical space by using a handheld projector. The features and effects of GENTORO are summarized as follows.

In GENTORO, children can express their story in an immersive environment where physical and virtual spaces are integrated.
Because children can use a robot that behaves in a physical space as a character of their story, they can express the story as if they were producing a film or a Tokusatsu (special effects movie).
The mobility of handheld projectors and robots enhances the children's embodied participation in their storytelling activities.

Figure 2 shows the system configuration for the current version of GENTORO. Children create their own story scripts and draw scenes with pen and paper or using tablet PCs. Each scene is projected through a handheld projector and a robot moves on the projected scene. Children's scene drawing (on tablet PCs or scanned when drawn on paper) is stored in the image database in Scalable Vector Graphics format. The robot path-setting module then loads each scene image and asks the children to set a path with a stylus pen on the tablet PC. The simulation module plays the story by changing scenes and moving the robot using the specified time for individual scenes.

The mobile equipment of GENTORO used by the children comprised a small lightweight LED projector (Mitsubishi LVP-PK, weight 0.5 kg) of reasonable resolution (800 x 600) and brightness, a lightweight USB camera (Logicool Qcam for Notebook Pro, weight 0.04 kg), and a mobile PC with sufficient computational power (Sony VAIO VGN-UX90PS, weight 0.52 kg). The children held the projector in their hands and carried the PC in a shoulder bag, as shown in Figure 3.

The GENTORO enables children to cooperatively manipulate a physical robot without any sensor and guide it to its destination using a handheld projector. In GENTORO, a robot is recognized by a camera carefully attached to a projector so that their optical axes become parallel and as close as possible as shown in Figure 4. With this alignment, it is possible to accurately estimate the position of the robot in the projected scene via an image captured by the camera. The robot is controlled by the mobile PC connected to the projector, to make the robot follow a path drawn on the projected image.

User studies of GENTORO with school children (age: 10 -12) indicate that it can effectively support the children in designing and expressing their creative and original stories. Robots and mobile mixed-reality technologies using handheld projectors can enhance the level of children's motivation of, engagement in, and embodied participation in their storytelling activities.

As the GENTORO project is work-in-progress and our initial targets at this moment is mainly to clarify the effects of the novel features of GENTORO and to explore its possibilities, several technical issues requiring investigation have not been addressed at this stage. The next step of the project is to improve and extend the functions of GENTORO, and to evaluate its ability to support children's creative storytelling via additional user studies.

Sugimoto, M.: A Mobile Mixed Reality Environment for Children's Storytelling using a Handheld Projector and a Robot, IEEE Transactions on Learning Technologies, No.4, Vol.3, pp.249-260 (2011).
Sugimoto, M., Ito, T., Nguyen, G.T., Inagaki, S.: GENTORO: A System for Supporting Children's Storytelling using Handheld Projectors and a Robot, In Proceedings of IDC 2009, Como, Italy pp. 214-217 (2009).
Ito, T., Nguyen, G.T., Sugimoto, M: A Storytelling Support System using Robots and Handheld Projectors, In Proceedings of IDC 2008, Evanston, IL (2008).