We have developed powerful software that “drives” the animation – this software enables the rapid and realistic development of new animations and has the potential for application within existing animations or to aid the development of new materials.
The LinguaSign resources are videos containing animated stories produced using the 3D animation tool, Maya. The stories have been written and translated into English, French, Portuguese and Dutch. Each story is a sequence of clips, some of which present the action of the story, while others present the story dialogue.
The process starts with the description of the stories (Story development) in the form of a written document (Story script) with dialogue and an indication of the actions of the characters. The second step is the creation of backgrounds and characters that will be the source material for subsequent activities. Once this is done it is possible to generate the Storyboard (Storyboard creation) that is a document that incorporates the Story Script together with indications of camera setting, dialogues, characters’ positions etc.
The storyboard is the key document of the production process, once it is ready the non-signing animation production, sign choice and gesture and speech animation production processes can start.
The dialogue clips are different for each language but tell the same story using speech from native speakers combined with signing gestures. The clips are performed by the LinguaSign characters using a combination of signing avatar technology commercial software for lip sync, and real-time capture of head movements using an adapted Nintendo Wii controller.
These three animation elements for dialogue clips are combined into body movements (bone animation) and facial gestures (morph target animation) compatible with Maya, synchronised to a speech recording, providing a realism that matches the action clips. The combined action and dialogue clips for a version of a story are rendered to a movie file using commercial production tools.
Once all clips and audio files are ready, they are assembled to produce the final movie.
This technology can be applied to other computer animations to cost-effectively generate films in different languages with different signs associated with them.
If you would like any further information regarding the technology behind LinguaSign, please contact us.