Digital comics

Researchers at IIT Jodhpur Create Framework for Creating Digital Comic to Video Conversion Software (C2VNet)

Digital humanities researchers (DH) at the Indian Institute of Technology in Jodhpur are developing a framework for creating software Comic-to-Video Network (C2VNet) to convert born comics to digital or digitized to video.

The increasing digitization has increased the need to digitize the cultural heritage of our country. This framework revolves around the creation of an audio-video file storybook.

Primary objective

DH at IIT Jodhpur focuses on and contributes to a set of approaches (ideas and methods), rather than different approaches, which emphasize the preservation, reconstruction, transmission and interpretation of human records of historical and contemporary way.

This, in a major way, answers epistemological questions on the production of knowledge on the generation of digital data from material objects, and on rethinking the existing processes of knowledge production.

The methods and methodologies for creating the desired multimedia content have developed thanks to technological advancements.

One such example is “automatic image synthesis”, which has received a lot of research attention. On the other hand, the synthesis of audio-video scenes, such as that based on images of documents, remains difficult and little studied.

This area of ​​HR lacks a sustained analysis of multimodality in automatic content synthesis and its growing impact on digital scholarship in the humanities. C2VNet is a step towards bridging this gap.

How? ‘Or’ What C2VNet works

The C2VNet panel-by-panel in a comic book and ultimately produces a full-length video (with audio) of a digitized or digital-born storybook.

The objective was to design and develop software which takes as input a comic strip of digital or digitized origin and produces an audiovisual animated film.

With the software, Researchers at IIT Jodhpur came up with a data set titled “IMCDB: Digitized Indian Comics Storyboo Indian Mythological Comics Datasetk in the English language.

This contains full annotations for the panels, bitmask of the text bubble and text files for each speech bubble and narration box in a panel and plans to make the dataset accessible to the public.

Dr Chiranjoy Chattopadhyay, Assistant Professor, Department of Computer Science and Engineering, IIT Jodhpur, said as the panel extraction model C2VNet has two internal networks to support video creation.

CPENet developed by the team offers an accuracy of over 97%, and the SBSNet speech bubble segmentation model gives an accuracy of 98% with fewer parameters.

“C2VNet is the first step towards the great future of automatic comic book multimedia creation to bring new comic book reading experiences”, Chiranjoy Chattopadhyay added.

This is a one-of-a-kind study to discuss the automation of audiovisual content creation from images of digitized documents.

In the future, the team is working on improving the software so that these multimedia books become more immersive and engaging for the target audience.

Usually this kind of work takes more time and effort, but with this software it can be done quickly and in a more interactive way.

Read: The mapping of teachers by subject for schools in top priority, according to Andhra CM

Read: Joint agenda for JEE Advanced 2023 can reduce burden but increase competition