Creative storytelling with subtitles: Is artificial intelligence up for the task – ETBrandEquity.com

By Jyothi NayakTo err is human but just how true is this in the case of subtitling and captioning?

Recently a friend of mine asked me why we dont use automatic subtitling tools. Little did she know how excited I was when I heard about these tools a couple of years ago! After all, wouldnt it be wonderful to get machines to do all the hard work while we humans multi-task?

Lets take a step back and use a real-life scenario to analyze this. Platforms like YouTube have for long offered automatic captions for videos, but they are notorious for delivering sentences studded with nonsensical or occasionally obscene phrases. For hearing impaired viewers, however, this is no laughing matter, as they often depend on subtitles to decipher spoken words within a video. To address this issue, social media campaigns like #NoMoreCRAPtions have emerged which focus on ditching automatic captions.

This article is all about how subtitling is becoming increasingly relevant today, why its imperative, and what role technology can play in the evolving industry landscape. A recent study in the UK showed that more than 63% of Gen Z, who are digital natives, end up using subtitles as it not only helps them watch content on-the-move, but also aids in better comprehension.

Recent experiments in India and a few other developing countries have proved that Same Language Subtitles (SLS) have improved reading literacy. SLS causes automatic, inescapable reading engagement even among weak readers, and over a period of time has a bigger impact than conventional print media. Even developed countries plan to make SLS a default option for childrens content, in order to help young viewers develop reading skills in their early years.

As the boom in the subtitling industry fuels new business opportunities, large volumes and tight deadlines are making content creators look towards AI-based solutions. Like most other industries, AI has penetrated the translation and localization space and unlocked exciting possibilities. Today, there are several AI-based solutions that not only understand spoken words and convert them to text, but also translate them to a target language.

But the million-dollar question is are these machine-generated results as good as human translation? No, not yet! While AI can assist in the overall process of subtitling, actual translation by humans is far more impactful for local audiences, as such translation is creatively generated by native speakers of that language.

AI tools, in my view, still have several limitations. When working on genres like mythology or content with considerable background noise, heavy accents and high context content (like sarcasm or humor), the use of AI tools becomes challenging and the results are hard to work with. Within text translations as well, complex sentences can result in gibberish. For example, when translating from Hindi to English, an experienced translator would translate the reference of romantic Indian duo Laila Majnu to Romeo and Juliet something a machine would be able to do only after considerable learning. Creativity plays an intrinsic part in translating content and generating impactful subtitles.

When it comes to subtitling, the context is as important as the content. While words like mom/mother can be used interchangeably, the usage of mother is more appropriate in the context of a religious mention, which the machine will not be able to decipher automatically. Similarly, there are many common idioms and culture sensitive languages (Arabic for instance) which, when translated literally, yield hilarious and sometimes offensive results! AI tools tend to struggle with unclear contexts, new slangs and specialized subjects that require a lot of research.

So, does it mean the world of subtitling will remain human-driven even with the advent of AI? It certainly will not, as machines start learning the nuances and growing intelligence. There are many areas where automation can help reduce manual effort and increase speed right away. Examples include time-code shifting, workflows for Quality Check (QC) and auto check for compliance issues (usage of restricted words etc.) which can creep in through human errors. The good news is that theres no need to follow an all-or-nothing approach. You can choose a hybrid workflow where machine transcription takes place first, and QC is performed on this output by native translators, who correct all mistakes (and dont just laugh at them!). These corrections should ideally be fed back to the machine, so that it continues learning and eventually generates better quality subtitles. It also helps to use advanced, end-to-end AI tools that not only create transcripts, but also sync these to the prescribed number of words per second/minute, as well as to the shot boundary. Such tools deliver subtitles that are far more accurate.

Another factor to consider is that since most off-the-shelf subtitling tools have several limitations, vendors who deal in large volumes can look at building their own machine learning tools that are trained with past data to fit a particular genre/style of subtitling. This can help you generate high quality results, suited to your specific needs. Alternately, you could even consider using specialized AI tools which go a step further by using the output of multiple best-in-class engines and smartly extract the best from all of these to deliver better results.

As you can see, there is a lot of potential for automating the subtitling process, its just not completely foolproof yet! For now, leveraging an optimal mix of human talent and cutting-edge technology seems to be the best answer. AI-led automation augmented with the creativity of native speakers is the best way to meet the need for speed and volume that the subtitling industry demands today. Getting this blend right is the key for delivering multi-platform, multi-language content to worldwide audiences and increasing global market share.

-The author is SVP global localization, Prime Focus Technologies. Views expressed are personal.

Continued here:
Creative storytelling with subtitles: Is artificial intelligence up for the task - ETBrandEquity.com