What are the most interesting applications of machine learning in unexpected spaces?
Creating a movie trailer!Last year, 20th Century Fox released a trailer (watch here), claiming to be the first one created by AI/Machine Learning (IBM Watson). The movie was an AI horror thriller, Morgan.In spite of headlines like Watch: IBM's Watson Created a Super Creepy Movie Trailer All By Itself, Watson did not do it all by itself. All Watson did was choosing 10 scenes (totaling 6 minutes of footage) from the movie that should appear in the trailer, and a human film maker then edited the scenes together and added the musical overlay.Nevertheless, it’s impressive. On a high-level, here’s how Watson chose those 10 scenes:Humans first looked at the trailers of 100 horror movies and labeled each scene of each trailer as ‘eerie’, ‘frightening’, ‘tender’, etc.Watson then analyzed the audio and video of those scenes to learn what the scenes and the corresponding labels mean. For example, it could have learnt something like:Soothing music + Child smiling = Tender scene.Silence + Person crying = Sad scene.Creaking sound + Door opening slowly = Suspenseful scene.Scream + Gory face for 1 millisecond = Scary scene.Watson now knows what kind of scenes typically appear in a horror movie trailer, in terms of audio and video features.Watson was then fed the full-length movie (Morgan). It analyzed the audio and video features of all scenes in the movie, and identified 10 scenes that would be the best candidates for a trailer.That's how the 10 scenes were chosen. And according to IBM’s blog, Watson helped reduce what could be a weeks-long process to just one day.But make no mistake, this is not equivalent to machines learning creativity or even just fear. As far as most visual features are concerned, any ML system probably treats Samara coming out of the TV in The Ring and a baby crawling out of a crib as almost the same thing.