In this exclusive interview with CrazyEngineers, Jithendra Vepa, Chief Scientist at Observe.AI tells us about his journey into the world of machine learning and AI and how their Voice AI platform in unlocking sentiment insights to help support teams improve agent performance. 

Backed by Scale Venture Partners, Nexus Venture Partners, Y Combinator, and Emergent Ventures, Observe.AI's headquarters is located in San Francisco with an office in Bangalore.

Read on to hear directly from Observe.ai's Chief Scientist Mr. Jithendra below -

CE: Tell us about your growing up years and family background.

Jithendra: I grew up in Visakhapatnam, a coastal City on the Bay of Bengal, my father was a govt. employee, and mother, a homemaker. We moved a few places around Visakhapatnam before settling down there. I come from a middle class family and we are four siblings. Like all the middle class families in that generation, my parents stressed on the importance of education, which fuelled my path towards engineering.

Both my graduation and masters degrees are in Electrical Engineering. I studied Signal processing and Machine learning during my Masters studies and applied some of these techniques to hard-core Electrical Engg for my thesis. From there, my journey started in the direction of machine learning and eventually into the realm of Voice AI.

CE: How did the Institute of Science (IISc) happen? What part of your journey in IT do you remember the most?

Jithendra: Though my dream was to study at one of the IITs, I got an admission for Masters in Indian Institute of Science (IISc) and I accepted. This decision definitely is a turning point in my career and laid a strong foundation for my Speech and NLP career. IISc is the place where for the first time I got exposed to Signal processing and Machine learning. Instead of opting for a software career through campus placements, I chose to stay back in the campus and  took a research intern position to work on Text-to-Speech technologies.  

This experience greatly helped to secure PhD admission in Univ. of Edinburgh, not only with full scholarship but also part-time position at my advisor’s start-up.

CE: How did you land a job with Samsung? 

Jithendra: I got an interview call from Samsung R&D for a Speech technology lead role while I was working on Medical Signal processing at Philips Research. Given my extensive experience in Speech and NLP technologies during my PhD and Postdoctoral experience at EPFL, Switzerland,  I took this opportunity to come back to my native domain.

Initially at Samsung, I worked on research projects on Text-to-Speech, Speech enhancement and, then started building a strong speech recognition team to build ASR systems for SVoice and later for Bixby.  

CE: When did you realise it was time to take a leap with Observe.AI? How did you plan and make the shift?

Jithendra: I met Swapnil, CEO and co-founder of Observe.AI, for a casual discussion on a Sunday afternoon onMay 6th, 2018 just after lunch at a coffee place in a Hotel. It was a great meeting where he walked me through his journey until then and the problems they were planning to solve. I was very impressed with his passion and the vision of building Voice AI for contact centre, and my thoughts echoed well with his on how to  build the team in India.  

Till then, I had never considered that I would move out of Samsung, as I was doing very well and had built a great team. After working in consumer electronics for almost a decade,  I truly felt this was a great opportunity to make a real impact in call centre space with sentiment analysis speech technologies that had yet to be introduced in the industry.

CE: What was the initial idea behind the ML engine behind Observe.AI?

Jithendra: I am fascinated by speech signals. My initial week or so at Observe.AI was spent listening to several calls from our initial customers. What was most fascinating was that these were completely different in quality from what I was used to listening to before, and had more nuances like prosody/tone, pauses, and disfluencies. Essentially, the customer calls being made across these various contact centres consisted of a very wide range of accents, verbiage, tone, word use, which posed an interesting opportunity to develop better accuracies across so many different types of calls.

These conversational characteristics made me think of analysing audio directly and extracting acoustic cues and using them along with textual features for improving the accuracies of downstream tasks like Sentiment detection and Silence-type identifications.

CE: How does Observe.AI’s ML engine fit into the current landscape of speech analytics/sentiment analysis? What makes it unique?

Jithendra: Over the last two years or so, we have across customer calls that have only increased in the breadth of variety in emotional patterns, vocabulary, and speaking styles. Though initially these varying characteristics of data challenged our ML engineers, they helped us make our ML models much more robust and capable of drawing valuable and nuanced insights from millions of calls.

What part about your career so far do you enjoy the most? What part do you find most challenging?

Jithendra: I have really enjoyed working with young engineers (fresh out of IITs) to build state-of-the-art ASR systems because discussions with fresh minds have helped me to seed many unique ideas that I wouldn’t have thought of. At the same time, coming up with challenging tasks to engage their sharp minds and manage them is definitely not easy!

CE: What is one piece of advice/life lesson you have for young aspiring ML scientists?

Jithendra: The machine learning space is very dynamic and evolving with new techniques and tools emerging each day.  To become a best Machine learning scientist, one of the important things is to keep up-to-date with the latest research and tools. 

Fall in love with the domain, not with the tools and techniques, as they will always be evolving.

Another major trend I have seen among young engineers is they directly jump to the latest work in Deep learning. Yet, it is important to develop a foundational knowledge consisting of disciplines such as Linear Algebra, Probability Theory and traditional ML, which will not only help in understanding deep learning concepts better, but also in solving real world problems. 

CE: What are some of your favourite ML/DL books?

Jithendra: Three classic books from “Hoel, Port and Stone”, books by Chris Bishop, 'Neural Networks for Pattern Recognition' and 'Pattern Recognition and Machine Learning', Finally “Deep learning” book from Ian Goodfellow, Yoshua Bengio and Aaron Courville.