Machine Learning in Product: How ML is Improving Communication

Andrew Blumenfeld, co-founder of Telepath spoke with Eugene Joseph, founder of Overtone Labs, about the future of machine learning and the impact of machine learning on product development.

Below is the transcript of the conversation:

Eugene. Hey, how's it going? 

Andrew. Hello Eugene. I'm doing well. How are you?

Eugene. I'm good. Doing good. 

Andrew. I just wanted to chat with you, as we are chatting with a handful of other business and product leaders, about how they think about using machine learning in the work that they're doing, how they think about it adding value to their business, how they think about adding value for customers both things that they are doing now as well as things that they would like to be doing now but maybe aren't for whatever reason and you know maybe what the future brings. So maybe with that, you can just start by telling me a little bit about what you're working on now like what's your current project?

Eugene. Yeah for sure so the current project is kind of, I have a company called Overtone Labs and it was kind of founded or created to be this company where I could build and experiment on products and three different areas that I’m really interested in which are health, communication, and music, and so a fair bit of my time is spent building products in-house and so one of them is Radical which I’ll tell you more about and that's why I’m really using AI to help improve the way we communicate but I also do some work as a consultant or a contractor doing software development and kind of machine learning work for other companies

Andrew. Very cool why don't you tell me a little bit about Radical and what that's all about and how you're using AI and ML in that context.

Eugene: Yeah, for sure. So it's basically a platform where you can sign up and have live audio conversations with other users and gain insight about how you're coming across so things like, how long you've been speaking in relation to your partner, what disfluencies things like um's, likes, uh-huhs things you're saying how often you're using them when you're using them and also understand how topics in the conversation are evolving over time so maybe we start talking about politics but maybe it drips into music and maybe that turns into remote work and so kind of being able to understand how themes are emerging in a conversation and where they're starting and where they're ending and how that is kind of evolving as the conversation flows so that's one set of functionality but when two people are done talking if they both feel comfortable they can make the conversation public and that adds another layer of functionality where now you can share these conversations and you can get kind of all of the social engagement that we're used to things like likes, comments but you can also receive tips for the knowledge or entertainment that you're sharing with this conversation so that's kind of the functionality in a nutshell.

Andrew. Very cool. I’m curious how it is that you decided to focus on this as like the centerpiece this being conversations -- what about sort of the audio of conversations was either interesting to you or that you saw a technical opportunity. How did this become kind of your current passion?

Eugene. Yeah, for sure so I think that just from several earlier kinds of entrepreneurial experiments and products that you know many of which that I explored with Adam who's working with you guys now at Telepath there was this notion that -- I apologize my dogs are barking there's probably someone that came to drop off a package.

Andrew. That's no problem

Eugene. Yeah, they'll stop in a minute but the interest kind of stems from how the medium affects the content that's created and the way that you communicate and so you know in theory if you’re thoughtful enough you can create this very virtuous medium that helps people communicate more effectively and that you know lowers misunderstandings and lowers kind of a lot of the toxicity that we're seeing.

So specifically for Radical this kind of started post-election and it became pretty clear that the prevalent social media you know things like Twitter and Facebook were failing us and bringing out the worst in us I would say and audio seem to be much more conducive to constructive conversation, it's how human have communicated for thousands of years it's like arguably the most natural way of communication and with that I started building out this audio communication platform where strangers could just connect over topics they were both interested in and that evolved into what Radical is today but to answer your question from a slightly different angle I think that poor communication is - a it's a disease from like one standpoint and when you add up all of the suffering everything from broken families to countries going to war that has resulted purely from just bad communication you can kind of see it as a very serious problem and so you know that in itself is something to try and work on.

Andrew. What role is machine learning playing in all of this? You kind of have described a little bit of sort of what the user experience is as far as you know maybe getting tips and insights can you talk a little bit more about the role machine learning is playing and AI is playing and also I guess I’m curious to some degree like how centrally important you find that or how sort of ancillary that is to kind of the core offering of Radical

Eugene. Yeah, I think machine learning is increasingly going to become more important as the product evolves the way that I see it right now like I mentioned it's driving all of the conversation insights features so like I mentioned things like determining the major themes of a conversation based on the contents of the speech or determining speech disfluencies, right now this happens after the conversation is finished and it's been archived on an audio file but moving forward I’d like to offer you know all of those types of insights in real time so maybe every time you say "um" you know a little your phone vibrates or a little thing pops up on your screen to let you know but kind of going deeper into things like spoken language processing being able to tell you when you are you know starting to sound aggressive maybe tone it down or being able to tell you in real time that you're being rude if you didn't if you didn't realize that, things like that so you know I think that functionality is just going to get more and more important.

There are several other aspects of the app that I think you know are going to be similar to most businesses in terms of where machine learning could be useful and where it's already being used but one of them is just who are the people that we recommend that you talk to based on similar interests that you have that you've told us, right so some sort of matching service and what is the most relevant content that we should show you like of the public conversations what should we show you that we think you'd enjoy listening to so I think across all those verticals it's just going to become more and more important

Andrew. What challenges have you encountered so far when it has come to operationalizing machine learning in the ways that you currently are and maybe as you also think about some of those future growth areas like what are some maybe some challenges you foresee encountering when it comes to making that all a reality?

Eugene. Yeah, I would say the biggest challenge is that it just takes time so right now as a solo entrepreneur I’m doing everything from the design to the product strategy to software engineering on both the back end and the client and so now to start developing like a robust machine learning pipeline like that just it's its own world with its own set of rules its own set of best practices that people spend their whole careers on so you know the biggest challenge has just been finding the time to do it and fortunately for me that's it is a core part of the user experience so you know I do have to prioritize that but I imagine for a lot of especially startups you know there are so many other things that could get in the way of finding the time to set up those pipelines that you really see the benefit of overtime as opposed to kind of immediately getting a bunch of value right away.

Andrew. I’m curious because that is something I've heard that a lot and it's very I’ve lived it to some degree on previous work that I’ve done and it makes a lot of just logical sense to me you know that the whole notion of you know a lot of the kind of underlying principles of startup growth is, exactly what you just said, like make sure you attack the things that are right in front of you and it's how it can be highly tactical and this whole sort of lean methodology tries to turn you away from things that may not yield value for a long time and you may not know for a long time what that value is, if anything, so given all of that I guess how did you come to prioritize it you said it is a central part of the user experience and so you do prioritize it how did you come to that conclusion that that was something that needed to happen even seeing all of those obstacles as they were and that made you say well I’m going to need to tackle these anyway.

Eugene. Yeah I think in the case of those conversation insights there was no other way that I could deliver that functionality without tapping into machine learning so the decision was a little easier there but yeah that's what you said is that's always the struggle as an early stage entrepreneur where like are we investing too much in this do we need to get a little bit more feedback whether people like this well maybe this is you know to like it's complex enough that you kind of have to spend a couple of months building something out before you could even get that feedback are there ways to fake it like there are all those questions that come up but I think it was also I mean this is this is a little different than the usual kind of entrepreneurial zeitgeist but I think I also just kind of wanted this to exist so yeah you know going back to saying that this was a kind of a an experiment right I wanted I wanted to put it out there and see how people would use it and you know I have my theories for how it could help people right if you can understand we never see that data when we're speaking right like after this conversation we're like unless you run it through some analysis you're never going to see how long I was speaking and how long you were speaking and you know how we interact with each other and what words we used and how the conversation evolved and because we never see that we never get that feedback that that's something we could improve. So just you know deciding that this was a base level of functionality that I wanted to give to my users in an area where I see just a huge potential for growth and as these systems mature like a huge potential for just business opportunity as well. 

Andrew. Yeah I think at the end of the day that level of sort of commitment to a vision is obviously super powerful and ends up sometimes, as it sounds like in this case, like pushing you to places that maybe just you know as you put it like the zeitgeist or the orthodoxy wouldn't necessarily have you go and that's really neat and I also think it's just from like an end user perspective as I think about exactly what you were saying about that feedback that in the absence that that even having some sort of systematic like in this case the use of machine learning is maybe especially helpful because it's not another person, it is a machine you know, and it's not a pre-programmed sort of static rules-based algorithmic predetermined outcome where someone's just sort of normatively deciding for me this is good this is bad you should talk about this you shouldn't talk about this right if a person said that to me or they made a rule-based system that was directing me that way I think a lot of people you know might be more resistant to that and certainly like egos get in the way when you're when you're giving people feedback something is like intimate is how they speak you know and right and somehow it seems like this might be a space where the you know a lot of people talk about the places in machine learning where you want there to be a human in the loop this seems like actually really kind of nice to divorce it from some of the emotions and the other kind of baggage that humans might bring to a conversation about hey this is just an analysis of what we're seeing and these are maybe some tips and tricks to move forward. I think just even I imagine that people just even looking at it at the end of a conversation is pretty you know has quite an impact. 

Eugene. Yeah, all of my users so far really enjoy that, and they kind of eagerly refreshing the page waiting for those insights to pop up so that's been nice to see but going back to what you were saying yeah it's even within machine learning it can quickly become political and I briefly mentioned kind of auto-detecting emotions that's politically fraught right and that's kind of an area that I was more interested in maybe two years ago and I’m less interested in now just because it's really hard to even for me to say like you're starting to sound a little aggressive here well according to whom right maybe in this country or in this cultural tradition that that sounds like friendship and you can quickly get into all of the other kind of ethical you know complexities of machine learning and just managing that, but yeah as of now I’ve tried to set it up in a very neutral kind of reporting style way where I’m just telling you what happened.

Andrew. I’m curious then how do you train the models that you're using to derive these insights and to sort of offer the whatever recommendations or whatnot that it may be outputting for the user is it just based on the body of conversations that are just growing every single day, did you start with some like pre-existing repository because to your point about how there are certain things that may make sense in one culture or another or may be interpreted in one culture another I assume that a lot of that would be driven ultimately by well who's whose data is teaching this model you know what's going on.

Eugene. Yeah so right now like going into the details of the kind of language model and like the translation you know the speech-to-text model that I’m using I have two things running in parallel one of them is Mozilla created this open-source library for just speech-to-text recognition that I’ve been running and then I’ve also been kind of running that in parallel with Amazon and they have a model that they set up to do the exact same thing and it just you feed it an audio file and it gives you a transcript of all of the text.

The reason to use Amazon is that that's just a good gold standard in terms of performance and in terms of which I can kind of benchmark this in-house kind of based on open source code model that I have running as well the amazon model is better there's no two ways around that but it's also nice to have something custom that I have a lot more control over and that I can modify more specifically in that you know eventually right you can start to modify at the level of the user, right, maybe they say particular words or they say things in a particular way which would be great once the contents are kind of transcribed to the text then then you go into like another NLP layer of the pipeline where now you're starting to parse like what are the themes of the conversation and that's just based on like all of like the classic grammatical rules and the machine learning there is really just telling you what role does this word play in this sentence and kind of based on that you can start to infer themes about what's being discussed, how it's being discussed, 

Andrew. Yeah, very cool. The last question for you here is about you you've touched on this a little bit and I’d love to hear your thoughts on the future of machine learning in your business but also just generally I think you know this is obviously something you've spent a lot of time thinking about obviously something that you've prioritized for the work that you do every day, and so if you're anything like me as soon as that starts to be true you see it everywhere and you start to imagine sort of the role of it anywhere and everywhere. What do you imagine are some of the more exciting things on the horizon again for Radical and for Overtone Labs and maybe even beyond that? 

Eugene. Yeah totally, I think it's going to unlock so many opportunities to just give us more awareness of how we're coming across and improving the way we're going to communicate kind of and we're already seeing this in so many ways with products like Grammarly that are that are you know helping with our written communication and everything from like the autocomplete function in Gmail when you when you type something it just finishes it for you and so I think the general trend is that as computation becomes cheaper and as it becomes more accessible and as the tools to build these systems become more accessible to everyone you know maybe to developers at first, I think we're going to see so many more creative and humanistic applications of machine learning because right now a lot of what I feel is driving exploration in this space is the bottom line right can we can we build a model that you know is 10% faster or is you know increases the odds of detecting something correctly you know or labeling something correctly from like to 55% - 56% right because that can lead to, that can make a huge difference to your bottom line but I think just as these things become more prevalent and they get in the hands of all sorts of different people we're just going to see increasingly cooler and nicher applications of this and it's changing yeah it's changing it's going to change and it is changing so many industries.

Andrew. Very cool well thank you so much for taking the time to chat with me today I guess completely unsurprising given your line of work this was a very pleasant conversation for at least me I hope it was for youth as well like and I’m looking forward to signing up for a Radical. Yeah, I will have to we'll have to stay in touch and hear about all the cool things that you all do with machine learning and beyond in the months and years ahead 

Eugene. Likewise, I really enjoyed the conversation and am excited to hear about the latest from Telepath.

Andrew. Awesome. Well, have a great rest of your day. Thanks for taking the time later all right take care. Bye.