Mastering Data for Colorado (Guest: Suma Nallapati)
Mastering Data for Colorado (Guest: Suma Nallapati)
Suma Nallapati was appointed Secretary for Technology and Chief Information Officer for Colorado, a state cabinet-level position, by Governor Hickenlooper in 2014. Under her leadership, her team has won both statewide and national awards, while also revolutionizing digital services for the state of Colorado.
We caught Suma in her last week as State CIO before she transitions her responsibilities to the new administration and takes a new leadership position at DISH.
Ben: Welcome to the Masters of Data podcast, the podcast that brings the human to data, and I'm your host, Ben Newton. In honor of the Thanksgiving holiday, we have a special episode for you.
Ben: Who knew that the food and beverage industry was ripe for disruption, using the power of artificial intelligence, machine learning, and data? Well, Jason Cohen sure did. Jason Cohen is the founder, and CEO of Analytical Flavor Systems, a startup, focusing on bringing the power of AI, and machine learning to the world of flavors, and sensory science.
Ben: Jason started his company after studying sensory science, and computer science at Penn State, as well as founding the Tea Institute, a research group devoted to tea tasting. He's even a professional coffee, tea, and beer taster, so he knows his stuff. So, without any further ado, let's dig in.
Ben: Welcome everybody, to the Masters of Data podcast, and I am very excited to finally have Jason Cohen, the founder, and CEO of Analytical Flavor Systems on here. We've been trying to get together for a few months now, so great to have you on, Jason.
Jason: Thanks for having me.
Ben: You, and I had a conversation one late night in New York, kind of getting to know you a little bit, I learned a lot about your background, and you know, like we always do on this podcast, we like to humanize it, by understanding who the people are that we're talking to, and kinda how you got to where you were.
Ben: So, tell us little bit of your journey. How did you get into, and we'll talk more about this later, but into sensory science, and into the science of flavors. How'd you arrive with that?
Jason: Yeah, I started. I started as a professional tea taster. I went to mainland China in 2007, originally to study politics, and it turns out, blonde hair, blue eyes, bad Chinese doesn't really endear me to asking about the government, and I just kind of fell into tasting tea, and working in the tea market in Kunming, and talking with these older men, and I got really, really into it, and then went, and lived on a Makaibari tea plantation, to pick, and harvest my own tea.
Jason: And then, when I went to Penn state, I started the Tea Institute of Penn State, and Interdisciplinary Research Institute for the study, and preservation of tea, tea ceremony, and tea culture, and I started doing research on the flavor of tea, and what people were tasting, and I pretty quickly found that sensory science wasn't a predictive science.
Jason: It was only explanatory science, and you had to collect empirical results through frequentist, statistical hypothesis testing, in order to make any inferences, and I really wanted to be able to make predictions about new products, and new flavors, and new sets of flavors.
Jason: So, I moved from sensory science, to machine learning in department of computer science in IST, and I did that for about three, and a half years before starting this company.
Ben: And so, what interested you so much about flavor in particular? You just enjoyed the experience of tea tasting so much, you just wanted to kind of, go deep on that. I mean, what really fascinated you about that discipline?
Jason: A few things. So, I started in tea, and the majority of my research was in tea, but pretty quickly, my interest broadened. I started looking, when I was looking for inspiration about systems, I would work to understand what people were tasting.
Jason: First, I looked at coffee, and realized that coffee didn't have anything, and then I looked at wine, and beer, and realized that they didn't have anything. I realized that no one had anything that worked, that was predictive, and it became, you know, a part of it became a quest in itself.
Jason: Everyone eats, you know, most people really enjoy flavor, aroma, and texture, it's a universal human experience, but we don't understand it. You know, one of the biggest mysteries still, is how the physiology of smell, you know, we don't know if it's lock, and key.
Jason: We don't know if it's vibrating strings, we know that humans can tell the difference, between billions, the most recent publications said trillions of different smells, but we have no idea how it works.
Jason: And so for us, working on the other side of it, trying to quantify what someone tastes, or what someone smells, that was really difficult, because it's not something that has an easily verifiable, empirical ground truth, right?
Jason: You can't go back to the test, and say, 'Okay, well you know, true value was a three, and we predicted a two point nine', there's no way to do that in ...
Ben: Is it because it's so subjective? Because flavor is subjective? 'Cause it's hard to measure.
Jason: A lot of people think that flavor is subjective, but we look at it, is that there's large scale convergence, across certain demographics, or certain consumer cohorts, and what someone is going to perceive in a product.
Jason: And there's very obviously, there's large scale convergence in what people are going to like in products, that's why nationally successful products exist. So, we know that it's not just, you know, a personal unquantitative experience.
Jason: We know that there's general agreement about what a lemon tastes like, and there's general agreement what flavors are good, and bad, but that doesn't make it an easily verifiable, quantifiable number, and so for us, turning it into those types of numbers, in order to be able to make predictions about what people taste, and smell, that's been a really long journey, and it's been a hard, and interesting problem to work on that, that we think is really valuable.
Ben: Yeah, I think it's fascinating. So, you basically applied the things you were learning on the computer science of the spectrum, with the machine learning, and with data mining, and all those disciplines, to this area of sensory science, and flavor, and you started Analytical Flavor Systems.
Jason: Yeah, exactly, and I went the other way actually. I come from an entirely applied background. When I went into the machine learning, I was doing machine learning, specifically to model human sensory perception. So, I went backwards.
Ben: I mean, that's how you make these connections, because you a very tight focus area. So, you were seeing what could help you, you know, what disciplines you can bring to the table, to help you solve a specific problem. I think that's pretty cool, and playing into your passion.
Jason: Yeah, and so when all of those models started to work, then we started Analytical Flavor Systems together, and that was in June, 2013.
Ben: And when you actually formed the company. So, what does the company actually do? So did you guys build a system that you were using to kind of, codify this, and productize. I mean, what did you guys actually end up building?
Jason: What it's evolved into, I'll tell you what we have now, which is not what we were originally building of course, but we have an AI platform, that models human sensory perception, to predict consumer preference, and we use that to help food, and beverage companies develop new products, optimize their existing brands, and to help them enter new markets with new consumers, and understand their preferences, so that they could develop more successful, better tasting products for everyone.
Ben: Is that the thing that you called the gastro graph, is that right?
Jason: Yeah, gastro graph AI is the platform.
Ben: So, how's that going to work for these customers? You, and I talked a little bit about some of the stories about what goes into creating some of these products, and I know most of the people listening to this, have no idea about the level of effort that food companies going into creating new products.
Ben: I mean, talk a little bit about that. I mean, what does that actually look like for some company wanting to create a new product? What's the effort, and why would what you guys built actually help with that?
Jason: Yeah, billions of dollars. People can't believe sometimes, when they find out if they're outside of the food, and beverage industry, that CPG companies, consumer packaged good companies spend billions of dollars a year, in order to develop new products, and bring them to market, and just a great majority of them fail, the individual products.
Jason: It is increasingly difficult to create a competitive product, that differentiates itself with flavor, aroma, and texture, and so, companies go to great extents to figure out if their products are going to be successful.
Jason: They have professional tasters on staff, they have product formulators developing products, and testing the perception against professional tasters. If the professional tasters give them positive feedback, then it can go to a consumer panel where companies can recruit 20, or 30, or 40 different you know, plus sometimes more, consumers to taste a product, and tell them if they like it, or dislike it.
Jason: So, a preference in hedonic test, and then they can go to a market test, or they could go to a competitive test, like a 60 40 test, where you take your product that you're developing, and a competitor product, and you say, 'You know, which product do you like?', and they're doing this blind, and your product needs to win 60 40, so that you think that you have a 60, you know, 60 percent of consumers prefer your product, which gives you a better chance of success.
Jason: Most of these things, these are not predictive methods, right? You have to go through all the work upfront. You have to develop the product upfront, you have to recruit the people up front, you have to train the people up front, and let's say that you do that, and the majority of the time, it results in a failed test, right, that product is not preferred.
Jason: You have to do that all over again, right? You have to start from square one, and do that all over again, in order to try to bring a product to market. So, companies are spending billions of dollars every year, and I think that there's a better way.
Ben: If I understood right, what you guys are doing with this, is that you can basically model how people would respond to a certain set of flavors, right? Without actually having to go ask somebody to taste it, you could say that this group of people would feel this way about, you know, this, I guess chemical composition. Is that what it really comes down to?
Jason: Yes, exactly. So, what we would do, is we would take what a professional taster inside one of these CPG companies, what they taste of the product, and then just like Google translate can translate French into Italian, Italian into Chinese, we can translate what a professional taster tastes, and translate that to any target demographic.
Jason: So, if you want to develop a product that's targeted at the Northeast United States, or you wanted to develop a product for Hispanic Americans, we could model that consumer demographic, and show you how your product is going to be perceived, or show these companies how their product is going to be perceived, and who's going to like them, and how much they're going to like them, and because that's all predictive, right?
Jason: They can actually start with verifiable [inaudible 00:10:10], and we can predict an optimal flavor profile, and they can go develop that product, and then they can test it immediately against a consumer panel, and prove that our model is accurate, and that it's going to result in more successful product developments.
Ben: Where does that understanding of these different subgroups come from? Is that something that's kind of well known in the industry, or did you guys actually have to go out there, and figure that out?
Jason: No, we had to do it the hard way. So, we actually go out, and do our own large scale demographic surveys, where we recruit consumers to come in, and taste products for us, and we use that to build a translation layer, so that any new products that are developed, and any new data that comes into our system, we can then translate that, and use that translated data, to make further predictions for that consumer group, and so, we've done that around the world now.
Jason: We've done that in North America, and Europe, and Japan, in Indonesia, New Zealand, Australia, and we continue to do large scale demographics race. You know, I think one of the things that's really different about our system is that, that data continues to build on itself, right?
Jason: It's not snapshot data, it's training data for the model, and we can always go back, and engineer new features, and engineer new interesting features, and search for new patterns in the data, that makes our overall model stronger.
Ben: You know, as you're explaining this, I remember I hearing on the news a few days ago, that yeah, I believe it was Coke had bought some very niche soda product from Maine, Did you hear about that? I forget the name of the soda, but apparently only people in Maine like it, and I think about where I grew up in North Carolina, there's this product called Cheerwine, that apparently people only in the Carolinas like.
Ben: And so, it seems like that's part of what you're talking about, is that these bigger companies, they're having a hard time getting new customer bases. So, maybe the best way for them to do, is actually go out, and buy these clearly successful niche products, and understand why those are successful, is that right?
Jason: Well, you know, we see a lot of that MNA strategy, and I think that, that MNA strategy is interesting. I think sometimes, it can result in success, but you know, just as frequently as we see a successful acquisition, you know, we see unsuccessful acquisitions.
Jason: So, if you're a CPG, and you're willing to spend 20, to 30, or X revenue, forward looking revenue multiple on a breakout brand, you still need the infrastructure to manage that brand, to understand why that brand is successful, and to continue to grow that brand, right?
Jason: Otherwise, you just spent a lot of money on a product that's going to move out of a preference. A lot of products, you know, trend for a while, and then begin to decline if there's no new innovation. So, that type of MNA isn't a cure all. It can be used, if you're willing to trade cash for growth, right?
Jason: Then it can work in the short term, but I think that CPG companies really need to invest in their own innovation strategy, and their ability to create good products in house, because if not, then what are they, right? And what are they doing? They can't rely on there always being someone else to do the innovation.
Ben: No, that definitely makes sense, and you know, one day I was reading an interview that you did, and you were talking about how a lot of food today is engineered for the lowest common denominator.
Ben: So it's engineered for something that's going to clearly have the widest appeal, and I think what you were saying in your article, and I'd be definitely interested in hearing what you think about it now, is that what you guys are developing in this new approach, would allow companies to tailor products through a much tighter audience, and then get better, actually, have those products do better in that particular demographic? Did I understand that right?
Jason: Exactly, exactly. You know, it used to be that you could win a national market, or an international market with a single product, right? You had new products like Coca Cola, which is available all over the world, and you have products like Budweiser, which is available all over the world, and it used to be that, that was, you know, enough.
Jason: You could have a single driver, but that's no longer the case. The markets are more competitive, than ever before. Where Coke used to be a singular soda, and Budweiser used to be a singular beer, right? You now have thousands of products that are each a better fit for a narrower, consumer demographic, than a single product can be a fit for everyone.
Jason: And so, the larger your target demographic, the larger consumer cohort, the less optimized your product can be, and the more likely that you're going to face the innovator's dilemma, right? Where there's no one product competitive with you, but there are hundreds of products, that are competitive with you, and so it's a death by a thousand cuts.
Ben: Yeah, well, you know what it reminded me of, you tell me if this analogy make sense to you, but you know, we've moved in like, the media market from having mass marketed, lowest common denominator shows that were on, you know, a few channels on the TV, maybe a few cable channels, to the point where it seems like, you know, Netflix is basically using effectively similar technologies, to profile sections of their customer base, to target very specific movies, and TV shows that some other group over here just thinks is crap, but this group loves it, and so they can target very, very specifically to small sub group.
Ben: So, is that kind of a good analogy for what's going on here? Is that you could basically, effectively on one of these companies, if they did it right, they could effectively start doing what Netflix did for movies, for flavors, and for food.
Jason: Yeah, that's a perfect analogy, and we actually use an example like that in our presentations, where we talk about how Netflix has been successful, because it has a model of consumer preference, and it's able to continue to develop new movies, and TV shows, new content to meet changing consumer preference.
Jason: So, you know, when it comes to food, and beverage, your preferences today, are not the same as your preferences a few years ago, and they're not going to be the same as your preferences, in a few years. In order to keep up with changing consumer preference, you have to have an innovation strategy that's going to develop products, that people are going to like today, that people are going to like in a year, and the people are going to like in two years.
Jason: And in order to do that, you really have to understand how preferences are changing, where preferences are changing, how it's different between different demographic groups, and that different consumer cohorts, right? With people who with different experiences.
Jason: And so, exactly, you know, being able to tailor products to specific people, you know, that the most amazing thing about Netflix, is that not everyone likes the same types of movies, not everyone likes the same type of TV, and it used to be that you had to go for lowest common denominator movies, and TV shows, in order to build prime time space, right?
Jason: You needed to have things that were generally well liked, but generally well liked, non polarizing products, or generally well liked products, means that no one loves them, because no one dislikes them. Right?
Jason: Frequently, the most competitive products are the most polarizing, and Netflix, because it's a streaming platform, because it's always available, and there's no schedules, it can have polarizing shows, and polarizing content, that maybe 50 percent, maybe 80 percent don't like it, but that 20 percent is willing to stay for it.
Jason: We think that you can do the same thing in food, and beverage, where you can create brands, or you can create product portfolios, where each product in that portfolio is targeted at a specific cohort, or demographic that's going to love it.
Jason: Right? And it's okay if no one else loves it, because you'll have a different product for them, but traditional sensory science, and traditional product development practices can't do that, right? It drags you to some generally acceptable, low polarization product.
Ben: What's the general response been? 'Cause I mean, as you just stated, kind of like traditional way of approaching this, is to do more of these lowest common denominator.
Ben: Do you feel like you're starting to penetrate the veil for some of these, the industry to think about this, or you know, people still, 'No, I'm not ready for that'. I mean, what's the response been?
Jason: No, the response has been amazing. We are overwhelmed by the acceptance of this. You know, one of the things about CPG's is, if you ask them in confidence, just about all of them will tell you that they have a 90 percent failure rate, and 95 percent failure rate, right?
Jason: Give, or take a few percentage points, but everyone freely admits, right? That the majority of their innovations are not successful. Whether that's from concept development, to initial prototyping, to professional panels, to consumer panels, to limited [inaudible 00:18:31] response, to first three years on the market, right?
Jason: We what we see on average, is a 95 percent failure rate, and because of that, everyone is looking for new technologies. Everyone's looking for better, smarter ways, and you know, we happen to be now, when we started, we weren't, but now we're in a buzzy market for machine learning, and artificial intelligence, and everyone's looking for these types of solutions.
Jason: But you know, we started before machine learning, and AI was a buzzword. We started, back when it was a niche technology that was just gaining some traction after the AI winter.
Jason: And so, as an early mover on this technology, we have a lot more credibility now, and a lot more proof points for these companies when they say, 'Well, you know, how can you do this? Prove to us that you can do this. Prove to us that this is better, than what our in house development models can do'.
Jason: It's become increasingly easy for us to do that, and to do that quickly, and to start making a real impact with the CPG companies that we work with.
Ben: I mean, one thing that I know, I've done a lot of interviews with people talking about AI, machine learning. The thing that always comes up is, a bias, 'cause there's kind of an inherent bias sometimes, in the way data is collected, and put into the machines, you know, into the actual algorithms, and how they're predicting stuff.
Ben: But it's in some sense, I'm almost wondering if this actually could be a way to allow these food companies to actually reach some of these, you know, very particular target markets that have been largely ignored. 'Cause you know, if you think about even a country like US, don't even go outside of the US, there's so many people coming in.
Ben: There's actually a lot of subgroups already, that maybe aren't, you know, they're not really maybe getting the love from the larger food market. I mean, does that sound right to you? You think it would actually help with that, or I mean, how do you think about that problem in particular?
Jason: I do think it will help with that. I think that it will seriously help underserved markets, and you know, I even have a personal example of that. I'm Eastern European Jewish, right? Blonde hair, and blue eyes, 'cause the Vikings invaded the Ukraine in the 1400's. Someone in my family didn't have a great time.
Jason: No one develops products, right, for my demographic, for my genetic profile, right? And even a larger demographic, like, there's not a beer in the United States developed for the Chinese American Palate, and that's a group that buys beer.
Jason: So, you know, you look at that, and you say, 'Is machine learning gonna cause bias? Is it going to give people who like sweet things, more sweet things?' I don't think that, that's what's going to happen. One, because it's not the case that, if you consume more sweet things, you always want more sweet things, right?
Jason: Two, because you can, you know, just like no one starts off drinking a double imperial IPA, right? There's always going to be a progression in these products, and you can create progressions to help consumers appreciate better products, right?
Jason: In the CPG industry, you call that a 'value ladder', and help to move them to more interesting, more innovative products, and third, is because there's so many underserved demographics, there's so many underserved preferences, that this technology is a net benefit one, and two, it's not like we're going to prevent anyone from exploring, right?
Jason: If someone wants to grab a product that's different, or that's not recommended for them, that's great. They should have that experience, and maybe they'll develop new preferences based off of it.
Ben: You know, one thing that you said, kind of stood out to me too, is you know, not knowing your area nearly as deeply, obviously. Is there a sense, for how much of this is genetic, versus learned?
Ben: Just for example, like, I read a article this the other day, I think it was, it was about, you know, that you know, kids in Mexico, didn't grow up liking really hot flavors, and their families kept feeding it to them until you know, they were like, I don't know, like 11, or 12, and now suddenly like really hot flavors.
Ben: So I mean, what's the sense of like, how much of this is genetic, versus how much of this is kind of culturally learned?
Jason: So, for the most part, perception is genetic. Your perception can go through some adaptation, due to repeat exposure to certain flavors. Your perception can go through transformations, due to an increase in your experience, which increases your ability to identify, and thus, appreciate subtlety, and nuance, but for the most part, perception is nature. Whereas preference, is nurture.
Jason: Right? Individuals who grow up, say with fermented, and spicy foods, are going to result in different preferences, than someone who grows up in Beijing, or Paris, or New York. Someone who grows up with mostly seafood, is going to result in different preferences, than someone who grows up with mostly beef.
Jason: That example is less United States centric, because United States tends to be more omnivorous than many countries, but for the most part, you can attribute a majority of perception to genetics, and a majority of preference to culture, and upbringing, and past experience.
Ben: I guess that yeah, it does make sense, 'cause I can even see for myself, when I grew up in the south, and went to school up in the northeast, my preferences changed.
Ben: I ended up liking different foods, but I guess what you're saying is, that maybe I was tasting them all along, but I didn't have an appreciation for them, until I got more experienced.
Jason: Yeah. Well, you must've been in for quite a shock.
Ben: Well, there was a Thai restaurant with 13 levels of hot, outside of my grad school, and I was like, oh, I'll try this. I'm like, holy-, that was a lot harder, than I'm used to, and now it's like no big thing, but yeah, it was definitely a bit of a culture shock at first, but that's one of the reasons why I try to get my kid to eat different things now, 'cause you know, the earlier you get-, taste it I guess, the better.
Ben: This is really, really interesting, and so I guess for you, I mean, where's this kind of going now? Well, particularly, with running the company now, what's been the hardest thing so far? I mean, what's the big challenge that you guys are working through, to make this technology a success?
Jason: The biggest challenge right now, is scaling. We have a lot of different ways that we can go. We have a lot of different markets that we can enter, but the biggest problem right now, is definitely scaling.
Jason: It's, we have demand, we have successful implementations under our belt. We have repeat CPG's that we're working with, but now, it's really out, going from, you know, a six person company, to building out an executive team, to building out a client success team, and to make this the standard way of developing new, successful products.
Jason: There's a lot of course, that goes on behind the scenes in building any company, but I think we have some particular complexities, in that we mostly work with large enterprise, you know, multinational CPG's that we have, that, you know, the machine learning is core to what we do, and that, you know, we work on really tight timescales, to make product launches successful.
Ben: I guess on some level, that's a pretty normal, but extremely difficult startup problem, is now you've found your market, you're finding success, people want it, they want a lot of it, and how do you scale up from that core team? So, I think that's a good place to be.
Jason: Yeah, it's better than the alternative, but it doesn't always feel good in the moment.
Ben: You will look back on it later, and laugh. Right now, not so much.
Jason: Yeah. I don't know if I'll laugh about all the times I flew coach, but as soon as I can stop sleeping on airplanes, or start sleeping in a business class seat, then I'll smile to myself, and say it was all worth it.
Ben: Oh yeah. I don't even know if I like flying business class anymore. Flying is not what it used to be, but I hear you. As long as it's not the middle seat, that's the thing, that's good for nobody.
Ben: Well, this was super interesting, Jason. I think what you guys are doing is fascinating, and it's probably the most interesting application of artificial intelligence, machine learning I've come across so far.
Ben: So, I wish you guys all the luck. I'm excited to see where you guys go with this.
Jason: Thank you. Thank you so much for having me. This has been great.
Speaker 5: Masters of Data, is brought to you by Sumo Logic. Sumo Logic, is a cloud native, machine data analytics platform, delivering real time continuous intelligence, as a service to build, run, and secure modern applications.
Speaker 5: Sumo Logic empowers the people who power modern business. For more information, go to sumologic.com. For more on Masters of Data, go to mastersofdata.com, and subscribe, and spread the word by rating us on iTunes, or your favorite podcast app.