How Stitch Fix uses Data to Personalize Fashion (Guest: Brad Klingenberg)

Ben Newton: Welcome to the Masters of Data podcast. The podcast that brings the human to data. And I'm your host, Ben Newton. Data scientists are using machine learning and artificial intelligence to upend entire industries, and fashion is no exception. Our guest this episode is Brad Klingenberg, the VP of algorithms at Stitch Fix.
Ben Newton: If you aren't one of the few people left that hasn't actually heard of Stitch Fix, Stitch Fix is an online personal styling service that both recommends additions to their clients' wardrobes, but actually commits to those recommendations by physically delivering that inventory to their clients. Brad and his team use statistics machine learning and human in the loop algorithms to deliver an amazing experience for their customers, optimize inventory and make great fashion recommendations for their customers. This human in the loop part means that they've actually cracked the code for using NextGen machine learning to super charge their network of thousands of personal stylists to work faster and deliver great results at the same time.
Ben Newton: So without any further ado, let's dig in.
Ben Newton: Welcome everybody to the Masters of Data podcast and I'm your host, Ben Newton. And I'm very excited about my guest today, Brad Klingenberg. He's the VP of Algorithms, which sounds very fancy, at Stitch Fix, and I'm very excited to talk to you today Brad. Welcome.
Brad K.: Well thank you Ben. It's a pleasure to be here.
Ben Newton: I think Stitch Fix is one of the first companies that my wife was actually excited I was talking to somebody from there. Some of the companies I talk to, she's like, "I don't care."
Brad K.: Well, that's great to hear and it's fun as we've grown to have more and more people recognize the brand and company.
Ben Newton: Yeah, absolutely. You guys are doing fantastic things. We're going to talk a little bit more later about what you actually do and what Stitch Fix actually does, but we always do on Masters of Data, we want to humanize you. Hear more about the person that is Brad, talk to me a little more about how you got the way you are. How'd you end up in data science? Was there some sort of childhood trauma or, you know do you-
Brad K.: No! Through a lot of happy accidents. I studied applied mathematics as an undergraduate and found problems with data and problems about the real world the most compelling and engaging That led me to study statistics in graduate school and it happened to be a very good time to be studying statistics and data. I went to graduate school anticipating that I'd become an academic, and a researcher. I spent a summer at Google and up the private sector bug and then, seeing the impact that data and algorithms could have on an operating business, really that tangible impact got me really excited and from there a kind of, few other internships and, part of which was working part-time at Netflix, while I was in graduate school, where I met Eric Colson who founded the data science team here. Few years later, joined Stitch Fix.
Ben Newton: Right, you've actually had some pretty interesting experience. Google and Netflix, you really got to see some of the cutting edge data science, I would expect, you got exposed to some really interesting stuff.
Brad K.: Yeah, absolutely! I was even kind of slow to take to the label of data science, thinking to myself as an applied statistician forever, and I think data science is interesting because it is such an umbrella term and can be organized so differently at different companies and the approach is far from uniform across the industry.
Ben Newton: Yeah. Well, actually let me ask you, 'cause it is funny, I remember I interviewed another guy that said most of his time A.I. is just statistics so ... it forms your experience. How do you define data science now that this is really your field?
Brad K.: I think it can be a tricky thing to define. There's a great characterization from, I think Josh Wills, who said that "data scientists are better at statistics and most engineers are better at engineering then most statisticians," and [inaudible 00:03:50] a blend of enough computational sophistications to really be able to leverage and issue warning and do to things at scale, but then also just a very principled approach to modeling and thinking about uncertainty. Part of the fun though is it is a big umbrella and there's a lot of different activities that can be called data science and the different companies.
Ben Newton: Yeah, no that actually makes a lot of sense and it's, I think of all the people I've met in data science, they come from a pretty wide array of background. I've met physicists and mathematicians and statisticians or I guess that speaks to the fact that it's kind of something you gravitate towards. So, what's the story of how you ended up at Stitch Fix? How did that happen?
Brad K.: I was working part-time at Netflix with the data science organization there as graduate student. One of the folks I got to meet was Eric Colson, who at the time was the VP at Netflix, running the data science team and working on recommendations among other things. And got to know the team there, had a great experience, went back to focus on school and it was a couple years later, Eric reached out and let me know that he left Netflix for a little start-up. And I knew his involvement signaled something interesting was happening. Though, I do remember my wife laughing out loud when I shared the prospect of me contributing to the fashion company in some way. (laughing) But me and Eric came out to visit the team and met Katrina and the leadership team, but even at that very early stage, it was put together and it was a very easy decision to join.
Ben Newton: That's cool. When I was looking more into your story, it seems like a really interesting story how Stitch Fix came about. Tell us a little bit more about what's the background. What is Stitch Fix actually trying to do and in particular, how you and your team fitting into that?
Brad K.: Absolutely, so Stitch Fix is an online, personal styling service. We're a retailer of mostly apparel for women's, men's, and most recently for kids. And it really is a personal styling service in a sense that it is a push model of retail. So the way it works is someone will sign up for the service and tell us about their preferences, how they like things to fit, and their style preferences and we'll take that information and then send things to the clients so the key distinction from traditional retail is actually having Stitch Fix acting as a personal stylist choosing the things to send to you, rather than you picking them yourself. This is in a sense, a very literal bet on the recommendation. Rather than just rendering something on a webpage, actually putting it in a box and paying for shipping and sending to a client.
Brad K.: And for that to work, we use a combination of algorithmic recommendations and data science alongside expert human curation. So one of the most interesting things about Stitch Fix is this blend of art and science and human and machine and in particular, the way it works for choosing items for clients is that we take all the information we have about preferences our clients have shared with us, things we know about our inventory, and critically, feedback the clients have shared with us. And we still it down into a set of algorithmic recommendations but each one of those is actually curated by human stylists. So every time you receive a shipment from Stitch Fix, it's the result of this human-machine hybrid, but also every single item we send to clients has been picked by a stylist.
Ben Newton: I did find when I was reading up a little bit more how the process works, I did find it really interesting how you guys have managed to find a really interesting blend of human interaction with the algorithm. 'Cause I think when I first started reading, I'm like okay, "this is an artificial intelligence stylist," but it's really not, it's a more complex process that makes sure the human stays part of the end-to-end process. Do you feel like that's something that really sets you guys apart or?
Brad K.: Absolutely, it shows up in other areas of our business. We've talked about stylists working with algorithms. We also have data scientists focused on [inaudible 00:07:34] and how much to buy and even designing new clothes. All of that is done in close partnerships with expert merchants and designers. The most universal theme here is probably humans and machines working together. And certainly our experience has been in multiple settings. You get a lot more from the two combined than you would from either alone.
Ben Newton: It really makes sense. It's interesting one of the themes we have on this podcast has been undeveloped over time was bringing human to data and I think that's one of the- When I was really reading this I was really excited to hear you talk about this because there's a sense when most people hear artificial intelligence they hear data science, they're always thinking of it in ways that like, "okay, this machine is replacing something." And in some senses, of course that is right but you really came back to what I'm hoping is going to be the case, so it's more about extending and complementing humans and making them more effective and helping them scale.
Brad K.: Absolutely, I think probably one of the most frequent questions I get at speaking events and conferences is, "is the goal just to automate all of this and to take the human out of the loop?" Certainly it's not. I mean as someone who has worked on this problem for many years, I think the role that human judgment and just holistic understanding of the client plays is really complementary to many of the things you can do algorithmically. And so just as you're saying, I think over time if you think of there being like a boundary between things like be done best algorithmically and things that humans are better at. That's something that will evolve over time, but the spirit is really using algorithms to let humans do things better than could alone rather than automating people away.
Ben Newton: [inaudible 00:09:16] I use sometimes use now is Ironman. You know it's like the suit that makes Ironman better at what he does, right?
Brad K.: Yep.
Ben Newton: It's not how nine thousand throwing ya out the airline.
Ben Newton: One thing before we dig in a little bit more on that, there was also something that you said a couple different places which really hit me is that, in particular Stitch Fix is in the middle of what I- There seems like a bit sea of change going on in the industry right now. There's really something changing ‘cause if you look at what's happened for so long in the general industry of retail how did Wal-Mart win and then Amazon win and initially, right, it was logistics and price. You know it's the same customer experience but really it's logistics and price. Like how can I get more stuff to you more cheaply and give you a better price? And basic get my service to more people and you talked about a couple different places that you needed to be able to compete on something else and you talked about personalization and skill. Talk a little bit about that. How do you- ‘cause that really resonated with me, how do you think out that?
Brad K.: Yeah absolutely, so I mean as you say, the bar for personalization is rising in retail and likely many other industries as well. There's intense competition for logistics and providing retail at low prices. But if that's not the surface you want to differentiate on, if that's not the value proposition you offer to clients, then it needs to be something else.
Brad K.: At Stitch Fix, for different clients we have different value propositions but it's really about the kind of surprise and delight of having something you love picked for you and the convenience of not having to scroll through a website and you apply a bunch of filters to try to find what you want. But actually just having it come to you and loving it and getting to try it on at home. There's many different ways that clients experience Stitch Fix and I think really the common denominator is though personalization. And what enables us to do that is both the way that we use data and science and in combination with human curation. And it's also enabled by our business model in a particular way.
Brad K.: One of the interesting things that sets us Stitch Fix apart from traditional retail is the push nature of the business. Us getting to choose what we send to clients. That makes the business even more different from traditional retail in obvious ways but also some subtle ways. One of them is that there are feedback loops that are available to us that aren't available to most traditional retail models. If we go into a store and try on a pair of pants and you know it doesn't fit quite right or you really don't like the style, nobody really finds out about that. Nobody gets to the feedback that like- Fit a little differently or slightly different color. You mean, even if you buy it, you don't typically, as the business you might see that something's selling but you really don't know why.
Brad K.: With Stitch Fix we really close this feedback loop by having clients tell us implicitly through what they choose to keep and what they choose to return, but also explicitly through structured feedback, you know, "this is too small or I love the style." Even free form text which is really fun, just because people are so excited to share with us and you know infusive in the feedback they share. And that feedback loop is really an important part of what enables the personalization because when you share that you love something or is a little too big for you or too small, we learn about you and can better serve you over time.
Brad K.: That's what excites clients to share with us, but we also learn about the preferences of other people and about our inventory. For example, if we send a blouse to a thousand people and 900 say that it's too small, we learn something about the blouse and we can use that not only recommend it more effectively to clients in the future. In many cases we can actually, through our own exclusive brands, just change the blouse or figure out what we should buy for market that would better serve our clients. It's really kinda just this flywheel of feedback that empowers a lot of the personalization. An important part of that is having or the incentive of our clients in wanting to share with us, we really strongly aligned with our incentive as a business.
Ben Newton: Yeah.
Brad K.: Trusting us to be good stewards of everything they share with us and wanting us deliver on improving their experience over time as we get to know them.
Ben Newton: That's really cool. I don't know if you picked up on it, but it feels like a collaborative process with your customers, that they feel like they're a part of it. Which is very different-
Brad K.: Absolutely.
Ben Newton: Than anything else I've seen. Even with retailers that have really good customer service, you have that aspect of [inaudible 00:13:41] take care of you. But it's different to feel like you're actually part of the process. I think that's pretty fascinating. How quickly are you guys able to respond or do you want to respond when you're getting this feedback?
Brad K.: Yes, I mean in principle we wanna use all the information we have all the time to make better decisions about stylings to making choices on what to send to clients but also in managing our inventory and thinking about our engagement strategies with clients. Of course, many of these decisions happen over many different time scales, so if you have a fix and provide some feedback, we can use that more or less immediately in understanding your preferences and thinking about what to send you in the future.
Brad K.: That feedback can also be quite important to understand how different items in the inventory fit into the portfolio of styles that we carry and whether we should maybe buy more of something or less of something. Particularly, on the inventory side, there's a lot of decisions that happen in longer time scales. It's a general rule, try to use all the information that's available to us as fast as we can, but certainly some of these feedback cycles play out over longer periods of time.
Ben Newton: Yeah, that makes sense. I'm assuming you're balancing the idea of the hyper personalization where you're really, it's that one person versus you're associating with some sort of, I guess the word would be cluster, but like some group of people that have similar tastes.
Brad K.: Absolutely, yeah one thing that I like to say a lot is that we should treat every client as their own segment. More traditional approaches to retail that don't have necessarily have the data or the data science capabilities that we're lucky to have, have to work with averages. If you're not just targeting the model client, you might have a small handful of different clusters or segments that you try to build for.
Brad K.: At Stitch Fix, I think that one thing that does differentiate us is, the fact that we do know our clients to a person and get feedback from every client. This really enables us to think about not just an average but actually a whole client population. Likewise, we have that feedback on every item in our inventory. It's really this one to one personalization opportunity. It's a product of both taking a data science approach and an algorithmic approach that enables that to scale, but also just fundamentally having the data. A lot of that comes from the nature of our business model.
Ben Newton: You know, one thing I think about when I see this- I mean you- What I'm seeing you talk about, I'm sure there's layers of sophistication beneath this, but you guys are pretty sophisticated about how are you using this data and how you're incorporating human feedback. I'm assuming that didn't happen day one, so has this been a processes over years of this continuing to refine the process and learning how to balance things? I'm assuming it is.
Brad K.: Oh absolutely. It's a journey, not a destination. (laughing) Continuing to get better and more sophisticated and more ambitious over time. If you look back, Eric Colson who is the data scientist here among the first problems he worked on was the first version of the recommendation algorithm that we use to help style things for clients. When I joined Stitch Fix, which is almost six years ago in June, that was the problem that I worked on. Overtime, we started to work on different areas of the business. Today, I think, actually one of the things that is striking about the team, the algorithms team at Stitch Fix is really how broadly engaged it is with virtually every aspect of the business from inventory to designing new things to how we all keep marketing spend and strategies for engaging with clients. Of course, some of the core problems related, "what do we send to clients and how do we pick the best stylist that should style a client?"
Brad K.: It's certainly been a journey, I think we overtime, continued to take on bigger and harder problems.
Ben Newton: One thing you made me think about is that it sounds like the data science approach was a core part of Stitch Fix' being able nail it early on. Did your finder, Katrina Lake, did she actually come from that background? Did she already have like an appreciation for that? It sounds like this was almost from the beginning pretty much, right?
Brad K.: Absolutely, I think much to her credit, I think Katrina very early on had a vision for the role that data and data science can play in the company. And her background is in economics and she started the company as an MBA student at Harvard. One of the many remarkable things about Stitch Fix is how early she assembled a very heavy hitting executive team that included Eric Colson.
Ben Newton: Netflix!
Brad K.: Like Smith from Wal-Mart and many others and I think she saw this opportunity very early and- From the very early stages of the company so she saw the opportunity for building the company around data. As the team has grown, we've really been able to bring that to life. But a lot of credit is due to Katrina for having that vision initially.
Ben Newton: That's cool and I mean it sounds like part of that was what lead to you guys' success is that you just have such a variety of inputs and different people coming in from different directions to where they build a new innovative way. It's not just a bunch of people just, "let's repeat," you know rinse and repeat- You're really coming out from a different direction which is pretty awesome.
Brad K.: I have to say one of the most fun aspects of actually working here is the intellectual diversity that we have. Getting to work with stylists and merchants and engineers and financial analysts and of course, data scientists. It's really a broad spectrum of talent that comes together to make the company work.
Ben Newton: I wanna run something past you that I've been thinking about and what you just said reminded me about is that I- Particularly, the US economy, but over all in the world changes, the problems we're tryin to solve change, and in particular, what we were just talking about here, that the field of competition is changing. It seems to me like both the people in the company, it's gonna be the people that are successful are the ones that are managed to combine- I think you said a couple times art and science. I'm assuming you're in the range of my age. When I started, when I was young, in particular when I went to college, it was, "you're going sciences." Science all day all the time. I had to fight to be able to take Liberal Arts classes. The idea was to get out of those as quickly as possible. They didn't encourage you to broaden yourself out and I worry that's even gotten worse. What you're describing is a company that is- You personally are succeeding because you're embracing the art as part of your science. What are you seeing particular because you bring in some of these more junior people? Do you feel like that deserves recognition that this is important or are you guys kinda having to instill that?
Brad K.: It's a great question. Especially a lot of the coverage of technology trends with automation, or AI hype, it has like a very art versus science tone.
Ben Newton: It does.
Brad K.: [inaudible 00:20:18] We really- It's art and science. In almost every setting there's some combination of human judgment and machines or algorithms is better than neither alone. It's interesting to make that work. You do work with people with a variety of different backgrounds, from educations and skillsets. That's even true within the algorithms [domestic checks 00:20:38]. If you look at the backgrounds of the scientists here, certainly we have people like myself, who're mathematicians by training or computer scientists. We also have people from all over the sciences, from neuroscience or physics or chemistry and I think there's value in that diversity of perspective even within the technical functions of the company.
Ben Newton: I think that's really good and I really do feel like that's going for the people that are gonna succeed are the gonna be the ones that appreciate that. And the company's, obviously they're gonna appreciate that.
Ben Newton: One thing you touched on a little earlier that I wanna go back to, you talked about data and you talked about the expectation from your customers that you were gonna be good stewards of their data. One thing that strikes me, I think I even saw some things that you said about your business runs on data, and clearly one of the reasons you guys are being so successful is because you are getting a lot of data about your customers but then, in today's day and age that also carries a lot of risk with it. How do you guys think about that and approach it to make sure that you're maintaining that trust?
Brad K.: Yeah, absolutely. As we discussed, having clients that are not only willing but are excited to share with us is a critical part of our business. It's really important that we have their trust to be good stewards of their data from a technological perspective, in thinking, look carefully about how we store things and using encryption and modern multi-layer approaches to being good stewards of data. Also, at the business level where I think an important part of people feeling comfortable sharing with us is because it's in their interest for us to get to know them and to improve the service over time. Also, just the nature of our business that we don't sell data to other companies. It's just not- It's not an aspect of our business model and I think it's quite important that we have that trust and alignment to keep the feedback flowing. I like to say that, "data isn't our product, but in many ways, it's our business as we use that data to get better at everything that we do."
Ben Newton: One thing I've heard a couple different ways, we actually say here at Sumo Logic, where I'm at is that we talk about an analytics economy but what we mean by that is that it's those that take the best advantage of the data and deliver better products because of that are the ones that are gonna win. That's really fundamentally what you guys- You guys have found a better way to get better data, more comprehensive data, and then you are making the best possible use of that to deliver better product back to your customers. It's kinda this virtuous circle, which means you're probably gonna get more data. And so you continue to enrich the experience, right?
Brad K.: Absolutely, and I think a fun recent example of that is a feature we rolled out in our iOS application called Style Shuffle, which is basically a rating game where you can look at different things and provide feedback on what you like and what you don't like. It's been both a really fun way to engage with clients and we've gone well over our ability of ratings on the platform so far. Also, another great way to learn from clients, another great way that they can share their preferences and this data has been very impactful in improving the client experience in very concrete, measurable ways. So, yes, certainly, I think, we'll continue to get better at using data, which is kinda of the core charter of the algorithms team at Stitch Fix, but also continue to get rooted and engaged with our clients in different ways and that'll help to make us continue to get better and better over time.
Ben Newton: One thing that comes to mind too is a theme that has been in a general consciousness for a while, but particularly in some of people that we've talked to on this podcast has been biased to extent that flip side, you've got that trust you have a data but also there has been instances, like we interviewed Cathy O'Neil who wrote the, "Weapons of Math Destruction." She talks about bias. And it's usually more about these kind of public services and all that, but I mean, do you feel like that's actually been something that you guys have had to confront and get your hands around bias in the platform? Or has that been less of a problem for you guys?
Brad K.: Well, I think there's a difference in severity between maybe using AI in the criminal justice system and we're about systematic bias there and being a retailer, a personal styling service. We do think quite about getting good feedback from our clients and many economists and behavioral scientists will tell you that there can be a difference between what people say they want and how they actually [crosstalk 00:24:51]
Ben Newton: You know what.
Brad K.: Yeah, absolutely. We think about that a lot. And also, just an aspect of the human in the loop nature of a lot of what we do, particularly in styling involves human stylists making decisions. We think a lot about how to match clients and stylists together, how to build tools that make styles more effective. Part of that is understanding different stylists have different stylistic preferences and can we actually not only be aware of it but use it to improve the service for our clients.
Ben Newton: That's interesting, so even you're running your algorithms through the filter of a person and you can see. And you guys have got thousands of stylists, right? You have quite a few of these stylists across the country, right?
Brad K.: Absolutely, yeah, so we have over 3,000 stylists across the country. They're mostly working remotely and part-time. They're actual Stitch Fix employees and we invest quite a bit in training and development and retaining them. Yeah, so it's a great workforce and we certainly, from an algorithm's perspective, learn a ton by the decisions the stylists make for clients and that helps us to make algorithms better. Stylists are also human and human preferences, I might love blue polo shirts and I'd assume everybody else would too. (laughing) We can look at preferences our stylists have and understand their strengths and weaknesses as humans in part of the larger system to help optimize the experience for everybody.
Ben Newton: Yeah, [inaudible 00:26:12] You reminded me, I remember when my wife helped me understand it, I could actually wear brown and [crosstalk 00:26:16] I was allowed. I didn't have to wear blue all the time. Just to kinda put a bow on all this, you guys are doing some pretty amazing things, so what's next? What's the next frontier that you can actually talk about?
Brad K.: There's an enormous variety of exciting things that we're working on as a team. I think, we talked about before, we're just getting started. There's so much more opportunity for just getting better fundamentally and understanding what people like and love and how to better manage inventory and develop new products. There's a lot of sustained innovation there that gets me excited to come to work every day. I think over time there's a lot of interesting ways of engaging with clients that we'll get to explore as things like computer vision and even things like 3D scanning become more accessible. What role could that play in the future of a business like Stitch Fix? I'm excited about how the ways in which we engage with clients, get data from clients will continue to evolve over time and it'll be an exciting journey.
Ben Newton: Absolutely, I would expect so. Well, Brad this has been a lot of fun. I think it's you guys really are a perfect example of- The company that I think is so interesting like we said at bringing together art and science and particularly using data science for good not for evil. (laughing) Making people's lives better so I think that's great and I wish you guys all the luck and thank you for taking the time to come on the show, I really appreciate it.
Brad K.: Absolutely, Ben, my pleasure.
Speaker 3: Masters of Data is brought to you by: Sumo Logic. Sumo Logic is a cloud native, machine data analytics platform, delivering real time, continuous intelligence as a service to build, run, and secure modern applications. Sumo Logic empowers the people who power modern business. For more information, go to SumoLogic.com
Speaker 3: For more on Masters of Data, go to MastersofData.com and subscribe and spread the word by rating us on iTunes or your favorite podcast app.
Page of