Transcript for "How to Scale Win-Loss Research with Klue's AI Interviewer": Hello, everyone. I am Matthew Reeves, and here we are with the webinar today, how to scale win loss research with Clues AI interviewer. So I'll bring Jenna Bugiardini on in a second, but maybe we could start with people leaving in the messages in the comments. Where are you today? We have a bunch of people joining. I can see me and Jenna Bugiardini are here in the Klue headquarters in Vancouver. So, yeah, drop that in. And if you have questions along the way, put them in the chat. We're going to be looking at these and answering as we go. Let's jump in right away. We only have thirty minutes, so I'm just going to kick off. So here's what we're doing today. Let's talk about how to run seller win loss. I can see people already from New York, from, LA, from Kentucky. Okay. We've got people all over The US. So today, we're gonna talk about seller win loss. We're gonna talk about what that is, what that involves, and Jenna Bugiardini gonna help us, with how we've run this at Klue. I'm excited to get to that part because we can get really tactical talk about how we set it up internally, what sellers were expected to do, the type of questions we asked them, and what we did with all the data and insights afterwards. Before we get there, let's just have a little bit of background about win loss. Most people have heard of win loss, and usually, when I'm talking win loss, I'm talking about buyer win loss. So you've won a loss of deal, and you reach out to that prospect or that new customer, and you interview them. You say, why what like, what was your buying experience? Why did you choose the vendor you did, and what was the right solution for you? What we're talking about is similar for interviewing your own sellers about deals they have recently won and lost, and we'll get into more about the type of information you get from them. But just to ground this conversation, you can, of course, sit and interview your sellers. What we're talking about is using Klue AI interviewer to conduct these interviews. So our AI interviewer is available on the website. There is a live link. You can take a link and take it for a test drive after this. But we're going to be giving you a little live demo just so you can picture it in your mind. So it's a short voice interview by AI. There's no creepy avatar you have to talk to. It takes about five to seven minutes. That's pretty short, and the interviewer conducts probing questions, digs into areas you would like to learn more about, and follows up with questions if the seller was really unclear. So I am going to give you a quick live demo. Let's hope the demo gods are on my side. I'm gonna share my screen, and let's see how we do. So when sellers take this interview, and we'll cover how they get this link and they get to this page in a second, but they land on a entry page just like any kind of survey that they're used to, like a Google Form. They put in their email so that we can link the response back to them in future, and then we're gonna start the interview. This, I'm going to go silent at pieces so that I can interact with the interviewer. You should hear the interviewer talking to me in a second. Hello. I will be your AI interviewer. Please adjust your speakers until my voice is at a comfortable So there you go. And then I can conduct a mic check so it can I can test that it can hear me? Testing, testing, one, two, three. Hi there. I'm Klue AI interviewer. I'm here in a lightweight demo mode to learn how people currently run win loss interviews and how they think about using AI for them. This isn't the full experience our customers get, but it should give you a sense of how an AI interviewer asks questions and follows up. Ready to get started? Yep. I am ready to get started. Great. What's your experience with win loss or deal reviews in general? Have you or your company done them before? Yeah. So we have. I have sat with sellers and kind of asked them about a recent deal they've won or lost, but it was really just when I had time. Got it. When you've done those, how did you usually capture or share what you learned from the seller? Okay. I've muted myself now, so it's not gonna carry on. But you can see it reads my answers, it understands, and it follows a set conversation path, but it's allowed to deviate and ask follow-up questions. So just to give you a little taste of what it's like, this is the demo instance. This is on our website. So like I said, visit clue.com. You can, take it for a test run yourself. For now, though, we're gonna bring out Jenna and jump into q and a about how we set this up and run this at Klue. Jenna, Hey, ma'am. welcome. How's it going? you good. Like, okay. So why don't you introduce yourself? I realized I didn't introduce myself at the start at all. So. my name you introduce yourself. Totally. So my name is Jenna. I, run enablement here at Klue, and I also lead our internal win loss program that we do. Great. Maybe we ground this in just what we were doing before any AI interviewing. You obviously work with sellers a lot. What were you doing internally with seller deal reviews at all? Like, were you doing them at all, or was this just kind of a nice to have? Yeah, tell us a bit. Yeah. Totally. I think, before we started using the AI interview, it was really ad hoc. Obviously, as most companies, when a deal closes, we'll ask our sellers to fill out the CRM fields, and they'll put in their notes. And I'll try if we have key deals. We'll do deal reviews with our sellers. Managers will sit down and do them. Me as enablement, I would occasionally try to sit down on some key deals, but it's really fragmented overall. So when you say fragmented, what do you mean? Is that like you did them when you had time, or, it was just when there was kind of the key account that was lost? What did it look like for you? Yeah. More the latter. It was like we'd have these deals that people wanted to hear more about, and that's when I'd reach out to the seller and, like, sit down and learn a bit more. There wasn't any system to it. It was more reactive. Yeah. So then, obviously, we internally we've been building our AI interview. We've been using this ourselves for buyer win loss and seller win loss. We're only talking about, seller win loss today. So how did you set that up? Meaning, what was let's start with what was the expectation for sellers? What types of deals did you ask them to talk about? How how did you go about that? Yeah. So we, specifically looked at all the deals across the board. And so with our seller interviews, we wanted to see across the team, like, what are what's actually happening and understand that a bit better. Did you but so what information were you trying to find from them? So you you could ask a lot about the deal. What were the key questions you were interested in? Yeah. Mostly, we wanted to understand from a number of different because we have many stakeholders. We speak regularly with product. We work with sales leadership, executive leadership as well. And so we had a number of different purposes we would try to serve with these interviews. We'd wanna get an understanding everything on what's the problem they're trying to solve, what was their perception of Klue in our product overall competitors. Like, what came up in the deal that they liked about competitors? What alternatives did they do? And then ultimately, like, seller performance as well. When they looked back and reflected on their deals, what did they think they did well? If they could go back and do it again, like, what might have they changed in the next time? Yeah. So okay. So it was very, like, a a kind of deal debrief looking at what the buyer wanted. How did you find the results in comparison to CRM? And, obviously, that's a loaded question because everybody knows, like, CRM notes are famously terrible. They're, like, one liners. But was there anything surprising about how reps responded when talking in voice? Yeah. Definitely. So, again, CRM notes, couple of lines, few seconds. They just wanna get that deal across the line. They're not spending a ton of time doing that. And so what we found when we started doing this is I was able to actually get, like, a full story of what was happening in the deal rather than just getting their perspective on exactly why they lost or what happened in that moment when they're closing out the deal. Doing these retroactively created a bit more of a narrative of what happened across the entire cycle, and it gave us more to work with, and we could be a little bit more thoughtful with how we use that information. Did you find that when they're kind of giving their narrative of what happened in the deal, did you find sellers were defensive, or, like, or or were they kind of straightforward? Like, how did their responses look to you? Yeah. They were actually very receptive to speaking with an AI interviewer. I think we were at first, we were like, are they gonna enjoy this process? Are they gonna be like, I don't I don't wanna do this? But we got a very good response to actually interacting with AI. And you could even see in some of the tone of what they were saying in the interviews, It was very relaxed. They're like, yeah. You know, when I came across this, I saw a little bit of defensiveness from my prospect. Like, they were very open and relaxed in the conversations. Okay. And then I suppose one thing I would be nervous of if I was setting this up for sellers, I would want to know, like, how often are they expected to do this? How did you structure the expectations with them? Were the rules that they had to follow, like, they have to do this for every deal? What what did that look like? Totally. So I can actually back up a bit in terms of how we thought about setting it up. When the AI interviewer, we started experimenting with it. I reached out to our CRM admin and I set up an automation through we use HubSpot. So through HubSpot, I went in with our CRM admin and every time a deal closed, our seller got an email with a link to the AI interviewer. So this became a regular pattern of behavior that they are used to seeing. Deal closed, they get a link to an AI interviewer. And then what we did to really drive adoption and engagement is we incorporated it into our QBR. So I asked all our sellers ahead of QBR, can you pick three deals this quarter? And, in that particular QBR, we are focused on losses. So I said pick key three key losses and do an AI interviewer. And so it was a requirement for all of them to do. And, yeah, it just got them thinking about their deals ahead of QBR so we could use that in conversations in that moment. Nice. I like that approach where I'm assuming there's if you send it out for every deal, there's going to be they're gonna get lots of opportunities to do these. It's made really easy for them. But there's also a link to an actual review of the QBR review. I I wanna come back to that, but I'm just looking at the messages in the chat, and I wanted to answer a few questions that we are getting. If I miss them, I will try and answer these afterwards too, or reach out personally. But, do reps have to go through the Klue platform to do this? So that's a great question. I think what Jenna said is, they could, but what happens is there's a link, and we send that out via email. We use HubSpot automations, but other people use Salesforce or just put a link in Slack, do other or Teams, things like that. The data, goes back to the Klue platform into the win loss part of our app. So it automatically, after an interview's been complete, feeds back into win loss. And somebody else asking, does the interview have insights into the deal before the interview? So, actually, not right now. We've kept that really segmented so that, the interview is, like, not really biased to an area. That said, we can like, Jenna or somebody setting it up can tune it to really focus on a key area. So should we have, like, kind of a quarterly focus on pricing and packaging, just making that example up, we could tailor the interview to focus down on that area. Okay. Jenna Bugiardini, I wanted to get back to the QBR piece. So, obviously, QBR, different companies do different kind of activities, different training, with their QBRs. How did you use this type of win loss information, in our QBRs? Yeah. So, sellers, I asked them to complete it, giving me about a week of time to do the analysis. And the really nice thing about Klue is I was able to get these interviews directly in Klue and the the beautiful it just I didn't have to do the analysis myself. So Klue did some auto tagging for me, and it picked out, like, key moments and snippets in the conversation. So from that, I was able to figure out, okay, here are the key themes that showed up across all the deals for our sellers. And that was what I based our activity that we ran at QBR about. So we held a discussion with the team where we I I saw there was a couple use cases that we hadn't really come across before that showed up in some of these interviews. And so in our QBR, we use the time to create a playbook around that use case. And it it was really powerful because sellers I think one thing about sellers is is they love to know why are other people winning and losing deals. Like, what are my teammates doing that I also could be doing? And so to have that point of view of all the sellers together sharing, like, their top stories, in QB in a QBR setting, it made for a really, really valuable discussion, and we came away with a tangible playbook, that we we could start to apply. Nice. And by the way, if there are more questions, I love seeing them. So so put more in the chat if you have more to come up. So in the QBR, going back to that, I always find this really interesting because I think a lot of, PMMs, a lot of CI professionals, they really want to help sellers. But sometimes sellers are, like, kind of laser focused on the deal and they shut out all the noise. How did you find that discussion went? Like, are sellers receptive to it or or not really? Yeah. Definitely. And and I think what was interesting is sellers are they're the first ones to hear, what's going on in the field, like, what competitors are coming up, like, what objections are common. They're gonna hear that first. And so their peers have information to share about things that, like, we, as an email team or sales leadership team, might not have had a perspective on. And so we could take that information and and spread it widely throughout the team. And so they were really wanted to know. It's almost like the inside information is, like, what's actually happening? We could lift the curtain a little bit and gives them a bit more insight into that. Yeah. That's one I've actually heard. So working with customers, I've heard that before where we're in this era of, like, AI first startups popping up everywhere. And a lot of companies are having this kind of shift where there's new startups appearing, and PMMs just don't know about them or don't know which to take seriously. And they found that sellers are that first line who start to notice them actually tangibly in deals as opposed to just kinda like reading a press release or seeing it on kinda LinkedIn. So I find that, that side of it really interesting. You also managed our, buyer win loss side. Maybe you could talk a little bit about what's the different type of information you got from buyer win loss versus seller. Yeah. Definitely. So we have a number of, I would say, point of views that all feed into our win loss program. For those of you who use Klue, you know our win and loss stories, coming from call transcripts. That's a really powerful one to hear. Like, what's being said in conversations and calls? So that's a buyer voice of what's happening on calls. We also run human win loss interviews. I typically use those for more in-depth strategic deals. So ones that maybe were complex, they're highly competitive, Maybe we ran a full POC with those deals. I wanna go and actually interview those people. I do some interviews myself. We also have our team to get a bit more of an unbiased perspective of researchers who do those. And so, really, we we have a really comprehensive view. And what the AI interviewer adds on to that is just scale. I we don't have to spend the time to go do all those interviews. We can get the seller's perspective in that story as well, without having to sacrifice a ton of time and effort. Yeah. And I think some of the questions I'm seeing in the chat, I'm just gonna answer a few more as well. So folks are asking, is this available? When is this available? So, yes. Like, this is available. Include today. The use cases we can work with are seller debriefs on very specific deals, and as mechanisms to kind of for them to link it to their Salesforce deal. That's where they include referencing which deal they're talking about in Salesforce. There's also more general use cases where, folks use this to put this into, like, Slack or Teams, and sales sales folks periodically talk about what they're seeing in the marketplace in general. So they might do a debrief on, you know, this month's objections that they're struggling with, and it's used as a bit of a a training piece. I'm just reading some more questions. Matthew, I can speak to one more thing on buyer interviews. Yeah. So right now, our seller interviews are the primary use case for, our AI interviewer. Like, in the future, we will be expanding that to buyer interviews, and they've started to experiment with some of that. I think the difference between an AI interview for buyer interviews versus a human interview, is really just in typically, I for me, what works best was in deals that maybe I just wanted to get a specific point of view. So they didn't do a full evaluation. Maybe they just did one demo, and I wanna get feedback on that that demo. Sending an AI interview to those folks is a quick way to get touch touch points and feedbacks on those early stage conversations, versus running a full human interview on a later stage deal. Yeah. And I think one element of how we designed this was it's designed to be able to work on phones as well. So we designed it in a way that's, like, five to seven minute interviews that people can kind of do sitting on the couch or in the break room as opposed to kind of a more formal thing. So it's fairly easy to get going. I think so one question I think is really interesting is how do we think about this? And I have my perspective, maybe, Jenna, of yours, is how do we think about this, when you have a perspective from Gong, a perspective from a seller, and a perspective from the buyer? And Jenna and I were talking about this yesterday, and I have this kind of analogy, which is, it's a couple going on a first date. You could be listening to their conversation. That's kinda like the gong call, how those folks talk to each other. But then they go back to their their friends, and they talk about how that date went. And both people are gonna have different perspectives. And and my view is there's never really a right and wrong. Like, I've spoken to customers who who say, we wanna see if our CRM details are right or wrong. And, personally, I don't see it like that. Like, they are likely going to be different, but I don't necessarily see it as aiming to say our CRM details are wrong. They they probably are wrong because buyers have just a different perspective to sellers, and buyers may have given the, it's not you. It's me and my budget type of excuse. Jenna, maybe you have a better kind of analogy that's less cheesy in dating. No. I love that analogy. I think that's perfect. So everyone's gonna have a different story when they leave a date, and you gotta hear all the stories to actually understand and form your own point of view on what's going on. I can give a bit of, like, a tactical answer of how I actually combine all these different source information. I think it was Greg asked in the chat, looking at specific deals that where you're getting the seller feedback. I'm getting the Gong call transcripts. I'm getting the buyer interviews. I use Ask Klue research mode to combine all that information. So I work within, like, our platform and I use I talk to it like I would talk to a chat GPT. So I'd say, hey. Look at these interviews that have been done in the last three months. And I asked her to pull out key themes that we're hearing. And so it's a really powerful tool for combining all those different data sources. And then I can do what I want. So last quarter we rolled it up into our board report and that was a foundation for how we did our win loss findings at the board level. So I love that call out, like, combining those data sources and, like, multiple points of view. And then you can use research mode to do that. It can be really, really powerful too. And when you do those board reports, how do you, how do you, like, make sense of the data? Like, are you trying to pull together a, like, unified story, or do you present them as, like, just perspectives we hear from Yeah. So we do audiences? a roll up, I would say. Like, a high level here are the key themes. Like, I try to distill it down into a couple slides ultimately. It's like our wins and our losses. What are we seeing if we can do that roll up? That seems to be, like, really well received by the leadership team, and then they I've seen it used. Like, they'll incorporate into other parts of, their work that they're doing too. So it's it's a good way to get eyes from the executive level. I think win loss, is very powerful. It's things people the execs wanna know about. Like, what's happening in deals? Why are we actually winning and losing? So they can stand in front of the board and sort of, speak back to those points. Yeah. And then we're very close to time, so I think I'm gonna do one or two more questions. And I'm gonna put my British cynicism on, and let's let's get negative. Is the how should we not do these? How should we not run them? Were there any mistakes? Were there was there anything you would redo differently if we were to do this again? I have some from a product perspective, but maybe you can kick us off. Yeah. I think, a biggest one when we were running these interviews. I I do wish I like the idea of getting sellers to do, every single deal, but I I do wish that come QBR time, I had focused it a bit more on a specific topic that we wanted to do. I kept it very broad. So I was like, woah, this is a lot of themes that have come up. But I think if we had narrowed it down into a bit more of, something strategic that we maybe we're working on as a leadership team, we would have been able to have a a bit more of a focused conversation. It was still very effective, but there were so many topics I couldn't really get to, because we kept it really broad at the beginning. Yeah. I think that's a really common thing I see from the product side when we're testing is that, the key to getting really good answers, in my opinion, or one of the keys is to keep this, like, short as you can. And that's a real challenge because we want to ask loads of questions. We we wanna know everything. And I'm saying we as, like, product marketers or CI professionals, but we will also wanna keep it engaging for sellers. And it's similar to survey design. You want to keep this short, punchy so that they kind of, they don't dread doing this. So I I really like that approach, and what I've seen work for lots of companies is do one round where it's fairly broad. And then you start to realize, like, I wish I had dug into a single competitor or focused on pricing or whatever issue is kind of top of mind in the business at the moment and then kind of tailor it down. So I think that's my kinda, like, pro tip to people out there. It's like, don't be afraid of, like, tailoring this down. Or if it's the first run at any sort of seller win loss, start really broad and let that kind of data and insight guide you. Let the rest of the company say, hey. I wish we could have asked about this and just do another run. Okay. Jenna Bugiardini, you've answered all of our questions. There's no more in the chat. Zero at all. But if people did have questions, go to klue.com. You can either request a demo or you can head to the AI interview page, take this for a test run yourself. You can also just email me. I didn't introduce myself. Matthew Reeves. You can email me as matthew@clue.com or connect with me on LinkedIn, the more cheesy route. This has been a lot of fun. Do send in questions. I love them, and we'll see you again next time when hopefully we talk future use cases for AI interviewer. Love Thanks. it. Thanks. Bye.