Karim Galil: Welcome to the Patientless Podcast. We discuss the good, the bad and the ugly about real world data and AI in clinical research. This is your host, Karim Galil, Cofounder and CEO of Mendel AI. I invite key thought leaders across the broad spectrum of believers and descenders of AI to share their experiences with actual AI and real world data initiatives.
Welcome to another episode. When we, when we first started Mendel, we acknowledged that there's two key challenges. One of the technical challenge, can you actually get the machine to read medical records like human beings? The second challenge is actually distribution. Can you convince the market that machines can do that? The state of the AI today is the boy who cried the Wolf. So there is a lot of companies that have promised AI didn't really deliver on those. Which makes the job of selling and actually working AI very challenging. So we acknowledged that from the get go it's like, you need to build a tech, you need to build a very unique go to market motion for it. So when we were out in the market trying to hire our first chief commercial officer, it was as hard as hiring my co-founder and, we got to meet Hilton we saw the signs like this is the right hire, one, like the story about how he got into healthcare and what's the mission that he's after in healthcare kind of aligned with us second, his background, third, he made this really big statement. Listen, I'm not a sales man and that's exactly what we were looking for. And that's counterintuitive when you hire your Chief Commercial Officer. Welcome Hilton to our podcast. I thought this, we, we had a press release, but I thought like it's only official when we do the podcast. So the idea is to get to introduce yourself, your background, why you joined Mendel. This is our first broadcast where we actually interview someone in Mendel. So no pressure. Your father is a physician, you actually were born in South Africa. I only got to know that last week or few days ago. And that's how you got into healthcare. So why don't we start from that?
Hylton Kalvaria: Yeah, sure. Well, my dad actually told me don't go into healthcare. that's exactly what he said. He's a, now retired gastroenterologist and, I was pre-med in college and he is like, do not go in into healthcare. And I think for him, he loves treating patients. He's great at the, the diagnostic parts of it. But actually the business of medicine got in the way for him and he said, don't. And actually for a long time, I had nothing to do with healthcare. So I started my career as, an executive recruiter, actually. That's what I did as my, my first job did that for a couple years. Joined, financial technology company. After that I did two real estate technology companies. And then post-business school. I, I said, well, now now's the time I really wanna do something in healthcare. So that's really what I've been doing for the last 15 years now.
Karim Galil: So it all started by an advice not to get into healthcare.
Hylton Kalvaria: That's what he said. Yeah. And he still stands by that.
Karim Galil: All right. So what was your first healthcare gig? Like how you went from finance to, to healthcare. I believe you were at Zs.
Hylton Kalvaria: Exactly. ZS Associates. My first job there was actually doing, qualitative interviews for a multiple myeloma drug and so we had to do literally a hundred interviews with multiple myeloma experts. All over the world. And if you've ever done these kinds of interviews, before, by the time you get to the fifth one, they become super boring. So you try to figure out ways to make them interesting as much as possible and infuse your own personality into it. I really enjoyed that part. Cause you just end up talking to amazingly smart people on a topic that you had no idea about before. And I learned about multiple myeloma.
Karim Galil: So how does that work? Do they first get you up to speed on the clinical aspect of multiple myeloma, or it's only more about like the go to market or is the mix of both?
Hylton Kalvaria: It ends up being a mix of both. I mean, if they just throw you in the deep end with your first interview with some multiple myeloma key opinion leader out there, the interview will go really poorly. They will prep you, allow you to speak to some experts and this was really clinically detailed, the thing that we were doing there. So you have to be very well prepped for that.
Karim Galil: So when I first got introduced to you, Here is what I was told. I was told like you, had a call with the founders of Flatiron and few years later, this call ended up with like 1.9 billion dollars almost so like can we talk about that?
Hylton Kalvaria: Yeah. Which part of it that sounds like it might be overstating did
Karim Galil: When I heard that, I thought like, all right. If, if this call ended by that, let's say he got half percent of this. So I expected you coming in Maybach or some crazy car. So I wanna make sure I give the disclaimer, you're not a billionaire or a millionaire yet.
Hylton Kalvaria: Not yet. I arrived in my 2014 Hyundai Genesis, so really sexy car
Karim Galil: What's the story? I know that you were working for Roche and, I believe you were tasked to find unique data assets. That kind of changed life for a lot of different folks, including Roche, Flatiron yourself, a lot of other people in the industry, actually.
Hylton Kalvaria: Yeah. So the way this happened was I was sitting at my desk at Genentech and one of my good friends came by and said, they want me to work on this magic database and it sounds really stupid. I don't want to do it. Do you want to do it? And I was like, well, that actually sounds awesome. I wanna work on that. And so it became this project that I was almost voluntold to do. It was supposed to be a 5% project and the name of the series of projects was called work stream 2020. So this is back in 2011, 2012. And somebody smart at Genentech said, there's a whole bunch of things we need to prepare our organization for the year 2020, we think data's gonna be an interesting thing. And so, my project was data and they said, go find interesting sources of data that can support this growing oncology portfolio. And one of the first meetings I took was with the two Flatiron co-founders.
Karim Galil: And I believe this was also their first customer?
Hylton Kalvaria: We were their first or second customer at the time. And this was when they were still visiting everybody. Every meeting I had with them, Nat and Zach were there, they made me feel special, brought me In n Out. They were really trying to figure out what their go to market motion was going to be not only in Genentech, but then how do you generalize that to lots of other pharma companies out there?
Karim Galil: Interesting. So, what is it that magic database. What was the thesis at the time? I mean, obviously we know how it ended. It became manually abstracted medical records, but what was the initial thesis at Roche for this magic database?
Hylton Kalvaria: Well, initially, most of my career at Genentech was in the commercial organization and so that's why they thought I was particularly interesting. If you know the whole story, actually it ended up being less about the commercial organization. More about clinical development, real world evidence, post-marketing studies, those kinds of things. But initially the thought was, well, if you're going to track your market based on claims data, why couldn't you do the same thing with the HR data? And so that's initially why they were interested in talking to me and as we started to really get into what are the real strengths of EHR data, the outcomes, right? The outcomes are the thing that you cannot get anywhere else. That's what really led us to think about other parts within the company that could really benefit from this kind of data.
Karim Galil: So would you say the bigger usage of real world data today or HR data is on the R and D side or on the commercial side? When it comes to the pharma.
Hylton Kalvaria: Because commercial organizations care about getting the biggest data cut possible. I think they still tend to use claims databases and those kinds of things. So for me personally, I've seen much more activity in real world evidence groups, H E R groups, medical affairs, and then bleeding into some clinical development also now,
Karim Galil: Before we'll come back to that like just to finish your background. So from there. You actually went from a buyer to become an employee in Flatiron. Right? Were you one of the first 50 or something, or you were later on in the journey of Flatiron.
Hylton Kalvaria: So what happened was, I think I got my offer and I would've been number 50 and then they acquired a company, and got 60 people overnight. So I went from being like number 51 to being 111, which is like a bit of a bummer, but yeah.
Karim Galil: Okay. Well, that still counts. So that, that was like early on. Right? So what's the story? Because at the time flat iron was still a startup, right? Like it wasn't Flatiron that we know today. So what made you make the leap from a big company? Like Roche super established company to like a Flatiron.
Hylton Kalvaria: A lot of it was how fast you can move. So my special project success for me was defined as I got one SOW in place. With, an outside partner. And to measure yourself based on that was actually fairly sad for me. So I wanted to move to a place where we could move incredibly quickly. And when I got there, it was exactly as promised everything moved at 10 X the pace, information is flowing everywhere in a way that I just had not seen anywhere else. So actually, initially it was very disorienting to go from a place like Roche, where. Communication and information is metered out. If you will, to a place where it's just like free for all. It was awesome.
Karim Galil: Yeah. It's interesting. A lot of folks start in startups and end up in startups, some start in corporate and end up in corporate, but it's usually the most interesting conversations is when someone saw the two sides of the coin, basically like a small company and a big company kind of a thing.
Hylton Kalvaria: Yeah. The biggest thing I noticed in my first week was, people were routinely sending emails to Flatiron all like the Flatiron all email address. And this to me was like, you never do this at Genentech. I mean, if you send an email to more than 20 people multiple times a week, like you're probably doing it wrong. Here you have this information flow to a hundred people, 150 people, and people would regularly do this without even thinking about it. So the information flow was very disorienting to begin with.
Karim Galil: But at the time, like have you ever thought that you're eventually going back to your first employer through an acquisition? Like, would that even cross your mind at the time?
You left Roche, then Flatiron, but you ended up in Roche somehow again, like through that acquisition.
Hylton Kalvaria: Yeah. And you know of all the large companies, you could go back to their top, top of the list. I mean, they really aren't an amazing company, but once you've experienced the speed of going to a small company, there's almost no way you can go back to a large company again, it would take a very unique circumstance.
Karim Galil: So is that why you joined Verana after that?
Hylton Kalvaria: Exactly. Yeah. So went back to large company. Realized I wanted to do something small again, and then join Verana at probably number 25.
Karim Galil: Oh, so with ver you're one of the first 50. Oh yeah. Yeah. Okay. So I just, for folks who don't know about Verana. Verana is basically, almost like a version of Flatiron, but outside oncology, right? Like other therapeutic areas that is not really covered by real world evidence.
Hylton Kalvaria: Exactly. And that was part of the appeal for me is oncology has such an amazing amount of real world evidence today, but, ophthalmology, which is really what Verana was focused on at the time, there wasn't a whole lot of this information. So why does oncology get all of the cool data sets, right? Why can't we have it in these other areas? So I've been doing oncology for the better part of 10 years and wanted to learn something new.
Karim Galil: Interesting. So what brought you to Mendel, right? We're obviously like to the size of companies where one of the smallest companies that you have joined, right? What was your. I can leave the room if you don't want. No, but like what attracted you to Mendel?
Hylton Kalvaria: Why this weird, like confluence of events that ended up happening, and maybe they're not so random, maybe you had something to do with it, or somebody else had something to do with it. But my intern, who I had at Verana from two summers ago reached out and was like, hey, you should come talk to us. Search firm reached out the same week and somebody who I really respected. Who is now our Chief Product Officer, Sailu, I heard that she was joining at the same time. I was like, well, that's three really interesting things. I should come talk to these guys.
Karim Galil: So here's the backstory.
Hylton Kalvaria: I want to hear this. Yeah.
Karim Galil: The backstory is, when we first engaged in that we engaged like a very reputable executive search firm and on the intake meeting, very first meeting, they were asking me like, who's your dream candidate? Like, can you describe us the profile. So I was like, I want someone who can do this. I can do that. And like as I was going through it, one of the partners was like, I know the guy that you should hire. I just don't think like you're too early for that hire. I was like, okay, fine. And he just mentioned your name and kind of gave me a brief and completely forgot about it. And we moved on with the search. And then he called me two weeks later and he was like, well, this same person called me. And he was like, what's up with this company called Mendel. I've been hearing their name almost three times in lesser than a week. I'm actually open for a meeting. And I think this is how the serendipity behind that story, how we ended up meeting.
Hylton Kalvaria: The other thing too is I thought I was actually coming in to get a demo to begin with which I was, yeah. I mean, we did the demo, but the demo was really when I saw it, I was like this. This is unbelievable. It's almost black magic in a way. And then it kind of turned into something else afterwards.
Karim Galil: Yeah. I believe after the demo, you sent us an email full of questions. There was like eight questions or nine questions. I don't know if you remember that email, but there were like to the point kind of questions you asked about dates you asked about, there are like very, kind of someone who have seen what is not working in AI can only ask those kind of questions. So maybe this is a good segue for that. One of the things that we as I said at the beginning of the podcast is we believe that the state of AI and healthcare in general is the boy who cried the world. But in healthcare, in specific, you hear AI all the time, but we haven't really seen the impact of AI yet on healthcare. So if we pick the area of electronic medical records in specific. What are your thoughts about that? What have you seen, what is not working? And what do you think is, or you believe is working here in Mendel. It's always refreshing to see the insights of someone who's outside the company. Like not anymore, but I've been doing this for 4 years, so I'm somehow in a bubble, but what was your first impressions? What impressed you and why.
Hylton Kalvaria: I think the first was honesty about what a can and can't do. Cause I think we can really hurt ourselves and it hurts the industry. When people come in and say, it can do literally everything because it's not, not credible. And you can kind of get into, you know, proof of concept or a contract with a customer and they very quickly figure out what it, what a can and can't do. Which exactly why. Now I wanna go check that email that I, I sent you guys to see what I'd written there, but I wanted to understand what, what worked and then where the holes were. And to have an honest discussion about, what has been done already? What's different from what other people have done in the past? And what do we still need to do as a group? And I've sat through many presentations with companies coming in to talk to Verana and even at my job at Genentech where people would promise things were that were just not credible. And I think there's a big danger in doing that.
Karim Galil: Yeah. So expectation setting. It's not only about like what we can do. It's also about what we cannot do today. Couple of arguments that we hear a lot in the industry, why do you need medical records? When claims has been basically doing the job for the last few years and the argument is like, you only need medical records in very specific situations. So it's kind of a niche thing. But claims is 90% of the times can cut it and can get the job done. What's your take on that?
Hylton Kalvaria: I think that's completely false. All you have to do is look at. What's happening with drug development in general, if it were the case that you were going to have many more, you know, blockbuster drugs that we saw in the nineties, where you could say every single patient who's got this cardiac condition should get this drug. I would agree with you. Right. But we're getting so specific in what people are, are studying here. And when it's not only how they're studied, but once they get on the market, it's not good enough to know that the patient has lung cancer. You have to understand that it's non-small cell lung cancer and then you have to understand which biomarker within non-small cell lung cancer. So drug development is headed into a direction where claims data just can't do it. So I think it's a fallacy to say that you have to have EHR data or figure out how to cobble together EHR plus claims plus genomic data, which I think is where everybody's headed now.
Karim Galil: So when it comes to the unstructured date or the medical records, there has been three kind of schools of thought one is that the only way to do it is manual abstraction. There is a lot of context. There's a lot of nuances that only humans can capture. Right. Then there is the other school of thought, which is AI will do it. We will build an AI. That is gonna understand medicine like humans, and it's just gonna structure the data, think of IBM Watson or Amazon comprehend, for example. Right. And then there is a third school of thought, which is we can build a machine that can understand a lot of things, but not everything. We will have some sort of a symbiotic relationship where AI increases the efficiency of humans and maybe even replace humans in some end points, but doesn't completely eliminate the human from the loop. What's your take also on this? What do you think is gonna end up being the go to approach?
Hylton Kalvaria: I think it's inevitable that'll always be an augmentation for humans. Like these two things have to work together. I think the Flatiron model was fairly heavy on the human side. With some technology to facilitate it, basically mechanical Turk for going through a patient record. What I found appealing here and what I find credible in the moment and why I'm enjoying what we're doing here. It's that model's flipped. Heavy on the machine with the appropriate amount of human intervention. So if we can figure out how to flip that model and say, these are the things that machines are really good at measure it really well. We think these are the variables that machines can do better than humans and those variables exist. And here are the ones that machines don't do particularly well at but we direct the humans to where they need to spend their time. That is a winning model.
Karim Galil: Obviously I agree. This is the thesis around the company. We, and, and maybe this is good, like for, for the audience of the podcast. So we ran this experiment, before you joined, we ran an experiment where we really wanted to understand how humans do against machines. We got 20 data variables that we asked. Two sets of abstracters to, to extract from the medical records. So like group one and group two, and we wanted to do enter annotator agreement between both of them, just to see like where did they agree? Right. Then we got a third group that we gave them AI to augment their abstraction. And then the fourth is AI only. Here is the very interesting finding on some data variables, like response or outcomes. Actually the chances for two humans to agree is against the gold set is almost a 50%. It's like tossing a coin. If you give them AI, what end up happening this jumps up to almost 80%. The reason is the AI is able to guide them to where in the record, there is most relevant information that can guide their decision making. The other key finding that we found is on some endpoints the AI outperformed humans. So the conclusion we came up with is like on some endpoints, AI is definitely better than humans on other endpoints. It is humans plus AI. There is no end point where humans only can actually get to the accuracy that that matches or comes close to a goal set. And we thought that was very intriguing finding that we didn't capture before when we first started Mendel actually.
Hylton Kalvaria: I think that makes a ton of sense. So somebody said here, the machine doesn't get tired. Right. So if you're trying to find all the mentions of something, all the instances of something complicated to narrow the task down and then have the human weigh in afterwards. I mean, it makes a ton of sense.
Karim Galil: Yeah. What are your key goals now in Mendel? Now that you have joined if you wanna share like, I mean, obviously I know all of them... it's just what would you like to see? Or how would you define success for the commercial organization in the upcoming few months.
Hylton Kalvaria: Yeah. So the challenge with the technology that we have is that, you can point it at so many different things in the healthcare industry. And as you kind of go in and this has even been eye opening for me to talk to customers now, The pockets of unstructured data that exist within the healthcare industry. It's vast, right? So it's a bit of a game of where do you go first? What I'm trying to figure out right now is where is the place where we could go to begin with where we can do the most good and figure out a repeatable way of engaging with customers that delivers them a ton of value and allows us to learn along the way. So we don't spread ourselves too thin. So it's really a game of where are we going to go first with this incredibly powerful tool?
Karim Galil: So obviously the word AI is something that's being repeated in the industry almost every day when you first joined? I believe, Wael, Wael is our co-founder and Chief Science Officer. Kind of gave you a crash course on AI, machine learning, symbolic AI, like all sorts of like AI things. Right. What was the most intriguing or surprising thing that you found out about AI that you didn't know before you joined the company?
Hylton Kalvaria: So I had almost no experience with this prior to coming here. And the most amazing thing about talking to Wael is the examples, the examples he gives, which are the real life examples, you get them immediately. They're so tangible and you can understand immediately why AI will struggle with certain kinds of concepts, but maybe the most amazing thing is, the NLP technologies that I knew, at least in the past. Pretty rudimentary. It's basically information retrieval. And when you actually walk through some of these examples, you're like, wow.
Karim Galil: Can you share some of those examples?
Hylton Kalvaria: Yeah. So when, Wael showed me is. We're trying to understand all of the words in the sentence or as many as possible with the relationships between them. It's both of them. It's not just individual words. It's all of the words and the relationships, and then try to model those relationships using a variety of techniques. When he showed me examples of what comes out of some of the other systems, it was one or two words that end up being pulled out. And sometimes the relationships between those words don't even come. So to me, I was like, Is this really what people call NLP? Because it felt like it was just on a search for very specific terms in returning those terms. It actually didn't make sense to me.
Karim Galil: Yeah. I mean, you find very basic things like fatigue, fatigue can be a symptom. It can also be a side effect. Yep. So how can the machine actually distinguish between those two things and actually in the same document, it can be both. It can be sometimes a symptom and sometimes a side effect for certain events. As a physician, I never thought that that fatigue can be such a complex thing because you take it for granted that your human brain can distinguish those almost instantly. But the amount of work that an AI team has to put in just to distinguish a side effect from a symptom is massive.
Hylton Kalvaria: Yeah. The other thing that I found amazing was how layered all these approaches are for us. And even something as simple as negation, right. You know, Wael has mapped out all of the different ways you could do negation in a sentence. And there's a whole class of algorithms just for that. And. You start to look at all of the different ways human sentences and medical sentences are constructed and you start to pick apart all the different pieces that require different kinds of frameworks and algorithms. That's what he and we have built. It's amazing.
Karim Galil: So usually when we approach a customer, they have a problem that they don't believe what we're saying. Like they believe it's too good to be true. That's like one of our key challenges in every customer interaction. I think when you joined, you had almost the same kind of reaction. So I remember you went in, there was a customer delivery. You actually went in and tried to corroborate the output of the AI with the actual medical records. And you wanted to see like, is the AI actually as good as it's being promised, but you've also seen some end points or you've seen some data variables where you didn't feel that the AI was doing well on. Can you share more on that? What are the challenges that we have today with the product, from your own observation and that we're working?
Hylton Kalvaria: Yeah. Great. Great question. So the one that I remember very clearly, and this is not like a class of problems, but, EGFR, if you're talking about kidney function or eGFR, if you're talking about cancer, right, they're spelled the same way. One is a lower KC, actually, almost every case is gonna have a lower KC. And, and so I found this example and, you know, it's the kind of thing you could fix it really easily. And while I pointed out that there's hundreds of these examples out there, so that was. One class of issue that I saw the other is around dates. Dates are hard. I think variables that are single points in time, I think tend to be a little bit easier, but understanding one, that this actually is a continuous variable where we expect to see lots of different measurements over time and then capture events, value and dates over and over and over again. I see that as an area that the team's working pretty hard at.
Karim Galil: One thing that a lot of our customers and potential customers don't understand is the actual difference between NLP and NLU. So it's one thing to be able to extract biomarkers or dates or whatever data variables that you want from a page or from a document, but it's a completely different set of challenges. To stitch those together to become a patient journey like a patient is, is what probably thousands of documents occurring over a span of like few years. And you have to extract events. You have to extract them and then you have to kind of put the puzzle together so that you understand what happened before what. You wanna speak on about that?
Hylton Kalvaria: Yeah, that was probably the other major eye opener event for me. To understand what NLP technologies today are doing. Some of them do a decent job at extracting all of the clinically relevant events. And the way I explain it to customers, when I talk to them now is, you know, if you were to work with some of those technologies, they will dump 3000 terms in your lap and basically say, okay, now you customer stitch it all together and turn it into a nice summarized version of that patient record. What was eyeopening was being able to take all of those different mentions of a particular term. Understand the trustworthiness of where all of those terms came from in the patient record and have algorithms that can decide based on the kind of document it is, where it found at the context in the sentence and come up with the consensus best answer for it. To take thousands of things and turn it into the summarized version of the patient record. That might be only the 30 or 40 data points that the customer truly needs to understand what happened with that patient that is completely groundbreaking.
Karim Galil: So you've mentioned earlier, we were getting into that thread of things that is not working right. And you've mentioned there are some data variables that remain very challenging, like dates, biomarkers , and some words that can be understood in different ways. What else? What else do you think is still missing today? That Mendel has to be investing and working on.
Hylton Kalvaria: I think one of the other pieces is actually structured data. Because I think we've tackled the hardest problem, which is unstructured data, but realistically speaking, there's many pockets of data out there that contain both the structured and the unstructured. So figuring out how to bring all the structured data in when we consider that consolidated version of the patient record, I think is something that we're gonna have to tackle over the next year.
Karim Galil: That's hundred percent. It's the merging of the structure, then the unstructured actually has a lot of challenges that a lot of people are not aware of because sometimes you have contradicting facts. Like you have something in the medical records that the patient was not built for or something that the patient was built for that actually does not exist in the medical record. And how to reconcile those is not an easy job.
Hylton Kalvaria: Yeah, it's hard. we talked at some point, well maybe, maybe you say the, the trustworthiness of these structured variables, you, you crank those up. And so as the system evaluates, it, that's the one that ends up picking, but sometimes you end up having things in the structured data that isn't necessarily right. Either. So it's hard to know which to go with.
Karim Galil: So in this day and age, work from home is the standard. Actually when we were negotiating the offer, we never talked about work from home or work from the office. But, I try my best, like before you join, I'm always the first person to step into Mendel like I'm the first person to turn on the boiler and the coffee. And since you have joined, you are the first person to come in. I don't know how you are doing it. Like today I came. Intentionally earlier than every time yet I come and I was like, today's the day I'm gonna be the first one I come in. You are in the meeting room. Like when do you come to the office? Like, I really wanna know that,
Hylton Kalvaria: I normally get in around 7:45am because my first meeting is normally at 8. And I figure if I'm not in the car and sitting down here, I kind of missed my window to travel. So my schedule in the morning, I have two kids, two girls, 10 and 13, try to help a little bit with them in the morning. So it's not all on my wife. And then I'm out the door.
Karim Galil: So obviously we have Benny who's recording a podcast. So there is two competing shows in Mendel. There's the podcast and then there is the vlog, and Benny's vlog is more popular as we speak today than the podcast. But one of his questions, I'm gonna borrow that question is who is your favorite employee in Mendel.
Hylton Kalvaria: Oh man. Actually easy, Thiago wow.
Karim Galil: I don't understand why Thiago is everyone's favorite employee.
Hylton Kalvaria: Well, a couple things
Karim Galil: So Thiago is actually just for context. Thiago is on our AI team. He just joined, he actually quit his PhD to come join Mendel, as one of our first 10 AI scientists.
Hylton Kalvaria: Yeah. So he's aside from me, he's first in, in the morning. So we always like congregate around the coffee pot over there as we're making our fancy coffee in the morning. Plus who's got a better accent than that. , it's the most amazing accent.
Karim Galil: So Thiago is originally from Brazil, I believe. Right. Everyone likes Thiago he cost us last month, I believe, close to $3,000 worth of like ping pong balls. What else did we get? Foosball we got a lot of ball stuff. So we haven't yet hired our first like head of finance. Like he's about to join in a few weeks and the joke is like, spend as much as you want before he comes. And it was a joke, but people are actually taking it serious. You're actually one of them.
Hylton Kalvaria: There was a need for t-shirts here. I gotta order the t-shirts.
Karim Galil: So when you first joined, like I think that was one of your questions, like why no one has any Mendel t-shirt here. And then one day I come in, everyone is dressed in Mendel t-shirt I still don't have one by the way, but like everyone had Mendel t-shirt. It was almost like synchronized, like where everyone came into that.
Hylton Kalvaria: I got a form you can fill out if you want one.
Karim Galil: That's what Benny was telling me. There's like a Google form or something to fill in there. Hey, thank you for being on our podcast, being the first, Mendel employee on the podcast. Thank you for joining the company. We're super pumped. And I think there is a exciting journey, ahead of us. If you have unstructured data, reach out to Hilton, we need your unstructured data.
Hylton Kalvaria: Yeah. That's firstname.lastname@example.org.
Karim Galil: Awesome. Alright man, thank you. Have a good one.
Hylton Kalvaria: Cool.
We’ve changed our look. Our goal remains the same: make medicine objective. The new site highlights the way our proprietary AI enables organizations to achieve quality and scale when structuring unstructured data. It comes down supercharging your clinical abstraction. We’ve validated that our human in the loop abstraction approach can support a machine that understands medical context like a physician. In our own experiments, the number of variables needing correction decreased by 40%. High quality abstraction = high quality data for cohort selection, real-world evidence, and registries.
The customer, a key player in the genomics space, had a strategic initiative to build a clinic genomic database to support their life sciences customers.
One clinical trial organization was using manual chart review and was looking to reduce the time it takes to find eligible patients.
From the Desk of the AI Team
Organizations that use patient data for internal or external research need to take steps to prevent the exposure of PHI to those who are not authorized to view it. They do this by redacting specific categories of identifiers from every patient document. Once the identifiers are masked, the risk profile of these datasets is significantly reduced. But how do you ensure that redaction engines are working to the highest accuracy?
The Mendel team is still buzzing from our week-long retreat in Cairo. The theme of the retreat was “coming together” and it was the first time the American and other remote employees were united with their Egyptian counterparts. Although there were many adventures–missing flights, seeing the pyramids, haggling at Khan el-Khalili–the highlight of the trip was collaborating together, as one global organization.
Competence via comprehension
Artificial intelligence (AI) is playing an increasingly important role in the healthcare industry. But to fully leverage the potential of AI, it must be equipped with clinical reasoning skills - the ability to truly comprehend clinical data, or in other words, to read it as a doctor would. When it comes to data processing tools, only a tool capable of clinical reasoning can effectively process unstructured clinical data.
Sailu Challapalli, our Chief Product Officer, spoke at a recent Harvard Business School Healthcare panel. The event brought together different healthcare and AI experts to discuss large language models and their impact.
Manually abstracting patient data at scale is an herculean task for humans alone. It is slow, expensive, difficult, and requires extreme precision and accuracy. Organizations have to choose between breadth and depth when it comes to making data useful for decision making. Because of these challenges, the Mendel team created Carbon. Carbon is an easy to use workspace that allows clinical abstraction teams to efficiently curate high quality clinical datasets at scale. The foundation of Carbon is Mendel’s AI. Carbon pulls directly from Mendel’s AI platform to give abstractors a headstart in identifying relevant data elements within a patient’s chart.
Within the real world evidence space, the generally accepted process for creating a regulatory grade data set is to have two human abstractors work with the same set of documents and bring in a third reviewer to adjudicate the differences. These datasets also serve a second purpose - as a reference standard against which the performance of human abstractors can be measured. Although this remains the industry standard, it is expensive, time consuming and difficult to scale.
From the Desk of the AI Team
AI projects have created tangible results for a wide range of industries. Despite the innovation, it is important to remember that AI is not a magic wand that will solve every problem in every industry with a single wave.
Before embarking on any new endeavor or enterprise, certain questions come to mind: How are we going to handle this? Does our team have the expertise, bandwidth, resources, and time to handle this undertaking on our own? When it comes to finding a scalable way to structure your unstructured healthcare data, the answers to these questions will impact when/whether you deliver a top-tier product for your clients.
Human abstraction has long been considered the gold standard for extracting high quality information from EHR data. With the rise of NLP and machine learning, how should we evaluate these new technologies and are human abstractors still the correct comparison?