Miscellaneous
Harry's guest this week is Jeff Elton, CEO of a Boston-based startup called Concert AI that's working to bring more "real-world data" and "real-world evidence" into the process of drug development. What's real-world data? It's everything about patients' health that's not included in the narrow outcomes measured by randomized, controlled clinical trials. By collecting, organizing, and analyzing it, Elton argues, pharmaceutical makers can it design better clinical trials, get drugs approved faster, and—after approval—learn who's really benefiting from a new medicine, and how. Concert AI, which has offices in Boston, Philadelphia, Memphis, New York, and Bangalore, specializes in providing “research-grade real-world data” and AI-based analytical services to companies developing cancer drugs. Before joining Concert AI, Elton was managing director of strategy and global lead of predictive health intelligence at Accenture, and before that he was a senior vice president of strategy and global chief operating officer at the Novartis Institutes of BioMedical Research. He’s the co-author with Anne O’Riordan of a 2016 book from Wiley called Healthcare Disrupted: Next Generation Business Models and Strategies.Please rate and review MoneyBall Medicine on Apple Podcasts! Here's how to do that from an iPhone, iPad, or iPod touch:• Launch the “Podcasts” app on your device. If you can’t find this app, swipe all the way to the left on your home screen until you’re on the Search page. Tap the search field at the top and type in “Podcasts.” Apple’s Podcasts app should show up in the search results.• Tap the Podcasts app icon, and after it opens, tap the Search field at the top, or the little magnifying glass icon in the lower right corner.• Type MoneyBall Medicine into the search field and press the Search button.• In the search results, click on the MoneyBall Medicine logo.• On the next page, scroll down until you see the Ratings & Reviews section. Below that, you’ll see five purple stars.• Tap the stars to rate the show.• Scroll down a little farther. You’ll see a purple link saying “Write a Review.”• On the next screen, you’ll see the stars again. You can tap them to leave a rating if you haven’t already.• In the Title field, type a summary for your review.• In the Review field, type your review.• When you’re finished, click Send.• That’s it, you’re done. Thanks!Full TranscriptHarry Glorikian: I’m Harry Glorikian, and this is MoneyBall Medicine, the interview podcast where we meet researchers, entrepreneurs, and physicians who are using the power of data to improve patient health and make healthcare delivery more efficient. You can think of each episode as a new chapter in the never-ending audio version of my 2017 book, “MoneyBall Medicine: Thriving in the New Data-Driven Healthcare Market.” If you like the show, please do us a favor and leave a rating and review at Apple Podcasts.Harry Glorikian: In the world of drug development, there’s a tendency to think that the only data that matter are the data that get collected from patients during randomized controlled clinical trials. That’s the type of study that drug companies use as the gold standard to test the safety and effectiveness of new drugs and that the FDA uses to make drug approval decisions. But it’s just not true. Way before clinical trials begin, there’s a ton of genomic or proteomic or chemical data that can go into identifying new drug candidates, as we’ve learned from many of our previous guests on the show. And today my old friend Jeff Elton is here to tell us about another important kind of data that get collected before, during, and even after clinical trials that can have a huge impact on how drugs are used.It’s called real-world data, and it basically means everything about a patient’s health that isn’t included in the narrow parameters and outcomes measured by clinical trials.Jeff is the CEO of a startup here in Boston called Concert AI that specializes in organizing and analyzing this real-world data. And his argument is that when you pay attention to real-world data, it can help you to design better clinical studies. It can help support the core clinical data that drug companies submit to the FDA when they’re applying for approval. And after approval, it can help show who’s really benefiting from a new medicine, and how. Jeff has been thinking about the importance of real-world data for a long time, at least since 2016, when he leading predictive health intelligence at Accenture and he published a book called Healthcare Disrupted. The book argued that real-world data from wearable devices, the Internet of Things, electronic medical record systems, and other sources could be combined with advanced analytics to change how and where healthcare is delivered. In our interview, I asked Jeff to explain how Concert AI is helping patients and how the predictions he made in the book are playing out today.Harry Glorikian: Hey, Jeff, welcome to the show. Jeff Elton: Thank you Harry. Pleasure to be here. Harry Glorikian: Yeah, it's been a long time since we've actually seen each other. I mean I feel like it was just yesterday. We were you know, interacting. Arshad was there and we were talking about all sorts of stuff. It's actually been quite a few years and, and, and you have now transitioned to a few different places and, and right now you're running something called Concert AI. And so, I mean, let's just start with what is Concert AI, for everybody who's listening. Jeff Elton: Yeah. So Concert AI is a real-world evidence company. We'll spend a little bit of time breaking that down. We are very focused on oncology, hematology, urological cancers. So we kind of tend to stay very much in that space.And within the real-world evidence area, we really focus on bringing together high credibility research grade data. This usually means clinical data. Genomic data can include medical images combined with technologies that aid gaining insights out of those particular data and that kind of align with our own various use cases.A use case could be designing a clinical study, it could be supporting a regulatory submission. It could be gaining insight, post-approval, about who's benefiting, who's not benefiting. And you know, our whole mission in life is accelerating needed new medicines and actually improving the effectiveness of current medicines out there.Harry Glorikian: So who's like, I don't know, the user, the beneficiary, in a sense, of this.Jeff Elton: So, you know, we like to think we have a very heavily clinical workforce. You know, we always put the patient first. So I'm actually gonna say that a lot of the reason why we're doing things is that we have the benefit to be stewards, combined with provider entities, of focusing on questions that matter for patient outcomes.So the first beneficiary is patients. I think the second beneficiary are biomedical innovators. We're trying to kind of support those innovations. We're trying to understand how to go into the clinic. We're trying to understand how to design those clinical trials to have them be more effective. We're trying to understand how to show that relative to the current standard of care, they offer a range of incremental therapeutic benefit. A lot of medicines become improved once they're actually already approved. And so we actually spend time doing a lot of post-approval research that actually begins to improve the outcomes by beginning to kind of refine the treatment approaches.And then the clinical communities we work very closely [with]. We're a very close working partner with American Society of Clinical Oncology and their canceling program. We're in a 10-year relationship with them that allows us to do work in truly high need areas. We did a COVID-19 registry jointly with ASCO that worked off of some of the data we brought together because it you know, COVID-19 uniquely hit cancer and particularly hematological malignancy patients.We do work with them in health disparities, making sure that racial, ethnic, and economic groups can be the beneficiaries of new medicines and are appropriately part of doing clinical trials, clinical studies. And then we work directly with provider communities who oftentimes are seeing the value of the work we're doing and making sure that for research purposes, we have appropriate access to data, information to conduct that research.Harry Glorikian: Yeah. I want to get into, you know, I think we're going to, I'm going to hit on some of that later, but I just want to make sure everybody's sort of on a level playing field with some of these wonky terms we use. How do you define real-world data and real-world evidence. I mean, I know what the FDA defines it as. I’m just curious. Jeff Elton: Yeah. So yeah. And FDA does have some, they have some publications really there that came out at the end of 2018 that actually began to lay out a framework around that, which I would encourage folks to reference. It's actually a very well-written document.So real-world data is sort of what it sounds like. It's the data. Right. And You know, if you were a clinician, if you were sitting in a clinical care environment, you probably wouldn't be using the word real-world data because those are the data generated through your treatment of the patient. So clinicians sometimes actually kind of pause for a moment to say, what's real-world? It’s the things I'm doing. And in fact, you know, real-world data would be structured data in a structured field. It’s a lab value that may have come in from the laboratory information system or a drop down menu. Did they smoke or not? Which can be a fixed field in an EMR. All the way over to physician notes, to appended molecular diagnostic reports, to imaging interpretation reports.So all those are forms of data. Now, evidence is a little bit about also what it would sound like. Data are not evidence. You have to actually, and in fact, to generate evidence, I want to have to trust the data. I have to believe those data are an accurate reflection of the source systems they came from. I have to believe they're representative or appropriate for the question that I'm actually trying to address. And then I have to make sure that the methodologies I'm using to analyze something, either comparing the effectiveness of two drugs relative to each other, actually then when I look at that analysis, I'm willing to either make a regulatory decision or a guideline modification.And the intent of evidence is either to support a regulatory decision or something that can inform practice of medicine or nature of treatment. So there's a bar, right, that one has to achieve to actually become evidence. But I think evidence is the right goal by what we're trying to do.Harry Glorikian: So you know, in the past, I mean, because I've, worked with companies like Evidation Health and so forth right there, some of this data was in paper form, right. Not in electronic form. So, what holes in the current system of, say, drug development would better real-world data or real world evidence help fill or, or drive forward.Jeff Elton: Yeah, that's a super good question. And, you know, Harry, you were kind of going back to your, I mean, you were one of the primary, leading individuals around that when the days of personalized and individualized and precision medicine, and even some of molecular medicine kind of came around. In fact, that's probably where you are my first point of interaction.And I come back to that concept because when you, when you're looking at data—and again, not all data are kind of created equal here—when I think about setting up and designing a clinical study, so now I'm with an experimental therapeutic or I'm thinking about moving it in. If it worked in one solid tumor and I suspect that same molecular pathway or kind of disease mechanism may be at work in another one. And so I want to kind of think about doing a pan tumor strategy or something of that nature. When I actually, when I, if I can bring together molecular diagnostic information, aspects of the individual patients, but do it at scale and understand the homogeneity, the heterogeneity and the different characteristics in there, I can design my trials differently and I can make my trials more precise. And the more precise the trials are, the higher the likelihood that I'm going to get meaningful outcomes. The outcomes here that are meaningful is what actually helps medicines progress. It's actually getting those questions to be as narrow and as precise and as declarative in their outcomes as possible.And so a lot of these data can actually be used to help guide that study design. Now, if I also have very rare cancers or very rare diseases—so this would apply even outside of oncology, although most of our work is oncology related—even if I'm outside of that, if I'm in very rare, oftentimes finding, you know, putting a patient on a standard of care therapy as a control oftentimes may not be in the patient's best interests. And so this notion of either a single arm or having an external control or having a real-world evidence support package, as part of that, may be part of what can occur between the sponsor and actually the FDA, et cetera, for kind of moving that through.But, you know, this has to be done individually around the individual program and the program and the characteristics have to kind of merit that, but these are big deals. So we feel that these are forms of data that can complement what would have been traditional legacy approaches to give more confidence in the decisions being made in the evaluation, the ones actually coming, too.Harry Glorikian: Yeah, I can hardly wait. I mean, maybe it's a dream, but I can hardly wait until we get rid of first-line and second-line and we just say, okay, look, here's a battery of assays or whatever. This is what you should be taking. No more first line or second line. I mean, these are sort of in my mind, I mean, almost arcane concepts from, because we didn't have the tools in the past and now we're starting to move in that direction.Jeff Elton: Yeah. So, Harry, just to, maybe to build on that a little bit. So if you look at some of our publications and things that we presented at this last ASCO, there's work one can do when you look at different features of patient response, et cetera. We're a company, but we also have a very strong data science backbone to what we do. And AI and ML applications. There are features that sometimes you can predict metastatic status. You can predict rate of response. You can predict progression. Now the very fact that I can make that statement kind of indicates that as you started thinking about the paradigm in the future, particularly when I start doing it liquid tumor, biopsies and surveillance mechanisms where I can see response much more rapidly in less invasive ways, you are going to start even over the course of this next five years, I think some of these will start to start influencing practice patterns in some very positive ways for patients, Harry.Harry Glorikian: From your lips to his or her ears. It needs to move faster. But, but it's interesting, right? I feel like you've been on this path for quite some time, like, I want to say since you're at least since your book in 2016, if not before. Jeff Elton: Yeah. So, you know yeah, you and I, in fact, you and I interacted first, I think we were kind of in the hallways, first interaction of what had been the Necco candy factory on Massachusetts Avenue in the Novartis building, where I was working in the Novartis Institute for Biomedical Research at the time.And Even prior to that, I think I did my first work back in the days of Millennium Pharmaceutical when it was still a standalone company, doing work in precision medicine and personalized medicine all the way through. And obviously Novartis's strategy was looking at pathway biology and actually using that as the basis of actually understanding where in a pathway system one could actually target and actually understanding that it is a system, it's got redundancy both in a bad, in a positive way. How do we use it to progress new medicines? So there's been an aspect of this that's always been kind of a little bit hard. I think I kind of made a decision to kind of pivot much more to a large scale data-centric, insight-technology-centric approach, and actually at scale, bring some of that back to the biomedical innovators. But yeah, it's been a progression over time and some of this it's a field that I feel, you know, strong passion around and will stay committed to for the duration of whatever my professional career looks like.Harry Glorikian: So can you give us maybe an example? I mean, I know some of it may be confidential. How does the data that you're providing, say, improve maybe drug safety or effectiveness? Jeff Elton: So you know, we're doing a project right now that that's safety related and I'll kind of try to keep it such that it I'm not betraying anybody's confidence. Eventually this will be in a publication, but it's not at the point yet. We're looking at a subpopulation that had severe adverse events, cardiac adverse events in the population. And originally the hypothesis was, it was a relatively homogeneous group. And we brought together some of our deepest clinical data, which means we have many different features of intermediate measures of disease, recurrence, progression, response, adverse events, severe adverse events. And we also brought some of our data science and AI solutions to it. And one of the major insights that came out of that is actually it wasn't a single homogeneous group. One group was characterized by having a series of co-morbidities that then linked to this significant adverse event and the other were purely immunological based.And so therefore actually in both cases, they're screenable, they're predictable. They're surveillable. And monitorable. And so therefore, but the actions would be very different if you didn't know what the two groups are. So in this particular case, we could discriminate that now. Well, we'll take that into more classical biostatistical analysis and do some confirmatory work on that, but that has significant implications on how you're going to kind of screen a patient survey of patients, look for whether or not they exhibit that area, and how you would kind of handle it, manage that. That would improve the outcome significantly of that subpopulation.So that's one example. In other areas, some of our data was actually being used as part of a regulatory submission. It was a very, very rare population in lung cancer. And it was unclear exactly how nonresponsive they were to the full range of current standard of care. And we were actually illustrating that there was almost a complete non-response to all current medicines that were actually used against this particular molecular target because of a sub mutation. And that actually was part of the regulatory submission. And that program both actually got breakthrough designation status, and that actually supported that and actually got an approval ahead of the PDUFA date. So when you start pulling some of these pieces together, they work to again, provide more confidence and interpretation and more confidence in decision-making. And in this particular case, certainly accelerated medicines being available to patients. Harry Glorikian: Oh yeah. Yeah. Drive value for patients and drive value for the people that are using the, the capability to get the product through. So, you know, we're talking about data, data, data. At some point, you've got to turn this into a product or a service of some sort or, or some, or maybe a SaaS as, as, as you guys might look at it, but you've got something called, you know, Eureka Health, right, in your product lineup. Can you give us an idea of what that is? I think it's a cloud-based SaaS product. You call it research-ready real-world data. So I'm just curious how that works. Jeff Elton: Yeah. So we do think.. So if you think about what we're trying to do, we're trying to allow a level of scale and a level of precision and depth on demand in the hands of individual researchers, from translational scientists, folks in clinical development, post-approval medical value and access. Kind of in that domain. And so each of those have different use cases. Each of those have different kind of demands that they'll place on data and technology for kind of doing that.We’re trying to move away from the world of bespokeness, because by nature of bespokeness, the question has its own orientation. The data is just unique to the question and that utility later is very low and, you know, in a way, what we'd rather do, what have we learned about what actually kind of create utility out of data, and let's make sure that we're covering the use cases of interest, but let's do it at very large scale. And that scale itself and the data we even represent at that very large scale is in itself representative and actually has significance whether it's on a prevalence basis of sub cohorts of disease or not. Now, the reason why I'm spending so much time developing that is when you put that in the hands of the right people, you're avoiding bias, but you're also giving utility at the same time and so you're actually improving their ability to conduct rapid question interrogation, but also structure really good research questions and have the discipline if I have a good research methods right around that. So we do structure those as products.And so, so actually one of the things we think of is, the work that we do in non-small cell lung cancer is an extremely large data set. It also has high depth on the molecular basis of non-small cell lung cancer. And it's created in a way that actually allows you to make those questions from translational through post-approval medical and doing that.Eureka is the technical environment. It is a cloud environment we are working in, and it actually allows you to do on-the-fly actually insights. So, outcome curves, which are called Kaplan-Meier and a few other measures. I can compare groups. I can compare cohorts. I can ask questions. It's actually exceptionally fast.And so this ability to navigate through a series of questions, its ability to make comparisons of alternative groups of patients on different classes of questions and finally get down to the patient cohort of interest that you may want to move into in the next phase, your research is done a lot faster. Now we took that, and now we're integrating more AI and ML into that. So we now have created probably what's one of the leading solutions for doing clinical study design. So we can optimize different features of that study design. We can actually release lab values. We can change parameters. There's a level of kind of fitness, ECOG scoring. We can actually modify that and show what the changes would be in the addressable patient population, and actually optimize that study design all the way down to the base activity level. And we're basically creating a digital object that's rooted on huge amounts of data. Underneath the 4.5 million records runs inside that particular area.There is no other solution in oncology, hematology that gets anywhere to that depth of information that can reflect, with different optimization, to the endpoint and even reflect statistical power. Now we're integrating in work around health disparities. How do you assure that if it's a disease like multiple myeloma, which may disproportionately affect black Americans, that I'm actually getting adequate representation of the groups that in fact, actually may be afflicted by the disease and actually assure the design of the study itself assures their representativeness actually in that work?Harry Glorikian: This dataset, what are some of the features of it? What is it? What sort of information does it have in it that you would be pulling from? Because my brain is like going on all sorts of levels that you would pull from, and some of it is incredibly messy.Jeff Elton: Yeah. So you are absolutely right. And so there have been expressions in the field of people who do work in real-world data that the real world's messy you know, fields may be empty. Do you know, as an empty field, because nothing got put there where's the empty field, because in that electronic medical record environment empty means it was not true of the state of the patient. That may sound like a nuanced thing, but sometimes empty actually is a value and sometimes empty is empty. And so you start getting into some things like that, which you start thinking about, like, those are pretty nuanced questions, but they all have to do with, if you don't know which it is, you don't know how to treat and move the data through.So back to your question here a little bit. What we actually, the sources of where we bring data from are portions of a clinical record. So, you know, we work under businesses, the work we do is either research- or quality-of-care-focused. And so, you know, we work actually, whether it's with the American Society of Clinical Oncology and et cetera, appropriately under all HIPAA guidelines and rules for how you interact with data around doing that. So I'll put that as a caveat because methods and how you do that security and everything else is super, super important. We have a clinical workforce. These are all credentialed people. Most of them have active clinical credentials. Most of them were in the clinic 10 to 15 years and even still interact on it. So a lot of my people feel they're still in clinical care. It's just happens to be a digital representation pf the individuals that are in there. And we're seeing, whether it's features of notes, depth of the molecular diagnostic information, radiologically acquired images that may show how the tumor progressed, regressed, et cetera, that's in there, any other, the medications, prior treatment history, comorbidities that may confound, actually, response. So all those different features are brought together, but if you don't bring it together consistently, we have tens of thousands of lines of business rules, concepts, and models that we try to publish around about how you bring a concept forward.So if you want to bring a concept forward, want to do it consistently, we come out of 10 different electronic medical record environments, and we're, we're actually interacting with the work of 1,100 medical oncologists and hematologists, et cetera. You have a lot of heterogeneity. Handle that heterogeneity with a clinical informatics team into a set of rules as it's coming forward so that everything comes to the point that you can have confidence in that, you know, in that particular analysis and that presentation.So there's something called abstraction, which is a term applied to unstructured data—and unstructured just means a machine can't read it on the fly. And so we're actually interacting with that, which could have a PDF document or something else. And from that, we use the business rules to then develop something that now is machine-readable, but actually has a definition behind it that one can trust, that one can, that kind of comes from some published basis about why did you create that variable? So I could measure outcomes of interest progression-free survival, adverse events, severe, whatever the feature of interests can. Help me answer the question we try to kind of bring through. So we're usually creating about 120 unique variables that never would have been machine-readable, in addition to the hundred, that probably were machine-readable when we bring that together. Harry Glorikian: So you're using a rule-based AI system, maybe not just a straight natural language processing system, to parse the words.Jeff Elton: Yeah. So natural language processing gets a little tricky. We do. We have, actually, excellent natural language processing. We'll sometimes use that for pre-processing, but you have to be careful with natural language processing. If it has context sensitivity, and if you're parsing for sets of reliable terms, it can actually be relatively accurate. If I'm doing something like a laboratory report that's so discreet, so finite, and it's so finite with how many alternatives you have with the same concept, it works really well. When you start getting into things that are much more nuanced, you actually start to have a combination of technology with the expert humans to actually have confidence in the ultimate outcome.Now we do have some very sophisticated AI models. Like I’ll give you an example. When you're looking at a medical record, usually metastatic status has just done a point of first but diagnosis in cancer care. So if the patient actually progressed and they made through there that they don't update the electronic medical record because they want to maintain what the starting point was when therapy was administered.But a biomedical researcher wants to know it at a point in time. So we have models that can literally read the record and bring back that status at any point in the time of disease progression. Now, would that work up to the grade of, say, for regulatory submission? No, but for a rapid analysis to pull back your question of interest and have it done in minutes, as opposed to weeks or months it works exceptionally well.Harry Glorikian: Understood. Understood. So now you and I both know that clinical trials, you know, are available only to a certain portion of the population really participate for a whole bunch of reasons. And then if you go down to sort of, you know, equality or, or across, you know, the socioeconomic scale, it, it gets even, it gets pretty thin, right? You guys, I, I think you've been pushing around inequality and cancer care and you have this program called ERACE which I think stands for Engaging Research to Achieve Clinical Care Equality. So help me out here. What is that? Jeff Elton: So we are, as an organization we're super privileged to have a very, very diverse workforce. And you know, men, women all forms of background races, ethnicities, and we really value that. And we've tried very hard to build that in our scientific committee. And I think when the public discourse around kind of equity, diversity, inclusiveness came forward, and you know, as you know, Harry, this has been a unprecedented period of time for just about anything, any of us. I mean, COVID-19 and social issues. You know, things of that nature. It's, it's really been a very, very unprecedented time in terms of how we work and how we interact and the questions.Our organization and our scientists actually came forward to me and said, you know Jeff, we have a tremendous amount of data. We have partners like American Society of Clinical Oncology and some of the leading biopharmaceutical researchers in the world. And we've got technology, et cetera. We want relevance. We really want what to make contributions back and we believe that actually, we can do some research that no one else can do. And we can actually begin to deliver insights that no one has the capability to do. Would you kind of support us in doing that? And so we put together the ERACE program and it actually was named by a couple of our internal scientists.And the program actually now is being collaboratively done. We've done a couple of webinars, with you know, some of our partners and that's included, you know, folks from, whether it's AstraZeneca, Janssen, and BMS, et cetera. It's become something around, how can we rethink how research takes place and actually assure its representativeness for all groups, but particularly in specific diseases. It impacts different groups differently. And so can we make sure it reflects that? Would we be generating the evidence so that they can in fact be appropriate beneficiaries earlier? And a lot of this came from when we looked at aspects of diagnostic activity we could say that, you know, black American women have a higher incidence of triple negative breast cancer and a few other diseases. When we look at patterns of diagnosis and activity, unfortunately, the evidence that we even have is not substantially in the practice of what we're actually seeing sometimes when we begin reviewing our data. And so we began confederating through our own work. We now have actually set up research funding. So we actually now will fund researchers who come in the academic community. If they come up with research proposals that have to do with, you know, health related disparities, whether it's economically based, or if it's racial, ethnically based. Those questions. We've got an external review board on those proposals. We'll provide them data technology and financial support to get that research done. We're doing it with our own group and we're doing it collaboratively with our own kind of biopharma sponsor partners kind of as well. So for us right now, it's about confederating an ecosystem, it's about building it into the fabric about how research questions are framed, research is conducted, clinical trials are conducted, and then actually those insights put into clinical practice for the benefit of all those groups. And so, you know, it's even changing where we get our data from now. So it's, it's like an integral part of how of everything we do. Harry Glorikian: So you saw, I don't want to say an immediate benefit, fooking at it this way or bringing this on, but I mean, you must have seen within a short period of time, the benefit of, of, I don't want to say broadening the lens, but I can't think of a better way to frame it. Jeff Elton: We were surprised how quickly, whether it was academic groups or others, rallied around some of the concepts and the notions. And we were surprised how quickly we were able to make progress in some of our own research questions. And we were pleased and astonished, only in the best ways, that we saw industry and biomedical research, the whole biomedical community, attempting to integrate into their research and the questions that they asked actually different ways of approaching that.And in fact, it's probably one of the most heartening areas. You couldn't have legislated this as quickly as I believe leading industry biomedical innovators decided it was time to kind of change portions of the research model. And you made a, Harry, you made a statement earlier on that. It's not just about kind of us analyzing data. Sometimes bow you find that to broaden actual, say, clinical trial participation, I actually have to go to sites that historically didn't conduct clinical trials. I may need to have investigators that are trusted, because some of the populations we may want to interact with don't trust clinical research and have a long history about why they didn't trust clinical research.So you're changing a social paradigm. You're changing research locations and capacity and capability for that research. So we're now moving research capacity out into community settings in specific communities with this idea that we actually, we actually need to bring the infrastructure to the people and not assume again, that people want to kind of go to where the research historically was conducted because that wasn't working before, you know? Harry Glorikian: At some point, you turn the crank enough, you start to influence, you should be able to influence, you know, standard of care and all that stuff, because if you're missing data in different places, you’ve got to make sure that we fill these holes. Otherwise we're never going to be able to diagnose and then treat appropriately.Jeff Elton: Generate the evidence that supports actually doing that and do it on an accelerated basis, but also that it gets confidence for those decisions. Absolutely. That's part of our goal. Harry Glorikian: Yeah. So I want to jump back in time here and sort of go back to your your Healthcare Disrupted book. You know, I feel like, you know, we're on the same page because I think the message was, you know, pharma, devices, diagnostics, healthcare, they need to rethink their business model to respond to this digital transformation, you know, which is obviously something in my own heart. I've been sort of banging that drum for quite some time.In particular, you argued in the book that real-world data from EMRs, wearables, the Internet of Things could be combined to change how and where healthcare is delivered. Is there a way in which like Concert AI's mission reflects the message of your book? Can I make that leap?Jeff Elton: I appreciate the way you asked the question and I think if you said our principles and perspectives about that, we need to kind of focus on value and outcomes, and then we're going to be bringing insights, digital cloud, and a variety of other tools to underpin how we work and operate. Absolutely.And in fact, I think, you know, positively. I had a lot of engagement and did a lot of interviews, even as we were putting the book together, which took place over a couple of months ago, it was probably, you've done your own books. Whatever you think it's going to be, it's a lot longer. So I'll leave it at that. I have recovered from the process now, but I think we had a lot of engagement, whether it was with medical community, biopharma, leadership, community, et cetera. And I think that alignment is some of the alignment we have with our partners today. It's actually around some of the same principles.What I couldn't have predicted, in fact, I was a couple of years ago and this probably would have been towards the tail end of 2019, I was already starting to think about, okay, I've recovered from the first writing. How did I do? And what would I say now? And at the time I was beginning to say certain things seem to be taking shape slightly more slowly than I originally forecast, but then COVID-19 happened. And all of a sudden certain things that we kind of had thought about and kind of had put there actually accelerated. And in fact, I think, you know, out of adversity, you'd like to say we bring sources of strength we didn't know we would kind of be beneficiaries of. But out of that, you could argue this concept of say a decentralized trial activity.So we have, let me pick up, you know, I'm one company, but let me pick a parallel company that I have respect for, say, Medable as an example, and Michelle [Longmire] leads that company, it does a very nice job, but that's the idea. Everything could be done remotely. I can actually do a device cloud around the individual. I can do a data collection and run RCT-grade trial activity. Now that doesn't work super well in oncology, hematology, et cetera, where I'm, you know, I'm doing chemo infusion and I have to do very close surveillance, but that concept is an accelerated version and got broader adoption and actually was part of some of the COVID-19 kind of clinical studies and capability. And it's not going to revert back. So actually what happens is you find it has a level of efficiency, a level of effectiveness and a level of inclusiveness that wasn't available before, when it had to do facilities-based only. Now we ourselves now we're asked to accelerate, we bring technologies and integrate them into provider settings for doing retrospective analysis. But actually during that period, not only did we bring our clinical study design tools and use AI and ML for doing that, which led to, we've supported the restart of many oncology studies now, and actually the redesign of studies to be able to move into different settings that they never were in before.And actually now we're beginning to use some of our same approaches for running prospective studies, but from clinically only derived data sources. It’s a very different paradigm about how you conduct clinical research. So when you think about this, there are unpredictable shocks, you know, which, you know, some of may have called Black Swan events or whatever you may ascribe to it, that actually are now consistent with everything we did. But actually accelerating it and in a weird way back on trajectory, if you will. But I think, yes, everything we're doing was informed by a lot of that seminal work and research and foundation about what worked in health system and didn't how are people being beneficiaries or not? How do we need to change how we do discovery translational clinical development? And we're very committed to doing that. Harry Glorikian: Yeah. I mean, it's interesting cause you almost answer my next two questions. I'm really hoping it doesn't slide backwards. That's one of my biggest fears is, you know, people like to revert back to what they were used to.Jeff Elton: But you know, maybe to encourage you and me. So one of the things, if you take a, let's take a look at a teleconsult. So during COVID-19, HHS opened up and allowed as a coded event, doing a digital teleconsult for kind of digital medicine, telemedicine, and that was put into place on an emergency basis by HHS. And then before the outgoing HHS had that, it's now made permanent. And it's now part of the code that actually will continue to actually be a reimbursable event for clinicians. That was actually super important during COVID-19. What’s not that well known is, not only did that allow people to be seen, but hospital systems were really financially distressed because most of their work was informed by kind of, you know, elective procedures and things of that nature. And that couldn't take place. But the teleconsult became a very important part of their even having economic viability, which you can't underestimate the importance of that during a pandemic. Right. So now that's part of how we're going to work. My personal view is, now that people are using digitally screening tools, they have decentralized trials, some of the solutions that we're putting into place, AI-based, bringing RWE as part of a regulatory submission, I don't see anything going back. And the work we're doing is if we can start putting 30 to 50% time and cost improvements and add more evidence around a decision, more robustly than we did before, that's not going backwards at all.Harry Glorikian: Good. That's that makes me. I'm hoping that we're all right, because we've been saying this and beating this drum for quite some time.It's interesting, right? Because I don't think I've gotten over the whole writing thing because I've got a new book coming out in the fall. So you know, I, I couldn't help myself. I hope, you know, we. We're able to give the listeners sort of a view of where this whole world is changing, how data's changing it.I mean, I've had the pleasure of talking to people about digital twins and that sort of data. And I believe that this, we're gonna be able to make predictions, as you say off this data almost proactively. It's interesting because I do talk to some people who are in the field that look at me strange when I say that, but after working with different forms of data in different places for so long, I can see how you can look at things predictively and sort of, you know, decide what's, you know, see what's going to happen almost before it happens for the most part, if you have a big enough data set. Jeff Elton: So we do a lot of prediction thing in the AI and ML world. And we predict, you can actually be relatively accurate on who's going to adhere and not adhere. You can begin to look at the biological response to being placed on a new therapy and understand whether that response is kind of in a direction that, that patient's going to remain on that therapy, or you need to discontinue to be placed on a new therapy.And you're right. And in fact, some of these features…well, the question, we use it from generating insights to design and hopefully improve outcomes, et cetera. That's a rapid process. I mean, I've seen things in the last three years in setting up Concert AI that would have taken me a decade to have seen in previous methods. But we're still not as fast and as effective as we can be.And the very fact that I can in my digital laboratory, if you will, create AI/ML to predict whether that patient is going to be discontinued or continue on to that course of therapy. Some of that needs to be brought into confidence tools that can start to inform parts of practice as well. They're not ready for that. They have to ascend to that. But when you look at these, some of these, whether it's coming in as software, as a medical device, sets and solutions to augment, are going to add a huge, huge amount of utility. And you're finding a lot of interest, even biomedical innovators are looking for predictive tools, too, complement their medicines.And you know, we're doing a couple of things that would be definitely considered in a more confidential area around doing that right now. And I have to tell you I've been so pleased and it's just for me, it's so, so catalyzing of our energy to be brought into this, to see people willing to reshape the paradigm about how they do things that actually will reshape how medicine's delivered and care provided too. Harry Glorikian: Oh yeah. I mean, look, ideally, right, I think every physician wants to give the patient the optimal therapy. Not pick the wrong one and have to redo it again. But, but I think a lot of these tools are also gonna lend themselves to adjudication.Jeff Elton: Absolutely. Harry Glorikian: Right? And that is a huge paradigm shift for everybody to wrap their head around. And I think we're going to get pushback from some people, but I can't see how you don't end up there at some point. You can see where it's going. You know, what's going to work, here's the drug. And if it doesn't work, here's the data to show [why] it didn't work.Jeff Elton: Well, and actually and Harry, to your point, right now you're thinking about how payers authorized the treatment that's proposed by our clinician for super expensive medicines. Right? But if I'm an oncology, I can tell you right now that claims data as a single data source can't tell you much about whether that patient responds, whether they're being treated according to NCCN ASCO guidelines or not. So you're wondering what's the basis of that. Whereas I can actually look at the data and I can understand how that patient presents and I can see what's actually the intended treatment. And you can immediately say that perfectly makes sense, given how everything's matched up and I can continue to kind of say what that response is it consistent with what I would have hoped for placed in that patient on that specific treatment. So to your point, this is going to change all sorts of things.Harry Glorikian: I love it when it changes on that level, it just makes me all happy inside. So, Jeff, it was great catching up with you. I hope when this pandemic is open, we can get together in person and you know, have a beer. Maybe we'll even bring Arshad because I think he's been working in this whole data area with a number of companies for a while now. Jeff Elton: Yeah. Would love it.Harry Glorikian: Excellent. Jeff Elton: All right. Harry Glorikian: Thank you.Jeff Elton: Thank you too.Harry Glorikian: That’s it for this week’s show. You can find past episodes of MoneyBall Medicine at my website, glorikian.com, under the tab “Podcast.” And you can follow me on Twitter at hglorikian. Thanks for listening, and we’ll be back soon with our next interview.