Are Palliative Care Providers Better Prognosticators? A Podcast with Bob Gramling

Eric: Welcome to the GeriPal Podcast! This is Eric Widera.

Alex: This is Alex Smith.

Eric: And Alex, somebody else is in our room with us.

Alex: We have another host with us today who’s been a frequent host and guest on this program. This is Anne Kelly, who’s a social worker here on our palliative care service. Welcome back.

Anne: Hi guys. Happy to be here.

Eric: And Alex, who is on Skype with us?

Alex: We have a dear friend from the other side of the country, University of Vermont. This is Bob Gramling, who is the Holly and Bob Miller Chair of Palliative Medicine at the University of Vermont. Palliative care physician, ethicist, really smart guy, outdoorsy guy. Welcome to the GeriPal Podcast, Bob.

Bob: A smart dresser, too, Alex, you were saying, right? You get that in?

Eric: This is on YouTube, so you can go to the GeriPal YouTube channel and see Bob’s smart dressing.

Alex: Midway through, Bob may pull out a sword and do a martial arts demonstration.

Eric: But we’re not here to talk about martial arts. We’re here to talk about Bob’s paper in Journal of Pain and Symptom Management titled, “Palliative Care Clinician Overestimation of Survival in Advanced Cancer: Disparities and Association with End of Life Care”.

Alex: What? We’re perfect prognosticators.

Eric: We are.

Alex: No. No way. I’m perfect.

Eric: Let’s pause. We’ll talk about that. But before we start going into this JPSM paper, Bob, do you want to give Alex a song he can lead us in with?

Bob: Ooh, Alex, I was going to maybe Big Yellow Taxi by Joni Mitchell? The world needs a little bit more Joni right about now.

Alex: Yeah. Is that why you chose that song?

Bob: That is. Absolutely.

Alex: All right, here we go.

Alex: [Singing].

Eric: Isn’t there like an ooh-bop-bop?

Alex: Yeah, that was your part!

Eric: Ooh-bop-bop?

Alex: Ooh-bop-bop?

Anne: At the end of the episode…

Alex: At the end.

Eric: Put me in, Coach, I’m ready.

Bob: Thank you, Alex. That was beautiful.

Alex: Thanks, Bob. Good request.

Eric: So, let’s talk about this article. Prognostication and whether or not palliative care providers are any better than past studies that show we’re not very good at prognostication. But before we do that, how did you get interested in this as a subject?

Bob: I think most of my career, I’ve been fascinated by concepts of predicting the future. Before I was a palliative care researcher, I did a lot with risk communication and predicting the onset or the risk for serious illnesses.

I think this concept of uncertainty, I love the philosophy of what prediction might mean. The future that has not happened yet. And how that impacts our well being, our perceptions of self, and our decisions in the context that we are in our medical world.

Bob: I’ve been pulling this string for a while, since probably the late ’90s, early 2000s in different contexts. I just find it super fun.

Alex: Clinically speaking, this is such an important question, because so many of our patients’ decisions hinge on prognosis. Near the end of life. I know we’ve done some work about prognosis and prognosis communication further out, on the order of years and decades.

Alex: But the kind of prognosis you’re interested in here is relevant to both geriatricians and palliative medicine providers. Probably even more the palliative medicine providers, although both. Maybe even primary care docs, too.

Sorry, that wasn’t a question.

Bob: The problem with getting a bunch of palliative care people in a room, we can all tolerate silence.

Eric: Silence.

Bob: For long periods.

Alex: That’s right.

Eric: I was waiting for Alex to finish it off with the question.

Alex: Yeah, yeah, yeah.

Bob: Tell me more, Alex.

Alex: Who does this study like this appeal to, I guess is the question.

Bob: For the audience for the paper?

Alex: Yeah, yeah, yeah.

Bob: … or the implication?

Alex: Yeah.

Bob: Well, I think for the audience, they’re for those who might be trying to help us scale palliative care services to larger contexts. One of the things I’m interested in … not just with prognostication specifically … but understanding what maybe some of the active ingredients about what we do. Despite the fact that we have a fair amount of observations in the literature about the frequency of overestimation, there really wasn’t too much out there about the empirical implications of that. Or at least the association with what decisions may unfold and what end of life might look like.

This paper, one of the things we were hoping to do is to identify whether there was any potential link between how frequently we overestimated someone’s survival, and the end of life course they might experience.

Alex: Right.

Bob: Manifest primarily by whether they enrolled in hospice or not, is one of the features we were able to measure.

Alex: Great. Before we get to the study, we just another setup question. In general, outside of palliative medicine, how good are doctors or other health professionals at prognostication?

Bob: Well, I think in defining “good,” if we separate out estimating what we would expect the future to look like, then how well we communicate it, I’ll stick just on the estimating piece for cleanliness of that answer.

Bob: I think we typically as human beings err on the more optimistic side. That’s what most of the literature around this in this field would suggest. Although not always.

Alex: Right. My recollection is Nicholas Christakis did a systematic review of physicians’ estimates of prognosis and actual survival. Physicians tend to overestimate prognosis by as much as a factor of five. Oddly, the longer that patient-

Bob: Yeah, we’re a pretty optimistic bunch.

Alex: We’re pretty optimistic.

Bob: We don’t like thinking of this mortality thing.

Alex: Yeah, but not palliative care docs, because we’ve been trained.

Bob: Oh, we are.

Alex: We are experts in the [crosstalk 00:09:26]-

Bob: … optimistic happy people, Alex.

Alex: Anne, you work with palliative care docs [inaudible 00:09:29].

Anne: You guys are always right.

Bob: Wow.

Anne: Is that how I keep my job here?

Bob: Gosh, that’s quite a burden if that was true.

Eric: What’s the old saying? Prediction is really difficult, especially about the future? Kind of something like that?

Bob: Absolutely. That was Niels Bohr, wasn’t it?

Eric: Niels Bohr. Everybody says Yogi Berra. Yeah, but Niels Bohr.

Alex: Yogi Berra gets credit for every smart saying out there.

Bob: It’s getting earlier later out there.

Eric: I’d love to talk a little bit about what you did in this article, and what you found. But maybe then also afterwards, talk a little bit about this issue of uncertainty, and how we deal with it in our field, and how we should think about it. Let’s set up the article. Real quickly, in a nutshell, how did you set up this article? What did you do to figure out are palliative care providers … are they any good at this?

Bob: The piece, the study is a cohort study where it’s not an intervention to the extent that any research is not an intervention. We’re observing what naturally happens in palliative care consultations. This was at two medical centers. One that you’re very close to at the moment, and one that I’m somewhat close to at the moment. We’re trying to ask the question, fundamentally, is what is a good conversation? And that’s a large challenging question.

But in order to do so, we both observed and recorded the conversations with the palliative care team, and patients who had advanced cancer who were referred for consultation. Before the consultation, had a brief interview with patients. After consultation, had a brief interview with patients, at least the following day. And I mean brief. 20 questions. We really wanted to try to not over select for people who had more energy and more time. We wanted to get as best a sampling as we could of the people we see in the palliative care service.

Immediately after the clinician exited the room, one of our research observers asked three questions. One of which was this prognosis question, the estimation of survival. And we asked them under the assumption that we’re commonly faced with hospice decision making. If we assume a natural course of disease. What is the most likely time interval we expect this person to live?

The intervals we chose were the ones that at the sites, were relatively commonly asked of us, so it wasn’t a foreign concept. For example, less than 24 hours, less than two weeks, less than three months, less than six months, or longer than six months. Because those thresholds are often useful for our colleagues in case management and social work to think about what might be some potential locations for receiving hospice-type care, beyond whether they were eligible prognostically. That’s how we chose these windows, because they had a familiar clinical feel to them.

We did pilot-test the idea of doing the surprise question, or some version of the surprise question. But in our beta testing of that, palliative care physicians aren’t surprised by much. It didn’t lead to much distinction, and it was not a very useful measure for that reason. So we asked people to put their nickel down on these time intervals.

Then we watched how long people lived. We defined accuracy based on whether people died during the interval selected, overestimation if they died sooner, and underestimation if they died after.

Eric: Did you think, for me, there’s a difference between whether or not they survived zero to two weeks, or two to four weeks versus … I thought they had days or weeks, and then they’re still alive six months later. Did you look at that issue? It’s not, “Oh, were they a little bit off, but were they really off?”

Bob: Yeah, the magnitude of error?

Eric: Yeah.

Bob: Yeah, I can’t pull it up right now, but I’m happy to … if your blog post posts some follow-up, I can give you the percentages and how far off people were. Most of the time it was one, maybe two categories off. But the problem is, in this measure, the categories get wider. So you have a much wider opportunity for accuracy the longer out you go.

Eric: Yeah.

Bob: Magnitude gets a little hard to judge in a precise sort of way.

Alex: Let me ask that again. How many clinicians did you have, and how many patients did you have in your study?

Bob: In the study we had 113 clinicians, about half of which were learners rotating with us. We had 54 palliative care clinicians, broken down between the most common discipline was attending physician, followed by fellow physician, and then nurse practitioner, and then nurse chaplain/social worker.

Bob: We had 240 patients, 231 of which had a estimate of survival by the clinician; nine that didn’t, were either died or discharged prior to the consultation happening.

Eric: All right, so what did you find in this article? What were the results?

Bob: We found that palliative care clinicians were really good at distinguishing who’s going to live longer and who’s going to die sooner, relative to one another. Actually, a quite striking pattern. It’s rare to see how well people can predict in a stratified way.

Bob: What we did find is in every category, we overestimated. And I say “we” because I was one of the clinicians in the study.

Alex: So Bob, how often overall, were the clinicians right?

Bob: Alex, it was a little over 4 in 10.

Alex: A little over 4 in 10. So that’s … is that-

Eric: Wait, that means 6 in 10 times we’re wrong?

Bob: 6 in 10 times we’re wrong.

Eric: That does not sound good. That sounds like we make a lot of errors. You’re saying most of those times we’re making errors, we’re overestimating the prognosis, right? This is not an underestimating of prognosis.

Bob: Yeah, of all of the errors we make, we’re about eight times more likely to overestimate than to underestimate.

Alex: Overestimate than underestimate-

Bob: In this study.

Alex: In this study. And we were, do we do better at shorter-term prognosis, or longer-term prognosis?

Bob: We get better at the challenge of answering the question, though I wanted to look at this the specific categories, Alex, is that we’re better earlier, at estimating survival. The problem is our last the longest category is just greater than six months. The call is a little bit easier there, on how long someone might live. So the overestimate does increase up until the three-month category. But then it dips down again because we have such a large window that we’re looking at.

Alex: So I’m looking at this Table 2. In the Observed Accuracy by Clinician Estimate: Where the clinicians survival estimate was three months to fewer than six months, there were 81 patients in that group. 58 of them, or 72% were overestimates. 52% ended up being overestimates. The patient lived less than three months, when the clinicians thought they were going to live three to six months. That’s the group where we did the worst, by far.

I guess what that raises for me is, when you’re making these groups, what’s the clinical decision that comes up that’s like that, three-to-six-months versus less than three months boundary?

Bob: Oftentimes it’s depending on your healthcare environment, it tends to be eligibility for acute inpatient hospice care, versus relatively subacute inpatient hospice care, where I practiced during this study.

We would have comfort care homes, is what we referred to them as, that they’re two-bed, philanthropically given homes, and there were 28 of them that could accept patients with hospice care. Typically shooting for a window of survival under three months was the population that they were trying to serve.

In the acute hospice setting for symptoms requiring inpatient and survival of less than two weeks, people would access some of the inpatient hospice units in the city more easily. That two-week to three-month had an implication for what was potentially going to be offered for that person, as a place to receive hospice services, if home was not a viable option.

Eric: Were there enough subjects in the study to see did prognostication depend at all based if you were a fellow, or if you’re a physician, or if you’re a nurse practitioner? Did it vary at all, or could you even look at that?

Bob: Yeah, we have that in Table 1. If you’re turning to that, it has the frequency of overestimation. That is also due to the number of estimates each group made. So it wasn’t that each person made the same amount of estimates. Which makes this a little bit hard to compare apples and apples.

However, we did find that the attending physicians were the least likely to overestimate, followed by the fellows, then followed by the nurse practitioners.

Anne: When you were asking them to offer an estimated prognosis, was there any questioning around what information they were using to come up with their estimate? Or how their estimates were informed?

Bob: No, they were never asked to justify why. Sometimes if you had a learner rotating with you, sometimes this question generated some conversation. I know our research observers would sometimes see a collaborative discussion about this, thinking about what might be the factors going into this judgment.

Bob: But we weren’t asking them to write that down or to give us a exact sense of that.

Eric: They were asked after the consultation happened. Not the first day of consult, but when consult discontinued, or when exactly were they asked?

Bob: All of the conversations that were recorded were within the first day. They were the initial touch with the patients. We had up to three visits to get a, what might be considered the initial consultation with them. For example, the person you went to say hi, but then they were being taken down to radiation; that didn’t count.

So these estimates were done after really the first assessment of the person when we met them.

Alex: That’s an important consideration, because I’m not perfect the first time I see them. But by the time…

Eric: By the second time, Alex is perfect.

Anne: Oh, boy.

Alex: But you do wonder, because in that first visit, you’re just getting this one-time snapshot. If you have two visits, then you can see somewhat of something of a trajectory. You get three, four, five visits, and then you’re really seeing, it’s like a time-limited trial. How are things going? Are they going in this direction or that direction? Is function getting better or worse? Is delirium resolving or not resolving?

Eric: Or is it the longer we get to know someone, the worse we get?

Alex: Yeah, could be. That’d be an interesting follow-up study.

Bob: Then there’s the argument that if our prognostication got better, at least our prognosis estimates got better, then we might be underestimating the potential relationship between error and hospice use, for example, as an outcome. Because if this self-corrected over the coming days, it would actually mitigate some of the potential causal relationship with the outcome.

Eric: Tell us a little bit more about this outcome. What did you find around hospice use?

Bob: The people whose survival was overestimated by the clinician had about between 2 and 2.5 times less likely to enroll in hospice before they died. About a similar estimate for enrolling in hospice for at least 72 hours before they died.

Eric: So this had potentially real clinical significance.

Bob: Yean, potentially, absolutely.

Alex: If it’s causal.

Bob: It’s causal.

Alex: Did you get any assessment of the patients’ estimates, or their loved ones’ estimates, of how long they might have to live?

Bob: It’s not presented in the paper, but yes, and it doesn’t have a large effect.

Alex: A large effect on hospice use?

Bob: On the estimate, it’s not a confounder; it doesn’t function as a confounder in this association.

Alex: Okay.

Bob: We do have a separate paper coming out about the potential factors influencing overestimation. Patient perceptions of their prognosis do have a potential impact there, as do personality types. Optimistic personality types do appear to be related to overestimation.

Alex: To physician overestimation.

Bob: Yeah, it’s a fascinating; the contagion of optimism is a little bit of what we’re looking at.

Alex: Oh, that is fascinating. So the physicians tend to be swayed by the patient’s perception of prognosis, and how optimistic the patient is about their future.

Bob: That’s what our estimation would suggest.

Eric: So I got another question. Thinking about this, and thinking about how when I see patients in the hospital, and if I was asked to give a prognosis to somebody, I can give a prognosis. But I also have varying degrees of certainty. Like my confidence around that prognosis. In some patients I may say, “Oh yeah, it’s going to be zero to two weeks.” But my confidence level is low.

Or another patient’s like, “Yeah. They’re gonna die in the next two weeks.” And my confidence level would be very high. Is that a potential flaw in this, is that we haven’t really assessed how confident are these clinicians in making these decisions? And does that vary as far as how good they are? Also, I think there is some data that around prognostication outside of our field, the more confident we are in our prognosis, the worse it is? What are your thoughts about that?

Bob: The trajectory of expertise as confidence continues to rise and accuracy tends to fall?

Eric: Yeah.

Bob: Well, I guess the first step is that at least just to acknowledge … and this is talking about estimations, it’s not talking about communication.

Bob: But if we follow that through with having a conversation with someone about how long we expect they might live … I just want to acknowledge with some people, that’s irrelevant information. If it’s not going to inform any of their decision making, if they don’t have prognosis-sensitive preferences. Then it may just be toxic information.

For some people, the bounds of uncertainty that might bother us are extraordinarily useful for their decision making. I think that what I find fascinating is that this finding an association, for me, does suggest the potential that our judgments do relate to how our conversations or the options that patients are presented may unfold in their end-of-life course.

But I don’t think it necessarily informs how and when we need to communicate prognosis, and how we do that. For example, I can be off by a category, but by the way that I’m talking about it, it may be sufficient for that human being to make the choices they need to make. Is that getting a little bit at what you’re suggesting, Eric?

Eric: Yeah, I think this is the reason I’m fascinated by this subject about prognostication and uncertainty. There’s so many different layers. Even when we break prognostication apart, there’s this initial estimation. Then there’s the communication of the prognosis. And then there’s the patient and family’s interpretation of that prognosis. I think the data would suggest that optimism on top of optimism on top of optimism bias for each one of those particular steps.

Bob: Agreed.

Alex: Yeah. So you also looked at differences between prognoses for minority patients versus white patients. What did you find there?

Bob: That for people who self identified as black or Latino, had a substantially higher risk of overestimation from their clinician, to the order of three to four.

Alex: Wow. So palliative care clinicians are more likely to overestimate for black and Latino patients, compared to white patients.

Bob: Yeah. In our study.

Eric: Why do you think that is?

Bob: Man, I wish I knew. One of the things we didn’t have, one of the weaknesses of this study, is that we weren’t able to look at concordant or discordant racially apparent pairs. Meaning our clinicians were all white, and self-identified as such.

One of the questions in the related work with how appearance may influence judgments, is that may be bilateral. So we can’t ask the question about whether the same phenomena existed for clinicians who may share a similar potential identity based on appearance?

But what I do wonder a little bit about is if … I guess I have to plead ignorance, Eric. I think that we’ve thought a lot about this, about why we think this might be. And I don’t have sufficient data or the types of questions in the study to be able to inform any of those guesses I might throw out there.

Eric: So there’s a lot of uncertainty.

Bob: The differences we’ve seen … What’s that?

Eric: So there’s a lot of uncertainty.

Bob: It is, but one of the things that’s intriguing to me is the question we do know the end-of-life course for people who self identify as black or Latino, often looks different. Whether that’s a disparity or whether that’s a difference, is certainly something that we haven’t had enough scientific clarity all the time.

But I think this adds to the fact that if the judgments are wrong, it’s more likely to lead to disparities than differences that are on warranted based on preferences.

Alex: So what’s the take-home message for practicing clinicians out there? Should we take our prognostic estimates and cut them in half? Or cut them by some fudge factor?

Bob: Well, I think probably the biggest thing for us … I think we do this anyway, and I think we’re reminded of it at all time, but bring the humility to these estimations, and know that where we are, we’re affected culturally, by similar factors that we’ve observed elsewhere. This study itself for me is, I’m more likely now to check my estimations more carefully with calculators, such as your ePrognosis, thank you very much. Also, with other colleagues, saying, “Am I nuts? Am I seeing something here that that you’re not seeing?” I find that to be helpful for me. Because these errors are not symmetric. Meaning for some people, an overestimation may not be a problem for them. For some, it may be a big problem.

I’m also thinking about the human being for whom this relates. Is an error in this direction going to lead to greater regret than an error in this direction? And try to bring that into how I de-bias my thinking.

Eric: Yeah, I often find it helpful if I give an estimate of prognosis and Anne looks at me, kind of cocks her head with a confused look in her eyes. I know I probably just messed up.

The couple of times we’ve actually tested this, by each of us putting our nickel down on prognosis, I think you got them much closer to right than both me and Alex. Would you agree, Alex?

Alex: Agree.

Anne: It’s sort of funny. I think that’s one of the things I was curious about around what goes into people forming these estimations. I can imagine that people coming from different vantage points might be using sometimes very often similar information, but also different.

Alex: It’s also different information. Yeah.

Anne: I’m wondering, for all three of you as medical providers, when you hear results like these, does it make you feel wary to offer an estimated prognosis? Or how does it contribute your thinking around delivering this kind of information to patients and families?

Bob: I’ll take a stab at that. It doesn’t dissuade me. I think I try to be humble in how I present it. But I also know culturally, this is the information that people avoid so much. That this is the elephant in the room, and that elephant can be a heck of a lot scarier if you’re not talking about it. So I’ll oftentimes start this thing. I’m just noticing that a lot of people are having a hard time thinking a little bit about how long your body might live with this. Is that important to you? Is that something you really want us to try to turn towards a little bit more? And some people say, “Nope. I’m fine. Day to day, and you’re doing your best. Totally get it.” Some will say, “Yeah, it really matters; I want to make it to for this.” I find myself comfortable to talk about it, and acknowledge that I’m doing my darnedest to be accurate, but you know what? We’re learning that we’re not so great at it as well.

Alex: Maybe as doctors, we’re not so great at it because we tend to focus on medical sorts of things. Palliative care clinicians, we tend to think, “Oh, we’re better than the primary medical team, the ICU team, who’s focusing on the sodium, the potassium. We’re seeing the patient.” But maybe we’re still influenced by some of those factors too, in a way that social workers aren’t.

I remember a previous social worker used to be very good at estimating prognosis on patients as well. Then there’s this story out of Brown, about that cat on the hospice unit.

Bob: The hospice cat?

Alex: Yeah, the hospice cat who would go and sit on the beds of patients who were dying. Just had some sort of sense about the patients who were dying. I think they were going to turn that to a book.

Eric: Yeah. I’m pretty sure that’s observation bias. Like, “Oh, look, the cat’s with this person. But the cat’s with everybody. We have a cat in our CLC, kind of the same thing, but the cat goes into everybody’s room. But you only notice when the cat’s in with the dying person.

Alex: I don’t know, I think there’s something there.

Eric: No.

Alex: You need to do another study on the cat?

Eric: I think for me, the key is that there’s so much more to learn. It helps me right my ship a little bit. But it doesn’t change my overall direction. But I also have to think about all of these other things that we can learn from other fields about risk and uncertainty and prediction.

How can we bring that into medicine, back into medicine, like we used to actually hold this as one of the most important parts of being a doctor is prognosis. And we’ve kind of lost that. How do we bring that back into the fold?

Even learning from other fields. Maybe having one person estimate prognosis. Like you’re okay, we’re still pretty good at estimating prognosis. There’s some difference in weeks or months. But pretty much, most people got weeks to months right.

But what if they combined different members of the team, and we combined those estimates together? Would it be any better? What if we included other things like family members in overestimation? Then that’s just the first part of prognostication. Then there’s the communication part. How do we deal with that? I love your statement of toxicity or information toxicity, when we give too much information. Like that one JAMA study of chronically critically ill patients, where everybody got prognosis, no matter what they wanted or not. There was higher rates of PTSD in those folks. It’s so complex.

Bob: Also, the introspection of to what extent of people’s perceptions of their own survival time; how frequently is that independently predicted? I find that quite fascinating, because there’s a related literature that suggests that it is, in ways that we can’t explain.

We may not want to rest too much on predictions that might seem really discordant with ours, but I find that somewhat interesting as well, about what people is going to happen in their bodies.

Eric: Great. Anything else you want to talk about, Bob, with us?

Bob: I’m appreciative of you taking the time, and that you’re interested in the topic, and I just … You took the time to talk with me today, and I really appreciate it.

It’s nice meeting you, and I know these other fellows. But it was really nice to meet you.

Anne: And you. Thank you.

Eric: Yeah, I always think about weather prediction. Just because sometimes we’re off with another weather prediction, doesn’t mean that we should stop predicting the weather. Just means we need to get better at it.

Bob: You know what we have a student doing right now? I just mentioned that as part of our related research is in natural language processing and machine learning, because we’re working on trying to automatically identify conversation features, to be able to scale up a lot of these studies of communication.

We’re now having them right now pulling up all of the forecasts from the National Oceanographic and Atmospheric Association, to use that as a training dataset, to be able to identify what is some of the uncertainty language that’s typically used in weather forecasting, that might be a way of, be a training dataset for how that functions in how our types of conversations. So it’s funny you should say weather. But there’s a lot of similarity and the words that show up in those things, which is interesting.

Alex: That’s fascinating.

Eric: Alex, you want to end us off with a little bit more?

Alex: Thank you, Bob. Appreciate you taking the time. This was great.

Bob: My pleasure. Thank you, everybody.

Alex: [Singing].

Eric: Ooh-bop-bop.

Bob: Feel the burn.

Alex: I was waiting for that. Ooh-bop-bop.

Eric: Bob, thank you very much. Thank you to all of our listeners this week. We look forward to having you listen to our podcast next week.

Alex: If you’re viewing this on iTunes or any other app, please rate us, give us the thumbs up, or don’t rate us.

Eric: No. Middle ground.

Anne: Thumbs up or nothing.

Eric: Goodbye, everybody.

Alex: Bye.